Part of course:

Linear Regression Tutorial with Example

- Example
- Linear regression
- Simple linear regression
- Multiple linear regression
- Cost functions
- Residuals
- Optimization using Gradient Descent

NaN.

Linear Regression Tutorial with Example[ Edit ]

Linear Regression is a simple machine learning model for regression problems, i.e., when the target variable is a real value.

Let's start with an example — suppose we have a dataset with information about the area of a house (in square feet) and its price (in thousands of dollars) and our task is to build a machine learning model which can predict the price given the area. Here is what our dataset looks like

If we plot our data, we might get something similar to the following:

In **simple linear regression**, we **establish a relationship between target variable and input variables by fitting a line**, known as the regression line.

In general, a line can be represented by linear equation **y = m * X + b**. Where, y is the dependent variable, X is the independent variable, m is the slope, b is the intercept.

In machine learning, we rewrite our equation as **y(x) = w _{0} + w_{1} * x** where w's are the

The above equation can be used when we have one input variable (also called **feature**). However, in general, we usually deal with datasets which have multiple input variables. The case when we have more than one feature is known as **multiple linear regression**, or simply, **linear regression**. We can generalize our previous equation for simple linear regression to multiple linear regression:

y(x) = w_0 + w_1 x_1 + w_2 x_2 + ... + w_n x_n

In the case of multiple linear regression, instead of our prediction being a line in 2-dimensional space, it is a hyperplane in n-dimensional space. For example, in 3D, our plot would look as follows

Different values of the weights (w_{0}, w_{1}, w_{2}, ... w_{n}) gives us different lines, and our task is to find weights for which we get best fit. One question you may have is, how can we determine how well a particular line fits our data? Or, given two lines, how do we determine which one is better? For this, we introduce a **cost function** which measures, given a particular value for the *w*'s, how close the *y*'s are to corresponding *y _{true}*'s. That is, how well do a particular set of weights predict the target value.

For linear regression, we use the **mean squared error** cost function. It is the average over the various data points (*x _{i}*,

J(w)=\frac{1}{n}\sum\limits_{i=1}^n (y(x^i) - y_{true}^i)^2

The cost function defines a cost based on the distance between true target and predicted target (shown in the graph as lines between sample points and the regression line), also known as the **residual. **The residuals are visualized below,

If a particular line is far from all the points, the residuals will be higher, and so will the cost function. If a line is close to the points, the residuals will be small, and hence the cost function.

Each value of the weight vector *w* gives us a corresponding cost *J*(*w*). We want to find the value of weights for which cost is minimum. We can visualize this as follows:

Note: Above we have used the word "global" because the shape of the cost-function for linear regression is convex (i.e. like a bowl). It has a single minimum, and it smoothly increases in all directions around it.

Given the linear regression model and the cost function, we can use **Gradient Descent** (covered in the next article) to find a good set of values for the weight vector. This process of finding the best model out of the many possible models is called **optimization**.

Read more…(666 words)

Mark as completed

Part of lists:

Previous

What is Machine Learning? Why Machine Learning?

Next

Gradient Descent, an intuitive introduction

About the contributors:

Savan VisalparaMachine Learning Practitioner

50%

Keshav DhandhaniaMSc in Deep Learning @ MIT (2014)

49%

Prabhav JainCo-founder, Commonlounge. MIT EECS.

1%

Loading…

Have a question? Ask here…

Post

Part of course:

Linear Regression Tutorial with Example

- Example
- Linear regression
- Simple linear regression
- Multiple linear regression
- Cost functions
- Residuals
- Optimization using Gradient Descent

Contributors

Savan VisalparaMachine Learning Practitioner

50%

Keshav DhandhaniaMSc in Deep Learning @ MIT (2014)

49%

Prabhav JainCo-founder, Commonlounge. MIT EECS.

1%

Ready to join our community?

Sign up below to automatically get notified of new courses, get **reminders** to finish ones you subscribe to, and **bookmark** lessons to read later.

Continue with Facebook

— OR —

Your Full Name

Email address

I have an account. Log in instead

By signing up, you agree to our Terms and our Privacy Policy.

Popular Courses

New Courses

Get in touch

Copyright 2016-18, Compose Labs Inc. All rights reserved.