Polynomial Features and Polynomial Regression

Polynomial features are higher power polynomial terms of the original features, which are added to the feature space of a model.

Let us understand this with a few examples.

Suppose we have a dataset with features x_1, x_2 and target variable y. A multivariable linear regression model for this set of data would be:

y_{pred}=w_1x_1+w_2x_2+b

Polynomial features are higher ordered values of x_1 and x_2 which we can add to this model, for eg. x_1^2, x_1^3, x_2^2 , etc.

Our new model would look like this:

y_{pred}=w_1x_1+w_2x_2+w_3x_1^2+w_4x_1^3+w_5x_2^2+b

This is known as a polynomial regression model.

We can also combine the features to create higher ordered terms like: x_1x_2 or x_1x_2^2 .

y_{pred}=w_1x_1+w_2x_2+w_3x_1^2+w_4x_1^3+w_5x_2^2+w_6(x_1x_2)+w_7(x_1x_2^2)+b

Important Note:

We treat the polynomial regression model just like a linear regression model, where each of the polynomial terms are additional features. Although new polynomial terms are added as features, the model is still a linear model in terms of the model parameters w and b. Hence, each of the new features are still linearly related to the outcome variable.

The training method is exactly the same as a linear regression model. We can use gradient descent to find the optimal values of each of the weights in our model.

Category: Machine Learning