Dimensionality reduction aims to reduce the number of features of a high dimensional dataset in-order to overcome the difficulties that arise due to the curse of dimensionality.

There are two approaches: feature selection and feature extraction. Feature selection focuses on finding a subset of the original attributes. Whereas feature extraction transforms the original high-dimensional space into a lower dimensional one. Ideally, some structure in the data should remain in order to preserve enough information. Algorithms can be unsupervised (principal component analysis or PCA, independent component analysis or ICA) or supervised (linear discriminant analysis or LDA). In feature extraction, transformations can be linear (PCA, LDA) or non-linear (t-SNE, autoencoders).

There are plenty of applications such as hidden patterns visualization (by removing highly correlated attributes), noise reduction (removing irrelevant features), further exploration, data compression and storage, etc. In fact, dimensionality reduction is usually applied as a preprocessing step for other machine learning and data ...

Dropout is a widely used regularization technique for neural networks. Neural networks, especially deep neural networks, are flexible machine learning algorithms and hence prone to overfitting. In this tutorial, we'll explain what is dropout and how it works, including a sample TensorFlow implementation.

If you [have] a deep neural net and it's not overfitting, you should probably be using a bigger one and using dropout, ... - Geoffrey Hinton [2]

Dropout algorithm

Dropout is a regularization technique where during each iteration of gradient descent, we drop a set of neurons selected at random. By drop, what we mean is that we essentially act as if they do not exist.

Logistic Regression is a variant of linear regression where dependent or output variable is categorical, i.e. it takes out of a few possible discrete values. Don't be confused by the name logistic regression, it's a classification algorithm.

In particular, we can use it for binary classification (two categories). For example, we might want to predict whether or not a person has diabetes, or whether or not an email is spam.

The logistic (or sigmoid) function

The term logistic in logistic regression comes from the logistic function (also known as sigmoid function), which can be written as:

The following is what we get if we plot f(z):

Overfitting is one of the most important problems (and concepts) in machine learning.

Generalization, Overfitting and Under-fitting

It's not a good idea to test a machine learning model on a dataset which we used to train it, since it won't give any indication of how well our model performs on unseen data. The ability to perform well on unseen data is called generalization, and is the desirable characteristic we want in a model.

When a model performs well on training data (the data on which the algorithm was trained) but does not perform well on test data (new or unseen data), we say that it has overfit the training data or that the model is overfitting. This happens because the model learns the noise present in the training data as if it was a reliable pattern.

Conversely, when a model does not perform well on tr...

Supervised Learning

Currently, most of the machine learning products use supervised learning. In this, we have a set of features or inputs X (for example, an image) and our model will predict a target or output variable y (for example, caption for the image).

In other words, our model learns a function that maps inputs to desired outputs. Features are independent variables and targets are the dependent variable.

Classification and Regression

Supervised learning problems can be further grouped ...

This tutorial describes the important components of a learning algorithm: representation (what the model looks like), evaluation (how do we differentiate good models from bad ones), and optimization (what is our process for finding the good models among all the possible models).

Linear Regression is a simple machine learning model for regression problems, i.e., when the target variable is a real value.

Example

Let's start with an example — suppose we have a dataset with information about the area of a house (in square feet) and its price (in thousands of dollars) and our task is to build a machine learning model which can predict the price given the area. Here is what our dataset looks like

Introduction and Overview

Gradient Descent is one of the most popular and widely used optimization algorithms. Given a machine learning model with parameters (weights and biases) and a cost function to evaluate how good a particular model is, our learning problem reduces to that of finding a good set of weights for our model which minimizes the cost function.