Dimensionality reduction aims to reduce the number of features of a high dimensional dataset in order to overcome the difficulties that arise due to the curse of dimensionality.

There are two approaches: feature selection and feature extraction. Feature selection focuses on finding a subset of the original attributes. Whereas feature extraction transforms the original high-dimensional space into a lower dimensional one. Ideally, some structure in the data should remain in order to preserve enough information. Algorithms can be unsupervised (principal component analysis or PCA, independent component analysis or ICA) or supervised (linear discriminant analysis or LDA). In feature extraction, transformations can be linear (PCA, LDA) or non-linear (t-SNE, autoencoders).

There are plenty of applications such as hidden patterns visualization (by removing highly correlated attributes), noise reduction (removing irrelevant features), further exploration, data compression and storage, etc.

In fact, dimensionality reduction is usually applied...

Dropout is a widely used regularization technique for neural networks. Neural networks, especially deep neural networks, are flexible machine learning algorithms and hence prone to overfitting. In this tutorial, we'll explain what is dropout and how it works, including a sample TensorFlow implementation.

If you [have] a deep neural net and it's not overfitting, you should probably be using a bigger one and using dropout, ... - Geoffrey Hinton [2]

Dropout algorithm

Dropout is a regularization technique where during each iteration of gradient descent, we drop a set of neurons selected at random. By drop, what we mean is that we essentially act as if they do not exist.

Logistic Regression is a variant of linear regression where the target variable is categorical, i.e. it takes one out of a few possible discrete values. Each of these values is called a label. Don't be confused by the name logistic regression, it's a classification algorithm.

In particular, we can use logistic regression for binary classification (two labels). For example, we might want to predict whether or not a person has diabetes, or whether or not an email is spam.

The logistic (or sigmoid) function

The term logistic in logistic regression comes from the logistic function (also known as sigmoid function), which can be written as:

In the last tutorial, we learned about generalization and overfitting. In this tutorial, we will discuss various methods to deal with overfitting.

1. Feature selection: In the previous tutorial, we saw that as the number of features grew, the model became more prone to overfitting. Hence, reducing the number of features, also known as feature selection, can reduce overfitting.
2. Regularization: Here, we penalize the model for having weights / parameters with large magnitude. This is achieved by modifying the cost function to include a penalty term based on the magnitude of the model weights. So minimizing the cost function also leads to a preference for smaller weights.

Another way to think about feature selection and regularization is as follows. Feature selection is equivalent to setting some model weights to 0. If a model weight is zero, the corresponding ...

There are three main types of learning algorithms in machine learning: supervised learning, unsupervised learning, and reinforcement learning.

Supervised Learning

Currently, most of the machine learning products use supervised learning. In this, we have a set of features or inputs X (for example, an image) and our model will predict a target or output variable y (for example, caption for the image).

In other words, our model learns a function that maps inputs to desired outputs. Features are independent variables and targets are the dependent variable.

Classification and Regression

This tutorial describes the important components of a learning algorithm: representation (what the model looks like), evaluation (how do we differentiate good models from bad ones), and optimization (what is our process for finding the good models among all the possible models).

Linear Regression is a simple machine learning model for regression problems, i.e., when the target variable is a real value.

Example problem

Let's start with an example — suppose we have a dataset with information about the area of a house (in square feet) and its price (in thousands of dollars) and our task is to build a machine learning model which can predict the price given the area.

Here is what our dataset looks like

Introduction and Overview

Gradient Descent is one of the most popular and widely used optimization algorithms.

Let's say we are given a machine learning model (parameterized by weights and biases) and a cost function to evaluate how good a particular model is. Then, our learning problem reduces to that of finding the values of the model parameters which minimize the cost function.