Part of course:
Tutorial: Linearly separable data
- Algebraic definition:
The idea of linearly separable is easiest to visualize and understand in 2 dimensions. Let the two classes be represented by colors red and green.
A dataset is said to be linearly separable if it is possible to draw a line that can separate the red and green points from each other.
Here are same examples of linearly separable data:
And here are some examples of linearly non-separable data
This concept can be extended to three or more dimensions as well. For example, below is an example of a three dimensional dataset that is linearly separable
In n dimensions, the separator is a (n-1) dimensional hyperplane - although it is pretty much impossible to visualize for 4 or more dimensions.
Algebraically, the separator is a linear function, i.e. if data point x is given by (x1, x2), when the separator is a function f(x) = w1*x1 + w2*x2 + b
All points for which f(x) = 0, are on the separator line. All points for which f(x) > 0 are on one side of the line, and all points for which f(x) < 0 are on the other side.