Can't separate your data with a line? Lift it to a higher dimension, without ever going there.
Hook
Impossible to Split?
This data cannot be split by any single cut. And yet one trick makes it easy.
- 1D data
- No single cut works
Problem
One Threshold Fails
Here is a line of points. Blue sits in the middle. Red sits on both sides. One threshold always fails.
- Blue in the middle
- Red on both sides
Lift
Lift With x Squared
So lift it. Give every point a second coordinate: x squared.
- Add a coordinate
- phi(x) = (x, x²)
Blue Low, Red High
Now the points sit on a parabola. Blue stays low. Red climbs high.
- Points on a parabola
- Height reveals class
Separate
A Line Splits Them
A flat horizontal line now splits them perfectly. Back in 1D, that line becomes two cuts.
- Linear in 2D
- Curved in 1D
Cost
The Cost of Lifting
But lifting costs. With thousands of features, the new space can become enormous, even infinite.
- Dimensions explode
- Even infinite
Trick
Skip the Lift
Here is the trick. SVMs only need dot products between points. A kernel computes that dot product without ever building the lift.
- SVMs need dot products
- k(x,z) = φ(x)·φ(z)
Payoff
Free Dimensions
Higher dimensions, for free. Lesson forty-three shows exactly how. Follow, so you don't miss it.
- Full proof in lesson 43