Almost every arrow turns when you transform space. Eigenvectors are the exception.
Hook
Almost Every Direction Turns
Almost every direction turns when you transform space. Almost.
- Transformations move arrows
- One direction is special
Shear
The Shear Transform
Take a shear. It pushes the top of a square sideways, like slanting a deck of cards.
- Bottom fixed, top slides
- Matrix: 1 1 / 0 1
Arrows Turn
Most Arrows Rotate
Watch a few arrows. Each one swings away from where it started.
- Angles change
- Directions are lost
The Fixed Direction
One Arrow Stays Put
But look along the horizontal axis. That arrow slides nowhere. It stays on its own line.
- Horizontal axis unchanged
- Stays on its own line
Eigenvector
Eigenvector Defined
That direction is an eigenvector. The transformation only scales it, never rotates it.
- Only scaled, never turned
- A v = lambda v
Eigenvalue
Eigenvalue: The Stretch
The scale amount is the eigenvalue. Here it is exactly one, so the arrow keeps its length.
- Scale factor is lambda
- Shear gives lambda equals 1
Stretch
A Bigger Stretch
Use a stretch instead, and that factor becomes two, or three. The eigenvalue is the stretch.
- Stretch scales the arrow
- Eigenvalue equals the factor
PCA Link
Eigenvectors Power PCA
This is how PCA works. The eigenvectors of the covariance matrix are the main directions of your data.
- Covariance matrix eigenvectors
- Main directions of data
Payoff
Follow for More
Follow for more machine learning, from zero.
- Machine Learning from Zero
- Next: SVD and PCA