Non-lineaer principal component analysis

A Dimensionality reduction technique similar to a regular Principal component analysis, but it can work with non-linear relations.

There are multiple methods for doing this. Here is one I prefer.

1. Start with your features.
2. Set error to your features.
3. Train two models, one that encodes error into 1 value, and another that turns that 1 value into error. This step is essentially an auto-encoder.
4. Take the difference of the predictions and error, this becomes the new error.
5. Do steps 3 and 4 for N iterations to end up with N principal components.

This gives us something like a PCA, or a greedy autoencoder.

Leave a Comment