Invertible-Manifold-Learning-for-Dimension-Reduction (ECML 2021)
Siyuan Li1,2, Haitao Lin1,2, Zelin Zang1,2, Lirong Wu1,2, Jun Xia1,2, Stan Z. Li†,1
The code is a deep neural network implementation of inv-ML, training and testing with toy datasets (Swiss roll, S Curve, Sphere) and real-world datasets (MNIST, USPS, KMNIST, Fashion-MNIST, COIL-20, etc.).
- pytorch == 1.3.1
- numpy == 1.17.2
- matplotlib == 3.1.1
- opencv-python == 4.4.0.42
- scikit-learn == 0.21.3
- scipy == 1.3.1
- imageio == 2.6.0
- ipython == 7.8.0
- main.py -- Train and test the model defined in the test_config.py.
- test_config.py -- Parameters to define i-ML-Enc models.
- dataset
- dataset.py -- Load data of selected toy datasets.
- dataloader.py -- Load data of selected real-world datasets.
- models
- InvML.py -- Define the MLP-based inv-ML-Enc model.
- loss
- InvML_loss.py -- Calculate losses of inv-ML-Enc: ℒLIS, ℒpush_away, ℒExtra, ℒOrth, ℒPadding.
- trainer
- invML_trainer.py -- Training loop for the network.
- invMLEnc_toy -- Toy version of i-ML-Enc for toy datasets.
- good_params -- Contain config file for all datasets, their results are given in Baidu Netdisk.
- scikit-learn_data -- Contain all datasets for testing i-ML-Enc. MNIST, FMNIST, KMNIST, and CIFAR-10 will be downloaded automatically; please download COIL-20, unzip it and put it in this folder.
- Install the required dependency packages.
- To get results on six real-world datasets (MNIST, USPS, KMNIST, FMNIST, COIL-20, CIFAR-10), run
bash run_test.sh
- To get results on two toy datasets (Swiss roll, S Curve), run
cd invMLEnc_toy
bash run_test.sh
Visualization results will be saved in the folder defined in run_test.sh, default are from "Test" to "Test8". You can try different datasets with config files in "./good_params".
-
Visualization of embeddings
We provide the config file and visualization results of eight datasets in [Data_baidu(code:5zfx)]. Take MNIST as an example: there are "+ExtraHead", "+Orth_loss", "+Padding", and "baseline" in the folder "./MNIST", which contains results and test_config.py.
-
Interpolation & reconstruction results
We also provide the interpolation results of MNIST, USPS, KMNIST, FMNIST in [Data_baidu(code:5zfx)]. You can find them under "./interpolation" and "./reconstruction".
- This repo borrows the architecture design and part of the code from MLDL.
If you are interested in our repository and our paper, please cite the following paper:
@inproceedings{Li2021InvertibleML,
title={Invertible Manifold Learning for Dimension Reduction},
author={Siyuan Li and Haitao Lin and Zelin Zang and Lirong Wu and Jun Xia and S. Li},
booktitle={ECML/PKDD},
year={2021}
}
Citation of the implementation of the MLDL paper.
@article{Li-MLDL-2020,
title = {Markov-Lipschitz Deep Learning},
author = {Stan Z Li and Zelin Zang and Lirong Wu},
journal = {arXiv preprint arXiv:2006.08256},
year = {2020}
}