The PyTorch implementation of Linformer from the paper Linformer: Self-Attention with Linear Complexity.
@article{wang2020linformer,
title = {Linformer: Self-Attention with Linear Complexity},
author = {Sinong Wang and Belinda Z. Li and Madian Khabsa and Han Fang and Hao Ma},
year = {2020},
journal = {arXiv preprint arXiv: Arxiv-2006.04768}
}
- Create a data folder:
mkdir data- Download the dataset compressed archive
wget $URL- Decompress the dataset compressed archive and put the contents into the data folder
unzip $dataset.zip
mv $datast ./data/$datast- Run the main file
python $dataset_main.py --task="$task"To install requirements:
pip3 install -r requirements.txt