From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting
Authors: 
Zhen Zeng, Rachneet Kaur, Suchetha Siddagangappa, Tucker Balch, Manuela VelosoAuthors Info & Claims
Pages 82 - 90
Abstract
Time series forecasting plays a crucial role in decision-making across various domains, but it presents significant challenges. Recent studies have explored image-driven approaches using computer vision models to address these challenges, often employing lineplots as the visual representation of time series data. In this paper, we propose a novel approach that uses time-frequency spectrograms as the visual representation of time series data. We introduce the use of a vision transformer for multimodal learning, showcasing the advantages of our approach across diverse datasets from different domains. To evaluate its effectiveness, we compare our method against statistical baselines (EMA and ARIMA), a state-of-the-art deep learning-based approach (DeepAR), other visual representations of time series data (lineplot images), and an ablation study on using only the time series as input. Our experiments demonstrate the benefits of utilizing spectrograms as a visual representation for time series data, along with the advantages of employing a vision transformer for simultaneous learning in both the time and frequency domains.
References
[1]
2023. Auto ARIMA. https://www.rdocumentation.org/packages/forecast/versions/8.21/topics/auto.arima. Accessed: 2023-06-20.
[2]
Alvaro Arroyo, Bruno Scalzo, Ljubiša Stanković, and Danilo P Mandic. 2022. Dynamic Portfolio Cuts: A Spectral Approach to Graph-Theoretic Diversification. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5468–5472.
[3]
Federico M. Bandi, Shomesh E. Chaudhuri, Andrew W. Lo, and Andrea Tamoni. 2021. Spectral factor models. Journal of Financial Economics 142, 1 (2021), 214–238. https://doi.org/10.1016/j.jfineco.2021.04.024
[4]
Wei Bao, Jun Yue, and Yulei Rao. 2017. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PloS one 12, 7 (2017), e0180944.
[5]
Shomesh Chaudhuri and Andrew W Lo. 2016. Spectral portfolio theory. Available at SSRN 2788999 (2016).
[6]
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
[7]
Naftali Cohen, Tucker Balch, and Manuela Veloso. 2020. Trading via image classification. In Proceedings of the First ACM International Conference on AI in Finance. 1–6.
[8]
Ingrid Daubechies. 1990. The wavelet transform, time-frequency localization and signal analysis. IEEE transactions on information theory 36, 5 (1990), 961–1005.
[9]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
[10]
Bairui Du and Paolo Barucca. 2020. Image Processing Tools for Financial Time Series Classification. arXiv preprint arXiv:2008.06042 (2020).
[11]
Mordechai Frisch and Hagit Messer. 1992. The use of the wavelet transform in the detection of an unknown transient signal. IEEE Transactions on Information Theory 38, 2 (1992), 892–897.
[12]
Everette S Gardner Jr and ED McKenzie. 1985. Forecasting trends in time series. Management Science 31, 10 (1985), 1237–1246.
[13]
Andre Gensler, Janosch Henze, Bernhard Sick, and Nils Raabe. 2016. Deep Learning for solar power forecasting—An approach using AutoEncoder and LSTM Neural Networks. In 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, 002858–002865.
[14]
Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I. Webb, Rob J. Hyndman, and Pablo Montero-Manso. 2021. Monash Time Series Forecasting Archive. In Neural Information Processing Systems Track on Datasets and Benchmarks.
[15]
Daniel Griffin and Jae Lim. 1984. Signal estimation from modified short-time Fourier transform. IEEE Transactions on acoustics, speech, and signal processing 32, 2 (1984), 236–243.
[16]
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The elements of statistical learning. Springer series in statistics. New York, NY, USA (2001).
[17]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
[18]
Charles C Holt. 2004. Forecasting seasonals and trends by exponentially weighted moving averages. International journal of forecasting 20, 1 (2004), 5–10.
[19]
Rob J Hyndman and Yeasmin Khandakar. 2008. Automatic time series forecasting: the forecast package for R. Journal of statistical software 27 (2008), 1–22.
[20]
Xixi Li, Yanfei Kang, and Feng Li. 2020. Forecasting with time series imaging. Expert Systems with Applications 160 (2020), 113680.
[21]
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2020. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting 36, 1 (2020), 54–74.
[22]
Massimiliano Marcellino, James H Stock, and Mark W Watson. 2006. A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. Journal of econometrics 135, 1-2 (2006), 499–526.
[23]
Fernando Moreno-Pino, Pablo M Olmos, and Antonio Artés-Rodríguez. 2023. Deep autoregressive models with spectral attention. Pattern Recognition 133 (2023), 109014.
[24]
Lasse Heje Pedersen. 2019. Efficiently inefficient: how smart money invests and market prices are determined. Princeton University Press.
[25]
Sharif Md Raihan, Yi Wen, and Bing Zeng. 2005. Wavelet: A new tool for business cycle analysis. Federal Reserve Bank of St. Louis Working Paper Series2005-050 (2005).
[26]
Pablo Romeu, Francisco Zamora-Martínez, Paloma Botella-Rocamora, and Juan Pardo. 2015. Stacked denoising auto-encoders for short-term time series forecasting. In Artificial Neural Networks. Springer, 463–486.
[27]
Alaa Sagheer and Mostafa Kotb. 2019. Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing 323 (2019), 203–213.
[28]
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181–1191.
[29]
Bruno Scalzo, Alvaro Arroyo, Ljubiša Stanković, and Danilo P Mandic. 2021. Nonstationary portfolios: Diversification in the spectral domain. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5155–5159.
[30]
Artemios-Anargyros Semenoglou, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2023. Image-based time series forecasting: A deep convolutional neural network approach. Neural Networks 157 (2023), 39–53.
[31]
Srijan Sood, Zhen Zeng, Naftali Cohen, Tucker Balch, and Manuela Veloso. 2021. Visual time series forecasting: an image-driven approach. In Proceedings of the Second ACM International Conference on AI in Finance. 1–9.
[32]
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems 27 (2014).
[33]
Souhaib Ben Taieb, Antti Sorjamaa, and Gianluca Bontempi. 2010. Multiple-output modeling for multi-step-ahead time series forecasting. Neurocomputing 73, 10-12 (2010), 1950–1957.
[34]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[35]
Daniel S Wilks. 2011. Statistical methods in the atmospheric sciences. Vol. 100. Academic press.
[36]
Peter R Winters. 1960. Forecasting sales by exponentially weighted moving averages. Management science 6, 3 (1960), 324–342.
[37]
Neo Wu, Bradley Green, Xue Ben, and Shawn O’Banion. 2020. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317 (2020).
[38]
Zhen Zeng, Tucker Balch, and Manuela Veloso. 2021. Deep video prediction for time series forecasting. In Proceedings of the Second ACM International Conference on AI in Finance. 1–7.
[39]
Zhen Zeng, Rachneet Kaur, Suchetha Siddagangappa, Saba Rahimi, Tucker Balch, and Manuela Veloso. 2023. Financial Time Series Forecasting using CNN and Transformer. arXiv preprint arXiv:2304.04912 (2023).
[40]
Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. 2022. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems 35 (2022), 3988–4003.
[41]
Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2019. Deeplob: Deep convolutional neural networks for limit order books. IEEE Transactions on Signal Processing 67, 11 (2019), 3001–3012.
[42]
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 11106–11115.
[43]
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. PMLR, 27268–27286.
Index Terms
- From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting
Index terms have been assigned to the content through auto-classification.
Recommendations
Visual time series forecasting: an image-driven approach
ICAIF '21: Proceedings of the Second ACM International Conference on AI in FinanceTime series forecasting is essential for agents to make decisions. Traditional approaches rely on statistical methods to forecast given past numeric values. In practice, end-users often rely on visualizations such as charts and plots to reason about ...
TSF-transformer: a time series forecasting model for exhaust gas emission using transformer
AbstractMonitoring and prediction of exhaust gas emissions for heavy trucks is a promising way to solve environmental problems. However, the emission data acquisition is time delayed and the pattern of emission is usually irregular, which makes it very ...
TS-Fastformer: Fast Transformer for Time-series Forecasting
Many real-world applications require precise and fast time-series forecasting. Recent trends in time-series forecasting models are shifting from LSTM-based models to Transformer-based models. However, the Transformer-based model has a limited ability to ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2023
697 pages
ISBN:9798400702402
DOI:10.1145/3604237
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 November 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICAIF '23
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 418Total Downloads
- Downloads (Last 12 months)418
- Downloads (Last 6 weeks)17
Reflects downloads up to 01 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format