research-article

AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting

Authors:

Christian S. JensenAuthors Info & Claims

Proceedings of the ACM on Management of Data, Volume 1, Issue 1

Article No.: 97, Pages 1 - 26

https://doi.org/10.1145/3588951

Published: 30 May 2023 Publication History

Abstract

Sensors in cyber-physical systems often capture interconnected processes and thus emit correlated time series (CTS), the forecasting of which enables important applications. The key to successful CTS forecasting is to uncover the temporal dynamics of time series and the spatial correlations among time series. Deep learning-based solutions exhibit impressive performance at discerning these aspects. In particular, automated CTS forecasting, where the design of an optimal deep learning architecture is automated, enables forecasting accuracy that surpasses what has been achieved by manual approaches. However, automated CTS solutions remain in their infancy and are only able to find optimal architectures for predefined hyperparameters and scale poorly to large-scale CTS. To overcome these limitations, we propose AutoCTS+, a joint, scalable framework, to automatically devise effective CTS forecasting models. Specifically, we encode each candidate architecture and accompanying hyperparameters into a joint graph representation. We introduce an efficient Architecture-Hyperparameter Comparator (AHC) to rank all architecture-hyperparameter pairs, and we then further evaluate the top-ranked pairs to select an architecture-hyperparameter pair as the final model. Extensive experiments on six benchmark datasets demonstrate that AutoCTS+ not only eliminates manual efforts but also is capable of better performance than manually designed and existing automatically designed CTS models. In addition, it shows excellent scalability to large CTS.

Supplemental Material

MP4 File

Presentation video for SIGMOD 2023

Download
30.02 MB

References

[1]

Mohamed S Abdelfattah, Abhinav Mehrotra, Łukasz Dudziak, and Nicholas Donald Lane. 2020. Zero-Cost Proxies for Lightweight NAS. In International Conference on Learning Representations.

[2]

Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting. In NeurIPS, Vol. 33. 17804--17815.

[3]

Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. 2018. Understanding and simplifying one-shot architecture search. In International conference on machine learning. PMLR, 550--559.

[4]

David Campos, Miao Zhang, Bin Yang, Tung Kieu, Chenjuan Guo, and Christian S. Jensen. 2023. LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation. SIGMOD (2023).

[5]

Wuyang Chen, Xinyu Gong, and Zhangyang Wang. 2021a. Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective. In International Conference on Learning Representations (ICLR).

[6]

Yaofo Chen, Yong Guo, Qi Chen, Minli Li, Wei Zeng, Yaowei Wang, and Mingkui Tan. 2021b. Contrastive neural architecture search with neural architecture comparators. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9502--9511.

[7]

Razvan-Gabriel Cirstea, Chenjuan Guo, Bin Yang, Tung Kieu, Xuanyi Dong, and Shirui Pan. 2022a. Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting. In IJCAI, Luc De Raedt (Ed.). 1994--2001.

[8]

Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. 2022b. Towards Spatio-Temporal Aware Traffic Time Series Forecasting. In ICDE. 2900--2913.

[9]

Razvan-Gabriel Cirstea, Tung Kieu, Chenjuan Guo, Bin Yang, and Sinno Jialin Pan. 2021. EnhanceNet: Plugin Neural Networks for Enhancing Correlated Time Series Forecasting. In ICDE. 1739--1750.

[10]

Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, et al. 2021. Fbnetv3: Joint architecture-recipe search using predictor pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16276--16285.

[11]

Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, and Quoc V Le. 2020. AutoHAS: Efficient hyperparameter and architecture search. arXiv preprint arXiv:2006.03656 (2020).

[12]

Lukasz Dudziak, Thomas Chau, Mohamed Abdelfattah, Royson Lee, Hyeji Kim, and Nicholas Lane. 2020. Brp-nas: Prediction-based nas using gcns. Advances in Neural Information Processing Systems, Vol. 33 (2020), 10480--10490.

[13]

Ori Bar El, Tova Milo, and Amit Somech. 2020. Automatically Generating Data Exploration Sessions Using Deep Reinforcement Learning. In Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14--19, 2020, David Maier, Rachel Pottinger, AnHai Doan, Wang-Chiew Tan, Abdussalam Alawini, and Hung Q. Ngo (Eds.). ACM, 1527--1537.

[14]

Christos Faloutsos, Jan Gasthaus, Tim Januschowski, and Yuyang Wang. 2019. Classical and Contemporary Approaches to Big Time Series Forecasting. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019. ACM, 2042--2047.

Digital Library

[15]

Chenjuan Guo, Bin Yang, Jilin Hu, Christian S. Jensen, and Lu Chen. 2020a. Context-aware, preference-based vehicle routing. VLDB J., Vol. 29, 5 (2020), 1149--1170.

[16]

Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun. 2020b. Single path one-shot neural architecture search with uniform sampling. In European conference on computer vision. Springer, 544--560.

Digital Library

[17]

Daniel Kang, Nikos Aré chiga, Sudeep Pillai, Peter D. Bailis, and Matei Zaharia. 2022. Finding Label and Model Errors in Perception Data With Learned Observation Assertions. In SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022. ACM, 496--505.

[18]

Tung Kieu, Bin Yang, Chenjuan Guo, Razvan-Gabriel Cirstea, Yan Zhao, Yale Song, and Christian S. Jensen. 2022a. Anomaly Detection in Time Series with Robust Variational Quasi-Recurrent Autoencoders. In ICDE. 1342--1354.

[19]

Tung Kieu, Bin Yang, Chenjuan Guo, Christian S. Jensen, Yan Zhao, Feiteng Huang, and Kai Zheng. 2022b. Robust and Explainable Autoencoders for Unsupervised Time Series Outlier Detection. In ICDE. 3038--3050.

[20]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[21]

Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR. 95--104.

[22]

Ting Li, Junbo Zhang, Kainan Bao, Yuxuan Liang, Yexin Li, and Yu Zheng. 2020. Autost: Efficient neural architecture search for spatio-temporal prediction. In SIGKDD. 794--802.

[23]

Yuening Li, Zhengzhang Chen, Daochen Zha, Kaixiong Zhou, Haifeng Jin, Haifeng Chen, and Xia Hu. 2021a. AutoOD: Neural Architecture Search for Outlier Detection. In 37th IEEE International Conference on Data Engineering, ICDE 2021, Chania, Greece, April 19--22, 2021. IEEE, 2117--2122.

[24]

Yang Li, Yu Shen, Wentao Zhang, Jiawei Jiang, Yaliang Li, Bolin Ding, Jingren Zhou, Zhi Yang, Wentao Wu, Ce Zhang, and Bin Cui. 2021b. VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition. Proc. VLDB Endow., Vol. 14, 11 (2021), 2167--2176.

Digital Library

[25]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In ICLR.

[26]

Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. DARTS: Differentiable Architecture Search. In ICLR.

[27]

Hao Miao, Jiaxing Shen, Jiannong Cao, Jiangnan Xia, and Senzhang Wang. 2022. MBA-STNet: Bayes-enhanced Discriminative Multi-task Learning for Flow Prediction. IEEE Transactions on Knowledge and Data Engineering (2022).

[28]

Xuefei Ning, Changcheng Tang, Wenshuo Li, Zixuan Zhou, Shuang Liang, Huazhong Yang, and Yu Wang. 2021. Evaluating efficient performance estimators of neural architectures. Advances in Neural Information Processing Systems, Vol. 34 (2021), 12265--12277.

[29]

Zheyi Pan, Songyu Ke, Xiaodu Yang, Yuxuan Liang, Yong Yu, Junbo Zhang, and Yu Zheng. 2021. AutoSTG: Neural Architecture Search for Predictions of Spatio-Temporal Graphs. In WWW. 1846--1855.

[30]

Cheonbok Park, Chunggi Lee, Hyojin Bahng, Yunwon Tae, Seungmin Jin, Kihwan Kim, Sungahn Ko, and Jaegul Choo. 2020. ST-GRAT: A Novel Spatio-temporal Graph Attention Networks for Accurately Forecasting Dynamically Changing Road Speed. In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19--23, 2020, Mathieu d'Aquin, Stefan Dietze, Claudia Hauff, Edward Curry, and Philippe Cudré -Mauroux (Eds.). ACM, 1215--1224.

Digital Library

[31]

Simon Aagaard Pedersen, Bin Yang, and Christian S. Jensen. 2020a. Anytime Stochastic Routing with Hybrid Learning. Proc. VLDB Endow., Vol. 13, 9 (2020), 1555--1567.

Digital Library

[32]

Simon Aagaard Pedersen, Bin Yang, and Christian S. Jensen. 2020b. Fast stochastic routing under time-varying uncertainty. VLDB J., Vol. 29, 4 (2020), 819--839.

[33]

Ragunathan Rajkumar, Insup Lee, Lui Sha, and John Stankovic. 2010. Cyber-physical systems: the next computing revolution. In Design automation conference. IEEE, 731--736.

[34]

Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International conference on machine learning. PMLR, 4334--4343.

[35]

Syed Yousaf Shah, Dhaval Patel, Long Vu, Xuan-Hong Dang, Bei Chen, Peter Kirchner, Horst Samulowitz, David Wood, Gregory Bramble, Wesley M. Gifford, Giridhar Ganapavarapu, Roman Vacul'i n, and Petros Zerfos. 2021. AutoAI-TS: AutoAI for Time Series Forecasting. In SIGMOD '21: International Conference on Management of Data, Virtual Event, China, June 20--25, 2021. ACM, 2584--2596.

[36]

Shun-Yao Shih, Fan-Keng Sun, and Hung-yi Lee. 2019. Temporal pattern attention for multivariate time series forecasting. Machine Learning, Vol. 108, 8 (2019), 1421--1441.

Digital Library

[37]

Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. 2019. Meta-weight-net: Learning an explicit mapping for sample weighting. Advances in neural information processing systems, Vol. 32 (2019).

[38]

Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2020. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 914--921.

[39]

Senzhang Wang, Hao Miao, Hao Chen, and Zhiqiu Huang. 2020b. Multi-task adversarial spatial-temporal networks for crowd flow prediction. In Proceedings of the 29th ACM international conference on information & knowledge management. 1555--1564.

Digital Library

[40]

Senzhang Wang, Meiyue Zhang, Hao Miao, Zhaohui Peng, and Philip S Yu. 2022. Multivariate correlation-aware spatio-temporal graph convolutional networks for multi-scale traffic prediction. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 13, 3 (2022), 1--22.

Digital Library

[41]

Xiaoyang Wang, Yao Ma, Yiqi Wang, Wei Jin, Xin Wang, Jiliang Tang, Caiyan Jia, and Jian Yu. 2020a. Traffic Flow Prediction via Spatial Temporal Graph Neural Network. In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20--24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 1082--1092.

Digital Library

[42]

Xinle Wu, Dalin Zhang, Chenjuan Guo, Chaoyang He, Bin Yang, and Christian S Jensen. 2022. AutoCTS: Automated correlated time series forecasting. Proc. VLDB Endow, Vol. 15, 4 (2022), 971--983.

Digital Library

[43]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. In SIGKDD. 753--763.

[44]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10--16, 2019. 1907--1913.

[45]

Sean Bin Yang, Chenjuan Guo, Jilin Hu, Bin Yang, Jian Tang, and Christian S. Jensen. 2022. Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning. In ICDE. 2873--2885.

[46]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In IJCAI. 3634--3640.

[47]

Zizhao Zhang, Han Zhang, Sercan O Arik, Honglak Lee, and Tomas Pfister. 2020. Distilling effective supervision from severe label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9294--9303.

[48]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In AAAI, Vol. 35. 11106--11115.

[49]

Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings. OpenReview.net.

Cited By

Shao ZWang FXu YWei WYu CZhang ZYao DSun TJin GCao XCong GJensen CCheng X(2025)Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity AnalysisIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348445437:1(291-305)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3484454
Qiu XHu JZhou LWu XDu JZhang BGuo CZhou AJensen CSheng ZYang B(2024)TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsProceedings of the VLDB Endowment10.14778/3665844.366586317:9(2363-2377)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.14778/3665844.3665863
Tang XZhang FZhang SLiu YHe BHe BDu XDu X(2024)Enabling Adaptive Sampling for Intra-Window Join: Simultaneously Optimizing Quantity and QualityProceedings of the ACM on Management of Data10.1145/36771342:4(1-31)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3677134
Show More Cited By

Index Terms

AutoCTS+: Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting
1. Information systems
  1. Information systems applications
    1. Data mining
    2. Spatial-temporal systems

Recommendations

AutoCTS++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting
Abstract
Sensors in cyber-physical systems often capture interconnected processes and thus emit correlated time series (CTS), the forecasting of which enables important applications. Recent deep learning based forecasting methods show strong capabilities ...
Correlated Time Series Forecasting using Multi-Task Deep Neural Networks
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

Cyber-physical systems often consist of entities that interact with each other over time. Meanwhile, as part of the continued digitization of industrial processes, various sensor technologies are deployed that enable us to record time-varying attributes (...
Time series forecasting by a seasonal support vector regression model

The support vector regression (SVR) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the applications of SVR models in a seasonal time series forecasting has not been widely investigated. This ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Management of Data

Proceedings of the ACM on Management of Data Volume 1, Issue 1

PACMMOD

May 2023

2807 pages

EISSN:2836-6573

DOI:10.1145/3603164

Editor:
Divyakant Agrawal
UC Santa Barbara, United States

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 May 2023

Published in PACMMOD Volume 1, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Author Tags

Qualifiers

Research-article

Funding Sources

Innovation Fund Denmark
Villum Fonden
Independent Research Fund Denmark

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
378
Total Downloads

Downloads (Last 12 months)184
Downloads (Last 6 weeks)5

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shao ZWang FXu YWei WYu CZhang ZYao DSun TJin GCao XCong GJensen CCheng X(2025)Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity AnalysisIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348445437:1(291-305)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3484454
Qiu XHu JZhou LWu XDu JZhang BGuo CZhou AJensen CSheng ZYang B(2024)TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting MethodsProceedings of the VLDB Endowment10.14778/3665844.366586317:9(2363-2377)Online publication date: 6-Aug-2024
https://dl.acm.org/doi/10.14778/3665844.3665863
Tang XZhang FZhang SLiu YHe BHe BDu XDu X(2024)Enabling Adaptive Sampling for Intra-Window Join: Simultaneously Optimizing Quantity and QualityProceedings of the ACM on Management of Data10.1145/36771342:4(1-31)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3677134
Li ZXu RHu JPeng ZLu XGuo CYang BSerra ESpezzano F(2024)Ocean Significant Wave Height Estimation with Spatio-temporally Aware Large Language ModelsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679973(3892-3896)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679973
Ren XZhao KTakova KRiddle PLi LSerra ESpezzano F(2024)Periormer: Periodic Transformer for Seasonal and Irregularly Sampled Time SeriesProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679720(1973-1982)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679720
Fan SHou MXi RMa WSerra ESpezzano F(2024)Precision Meets Resilience: Cross-Database Generalization with Uncertainty Quantification for Robust Cost EstimationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679632(581-590)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679632
Jensen CYang BGuo CHu JTorp K(2024)Routing with Massive Trajectory Data2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00442(5542-5547)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00442
Liu ZMiao HZhao YLiu CZheng KLi H(2024)LightTR: A Lightweight Framework for Federated Trajectory Recovery2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00337(4422-4434)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00337
Miao HZhao YGuo CYang BZheng KHuang FXie JJensen C(2024)A Unified Replay-Based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00085(1050-1062)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00085
Zeng HDuan XHuang XCui Q(2024)STA-former: encoding traffic flows with spatio-temporal associations in transformer networks for predictionCluster Computing10.1007/s10586-024-04462-y27:7(9693-9714)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1007/s10586-024-04462-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents