research-article

DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models

Authors:

Roger Zimmermann,

Yuxuan LiangAuthors Info & Claims

SIGSPATIAL '23: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems

Article No.: 60, Pages 1 - 12

https://doi.org/10.1145/3589132.3625614

Published: 22 December 2023 Publication History

Abstract

Spatio-temporal graph neural networks (STGNN) have emerged as the dominant model for spatio-temporal graph (STG) forecasting. Despite their success, they fail to model intrinsic uncertainties within STG data, which cripples their practicality in downstream tasks for decision-making. To this end, this paper focuses on probabilistic STG forecasting, which is challenging due to the difficulty in modeling uncertainties and complex ST dependencies. In this study, we present the first attempt to generalize the popular de-noising diffusion probabilistic models to STGs, leading to a novel non-autoregressive framework called DiffSTG, along with the first denoising network UGnet for STG in the framework. Our approach combines the spatio-temporal learning capabilities of STGNNs with the uncertainty measurements of diffusion models. Extensive experiments validate that DiffSTG reduces the Continuous Ranked Probability Score (CRPS) by 4%-14%, and Root Mean Squared Error (RMSE) by 2%-7% over existing methods on three real-world datasets.

References

[1]

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).

[2]

Ping Chang, Huayu Li, Stuart F Quan, Janet Roveda, and Ao Li. 2023. TDSTF: Transformer-based Diffusion probabilistic model for Sparse Time series Forecasting. arXiv preprint arXiv:2301.06625 (2023).

[3]

Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent neural networks for multivariate time series with missing values. Scientific reports 8, 1 (2018), 6085.

[4]

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. 2020. WaveGrad: Estimating Gradients for Waveform Generation. In International Conference on Learning Representations.

[5]

Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, and Liang Sun. 2022. Learning to rotate: Quaternion transformer for complicated periodical time series forecasting. In KDD. 146--156.

[6]

Jeongwhan Choi, Hwangyong Choi, Jeehyun Hwang, and Noseong Park. 2022. Graph neural controlled differential equations for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 6367--6374.

[7]

Yarin Gal, Jiri Hron, and Alex Kendall. 2017. Concrete dropout. Advances in neural information processing systems 30 (2017).

[8]

Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. 2021. Learning dynamics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions on Knowledge and Data Engineering (2021).

[9]

Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840--6851.

[10]

Aapo Hyvärinen and Peter Dayan. 2005. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research 6, 4 (2005).

[11]

Jiahao Ji, Jingyuan Wang, Zhe Jiang, Jiawei Jiang, and Hu Zhang. 2022. STDEN: Towards physics-guided neural networks for traffic flow prediction. In AAAI.

[12]

Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I Webb, Irwin King, and Shirui Pan. 2023. A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection. arXiv preprint arXiv:2307.03759 (2023).

[13]

Jaehyeon Kim, Sungwon Kim, Jungil Kong, and Sungroh Yoon. 2020. Glow-tts: A generative flow for text-to-speech via monotonic alignment search. Advances in Neural Information Processing Systems 33 (2020), 8067--8077.

[14]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[15]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.

[16]

Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2020. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations.

[17]

Lixin Li and Peter Revesz. 2004. Interpolation methods for spatio-temporal geographic data. Computers, Environment and Urban Systems 28, 3 (2004).

[18]

Yan Li, Xinjiang Lu, Yaqing Wang, and Dejing Dou. 2023. Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement. arXiv preprint arXiv:2301.03028 (2023).

[19]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In International Conference on Learning Representations.

[20]

Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. 2022. AirFormer: Predicting Nationwide Air Quality in China with Transformers. arXiv preprint arXiv:2211.15979 (2022).

[21]

Dachuan Liu, Jin Wang, Shuo Shang, and Peng Han. 2022. Msdr: Multi-step dependency relation networks for spatial temporal forecasting. In KDD.

[22]

Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, and Zhou Zhao. 2022. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11020--11028.

[23]

James E Matheson and Robert L Winkler. 1976. Scoring rules for continuous probability distributions. Management science 22, 10 (1976), 1087--1096.

[24]

Soumyasundar Pal, Liheng Ma, Yingxue Zhang, and Mark Coates. 2021. RNN with particle flow for probabilistic spatio-temporal forecasting. In International Conference on Machine Learning. PMLR, 8336--8348.

[25]

Hao Peng, Hongfei Wang, Bowen Du, Md Zakirul Alam Bhuiyan, Hongyuan Ma, Jianwei Liu, Lihong Wang, Zeyu Yang, Linfeng Du, Senzhang Wang, et al. 2020. Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting. Information Sciences 521 (2020), 277--290.

Digital Library

[26]

Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. 2021. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning. PMLR, 8857--8868.

[27]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning. PMLR, 1278--1286.

[28]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of CVPR. 10684--10695.

[29]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.

[30]

Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. 2019. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems 32 (2019).

[31]

David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 3 (2020), 1181--1191.

[32]

Jelena Simeunović, Baptiste Schubnel, Pierre-Jean Alet, and Rafael E Carrillo. 2021. Spatio-temporal graph neural networks for multi-site PV power forecasting. IEEE Transactions on Sustainable Energy 13, 2 (2021), 1210--1220.

[33]

Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2020. Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. AAAI 2020 34, 1 (2020), 914--921.

[34]

Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising Diffusion Implicit Models. In International Conference on Learning Representations.

[35]

Yang Song and Stefano Ermon. 2019. Generative modeling by estimating gradients of the data distribution. NIPS 32 (2019).

[36]

Yang Song and Stefano Ermon. 2020. Improved techniques for training score-based generative models. NIPS 33 (2020), 12438--12448.

[37]

Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. 2021. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems 34 (2021), 24804--24816.

[38]

Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. WaveNet: A Generative Model for Raw Audio. In 9th ISCA Speech Synthesis Workshop. 125--125.

[39]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

[40]

Pascal Vincent. 2011. A connection between score matching and denoising autoencoders. Neural computation 23, 7 (2011), 1661--1674.

[41]

Vikram Voleti, Alexia Jolicoeur-Martineau, and Christopher Pal. 2022. MCVD-Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation. In Advances in Neural Information Processing Systems.

[42]

Zhixian Wang, Qingsong Wen, Chaoli Zhang, Liang Sun, and Yi Wang. 2023. DiffLoad: Uncertainty Quantification in Load Forecasting with Diffusion Model. arXiv preprint arXiv:2306.01001 (2023).

[43]

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. 2023. Transformers in time series: A survey. In International Joint Conference on Artificial Intelligence (IJCAI).

Digital Library

[44]

Dongxia Wu, Liyao Gao, Matteo Chinazzi, Xinyue Xiong, Alessandro Vespignani, Yi-An Ma, and Rose Yu. 2021. Quantifying Uncertainty in Deep Spatiotemporal Forecasting. In KDD. 1841--1851.

[45]

Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In IJCAI 2019: 28th International Joint Conference on Artificial Intelligence. 1907--1913.

[46]

Huaxiu Yao, Fei Wu, Jintao Ke, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. 2018. Deep multi-view spatial-temporal network for taxi demand prediction. In AAAI conference on artificial intelligence, Vol. 32.

[47]

Xiuwen Yi, Junbo Zhang, Zhaoyuan Wang, Tianrui Li, and Yu Zheng. 2018. Deep distributed fusion network for air quality prediction. In KDD. 965--973.

[48]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. In IJCAI.

[49]

Yiji Zhao, Youfang Lin, Haomin Wen, Tonglong Wei, Xiyuan Jin, and Huaiyu Wan. 2022. Spatial-temporal position-aware graph convolution networks for traffic flow forecasting. IEEE TITS (2022).

[50]

Yiji Zhao, Youfang Lin, Yongkai Zhang, Haomin Wen, Yunxiao Liu, Hao Wu, Zhihao Wu, Shuaichao Zhang, and Huaiyu Wan. 2022. Traffic inflow and outflow forecasting by modeling intra-and inter-relationship between flows. IEEE Transactions on Intelligent Transportation Systems 23, 11 (2022), 20202--20216.

[51]

Yiji Zhao, Haomin Wen, Junhong Lou, Jinji Fu, Jianbin Zheng, and Youfang Lin. 2022. EasyST: Modeling Spatial-Temporal Correlations and Uncertainty for Dynamic Wind Power Forecasting via PaddlePaddle. In KDD workshop.

[52]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, Vol. 35. 11106--11115.

[53]

Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI Open 1 (2020), 57--81.

[54]

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. 27268--27286.

Cited By

Zou XYan YHao XHu YWen HLiu EZhang JLi YLi TZheng YLiang Y(2025)Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlookInformation Fusion10.1016/j.inffus.2024.102606113(102606)Online publication date: Jan-2025
https://doi.org/10.1016/j.inffus.2024.102606
Dai GHuang HPeng XZhang BFu X(2024)ARFGCN: Adaptive Receptive Field Graph Convolutional Network for Urban Crowd Flow PredictionMathematics10.3390/math1211173912:11(1739)Online publication date: 3-Jun-2024
https://doi.org/10.3390/math12111739
Li JZhang MLi NWeyns DJin ZTei K(2024)Generative AI for Self-Adaptive Systems: State of the Art and Research RoadmapACM Transactions on Autonomous and Adaptive Systems10.1145/368680319:3(1-60)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3686803
Show More Cited By

Index Terms

DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models
1. Applied computing
  1. Operations research
    1. Forecasting
2. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Probabilistic Weather Forecasting with Deterministic Guidance-Based Diffusion Model
Computer Vision – ECCV 2024
Abstract
Weather forecasting requires both deterministic outcomes for immediate decision-making and probabilistic results for assessing uncertainties. However, deterministic models may not fully capture the spectrum of weather possibilities, and ...
Graph Convolution Recurrent Denoising Diffusion Model for Multivariate Probabilistic Temporal Forecasting
Advanced Data Mining and Applications
Abstract
The probabilistic estimation for multivariate time series forecasting has recently become a trend in various research fields, such as traffic, climate, and finance. The multivariate time series can be treated as an interrelated system, and it is ...
Uncertainty forecasting system for tropical cyclone tracks based on conformal prediction
Abstract
Tropical cyclone tracks are one of the most critical factors in tropical cyclone forecasting, but there is an inherent uncertainty in their forecasts, but there are no relevant machine learning methods to carry out uncertainty studies. This study ...
Highlights
- The first study using machine learning for tropical cyclone track uncertainty forecasting.
- “Trajectory forecast ellipse” method proposed to characterise forecast area.
- ML-based regional forecasting effective complement to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGSPATIAL '23: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems

November 2023

686 pages

ISBN:9798400701689

DOI:10.1145/3589132

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Guangzhou Municiple Science and Technology Project

Conference

SIGSPATIAL '23

Sponsor:

SIGSPATIAL

SIGSPATIAL '23: 31st ACM International Conference on Advances in Geographic Information Systems

November 13 - 16, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
960
Total Downloads

Downloads (Last 12 months)960
Downloads (Last 6 weeks)95

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zou XYan YHao XHu YWen HLiu EZhang JLi YLi TZheng YLiang Y(2025)Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlookInformation Fusion10.1016/j.inffus.2024.102606113(102606)Online publication date: Jan-2025
https://doi.org/10.1016/j.inffus.2024.102606
Dai GHuang HPeng XZhang BFu X(2024)ARFGCN: Adaptive Receptive Field Graph Convolutional Network for Urban Crowd Flow PredictionMathematics10.3390/math1211173912:11(1739)Online publication date: 3-Jun-2024
https://doi.org/10.3390/math12111739
Li JZhang MLi NWeyns DJin ZTei K(2024)Generative AI for Self-Adaptive Systems: State of the Art and Research RoadmapACM Transactions on Autonomous and Adaptive Systems10.1145/368680319:3(1-60)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3686803
Ghosh SSharma AGupta JSubramanian AShekhar S(2024)Towards Kriging-informed Conditional Diffusion for Regional Sea-Level Data Downscaling: A Summary of ResultsProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems10.1145/3678717.3691304(372-383)Online publication date: 29-Oct-2024
https://dl.acm.org/doi/10.1145/3678717.3691304
Zhu YYu JZhao XLiu QYe YChen WZhang ZWei XLiang YBaeza-Yates RBonchi F(2024)ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion ModelProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671866(4676-4687)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671866
Wu LWen HHu HMao XXia YShan EZheng JLou JLiang YYang LZimmermann RLin YWan HBaeza-Yates RBonchi F(2024)LaDe: The First Comprehensive Last-mile Express Dataset from IndustryProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671548(5991-6002)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671548
Liang YWen HNie YJiang YJin MSong DPan SWen QBaeza-Yates RBonchi F(2024)Foundation Models for Time Series Analysis: A Tutorial and SurveyProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671451(6555-6565)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671451
An YLi ZLiu WSun HChen MLu WGong YSerra ESpezzano F(2024)Spatio-temporal Graph Normalizing Flow for Probabilistic Traffic PredictionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679705(45-55)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679705
Zhang KWen QZhang CCai RJin MLiu YZhang JLiang YPang GSong DPan S(2024)Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and ProspectsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338731746:10(6775-6794)Online publication date: Oct-2024
https://doi.org/10.1109/TPAMI.2024.3387317
Cao HTan CGao ZXu YChen GHeng PLi S(2024)A Survey on Generative Diffusion ModelsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336147436:7(2814-2830)Online publication date: Jul-2024
https://doi.org/10.1109/TKDE.2024.3361474
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents