Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Contextually enhanced ES-dRNN with dynamic attention for short-term load forecasting

Published: 04 March 2024 Publication History

Abstract

In this paper, we propose a new short-term load forecasting (STLF) model based on contextually enhanced hybrid and hierarchical architecture combining exponential smoothing (ES) and a recurrent neural network (RNN). The model is composed of two simultaneously trained tracks: the context track and the main track. The context track introduces additional information to the main track. It is extracted from representative series and dynamically modulated to adjust to the individual series forecasted by the main track. The RNN architecture consists of multiple recurrent layers stacked with hierarchical dilations and equipped with recently proposed attentive dilated recurrent cells. These cells enable the model to capture short-term, long-term and seasonal dependencies across time series as well as to weight dynamically the input information. The model produces both point forecasts and predictive intervals. The experimental part of the work performed on 35 forecasting problems shows that the proposed model outperforms in terms of accuracy its predecessor as well as standard statistical models and state-of-the-art machine learning models.

References

[1]
Aprillia H., Yang H.-T., Huang C.-M., Statistical load forecasting using optimal quantile regression random forest and risk assessment index, IEEE Transactions on Smart Grid 12 (2) (2020) 1467–1480.
[2]
Arora S., Taylor J.W., Rule-based autoregressive moving average models for forecasting load on special days: A case study for France, European Journal of Operational Research 266 (2018) 259–268.
[3]
Ben-Ari, I., & Shwartz-Ziv, R. (2017). Sequence modeling using a memory controller extension for LSTM. In NIPS 2017 time series workshop.
[4]
Bojer C.S., Meldgaard J.P., Kaggle forecasting competitions: An overlooked learning opportunity, International Journal of Forecasting 37 (2) (2021) 587–603.
[5]
Caruana R., Multitask learning, Machine Learning 28 (1) (1997) 41–75.
[7]
Charlton N., Singleton C., A refined parametric model for short term load forecasting, International Journal of Forecasting 30 (2) (2014) 364–368.
[8]
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proc. ACM SIGKDD.
[9]
Chen K., et al., Short-term load forecasting with deep residual networks, IEEE Transactions on Smart Grid 10 (4) (2018) 3943–3952.
[10]
Cho K., et al., Learning phrase representations using RNN encoder–decoder for statistical machine translation, 2014, arXiv preprint arXiv:1406.1078.
[11]
De Stefani J., Bontempi G., Factor-based framework for multivariate and multi-step-ahead forecasting of large scale time series, Frontiers in Big Data (2021) 75.
[12]
Dudek G., Pattern similarity-based methods for short-term load forecasting – part 2: Models, Applied Soft Computing 36 (2015) 422–441.
[13]
Dudek G., Neural networks for pattern-based short-term load forecasting: A comparative study, Neurocomputing 205 (2016) 64–74.
[14]
Dudek G., Pełka P., Smyl S., A hybrid residual dilated LSTM and exponential smoothing model for midterm electric load forecasting, IEEE Transactions on Neural Networks Learning Systems 33 (7) (2022) 2879–2891.
[15]
Eskandari H., Imani M., Moghaddam M.P., Convolutional and recurrent neural network based model for short-term load forecasting, Electric Power Systems Research 195 (2021).
[16]
Gao R., Du L., Suganthan P.N., Zhou Q., Yuen K.F., Random vector functional link neural network based ensemble deep learning for short-term load forecasting, Expert Systems with Applications 206 (2022).
[17]
Giacomini R., White H., Tests of conditional predictive ability, Econometrica 74 (6) (2006) 1545–1578.
[18]
Granger C.W.J., Investigating causal relations by econometric models and cross-spectral methods, Econometrica 37 (3) (1969) 424–438.
[19]
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proc. CVPR (pp. 770–778).
[20]
Hochreiter S., Schmidhuber J., Long short-term memory, Neural Computation 9 (8) (1997) 1735–1780.
[21]
Hong Y., Zhou Y., Li Q., Xu W., Zheng X., A deep learning method for short-term residential load forecasting in smart grid, IEEE Access 8 (2020) 55785–55797.
[22]
Januschowski T., et al., Criteria for classifying forecasting methods, International Journal of Forecasting 36 (2020) 167–177.
[23]
Kandilogiannakis G., Mastorocostas P., Voulodimos A., ReNFuzz-LF: A recurrent neurofuzzy system for short-term load forecasting, Energies 15 (10) (2022) 3637.
[24]
Ke, G., et al. (2017). LightGBM: A highly efficient gradient boosting decision tree. In Proc. NIPS.
[25]
Kim J., Moon J., Hwang E., Kang P., Recurrent inception convolution neural network for multi short-term load forecasting, Energy and Buildings 194 (2019) 328–341.
[26]
Kong W., et al., Short-term residential load forecasting based on LSTM recurrent neural network, IEEE Transactions on Smart Grid 10 (1) (2017) 841–851.
[27]
Li W., Shi Q., Sibtain M., Li D., Mbanze D.E., A hybrid forecasting model for short-term power load based on sample entropy, two-phase decomposition and whale algorithm optimized support vector regression, IEEE Access 8 (2020) 166907–166921.
[28]
Li, R., Wang, S., Zhu, F., & Huang, J. (2018). Adaptive graph convolutional neural networks. In Proc. AAAI, vol. 32, no. 1.
[29]
Lin, W., & Wu, D. (2021). Residential Electric Load Forecasting Via Attentive Transfer of Graph Neural Networks. In IJCAI (pp. 2716–2722).
[30]
Massaoudi M., et al., A novel stacked generalization ensemble-based hybrid LGBM-XGB-MLP model for short-term load forecasting, Energy 214 (2021).
[31]
Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2020). N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. In Proc. ICLR.
[32]
Pełka, P. (2021). Pattern-based forecasting of monthly electricity demand using support vector machine. In Proc. IJCNN (pp. 1–8).
[33]
Pełka P., Dudek G., Neuro-fuzzy system for medium-term electric energy demand forecasting, in: Proc. ISAT, Springer AISC 655, 2018, pp. 38–47.
[34]
Pełka, P., & Dudek, G. (2020). Pattern-based long short-term memory for mid-term electrical load forecasting. In Proc. IJCNN (pp. 1–8).
[35]
Piotrowski P., Baczyński D., Kopyt M., Gulczyński T., Advanced ensemble methods using machine learning and deep learning for one-day-ahead forecasts of electric energy production in wind farms, Energies 15 (4) (2022) 1252.
[36]
Qin Y., et al., A dual-stage attention-based recurrent neural network for time series prediction, 2017, arXiv preprint arXiv:1704.02971.
[37]
Salinas D., Flunkert V., Gasthaus J., Januschowski T., DeepAR: Probabilistic forecasting with autoregressive recurrent networks, International Journal of Forecasting 36 (3) (2020) 1181–1191.
[38]
Satorras V.G., Rangapuram S.S., Januschowski T., Multivariate time series forecasting with latent graph inference, 2022, arXiv preprint arXiv:2203.03423.
[39]
Sen, R., Yu, H.-F., & Dhillon, I. S. (2019). Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. 32, In Proc. NeurIPS, 2019.
[40]
Shi H., Xu M., Li R., Deep learning for household load forecasting—A novel pooling deep RNN, IEEE Transactions on Smart Grid 9 (5) (2017) 5271–5280.
[41]
Smith, S. L., Kindermans, P. J., Ying, C., & Le, Q. V. (2018). Don’t decay the learning rate, increase the batch size. In Proc. ICLR.
[42]
Smyl S., A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting, International Journal of Forecasting 36 (1) (2020) 75–85.
[43]
Smyl S., Dudek G., Pełka P., ES-dRNN: A hybrid exponential smoothing and dilated recurrent neural network model for short-term load forecasting, 2021, arXiv preprint arXiv:2112.02663.
[44]
Smyl, S., Dudek, G., & Pelka, P. (2022). ES-dRNN with Dynamic Attention for Short-Term Load Forecasting. In Proc. IJCNN (pp. 1–8).
[45]
Takeda H., Tamura Y., Sato S., Using the ensemble Kalman filter for electricity load forecasting and analysis, Energy 104 (2016) 184–198.
[46]
Tan M., et al., Ultra-short-term industrial power demand forecasting using LSTM based hybrid ensemble learning, IEEE Transactions on Power Systems 35 (4) (2019) 2937–2948.
[47]
Taylor J.W., Short-term load forecasting with exponentially weighted methods, IEEE Transactions on Power Systems 27 (1) (2012) 458–464.
[48]
Taylor S.J., Letham B., Forecasting at scale, The American Statistician 72 (1) (2018) 37–45.
[49]
Tran, A., Mathews, A., Ong, C. S., & Xie, L. (2021). Radflow: A recurrent, aggregated, and decomposable model for networks of time series. In Proc. WWW (pp. 730–742).
[50]
van den Oord A., et al., WaveNet: A generative model for raw audio, 2016, arXiv:1609.03499.
[51]
Wang S., Wang X., Wang S., Wang D., Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting, International Journal of Electrical Power & Energy Systems 109 (2019) 470–479.
[52]
Wang, Y., et al. (2019). Deep factors for forecasting. In Proc. ICML (pp. 6607–6617).
[53]
Wang Y., et al., Short-term load forecasting for industrial customers based on TCN-lightgbm, IEEE Transactions on Power Systems 36 (3) (2020) 1984–1997.
[54]
Wu, Z., et al. (2020a). Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proc. 26th ACM SIGKDD (pp. 753–763).
[55]
Wu, Z., et al. (2020b). Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proc. ACM SIGKDD.
[56]
Yang D., Guo J., Sun S., Han J., Wang S., An interval decomposition-ensemble approach with data-characteristic-driven reconstruction for short-term load forecasting, Applied Energy 306 (2022).
[57]
Zhou J., et al., Graph neural networks: A review of methods and applications, AI Open 1 (2020) 57–81.

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Neural Networks
Neural Networks  Volume 169, Issue C
Jan 2024
818 pages

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 04 March 2024

Author Tags

  1. Exponential smoothing
  2. Hybrid forecasting models
  3. Recurrent neural networks
  4. Short-term load forecasting
  5. Time series forecasting

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 29 Nov 2024

Other Metrics

Citations

View Options

View options

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media