Abstract
We propose a unified forecasting framework for accurately predicting carbon markets of EU Emission Trading Scheme (EU ETS) and Chinese Emission Allowance (CEA). Our framework utilizes a Time-Series Model (TSM) for initial prediction followed by applying a Large Language Model (LLM) to refine the forecasts. We prompt the LLM to refine the TSM forecasts by demonstrating an example pair of past TSM predictions and their corresponding true future prices to the LLM as a chain-of-thought. The in-context learning capacity of the LLM allows the LLM to rectify inaccurate predictions to reflect on TSM predictions and refine the forecasts. To further reduce the prompting delays and expenses involving LLMs, we innovate a post-finetuning approach to train a Gated Linear Unit (GLU) model to condense the LLM’s in-context learning capability. This enables direct fine-tuning of TSM outputs without the need for explicit prompting LLM during inference. Experimental results show that our method can refine the TSM prediction by 10% to 40% in various zones, as well as enhance transfer learning by 10% to 21% through the inclusion of market context of the source zone when predicting the target zone. Remarkably, our GLU model achieves comparable, and in some cases superior, performance compared to LLM prompting. It effectively combines the short-term forecasting capability of classical Time Series Models with the long-term trend prediction ability typically associated with the LLMs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Besta, M., et al.: Graph of thoughts: solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687 (2023)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Chen, R., Ren, J.: Do AI-powered mutual funds perform better? Financ. Res. Lett. 47, 102616 (2022)
Fan, C., Pang, T., Huang, A.: Pre-trained financial model for price movement forecasting. In: International Conference on Neural Information Processing, pp. 216–229. Springer (2023). https://doi.org/10.1007/978-981-99-8184-7_17
Hu, E.J., et al.: Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)
Jha, M., Qian, J., Weber, M., Yang, B.: ChatGPT and corporate policies. Tech. rep, National Bureau of Economic Research (2024)
Li, Y., Wang, S., Ding, H., Chen, H.: Large language models in finance: a survey. In: Proceedings of the Fourth ACM International Conference on AI in Finance, pp. 374–382 (2023)
Liu, X., Ji, K., Fu, Y., Tam, W.L., Du, Z., Yang, Z., Tang, J.: P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)
Madaan, A., Tandon, N., et al.: Self-refine: iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 (2023)
Ouyang, L., et al.: Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 (2022)
Pang, T., Tan, K., Fan, C.: Carbon price forecasting with quantile regression and feature selection. In: 2023 7th International Symposium on Computer Science and Intelligent Control (ISCSIC) (2023)
Pang, T., Tan, K., Yao, Y., Liu, X., Meng, F., Fan, C., Zhang, X.: REMED: retrieval-augmented medical document query responding with embedding fine-tuning. IJCNN (2024)
Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. arXiv preprint arXiv:1710.05941 (2017)
Ren, X., Duan, K., Tao, L., Shi, Y., Yan, C.: Carbon prices forecasting in quantiles. Energy Econ. 108, 105862 (2022)
Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., Yih, W.t.: Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652 (2023)
Shinn, N., Labash, B., Gopinath, A.: Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366 (2023)
Touvron, H., Lavril, T., et al.: LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)
Wang, S., Yuan, H., Zhou, L., Ni, L.M., Shum, H.Y., Guo, J.: Alpha-GPT: human-AI interactive alpha mining for quantitative investment. arXiv preprint arXiv:2308.00016 (2023)
Wei, J., et al.: Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022)
Wu, C., et al.: LLaMA pro: progressive llama with block expansion. arXiv preprint arXiv:2401.02415 (2024)
Wu, H., Xu, J., Wang, J., Long, M.: Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv. Neural. Inf. Process. Syst. 34, 22419–22430 (2021)
Wu, S.,et al.: BloombergGPt: a large language model for finance. arXiv preprint arXiv:2303.17564 (2023)
Xie, Q., Han, W., Lai, Y., Peng, M., Huang, J.: The wall street neophyte: a zero-shot analysis of chatGPT over multimodal stock movement prediction challenges. arXiv preprint arXiv:2304.05351 (2023)
Yao, S., et al.: Tree of thoughts: deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 (2023)
Zhou, F., Huang, Z., Zhang, C.: Carbon price forecasting based on CEEMDAN and LSTM. Appl. Energy 311, 118601 (2022)
Zhu, B.: A novel multiscale ensemble carbon price prediction model integrating empirical mode decomposition, genetic algorithm and artificial neural network. Energies 5(2), 355–370 (2012)
Acknowledgment
This work is supported by the National Natural Science Foundation of China (Project 62106156), and the South China Normal University, China. We also thank Tianqi Pang for providing the implementations of Lasso methods on EU ETS forecasting.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, H., Ding, Y., Chen, R., Fan, C. (2024). Carbon Price Forecasting with LLM-Based Refinement and Transfer-Learning. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15024. Springer, Cham. https://doi.org/10.1007/978-3-031-72356-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-72356-8_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72355-1
Online ISBN: 978-3-031-72356-8
eBook Packages: Computer ScienceComputer Science (R0)