Deep Reinforcement Learning for Smart Home Temperature Comfort in IoT-Edge Computing Systems
Authors: 
Marios Christopoulos, Sotirios Spantideas, Anastasios Giannopoulos, Panagiotis TrakadasAuthors Info & Claims
Pages 1 - 7
Abstract
In this paper, a novel IoT-Edge-Cloud (IEC) computing system designed for multiple Smart Homes is introduced, with a focus on supporting Home Energy Management Systems (HEMS) for temperature control within a defined comfort range. Leveraging model-free deep reinforcement learning, the proposed method, Smart Home Energy and Temperature Control (SHETEC), employs autonomous agents which are trained to manipulate the input power of Heating, Ventilation, and Air Conditioning (HVAC) systems and charging/discharging power of Energy Storage Systems (ESS) using Deep Deterministic Policy Gradients (DDPG). In addition, we present the Average Opinion (AO) method, a collaborative decision-making approach that combines the models of all Smart Homes in a distributed approach. Experimental results, conducted through simulation on three Smart Homes using real-world heterogeneous data, demonstrate the effectiveness of both SHETEC and Average Opinion in maintaining temperatures within the desired comfort bounds.
References
[1]
Panagiotis Trakadas, Xavi Masip-Bruin, Federico M Facca, Sotirios T Spantideas, Anastasios E Giannopoulos, Nikolaos C Kapsalis, Rui Martins, Enrica Bosani, Joan Ramon, Raül González Prats, et al. A reference architecture for cloud-edge meta-operating systems enabling cross-domain, data-intensive, ml-assisted applications: Architectural overview and key concepts. Sensors, 22(22):9003, 2022.
[2]
Panagiotis Gkonis, Anastasios Giannopoulos, Panagiotis Trakadas, Xavi Masip-Bruin, and Francesco D'Andria. A survey on iot-edge-cloud continuum systems: Status, challenges, use cases, and open issues. Future Internet, 15(12), 2023. ISSN 1999-5903. URL https://www.mdpi.com/1999-5903/15/12/383.
[3]
Angelos Angelopoulos, Anastasios Giannopoulos, Nikolaos Nomikos, Alexandros Kalafatelis, Antonios Hatziefremidis, and Panagiotis Trakadas. Federated learning-aided prognostics in the shipping 4.0: Principles, workflow, and use cases. IEEE Access, pages 1--1, 2024.
[4]
Konstantinos Skianis, Anastasios Giannopoulos, Panagiotis Gkonis, and Panagiotis Trakadas. Data aging matters: Federated learning-based consumption prediction in smart homes via age-based model weighting. Electronics, 12(14), 2023. ISSN 2079-9292. URL https://www.mdpi.com/2079-9292/12/14/3054.
[5]
Anastasios Giannopoulos, Sotirios Spantideas, Nikolaos Kapsalis, Karkazis Panagiotis, and Trakadas Panagiotis. Deep reinforcement learning for energy-efficient multi-channel transmissions in 5g cognitive hetnets: Centralized, decentralized and transfer learning based solutions. IEEE Access, PP:1--1, 09 2021.
[6]
William Valladares, Marco Galindo, Jorge Gutiérrez, Wu Chieh Wu, Kuo Kai Liao, Jen Chung Liao, Kuang Chin Lu, and Chi-Chuan Wang. Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm. Building and Environment, 155:105--117, May 2019. ISSN 0360-1323.
[7]
Zhiang Zhang, Adrian Chong, Yuqi Pan, Chenlu Zhang, and Khee Poh Lam. Whole building energy model for hvac optimal control: A practical framework based on deep reinforcement learning. Energy and Buildings, 199:472--490, September 2019. ISSN 0378-7788. Publisher Copyright: © 2019 Elsevier B.V.
[8]
Zeqing Wu, Yunfei Mu, Shuai Deng, Jiajun Wang, Yadi Bai, Juan Xue, Yang Li, Youtao Jiang, Xunda Zhang, and Weicong Xu. Towards comfortable and cost-effective indoor temperature management in smart homes: A deep reinforcement learning method combined with future information. Energy and Buildings, 275:112491, 2022.
[9]
H Yar, AS Imran, ZA Khan, M Sajjad, and Z Kastrati. Towards smart home automation using iot-enabled edge-computing paradigm. Sensors (Basel), 21(14):4932, 2021.
[10]
Yanjie Song, Ponnuthurai Nagaratnam Suganthan, Witold Pedrycz, Junwei Ou, Yongming He, Yingwu Chen, and Yutong Wu. Ensemble reinforcement learning: A survey. Applied Soft Computing, page 110975, 2023.
[11]
Mojtaba Yousefi, Amin Hajizadeh, and Mohsen N. Soltani. A comparison study on stochastic modeling methods for home energy management system. IEEE Transactions on Industrial Informatics, 15(8):4799--4808, August 2019. ISSN 1551-3203.
[12]
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, second edition, 2018. URL http://incompleteideas.net/book/the-book-2nd.html.
[13]
Liang Yu, Weiwei Xie, Di Xie, Yulong Zou, Dengyin Zhang, Zhixin Sun, Linghua Zhang, Yue Zhang, and Tao Jiang. Deep reinforcement learning for smart home energy management. IEEE Internet of Things Journal, 7(4):2751--2762, 2020.
[14]
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In Yoshua Bengio and Yann LeCun, editors, ICLR, 2016. URL https://dblp.uni-trier.de/db/conf/iclr/iclr2016.html#LillicrapHPHETS15.
[15]
Sotirios Spantideas, Anastasios Giannopoulos, Nikolaos Kapsalis, Alexandros Kalafatelis, Christos Capsalis, and Trakadas Panagiotis. Joint energy-efficient and throughput-sufficient transmissions in 5g cells with deep q-learning. pages 265--270, 09 2021.
[16]
Morten Herget Christensen, Cédric Ernewein, and Pierre Pinson. Demand response through price-setting multi-agent reinforcement learning. In Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings & Cities, pages 1--5, 2020.
[17]
P. Constantopoulos, F. C. Schweppe, and R. C. Larson. Estia: A real-time consumer control scheme for space conditioning usage under spot electricity pricing. Computers & Operations Research, 18(8):751--765, 1991. ISSN 0305-0548. URL https://www.sciencedirect.com/science/article/pii/030505489190013H.
[18]
Anupam A. Thatte and Le Xie. Towards a unified operational value index of energy storage in smart grid environment. IEEE Transactions on Smart Grid, 3(3):1418--1426, 2012.
[19]
Ruilong Deng, Zhaohui Zhang, Ju Ren, and Hao Liang. Indoor temperature control of cost-effective smart buildings via real-time smart grid communications. In 2016 IEEE Global Communications Conference (GLOBECOM), pages 1--6, 2016.
[20]
Virginia Pilloni, Alessandro Floris, Alessio Meloni, and Luigi Atzori. Smart home energy management including renewable sources: A qoedriven approach. IEEE Transactions on Smart Grid, 9(3):2006--2018, 2018.
Index Terms
- Deep Reinforcement Learning for Smart Home Temperature Comfort in IoT-Edge Computing Systems
Recommendations
Online management framework for building HVAC systems considering peak shaving and thermal comfort: an experimental study
MSCPES '21: Proceedings of the 9th Workshop on Modeling and Simulation of Cyber-Physical Energy SystemsIn this study, an online management framework for building HVAC (Heating, Ventilation, and Air-Conditioning) systems, which achieves peak shaving and thermal comfort improvement, has been designed and studied experimentally. We formulate a model ...
Leveraging Ontology to Enable Indoor Comfort Customization in the Smart Home
Flexible Query Answering SystemsAbstractThis paper introduces the Future Home for Future Communities’ Smart Home, a semantic-based framework for indoor comfort metrics customization inside a living environment. The Smart Home merges Ambient Intelligence, Ambient Assisted Living and ...
Grey prediction on indoor comfort temperature for HVAC systems
This paper describes determination of indoor comfort temperature for efficiently Heating, Ventilating, and Air-Conditioning (HVAC) system under dynamical environment. Making occupants' satisfaction on thermal comfort is still challenging by how the ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
53 pages
ISBN:9798400705434
DOI:10.1145/3642975
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- EU HORIZON EUROPE
Conference
EuroSys '24
Sponsor:
Acceptance Rates
MECC '24 Paper Acceptance Rate 7 of 15 submissions, 47%;
Overall Acceptance Rate 7 of 15 submissions, 47%
Upcoming Conference
EuroSys '25
- Sponsor:
- sigops
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 38Total Downloads
- Downloads (Last 12 months)38
- Downloads (Last 6 weeks)21
Reflects downloads up to 20 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in