Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article
Open access

I Know Your Intent: Graph-enhanced Intent-aware User Device Interaction Prediction via Contrastive Learning

Published: 27 September 2023 Publication History

Abstract

With the booming of smart home market, intelligent Internet of Things (IoT) devices have been increasingly involved in home life. To improve the user experience of smart homes, some prior works have explored how to use machine learning for predicting interactions between users and devices. However, the existing solutions have inferior User Device Interaction (UDI) prediction accuracy, as they ignore three key factors: routine, intent and multi-level periodicity of human behaviors. In this paper, we present SmartUDI, a novel accurate UDI prediction approach for smart homes. First, we propose a Message-Passing-based Routine Extraction (MPRE) algorithm to mine routine behaviors, then the contrastive loss is applied to narrow representations among behaviors from the same routines and alienate representations among behaviors from different routines. Second, we propose an Intent-aware Capsule Graph Attention Network (ICGAT) to encode multiple intents of users while considering complex transitions between different behaviors. Third, we design a Cluster-based Historical Attention Mechanism (CHAM) to capture the multi-level periodicity by aggregating the current sequence and the semantically nearest historical sequence representations through the attention mechanism. SmartUDI can be seamlessly deployed on cloud infrastructures of IoT device vendors and edge nodes, enabling the delivery of personalized device service recommendations to users. Comprehensive experiments on four real-world datasets show that SmartUDI consistently outperforms the state-of-the-art baselines with more accurate and highly interpretable results.

Supplemental Material

ZIP File - xiao
Supplemental movie, appendix, image and software files for, I Know Your Intent: Graph-enhanced Intent-aware User Device Interaction Prediction via Contrastive Learning

References

[1]
Khaled A. Alaghbari, Mohamad Hanif Md. Saad, Aini Hussain, and Muhammad Raisul Alam. 2022. Activities Recognition, Anomaly Detection and Next Activity Prediction Based on Neural Networks in Smart Homes. IEEE Access 10 (2022), 28219--28232. https://doi.org/10.1109/ACCESS.2022.3157726
[2]
Miao Lin andf Vincent W. Zheng and Shili Xiang. 2018. Sequential context modeling for smart devices by Collaborative Hidden Markov Model. In 4th IEEE World Forum on Internet of Things, WF-IoT 2018, Singapore, February 5-8, 2018. IEEE, 771--777. https://doi.org/10.1109/WF-IoT.2018.8355155
[3]
Dan Busbridge, Dane Sherburn, Pietro Cavallo, and Nils Y Hammerla. 2019. Relational graph attention networks. arXiv preprint arXiv:1904.05811 (2019).
[4]
Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 3438--3445.
[5]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597--1607.
[6]
Yegang Du, Yuto Lim, and Yasuo Tan. 2019. A Novel Human Activity Recognition and Prediction in Smart Home Based on Interaction. Sensors 19, 20 (2019), 4474. https://doi.org/10.3390/s19204474
[7]
Sean R Eddy. 1996. Hidden markov models. Current opinion in structural biology 6, 3 (1996), 361--365.
[8]
Araf Farayez, Mamun Bin Ibne Reaz, and Norhana Arsad. 2019. SPADE: Activity Prediction in Smart Homes Using Prefix Tree Based Context Generation. IEEE Access 7 (2019), 5492--5501. https://doi.org/10.1109/ACCESS.2018.2888923
[9]
Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. 2018. Deepmove: Predicting human mobility with attentional recurrent networks. In Proceedings of the 27th International Conference on World Wide Web (WWW). ACM, Lyon, 1459--1468.
[10]
Alex Graves and Alex Graves. 2012. Long short-term memory. Supervised sequence labelling with recurrent neural networks (2012), 37--45.
[11]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855--864.
[12]
John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics) 28, 1 (1979), 100--108.
[13]
Hyunsik Jeon, Jongjin Kim, Hoyoung Yoon, Jaeri Lee, and U Kang. 2022. Accurate Action Recommendation for Smart Home via Two-Level Encoders and Commonsense Knowledge. In Proceedings of the 31th ACM International Conference on Information & Knowledge Management (CIKM). ACM, Atlanta, Georgia, USA, 1--10.
[14]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE 18th International Conference on Data Mining (ICDM). IEEE, Singapore, 197--206.
[15]
Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Marcus Brubaker. 2019. Time2vec: Learning a vector representation of time. arXiv preprint arXiv:1907.05321 (2019).
[16]
Bisma S Khan and Muaz A Niazi. 2017. Network community detection: A review and visual survey. arXiv preprint arXiv:1708.00977 (2017).
[17]
Phil Kim and Phil Kim. 2017. Convolutional neural network. MATLAB deep learning: with machine learning, neural networks and artificial intelligence (2017), 121--147.
[18]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[19]
Ruoyu Li, Qing Li, Yucheng Huang, Wenbin Zhang, Peican Zhu, and Yong Jiang. 2022. IoTEnsemble: Detection of Botnet Attacks on Internet of Things. In European Symposium on Research in Computer Security (ESORICS). Springer, 569--588.
[20]
Ruoyu Li, Qing Li, Jianer Zhou, and Yong Jiang. 2021. ADRIoT: an edge-assisted anomaly detection framework against IoT-based network attacks. IEEE Internet of Things Journal (IoTJ) 9, 13 (2021), 10576--10587.
[21]
Chengwu Liao, Chao Chen, Suiming Guo, Zhu Wang, Yaxiao Liu, Ke Xu, and Daqing Zhang. 2022. Wheels know why you travel: Predicting trip purpose via a dual-attention graph embedding network. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 6, 1 (2022), 1--22.
[22]
Liwei Liu, Wei Chen, Lu Liu, Kangkang Zhang, Jun Wei, and Yan Yang. 2021. TAGen: Generating Trigger-Action Rules for Smart Homes by Mining Event Traces. In Service-Oriented Computing - 19th International Conference, ICSOC 2021, Virtual Event, November 22-25, 2021, Proceedings (Lecture Notes in Computer Science, Vol. 13121), Hakim Hacid, Odej Kao, Massimo Mecella, Naouel Moha, and Hye-young Paik (Eds.). Springer, 652--662. https://doi.org/10.1007/978-3-030-91431-8_41
[23]
Qiang Liu, Shu Wu, Diyi Wang, Zhaokang Li, and Liang Wang. 2016. Context-aware sequential recommendation. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, Barcelona, Spain, 1053--1058.
[24]
Knud Lasse Lueth. 2018. State of the IoT 2018: Number of IoT devices now at 7B -- Market accelerating. https://iot-analytics.com/state-of-the-iot-update-q1-q2-2018-number-of-iot-devices-now-7b/.
[25]
Mustafa A. Mustafa, Alexandros Konios, and Matias Garcia-Constantino. 2021. IoT-Based Activities of Daily Living for Abnormal Behavior Detection: Privacy Issues and Potential Countermeasures. IEEE Internet Things Mag. 4, 3 (2021), 90--95. https://doi.org/10. 1109/IOTM.0001.2000169
[26]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NIPS) 32 (2019).
[27]
Lakshmanan Rakkappan and Vaibhav Rajan. 2019. Context-aware sequential recommendations withstacked recurrent neural networks. In Proceedings of the 28th International Conference on World Wide Web (WWW). ACM, San Francisco, 3172--3178.
[28]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW). ACM, Raleigh, NC, USA, 811--820.
[29]
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. Advances in Neural Information Processing Systems (NIPS) 30 (2017).
[30]
Klaus U Schulz and Stoyan Mihov. 2002. Fast string correction with Levenshtein automata. International Journal on Document Analysis and Recognition 5 (2002), 67--85.
[31]
Amit Kumar Sikder, Leonardo Babun, Hidayet Aksu, and A. Selcuk Uluagac. 2019. Aegis: a context-aware security framework for smart home systems. In Proceedings of the 35th Annual Computer Security Applications Conference, ACSAC 2019, San Juan, PR, USA, December 09-13, 2019, David Balenson (Ed.). ACM, 28--41. https://doi.org/10.1145/3359789.3359840
[32]
Vijay Srinivasan, Christian Koehler, and Hongxia Jin. 2018. RuleSelector: Selecting conditional action rules from user behavior patterns. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2, 1 (2018), 1--34.
[33]
Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM International Conference on Web Search and Data Mining (WSDM). ACM, Los Angeles, California, USA, 565--573.
[34]
Niek Tax. 2018. Human Activity Prediction in Smart Home Environments with LSTM Neural Networks. In 14th International Conference on Intelligent Environments, IE 2018, Roma, Italy, June 25-28, 2018. IEEE, 40--47. https://doi.org/10.1109/IE.2018.00014
[35]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems (NIPS) 30 (2017).
[36]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. Proceedings of the 6th International Conference on Learning Representations (ICLR) (2018).
[37]
Qinghua Wu and Jin-Kao Hao. 2015. A review on algorithms for maximum clique problems. European Journal of Operational Research 242, 3 (2015), 693--709.
[38]
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI conference on Artificial Intelligence, Vol. 33. AAAI, Hilton Hawaiian Village, Honolulu, Hawaii, USA, 346--353.
[39]
Jingyu Xiao, Qingsong Zou, Qing Li, Dan Zhao, Kang Li, Wenxin Tang, Runjie Zhou, and Yong Jiang. 2023. User Device Interaction Prediction via Relational Gated Graph Attention Network and Intent-aware Encoder. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 1634--1642.
[40]
Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019. Graph Contextualized Self-Attention Network for Session-based Recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI), Vol. 19. Morgan Kaufmann, Macao, China, 3940--3946.
[41]
Masaaki Yamauchi, Masahiro Tanaka, Yuichi Ohsita, Masayuki Murata, Kensuke Ueda, and Yoshiaki Kato. 2021. Smart-home anomaly detection using combination of in-home situation and user behavior. CoRR abs/2109.14348 (2021). https://arxiv.org/abs/2109.14348
[42]
Qingsong Zou, Qing Li, Ruoyu Li, Yucheng Huang, Gareth Tyson, Jingyu Xiao, and Yong Jiang. 2023. IoTBeholder: A Privacy Snooping Attack on User Habitual Behaviors from Smart Home Wi-Fi Traffic. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 7, 1 (2023), 1--26.

Cited By

View all
  • (2024)Make Your Home Safe: Time-aware Unsupervised User Behavior Anomaly Detection in Smart Homes via Loss-guided MaskProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671708(3551-3562)Online publication date: 25-Aug-2024
  • (2024)Contextual Distillation Model for Diversified RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671514(5307-5316)Online publication date: 25-Aug-2024
  • (2024)Themis: A passive-active hybrid framework with in-network intelligence for lightweight failure localizationComputer Networks10.1016/j.comnet.2024.110836255(110836)Online publication date: Dec-2024

Index Terms

  1. I Know Your Intent: Graph-enhanced Intent-aware User Device Interaction Prediction via Contrastive Learning

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 7, Issue 3
    September 2023
    1734 pages
    EISSN:2474-9567
    DOI:10.1145/3626192
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 September 2023
    Published in IMWUT Volume 7, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Contrastive Learning
    2. Graph Neural Networks
    3. Human Device Interaction
    4. Smart Home

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)235
    • Downloads (Last 6 weeks)48
    Reflects downloads up to 16 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Make Your Home Safe: Time-aware Unsupervised User Behavior Anomaly Detection in Smart Homes via Loss-guided MaskProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671708(3551-3562)Online publication date: 25-Aug-2024
    • (2024)Contextual Distillation Model for Diversified RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671514(5307-5316)Online publication date: 25-Aug-2024
    • (2024)Themis: A passive-active hybrid framework with in-network intelligence for lightweight failure localizationComputer Networks10.1016/j.comnet.2024.110836255(110836)Online publication date: Dec-2024

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Full Access

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media