AutoConsis: Automatic GUI-driven Data Inconsistency Detection of Mobile Apps
Authors: 
Yongxiang Hu, Hailiang Jin, Xuan Wang, Jiazhen Gu, Shiyu Guo, Chaoyi Chen, Xin Wang, Yangfan ZhouAuthors Info & Claims
Pages 137 - 146
Abstract
In industrial practice, many bugs in commercial mobile apps manifest as self-conflicts of data presented in the GUI (Graphical User Interface). Such data inconsistency bugs can bring confusion to the users and deteriorate user experiences. They are a major target of industrial testing practice. However, due to the complication and diversity of GUI implementation and data presentation (e.g., the ways to present the data in natural language), detecting data inconsistency bugs is a very challenging task. It still largely relies on manual efforts. To reduce such human efforts, we proposed AutoConsis, an automated data inconsistency testing tool we designed for Meituan. one of the largest E-commerce providers with over 600 million transacting users. AutoConsis can automatically analyze GUI pages via a multi-modal deep-learning model and extract target data from textual phrases leveraging LLMs (Large Language Models). With these extracted data, their inconsistencies can then be detected. We evaluate the design of AutoConsis via a set of ablation experiments. Moreover, we demonstrate the effectiveness of AutoConsis when applying it to real-world commercial mobile apps with eight representative cases.
References
[1]
Accessed: 2023. Android Debug Bridge (adb). https://developer.android.com/tools/adb
[2]
Accessed: 2023. Differential testing. https://en.wikipedia.org/wiki/Differential_testing
[3]
Accessed: 2023. Introducing ChatGPT. https://openai.com/blog/chatgpt
[4]
Accessed: 2023. Layouts in Views. https://developer.android.com/develop/ui/views/layout/declaring-layout
[5]
Accessed: 2023. Meituan. https://about.meituan.com
[6]
Accessed: 2023. Meituan ANNUAL REPORT 2022. http://media-meituan.todayir.com/202304252152521735786604_en.pdf
[7]
Accessed: 2023. UI/Application Exerciser Monkey. https://developer.android.com/studio/test/other-testing-tools/monkey
[8]
Accessed: 2023. XML Path Language (XPath). https://www.w3.org/TR/1999/REC-xpath-19991116/
[9]
Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proc. of the 20th ACM SIGSOFT Symposium on the Foundations of Software Engineering, FSE. ACM, 59.
[10]
Zhen Dong, Marcel Böhme, Lucia Cojocaru, and Abhik Roychoudhury. 2020. Time-travel testing of Android apps. In Proc. of the 42nd International Conference on Software Engineering, ICSE. ACM, 481--492.
[11]
Mattia Fazzini and Alessandro Orso. 2017. Automated cross-platform inconsistency detection for mobile apps. In Proc. of the 32nd IEEE/ACM International Conference on Automated Software Engineering, ASE. IEEE Computer Society, 308--318.
[12]
Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. 2021. YOLOX: Exceeding YOLO Series in 2021. CoRR abs/2107.08430 (2021).
[13]
Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In Proc. of the 41st International Conference on Software Engineering, ICSE. IEEE / ACM, 269--280.
[14]
Wunan Guo, Liwei Shen, Ting Su, Xin Peng, and Weiyang Xie. 2020. Improving Automated GUI Exploration of Android Apps via Static Dependency Analysis. In IEEE International Conference on Software Maintenance and Evolution, ICSME 2020, Adelaide, Australia, September 28 - October 2, 2020. IEEE, 557--568.
[15]
Alain Horé and Djemel Ziou. 2010. Image Quality Metrics: PSNR vs. SSIM. In 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, 23-26 August 2010. IEEE Computer Society, 2366--2369.
[16]
Cuixiong Hu and Iulian Neamtiu. 2011. Automating GUI testing for Android applications. In Proc. of the 6th International Workshop on Automation of Software Test, AST. ACM, 77--83.
[17]
Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: using machine learning to synthesize robust, reusable UI tests. In Proc. of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE. ACM, 269--282.
[18]
Reyhaneh Jabbarvand, Forough Mehralian, and Sam Malek. 2020. Automated construction of energy test oracles for Android. In Proc. of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering,ESEC/FSE. ACM, 927--938.
[19]
Myeongsoo Kim, Qi Xin, Saurabh Sinha, and Alessandro Orso. 2022. Automated test generation for REST APIs: no time to rest yet. In ISSTA '22: 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, South Korea, July 18 - 22, 2022, Sukyoung Ryu and Yannis Smaragdakis (Eds.). ACM, 289--301.
[20]
Pingfan Kong, Li Li, Jun Gao, Timothée Riom, Yanjie Zhao, Tegawendé F. Bissyandé, and Jacques Klein. 2021. ANCHOR: locating android framework-specific crashing faults. Autom. Softw. Eng. 28, 2 (2021), 10.
[21]
Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test Transfer Across Mobile Apps Through Semantic Mapping. In Proc. of the 34th IEEE/ACM International Conference on Automated Software Engineering, ASE. IEEE, 42--53.
[22]
Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2023. Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing. In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023. IEEE, 1355--1367.
[23]
Michael R Lyu et al. 1996. Handbook of software reliability engineering. Vol. 222. IEEE computer society press Los Alamitos.
[24]
Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: an input generation system for Android apps. In Proc. of 21st ACM SIGSOFT Symposium on the Foundations of Software Engineering, FSE. ACM, 224--234.
[25]
Riyadh Mahmood, Nariman Mirzaei, and Sam Malek. 2014. EvoDroid: segmented evolutionary testing of Android apps. In Proc. of the 22nd ACM SIGSOFT Symposium on Foundations of Software Engineering, FSE. ACM, 599--609.
[26]
Tarek Mahmud. 2021. API Compatibility Issue Detection, Testing and Analysis for Android Apps. In 36th IEEE/ACM International Conference on Automated Software Engineering, ASE 2021, Melbourne, Australia, November 15-19, 2021. IEEE, 1061--1063.
[27]
Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18-20, 2016, Andreas Zeller and Abhik Roychoudhury (Eds.). ACM, 94--105.
[28]
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, 11048--11064.
[29]
Yun Peng, Shuzheng Gao, Cuiyun Gao, Yintong Huo, and Michael R. Lyu. 2023. Domain Knowledge Matters: Improving Prompts with Fix Templates for Repairing Python Type Errors. CoRR abs/2306.01394 (2023).
[30]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, Marina Meila and Tong Zhang (Eds.). PMLR, 8748--8763.
[31]
Dezhi Ran, Zongyang Li, Chenxu Liu, Wenyu Wang, Weizhi Meng, Xionglin Wu, Hui Jin, Jing Cui, Xing Tang, and Tao Xie. 2022. Automated Visual Testing for Mobile Apps in an Industrial Setting. In Proc. of the 44th IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP). IEEE, 55--64.
[32]
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580 (2023).
[33]
Ting Su, Lingling Fan, Sen Chen, Yang Liu, Lihua Xu, Geguang Pu, and Zhendong Su. 2022. Why My App Crashes? Understanding and Benchmarking Framework-Specific Exceptions of Android Apps. IEEE Trans. Software Eng. 48, 4 (2022), 1115--1137.
[34]
Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, Paderborn, Germany, September 4-8, 2017, Eric Bodden, Wilhelm Schäfer, Arie van Deursen, and Andrea Zisman (Eds.). ACM, 245--256.
[35]
Ting Su, Jue Wang, and Zhendong Su. 2021. Benchmarking automated GUI testing for Android against real-world bugs. In ESEC/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece, August 23-28, 2021, Diomidis Spinellis, Georgios Gousios, Marsha Chechik, and Massimiliano Di Penta (Eds.). ACM, 119--130.
[36]
Jingling Sun, Ting Su, Junxin Li, Zhen Dong, Geguang Pu, Tao Xie, and Zhendong Su. 2021. Understanding and finding system setting-related defects in Android apps. In Proc. of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA. ACM, 204--215.
[37]
Vidit Vidit, Martin Engilberge, and Mathieu Salzmann. 2023. CLIP the Gap: A Single Domain Generalization Approach for Object Detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. IEEE, 3219--3229.
[38]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2023. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023. IEEE, 7464--7475.
[39]
Wenyu Wang, Wing Lam, and Tao Xie. 2021. An infrastructure approach to improving effectiveness of Android UI testing tools. In ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, Virtual Event, Denmark, July 11-17, 2021, Cristian Cadar and Xiangyu Zhang (Eds.). ACM, 165--176.
[40]
Wenyu Wang, Dengfeng Li, Wei Yang, Yurui Cao, Zhenwen Zhang, Yuetang Deng, and Tao Xie. 2018. An empirical study of Android test generation tools in industrial cases. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE 2018, Montpellier, France, September 3-7, 2018, Marianne Huchard, Christian Kästner, and Gordon Fraser (Eds.). ACM, 738--748.
[41]
Wenyu Wang, Wei Yang, Tianyin Xu, and Tao Xie. 2021. Vet: identifying and avoiding UI exploration tarpits. In Proc. of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE. ACM, 83--94.
[42]
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In NeurIPS.
[43]
Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. 2023. Empowering LLM to use Smartphone for Intelligent Task Automation. CoRR abs/2308.15272 (2023).
[44]
Hao Wen, Hongming Wang, Jiaxuan Liu, and Yuanchun Li. 2023. DroidBot-GPT: GPT-powered UI Automation for Android. CoRR abs/2304.07061 (2023).
Index Terms
- AutoConsis: Automatic GUI-driven Data Inconsistency Detection of Mobile Apps
Recommendations
Appaction: Automatic GUI Interaction for Mobile Apps via Holistic Widget Perception
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software EngineeringIn industrial practice, GUI (Graphic User Interface) testing of mobile apps still inevitably relies on huge manual efforts. The major efforts are those on understanding the GUIs, so that testing scripts can be written accordingly. Quality assurance could ...
Early Detection of Spam Mobile Apps
WWW '15: Proceedings of the 24th International Conference on World Wide WebIncreased popularity of smartphones has attracted a large number of developers to various smartphone platforms. As a result, app markets are also populated with spam apps, which reduce the users' quality of experience and increase the workload of app ...
Spam Mobile Apps: Characteristics, Detection, and in the Wild Analysis
The increased popularity of smartphones has attracted a large number of developers to offer various applications for the different smartphone platforms via the respective app markets. One consequence of this popularity is that the app markets are also ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
480 pages
ISBN:9798400705014
DOI:10.1145/3639477
- Co-chairs:
- Ana Paiva,
- Rui Abreu,
- Maurício Aniche,
- Nachiappan Nagappan,
- Program Co-chairs:
- Abhik Roychoudhury,
- Margaret Storey
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
- Faculty of Engineering of University of Porto
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Shanghai Municipal Science and Technology Commission
- The Natural Science Foundation of Shanghai
Conference
ICSE-SEIP '24
Sponsor:
ICSE-SEIP '24: 46th International Conference on Software Engineering: Software Engineering in Practice
April 14 - 20, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 119Total Downloads
- Downloads (Last 12 months)119
- Downloads (Last 6 weeks)40
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in