research-article

Towards Efficient Record and Replay: A Case Study in WeChat

Authors:

Chunyang ChenAuthors Info & Claims

ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1681 - 1692

https://doi.org/10.1145/3611643.3613880

Published: 30 November 2023 Publication History

Abstract

WeChat, a widely-used messenger app boasting over 1 billion monthly active users, requires effective app quality assurance for its complex features. Record-and-replay tools are crucial in achieving this goal. Despite the extensive development of these tools, the impact of waiting time between replay events has been largely overlooked. On one hand, a long waiting time for executing replay events on fully-rendered GUIs slows down the process. On the other hand, a short waiting time can lead to events executing on partially-rendered GUIs, negatively affecting replay effectiveness. An optimal waiting time should strike a balance between effectiveness and efficiency. We introduce WeReplay, a lightweight image-based approach that dynamically adjusts inter-event time based on the GUI rendering state. Given the real-time streaming on the GUI, WeReplay employs a deep learning model to infer the rendering state and synchronize with the replaying tool, scheduling the next event when the GUI is fully rendered. Our evaluation shows that our model achieves 92.1% precision and 93.3% recall in discerning GUI rendering states in the WeChat app. Through assessing the performance in replaying 23 common WeChat usage scenarios, WeReplay successfully replays all scenarios on the same and different devices more efficiently than the state-of-the-practice baselines.

References

[1]

2022. Appium. http://appium.io/

[2]

2022. Culebra. https://github.com/dtmilano/AndroidViewClient/wiki/culebra

[3]

2022. Rendering - Android Developers. https://developer.android.com/topic/performance/rendering

[4]

2022. Selenium. https://www.selenium.dev/

[5]

2022. UI Automator. https://developer.android.com/training/testing/other-components/ui-automator

[6]

2023. Android Debug Bridge (adb). https://developer.android.com/studio/command-line/adb

[7]

2023. appetizer-toolkit. https://github.com/appetizerio/appetizer-toolkit

[8]

2023. Espresso Test Recorder. https://developer.android.com/studio/test/espresso-test-recorder.html

[9]

2023. HiroMacro Auto-Touch Macro. https://play.google.com/store/apps/details?id=com.prohiro.macro

[10]

2023. How Many People Use WeChat? User Statistics & Trends (May 2023). https://www.bankmycell.com/blog/number-of-wechat-users/

[11]

2023. monkeyrunner. https://developer.android.com/studio/test/monkeyrunner

[12]

2023. Ranorex. http://www.ranorex.com/mobile-automation-testing.html

[13]

2023. RepetiTouch Free. https://play.google.com/store/apps/details?id=com.cygery.repetitouch.free

[14]

2023. Unity Technologies. https://unity.com/

[15]

2023. WEditor. https://github.com/alibaba/web-editor

[16]

Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Bryan Dzung Ta, and Atif M Memon. 2014. MobiGUITAR: Automated model-based testing of mobile apps. IEEE software, 32, 5 (2014), 53–59.

[17]

Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. Surf: Speeded up robust features. In European conference on computer vision. 404–417.

Digital Library

[18]

Leo Breiman. 2001. Random forests. Machine learning, 45, 1 (2001), 5–32.

[19]

Chunyang Chen, Sidong Feng, Zhengyang Liu, Zhenchang Xing, and Shengdong Zhao. 2020. From lost to found: Discover missing ui design semantics through recovering missing tags. Proceedings of the ACM on Human-Computer Interaction, 4, CSCW2 (2020), 1–22.

Digital Library

[20]

Chunyang Chen, Sidong Feng, Zhenchang Xing, Linda Liu, Shengdong Zhao, and Jinshui Wang. 2019. Gallery dc: Design search and knowledge discovery through auto-created gui component gallery. Proceedings of the ACM on Human-Computer Interaction, 3, CSCW (2019), 1–22.

[21]

Jieshan Chen, Jiamou Sun, Sidong Feng, Zhenchang Xing, Qinghua Lu, Xiwei Xu, and Chunyang Chen. 2023. Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications. arXiv preprint arXiv:2308.05898.

[22]

Sidong Feng and Chunyang Chen. 2022. GIFdroid: an automated light-weight tool for replaying visual bug reports. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings. 95–99.

Digital Library

[23]

Sidong Feng and Chunyang Chen. 2022. GIFdroid: Automated Replay of Visual Bug Reports for Android Apps. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE). 1045–1057.

[24]

Sidong Feng and Chunyang Chen. 2023. Prompting Is All You Need: Automated Android Bug Replay with Large Language Models. arXiv preprint arXiv:2306.01987.

[25]

Sidong Feng, Chunyang Chen, and Zhenchang Xing. 2022. Gallery DC: Auto-created GUI component gallery for design search and knowledge discovery. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings. 80–84.

Digital Library

[26]

Sidong Feng, Chunyang Chen, and Zhenchang Xing. 2023. Video2Action: Reducing Human Interactions in Action Annotation of App Tutorial Videos. arXiv preprint arXiv:2308.03252.

[27]

Sidong Feng, Minmin Jiang, Tingting Zhou, Yankun Zhen, and Chunyang Chen. 2022. Auto-Icon+: An Automated End-to-End Code Generation Tool for Icon Designs in UI Development. ACM Transactions on Interactive Intelligent Systems, 12, 4 (2022), 1–26.

Digital Library

[28]

Sidong Feng, Suyu Ma, Jinzhong Yu, Chunyang Chen, Tingting Zhou, and Yankun Zhen. 2021. Auto-icon: An automated code generation tool for icon designs assisting in ui development. In 26th International Conference on Intelligent User Interfaces. 59–69.

Digital Library

[29]

Sidong Feng, Mulong Xie, and Chunyang Chen. 2023. Efficiency matters: Speeding up automated testing with gui rendering inference. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 906–918.

Digital Library

[30]

Sidong Feng, Mulong Xie, Yinxing Xue, and Chunyang Chen. 2023. Read It, Don’t Watch It: Captioning Bug Recordings Automatically. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 2349–2361.

[31]

Cuiyun Gao, Wujie Zheng, Yuetang Deng, David Lo, Jichuan Zeng, Michael R Lyu, and Irwin King. 2019. Emerging app issue identification from user feedback: Experience on wechat. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 279–288.

Digital Library

[32]

Lorenzo Gomez, Iulian Neamtiu, Tanzirul Azim, and Todd Millstein. 2013. Reran: Timing-and touch-sensitive record and replay for android. In 2013 35th International Conference on Software Engineering (ICSE). 72–81.

[33]

Jiaqi Guo, Shuyue Li, Jian-Guang Lou, Zijiang Yang, and Ting Liu. 2019. Sara: self-replay augmented record and replay for Android in industrial cases. In Proceedings of the 28th acm sigsoft international symposium on software testing and analysis. 90–100.

Digital Library

[34]

Matthew Halpern, Yuhao Zhu, Ramesh Peri, and Vijay Janapa Reddi. 2015. Mosaic: cross-platform user-interaction record and replay for the fragmented android ecosystem. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 215–224.

[35]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[36]

Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

[37]

Gang Hu, Xinhao Yuan, Yang Tang, and Junfeng Yang. 2014. Efficiently, effectively detecting mobile app bugs with appdoctor. In Proceedings of the Ninth European Conference on Computer Systems. 1–15.

Digital Library

[38]

Yongjian Hu, Tanzirul Azim, and Iulian Neamtiu. 2015. Versatile yet lightweight record-and-replay for android. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications. 349–366.

Digital Library

[39]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448–456.

Digital Library

[40]

Shachar Kaufman, Saharon Rosset, Claudia Perlich, and Ori Stitelman. 2012. Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data (TKDD), 6, 4 (2012), 1–21.

Digital Library

[41]

James M Keller, Michael R Gray, and James A Givens. 1985. A fuzzy k-nearest neighbor algorithm. IEEE transactions on systems, man, and cybernetics, 580–585.

[42]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

[43]

Sotiris B Kotsiantis, Ioannis Zaharakis, and P Pintelas. 2007. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering, 160, 1 (2007), 3–24.

[44]

Wing Lam, Zhengkai Wu, Dengfeng Li, Wenyu Wang, Haibing Zheng, Hui Luo, Peng Yan, Yuetang Deng, and Tao Xie. 2017. Record and replay for android: Are we there yet in industrial cases? In Proceedings of the 2017 11th joint meeting on foundations of software engineering. 854–859.

Digital Library

[45]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, 521, 7553 (2015), 436–444.

[46]

Cong Li, Yanyan Jiang, and Chang Xu. 2022. Cross-device record and replay for Android apps. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 395–407.

Digital Library

[47]

Li Li, Tegawendé F Bissyandé, Haoyu Wang, and Jacques Klein. 2018. Cid: Automating the detection of api-related compatibility issues in android apps. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 153–163.

Digital Library

[48]

Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge. 2020. Mapping natural language instructions to mobile UI action sequences. arXiv preprint arXiv:2005.03776.

[49]

Yi Liu, Jinhui Xie, Jianbo Yang, Shiyu Guo, Yuetang Deng, Shuqing Li, Yechang Wu, and Yepang Liu. 2020. Industry practice of javascript dynamic analysis on wechat mini-programs. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. 1189–1193.

Digital Library

[50]

Zhenyue Long, Guoquan Wu, Xiaojiang Chen, Wei Chen, and Jun Wei. 2020. WebRR: self-replay enhanced robust record/replay for web application testing. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1498–1508.

Digital Library

[51]

David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60, 2 (2004), 91–110.

Digital Library

[52]

Daniel Müllner. 2011. Modern hierarchical, agglomerative clustering algorithms. arXiv preprint arXiv:1109.2378.

[53]

Kevin P Murphy. 2012. Machine learning: a probabilistic perspective. MIT press.

Digital Library

[54]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, and Luca Antiga. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32 (2019).

[55]

Ju Qian, Zhengyu Shang, Shuoyan Yan, Yan Wang, and Lin Chen. 2020. Roscript: a visual script driven truly non-intrusive robotic testing system for touch screen applications. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 297–308.

Digital Library

[56]

Zhengrui Qin, Yutao Tang, Ed Novak, and Qun Li. 2016. Mobiplay: A remote execution based record-and-replay tool for mobile applications. In Proceedings of the 38th International Conference on Software Engineering. 571–582.

Digital Library

[57]

Alan Romano, Zihe Song, Sampath Grandhi, Wei Yang, and Weihang Wang. 2021. An empirical analysis of UI-based flaky tests. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 1585–1597.

Digital Library

[58]

Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An efficient alternative to SIFT or SURF. In 2011 International conference on computer vision. 2564–2571.

Digital Library

[59]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510–4520.

[60]

Connor Shorten and Taghi M Khoshgoftaar. 2019. A survey on image data augmentation for deep learning. Journal of big data, 6, 1 (2019), 1–48.

[61]

Wei Song, Xiangxing Qian, and Jeff Huang. 2017. EHBDroid: Beyond GUI testing for Android applications. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 27–37.

[62]

Donna Spencer. 2009. Card sorting: Designing usable categories. Rosenfeld Media.

[63]

Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. 2014. Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080.

[64]

Tao Wang, Qingxin Xu, Xiaoning Chang, Wensheng Dou, Jiaxin Zhu, Jinhui Xie, Yuetang Deng, Jianbo Yang, Jiaheng Yang, and Jun Wei. 2022. Characterizing and detecting bugs in WeChat mini-programs. In Proceedings of the 44th International Conference on Software Engineering. 363–375.

Digital Library

[65]

Wei Wang and Michael W Godfrey. 2013. Detecting api usage obstacles: A study of ios and android developer questions. In 2013 10th Working Conference on Mining Software Repositories (MSR). 61–64.

[66]

Wenyu Wang, Wing Lam, and Tao Xie. 2021. An infrastructure approach to improving effectiveness of Android UI testing tools. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 165–176.

Digital Library

[67]

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13, 4 (2004), 600–612.

Digital Library

[68]

Mulong Xie, Sidong Feng, Zhenchang Xing, Jieshan Chen, and Chunyang Chen. 2020. UIED: a hybrid tool for GUI element detection. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1655–1659.

Digital Library

[69]

Mulong Xie, Zhenchang Xing, Sidong Feng, Xiwei Xu, Liming Zhu, and Chunyang Chen. 2022. Psychologically-inspired, unsupervised inference of perceptual groups of GUI widgets from GUI images. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 332–343.

Digital Library

[70]

Jiwei Yan, Linjie Pan, Yaqi Li, Jun Yan, and Jian Zhang. 2018. Land: A user-friendly and customizable test generation tool for android apps. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 360–363.

Digital Library

[71]

Shengcheng Yu, Chunrong Fang, Yexiao Yun, and Yang Feng. 2021. Layout and image recognition driving cross-platform automated mobile testing. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 1561–1571.

Digital Library

[72]

Hrushikesh Zadgaonkar. 2013. Robotium automated testing for android. Packt Publishing.

Cited By

Zhang HLiao LDing ZShang WNarula NSporea CToma ASajedi SFilkov VRay BZhou M(2024)Towards a Robust Waiting Strategy for Web GUI Testing for an Industrial Software SystemProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695269(2065-2076)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695269
Feng SLu HJiang JXiong THuang LLiang YLi XDeng YAleti AFilkov VRay BZhou M(2024)Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChatProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695260(1973-1978)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695260
Feng SMa SWang HKong DChen C(2024)MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI ModelingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642350(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642350

Index Terms

Towards Efficient Record and Replay: A Case Study in WeChat
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Efficiency Matters: Speeding Up Automated Testing with GUI Rendering Inference
ICSE '23: Proceedings of the 45th International Conference on Software Engineering

Due to the importance of Android app quality assurance, many automated GUI testing tools have been developed. Although the test algorithms have been improved, the impact of GUI rendering has been overlooked. On the one hand, setting a long waiting ...
Versatile yet lightweight record-and-replay for Android
OOPSLA 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications

Recording and replaying the execution of smartphone apps is useful in a variety of contexts, from reproducing bugs to profiling and testing. Achieving effective record-and-replay is a balancing act between accuracy and overhead. On smartphones, the act ...
Versatile yet lightweight record-and-replay for Android
OOPSLA '15

Recording and replaying the execution of smartphone apps is useful in a variety of contexts, from reproducing bugs to profiling and testing. Achieving effective record-and-replay is a balancing act between accuracy and overhead. On smartphones, the act ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2023

2215 pages

ISBN:9798400703270

DOI:10.1145/3611643

General Chair:
Satish Chandra
Google, USA
,
Program Chairs:
Kelly Blincoe
University of Auckland, New Zealand
,
Paolo Tonella
USI Lugano, Switzerland

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '23

Sponsor:

SIGSOFT

ESEC/FSE '23: 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

December 3 - 9, 2023

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
118
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)8

Reflects downloads up to 16 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HLiao LDing ZShang WNarula NSporea CToma ASajedi SFilkov VRay BZhou M(2024)Towards a Robust Waiting Strategy for Web GUI Testing for an Industrial Software SystemProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695269(2065-2076)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695269
Feng SLu HJiang JXiong THuang LLiang YLi XDeng YAleti AFilkov VRay BZhou M(2024)Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChatProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695260(1973-1978)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695260
Feng SMa SWang HKong DChen C(2024)MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI ModelingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642350(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642350

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents