research-article

Testing of Mobile Applications in the Wild: A Large-Scale Empirical Study on Android Apps

Authors:

Fabiano Pecorelli,

Gemma Catolino,

Filomena Ferrucci,

Andrea De Lucia,

Fabio PalombaAuthors Info & Claims

ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

Pages 296 - 307

https://doi.org/10.1145/3387904.3389256

Published: 12 September 2020 Publication History

Abstract

Nowadays, mobile applications (a.k.a., apps) are used by over two billion users for every type of need, including social and emergency connectivity. Their pervasiveness in today's world has inspired the software testing research community in devising approaches to allow developers to better test their apps and improve the quality of the tests being developed. In spite of this research effort, we still notice a lack of empirical studies aiming at assessing the actual quality of test cases developed by mobile developers: this perspective could provide evidence-based findings on the current status of testing in the wild as well as on the future research directions in the field. As such, we performed a large-scale empirical study targeting 1,780 open-source Android apps and aiming at assessing (1) the extent to which these apps are actually tested, (2) how well-designed are the available tests, and (3) what is their effectiveness. The key results of our study show that mobile developers still tend not to properly test their apps. Furthermore, we discovered that the test cases of the considered apps have a low (i) design quality, both in terms of test code metrics and test smells, and (ii) effectiveness when considering code coverage as well as assertion density.

References

[1]

Jean-Yves Antoine, Jeanne Villaneau, and Anaïs Lefeuvre. 2014. Weighted Krippendorff's alpha is a more reliable metrics for multi-coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation.

[2]

Gergő Balogh, Tamás Gergely, Árpád Beszédes, and Tibor Gyimóthy. 2016. Are my unit tests in the right package?. In 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 137--146.

[3]

Gabriele Bavota, Mario Linares-Vasquez, Carlos Eduardo Bernal-Cardenas, Massimiliano Di Penta, Rocco Oliveto, and Denys Poshyvanyk. 2014. The impact of api change-and fault-proneness on the user ratings of android apps. IEEE Transactions on Software Engineering 41, 4 (2014), 384--407.

Digital Library

[4]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and David Binkley. 2012. An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In Software Maintenance (ICSM), 2012 28th IEEE International Conference on. LEEE, 56--65.

Digital Library

[5]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering 20, 4 (2015), 1052--1094.

Digital Library

[6]

Moritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, and Andy Zaidman. 2017. Developer testing in the ide: Patterns, beliefs, and behavior. IEEE Transactions on Software Engineering 45, 3 (2017), 261--284.

Digital Library

[7]

Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman. 2015. When, How, and Why Developers (Do Not) Test in Their IDEs (ESEC/FSE 2015). ACM, New York, NY, USA, 179--190. https://doi.org/10.1145/2786805.2786843

[8]

Blinded. 2020. Testing of Mobile Applications in the Wild: A Large-Scale Empirical Study on Android Apps --- Online Appendix. https://figshare.com/s/0800d00b1c2d67e15915.

[9]

Gemma Catolino. 2018. Does source code quality reflect the ratings of apps?. In Proceedings of the 5th International Conference on Mobile Software Engineering and Systems. ACM, 43--44.

Digital Library

[10]

Gemma Catolino, Dario Di Nucci, and Filomena Ferrucci. 2019. Cross-Project Just-in-Time Bug Prediction for Mobile Apps: An Empirical Assessment. In 2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems (MOBILESoft). IEEE, 99--110.

[11]

Gemma Catolino, Fabio Palomba, Andy Zaidman, and Filomena Ferrucci. 2019. How the Experience of Development Teams Relates to Assertion Density of Test Classes. In 2019 TFFF. 35th International Conference on Software Maintenance and Evolution (ICSME). LEEE, to appear.

[12]

Gemma Catolino, Fabio Palomba, Andy Zaidman, and Filomena Ferrucci. 2019. Not all bugs are the same: Understanding, characterizing, and classifying bug types. Journal of Systems and Software 152 (2019), 165--181.

Digital Library

[13]

Shyam R Chidamber and Chris F Kemerer. 1994. A metrics suite for object oriented design. IEEE Transactions on software engineering 20, 6 (1994), 476--493.

Digital Library

[14]

Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. 2015. Automated test input generation for android: Are we there yet?. In 2015 30th IEEE/ACM International Conference on Automated Software Engineering ASE. IEEE, 429--440.

Digital Library

[15]

Steve Counsell, Stephen Swift, and Jason Crampton. 2006. The interpretation and utility of three cohesion metrics for object-oriented design. ACM Transactions on Software Engineering and Methodology (TOSEM) 15, 2 (2006), 123--149.

Digital Library

[16]

Luis Cruz, Rui Abreu, and David Lo. 2019. To the attention of mobile software developers: guess what, test your app! Empirical Software Engineering (2019), 1--31.

[17]

Teerath Das, Massimiliano Di Penta, and Ivano Malavolta. 2016. A quantitative and qualitative investigation of performance-related commits in android apps. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 443--447.

[18]

Dario Di Nucci, Fabio Palomba, Giuseppe De Rosa, Gabriele Bavota, Rocco Oliveto, and Andrea De Lucia. 2018. A developer centered bug prediction model. IEEE Transactions on Software Engineering 44, 1 (2018), 5--24.

Digital Library

[19]

Dario Di Nucci, Fabio Palomba, Antonio Prota, Annibale Panichella, Andy Zaidman, and Andrea De Lucia 2017. Software-based energy profiling of android apps: Simple, efficient and reliable?. In 2017 IEEE 24th international conference on software analysis, evolution and reengineering (SANER). LEEE, 103--114.

[20]

Moritz Eck, Fabio Palomba, Marco Castelluccio, and Alberto BacchellL 2019. Understanding Flaky Tests: The Developer's Perspective. (2019), to appear.

[21]

Letha H Etzkom, Sampson E Gholston, Julie L Fortune, Cara E Stein, Dawn Utley, Phillip A Farrington, and Glenn W Cox. 2004. A comparison of cohesion metrics for object-oriented systems. Information and Software Technology 46, 10 (2004), 677--687.

[22]

Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic Test Suite Generation for Object-oriented Software (ESEC/FSE '11). ACM, New York, NY, USA, 416--419. https://doi.org/10.1145/2025113.2025179

[23]

Enrico Fregnan, Tobias Baum, Fabio Palomba, and Alberto Bacchelli. 2018. A survey on software coupling relations and tools. Information and Software Technology (2018).

[24]

Jerry Gao, Wei-Tek Tsai, Ray Paul, Xiaoying Bai, and Tadahiro Uehara. 2014. Mobile Testing-as-a-Service (MTaaS)-Infrastructures, Issues, Solutions and Needs. In 2014 IEEE 15th International Symposium on High-Assurance Systems Engineering. IEEE, 158--167.

Digital Library

[25]

Vahid Garousi and Baış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of systems and software 138 (2018), 52--81.

[26]

Franz-Xaver Geiger and Ivano Malavolta. 2018. Datasets of Android Applications: a Literature Review. arXiv preprint arXiv:1809.10069 (2018).

[27]

Franz-Xaver Geiger, Ivano Malavolta, Luca Pascarella, Fabio Palomba, Dario Di Nucci, and Alberto Bacchelli. 2018. A graph-based dataset of commit history of real-world android apps. In Proceedings of the 15th International Conference on Mining Software Repositories. ACM, 30--33.

Digital Library

[28]

Peter Gilbert, Byung-Gon Chun, Landon P Cox, and Jaeyeon Jung. 2011. Vision: automated security validation of mobile apps at app markets. In Proceedings of the second international workshop on Mobile cloud computing and services. ACM, 21--26.

Digital Library

[29]

Rahul Gopinath, Iftekhar Ahmed, Mohammad Amin Alipour, Carlos Jensen, and Alex Groce. 2017. Mutation reduction strategies considered harmful. IEEE Transactions on Reliability 66, 3 (2017), 854--874.

[30]

Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Code coverage for suite evaluation by developers. In Proceedings of the 36th International Conference on Software Engineering. ACM, 72--82.

Digital Library

[31]

Giovanni Grano, Adelina Ciurumelea, Sebastiano Panichella, Fabio Palomba, and Harald C Gall. 2018. Exploring the integration of user feedback in automated testing of android applications. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 72--83.

[32]

Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. Journal of Systems and Software 156 (2019), 312--327.

Digital Library

[33]

Michaela Greiler, Arie van Deursen, and Margaret-Anne Storey. 2012. Test confessions: A study of testing practices for plug-in systems. In Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 244--254.

Digital Library

[34]

Michaela Greiler, Arie Van Deursen, and Margaret-Anne Storey. 2013. Automated detection of test fixture strategies and smells. In Software Testing, Verification and Validation (ICST). 322--331.

[35]

Brian Henderson-Sellers, Larry L Constantine, and Ian M Graham. 1996. Coupling and cohesion (towards a valid metrics suite for object-oriented analysis and design). Object oriented systems 3, 3 (1996), 143--158.

[36]

Yue Jia and Mark Harman. 2011. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2011), 649--678.

Digital Library

[37]

Mona Erfani Joorabchi, Ali Mesbah, and Philippe Kruchten. 2013. Real challenges in mobile app development. In 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 15--24.

[38]

Hammad Khalid, Emad Shihab, Meiyappan Nagappan, and Ahmed E Hassan. 2014. What do mobile app users complain about? IEEE Software 32, 3 (2014), 70--77.

Digital Library

[39]

Heejin Kim, Byoungju Choi, and W Eric Wong. 2009. Performance testing of mobile applications at the unit test level. In 2009 Third IEEE International Conference on Secure Software Integration and Reliability Improvement. IEEE, 171--180.

Digital Library

[40]

Pavneet Singh Kochhar, Ferdian Thung, Nachiappan Nagappan, Thomas Zimmermann, and David Lo. 2015. Understanding the test automation culture of app developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation ICST. IEEE, 1--10.

[41]

A Güneş Koru, Dongsong Zhang, Khaled El Emam, and Hongfang Liu. 2009. An investigation into the functional form of the size-defect relationship for software modules. IEEE Transactions on Software Engineering 35, 2 (2009), 293--304.

Digital Library

[42]

Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology. Sage publications.

[43]

Daniel E Krutz, Mehdi Mirakhorli, Samuel A Malachowsky, Andres Ruiz, Jacob Peterson, Andrew Filipski, and Jared Smith. 2015. A dataset of open-source Android applications. In Proceedings of the 12th Working Conference on Mining Software Repositories. IEEE Press, 522--525.

Digital Library

[44]

Gunnar Kudrjavets, Nachiappan Nagappan, and Thomas Ball. 2006. Assessing the relationship between software assertions and faults: An empirical investigation. In 2006 17th International Symposium on Software Reliability Engineering. IEEE, 204--212.

Digital Library

[45]

Christoph Laaber and Philipp Leitner. 2018. An evaluation of open-source software microbenchmark suites for continuous performance assessment. In Proceedings of the 15th International Conference on Mining Software Repositories. ACM, 119--130.

Digital Library

[46]

Mario Linares-Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do developers test android applications?. In 2017 IEEE International Conference on Software Maintenance and Evolution ICSME. IEEE, 613--622.

[47]

Mario Linares-Vásquez, Kevin Moran, and Denys Poshyvanyk. 2017. Continuous, evolutionary and large-scale: A new perspective for automated mobile app testing. In 2017 IEEE International Conference on Software Maintenance and Evolution ICSME. IEEE, 399--410.

[48]

Qingzhou Luo, Farah Hariri, Lamyaa Eloussi, and Darko Marinov. 2014. An empirical analysis of flaky tests. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 643--653.

Digital Library

[49]

Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 224--234.

Digital Library

[50]

Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, 94--105.

Digital Library

[51]

Brian Marick et al. 1999. How to misuse code coverage. In Proceedings of the 16tk Interational Conference on Testing Computer Software. 16--18.

[52]

William Martin, Federica Sarro, Yue Jia, Yuanyuan Zhang, and Mark Harman. 2016. A survey of app store analysis for software engineering. IEEE transactions on software engineering 43, 9 (2016), 817--847.

[53]

Stuart McIlroy, Nasir Ali, and Ahmed E Hassan. 2016. Fresh apps: an empirical study of frequently-updated mobile apps in the Google play store. Empirical Software Engineering 21, 3 (2016), 1346--1370.

Digital Library

[54]

Patrick E McKnight and Julius Najab. 2010. Mann-Whitney U Test. The Corsini encyclopedia of psychology (2010), 1--1.

[55]

Ali Mesbah and Mukul R Prasad. 2011. Automated cross-browser compatibility testing. In Proceedings of the 33rd International Conference on Software Engineering. ACM, 561--570.

Digital Library

[56]

Gerard Meszaros. 2007. xUnit test patterns: Refactoring test code. Pearson Education.

[57]

Roberto Minelli and Michele Lanza. 2013. Software Analytics for Mobile Applications-Insights & Lessons Learned. In 2013 17th European Conference on Software Maintenance and Reengineering. IEEE, 144--153.

[58]

Audris Mockus, Nachiappan Nagappan, and Trung T Dinh-Trong. 2009. Test coverage and post-verification defects: A multiple case study. In 2009 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE, 291--301.

Digital Library

[59]

Israel J Mojica, Bram Adams, Meiyappan Nagappan, Steffen Dienst, Thorsten Berger, and Ahmed E Hassan. 2013. A large-scale empirical study on software reuse in mobile apps. IEEE software 31, 2 (2013), 78--86.

[60]

Rodrigo Morales, Ruben Saborido, Foutse Khomh, Francisco Chicano, and Giuliano Antoniol. 2016. Anti-patterns and the energy efficiency of Android applications. arXiv preprint arXiv:1610.05711 (2016).

[61]

Henry Muccini, Antonio Di Francesco, and Patrizio Esposito. 2012. Software testing of mobile applications: Challenges and future research directions. In Proceedings of the 7th International Workshop on Automation of Software Test. IEEE Press, 29--35.

Digital Library

[62]

Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.

Digital Library

[63]

Meiyappan Nagappan and Emad Shihab. 2016. Future trends in software engineering research for mobile apps. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 5. IEEE, 21--32.

[64]

Nachiappan Nagappan, Andreas Zeiler, Thomas Zimmermann, Kim Herzig, and Brendan Murphy. 2010. Change bursts as defect predictors. In 2010 IEEE 21st International Symposium on Software Reliability Engineering. IEEE, 309--318.

Digital Library

[65]

Maleknaz Nayebi, Bram Adams, and Guenther Ruhe. 2016. Release Practices for Mobile Apps-What do Users and Developers Think?. In 2016 ieee 23rd international conference on software analysis, evolution, and reengineering (saner), Vol. 1. IEEE, 552--562.

[66]

Business of Apps. [n.d.]. There are 12 million mobile developers world-wide, and nearly half develop for Android first.

[67]

Fabio Palomba, Dario Di Nucci, Annibale Panichella, Rocco Oliveto, and Andrea De Lucia. 2016. On the diffusion of test smells in automatically generated test code: An empirical study. In Proceedings of the 9th International Workshop on Search-Based Software Testing. ACM, 5--14.

Digital Library

[68]

Fabio Palomba, Dario Di Nucci, Annibale Panichella, Andy Zaidman, and Andrea De Lucia. 2019. On the impact of code smells on the energy consumption of mobile applications. Information and Software Technology 105 (2019), 43--55.

[69]

Fabio Palomba, Mario Linares-Vásquez, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Denys Poshyvanyk, and Andrea De Lucia. 2018. Crowdsourcing user reviews to support the evolution of mobile apps. Journal of Systems and Software 137 (2018), 143--162.

[70]

Fabio Palomba, Annibale Panichella, Andy Zaidman, Rocco Oliveto, and Andrea De Lucia. 2016. Automatic test case generation: What if test code quality matters?. In Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, 130--141.

Digital Library

[71]

Fabio Palomba, Pasquale Salza, Adefina Ciurumelea, Sebastiano Panichella, Harald Gall, Filomena Ferrucci, and Andrea De Lucia. 2017. Recommending and localizing change requests for mobile apps based on user reviews. In Proceedings of the 39th international conference on software engineering. IEEE Press, 106--117.

Digital Library

[72]

Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic test smell detection using information retrieval techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 311--322.

[73]

Fabio Palomba, Andy Zaidman, Rocco Oliveto, and Andrea De Lucia. 2017. An exploratory study on the relationship between changes and refactoring. In 2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC). IEEE, 176--185.

Digital Library

[74]

Mauro Pezze, Konstantin Rubinov, and Jochen Wuttke. 2013. Generating effective integration test cases from unit ones. In 2013 WEE Sixth International Conference on Software Testing, Verification and Validation. IEEE, 11--20.

Digital Library

[75]

Raphael Pham, Stephan Riesling, Olga Liskin, Leif Singer, and Kurt Schneider. 2014. Enablers, inhibitors, and perceptions of testing in novice software teams. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 30--40.

Digital Library

[76]

Nick Pidgeon and Karen Henwood. 2004. Grounded theory, na.

[77]

Pasquale Salza, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Filomena Ferrucci 2019. Third-party libraries in mobile apps. Empirical Software Engineering (2019), 1--37.

[78]

Davi Bernardo Silva, Andre Takeshi Endo, Marcelo Medeiros Eler, and Vinicius HS Durelli. 2016. An analysis of automated tests for mobile Android applications. In 2016 KLU Latin American Computing Conference CLEL IEEE, 1--9.

[79]

Davide Spadini, Fabio Palomba, Tobias Baum, Stefan Hanenberg, Magiel Bruntink, and Alberto Bacchelli. 2019. Test-driven code review: an empirical study. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 1061--1072.

Digital Library

[80]

Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the relation of test smells to software code quality. In 2018 WEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 1--12.

[81]

Starista. 2018. Number of apps available in leading app stores as of October 2018.

[82]

Mark D Syer, Meiyappan Nagappan, Bram Adams, and Ahmed E Hassan. 2015. Studying the relationship between source code quality and mobile platform dependence. Software Quality Journal 23, 3 (2015), 485--508.

Digital Library

[83]

Mark D Syer, Meiyappan Nagappan, Ahmed E Hassan, and Bram Adams. 2013. Revisiting prior empirical findings for mobile apps: An empirical case study on the 15 most popular open-source Android apps. In Proceedings of the 2013 Conference of the Center for Advanced Studies on Collaborative Research. IBM Corp., 283--297.

[84]

Michele Tufano, Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrea De Lucia, and Denys Poshyvanyk. 2016. An empirical investigation into the nature of test smells. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 4--15.

Digital Library

[85]

Bela Ujhazi, Rudolf Ferenc, Denys Poshyvanyk, and Tibor Gyimothy. 2010. New conceptual coupling and cohesion metrics for object-oriented systems. In 2010 10th WEE Working Conference on Source Code Analysis and Manipulation. IEEE, 33--42.

Digital Library

[86]

Arash Vahabzadeh, Amin Milani Fard, and Ali Mesbah. 2015. An empirical study of bugs in test code. In 2015 WEE international conference on software maintenance and evolution (ICSME). IEEE, 101--110.

Digital Library

[87]

Arie Van Deursen, Leon Moonen, Alex van den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering (XP2001). 92--95.

[88]

Bart Van Rompaey and Serge Demeyer. 2009. Establishing traceability links between unit test cases and units under test. In 2009 13th European Conference on Software Maintenance and Reengineering. IEEE, 209--218.

Digital Library

[89]

Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On the detection of test smells: A metrics-based approach for general fixture and eager test. WEE Transactions on Software Engineering 33, 12 (2007), 800--817.

Digital Library

[90]

Tony Wasserman. 2010. Software engineering issues for mobile application development. (2010).

[91]

Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2016. Taming Android fragmentation: Characterizing and detecting compatibility issues for Android apps. In 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 226--237.

Digital Library

[92]

Yi Wei, Bertrand Meyer, and Manuel Oriol. 2012. Is branch coverage a good measure of testing effectiveness? In Empirical Software Engineering and Verification. Springer, 194--212.

Digital Library

[93]

Chak Shun Yu, Christoph Treude, and Mauricio Aniche. 2019. Comprehending Test Code: An Empirical Study. (2019), to appear.

[94]

Nico Zazworka, Michele A Shaw, Forrest Shull, and Carolyn Seaman. 2011. Investigating the impact of design debt on software quality. In Proceedings of the 2nd Workshop on Managing Technical Debt. ACM, 17--23.

Digital Library

[95]

Jack Zhang, Shikhar Sagar, and Emad Shihab. 2013. The evolution of mobile apps: An exploratory study. In Proceedings of the 2013 International Workshop on Software Development Lifecycle for Mobile. ACM, 1--8.

Digital Library

[96]

Thomas Zimmermann, Rahul Premraj, and Andreas Zeiler. 2007. Predicting defects for eclipse. In Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007). IEEE, 9--9.

Digital Library

Cited By

Callan JPetke J(2024)Multi-objective improvement of Android applicationsAutomated Software Engineering10.1007/s10515-024-00472-732:1Online publication date: 4-Nov-2024
https://doi.org/10.1007/s10515-024-00472-7
Ramos MVicedo APadallan JVirtucio JBalatibat GLapinid JCatedrilla GAsor J(2024)Development of a Smart Pole System Utilizing Arduino Technology and the Internet of ThingsIntelligent Sustainable Systems10.1007/978-981-99-8111-3_7(67-77)Online publication date: 8-Mar-2024
https://doi.org/10.1007/978-981-99-8111-3_7
Rzig DIqbal NAttisano IQin XHassan FJust RFraser G(2023)Virtual Reality (VR) Automated Testing in the Wild: A Case Study on Unity-Based VR ApplicationsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598134(1269-1281)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598134
Show More Cited By

Index Terms

Testing of Mobile Applications in the Wild: A Large-Scale Empirical Study on Android Apps
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation
      2. Software defect analysis
        Software testing and debugging

Recommendations

GUI testing for mobile applications: objectives, approaches and challenges
Internetware '20: Proceedings of the 12th Asia-Pacific Symposium on Internetware

Graphical User Interface (GUI) is unavoidable in modern software apps. It facilitates the interactions between the users and the apps. As shown on the Google play store, some apps with higher downloads often have higher-quality, well-designed and ...
An Explorative Study of the Mobile App Ecosystem from App Developers' Perspective
WWW '17: Proceedings of the 26th International Conference on World Wide Web

With the prevalence of smartphones, app markets such as Apple App Store and Google Play has become the center stage in the mobile app ecosystem, with millions of apps developed by tens of thousands of app developers in each major market. This paper ...
Mobile Testing

Mobile apps are everywhere. Some apps entertain and others enable business transactions. Apps increasingly interact with complex IT landscapes. For example, a banking app on a mobile device acts as a front end that invokes services on a back-end server ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPC '20: Proceedings of the 28th International Conference on Program Comprehension

July 2020

481 pages

ISBN:9781450379588

DOI:10.1145/3387904

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung

Conference

ICPC '20

Sponsor:

SIGSOFT

ICPC '20: 28th International Conference on Program Comprehension

July 13 - 15, 2020

Seoul, Republic of Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
441
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Callan JPetke J(2024)Multi-objective improvement of Android applicationsAutomated Software Engineering10.1007/s10515-024-00472-732:1Online publication date: 4-Nov-2024
https://doi.org/10.1007/s10515-024-00472-7
Ramos MVicedo APadallan JVirtucio JBalatibat GLapinid JCatedrilla GAsor J(2024)Development of a Smart Pole System Utilizing Arduino Technology and the Internet of ThingsIntelligent Sustainable Systems10.1007/978-981-99-8111-3_7(67-77)Online publication date: 8-Mar-2024
https://doi.org/10.1007/978-981-99-8111-3_7
Rzig DIqbal NAttisano IQin XHassan FJust RFraser G(2023)Virtual Reality (VR) Automated Testing in the Wild: A Case Study on Unity-Based VR ApplicationsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598134(1269-1281)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598134
Wilder GMiyamoto RWatson SKazman RPeruma A(2023)An Exploratory Study on the Occurrence of Self-Admitted Technical Debt in Android Apps2023 ACM/IEEE International Conference on Technical Debt (TechDebt)10.1109/TechDebt59074.2023.00007(1-10)Online publication date: May-2023
https://doi.org/10.1109/TechDebt59074.2023.00007
Kuroishi PMaldonado JVincenzi A(2023)Towards the Implementation of a Mobile Application Testing Infrastructure at Von Braun Labs2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE59848.2023.00078(91-101)Online publication date: 9-Oct-2023
https://doi.org/10.1109/ISSRE59848.2023.00078
Fuad MSakib K(2023)WebEV: A Dataset on the Behavior of Testers for Web Application End to End Testing2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC)10.1109/ICPC58990.2023.00022(79-83)Online publication date: May-2023
https://doi.org/10.1109/ICPC58990.2023.00022
Martins LCosta HMachado I(2023)On the diffusion of test smells and their relationship with test code quality of Java projectsJournal of Software: Evolution and Process10.1002/smr.2532Online publication date: 18-Jan-2023
https://doi.org/10.1002/smr.2532
Liu ZDwyer M(2022)WoodpeckerProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings10.1145/3510454.3522681(334-336)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510454.3522681
Chen SChen CFan LFan MZhan XLiu Y(2022)Accessible or Not? An Empirical Investigation of Android App AccessibilityIEEE Transactions on Software Engineering10.1109/TSE.2021.310816248:10(3954-3968)Online publication date: 1-Oct-2022
https://doi.org/10.1109/TSE.2021.3108162
Liu Z(2022)Woodpecker: Identifying and Fixing Android UI Display Issues2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)10.1109/ICSE-Companion55297.2022.9793773(334-336)Online publication date: May-2022
https://doi.org/10.1109/ICSE-Companion55297.2022.9793773
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents