Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

The iMPAcT Tool for Android Testing

Published: 13 June 2019 Publication History

Abstract

This paper presents iMPAcT tool that tests recurring common behavior on Android mobile applications. The process followed combines exploration, reverse engineering and testing to automatically test Android mobile applications. The tool explores automatically the App by firing UI events. After each event fired, the tool checks if there are UI patterns present using a reverse engineering process. If a UI pattern is present, the tool runs the corresponding testing strategy (Test Pattern). During reverse engineering the tool uses a catalog of UI Patterns which describes recurring behavior (UI Patterns) to test and the corresponding test strategies (Test Patterns). This catalog may be extended in the future as needed (e.g., to deal with new interaction trends). This paper describes the implementation details of the iMPAcT tool, the catalog of patterns used, the outputs produced by the tool and the results of experiments performed in order to evaluate the overall testing approach. These results show that the overall testing approach is capable of finding failures on existing Android mobile applications.

References

[1]
Christoffer Quist Adamsen, Gianluca Mezzetti, and Anders Møller. 2015. Systematic execution of Android test suites in adverse conditions. In Proceedings of the 2015 International Symposium on Software Testing and Analysis - ISSTA 2015. ACM Press, New York, New York, USA, 83--93.
[2]
Pekka Aho, Tomi Raty, and Nadja Menz. 2013. Dynamic reverse engineering of GUI models for testing. In 2013 International Conference on Control, Decision and Information Technologies (CoDIT). IEEE, 441--447.
[3]
Christopher W. Alexander, Sara Ishikawa, Murray Silverstein, and Max Jacobson. 1977. A Pattern Language: Towns, Buildings, Construction 1 ed.). Oxford University Press, New York, New York, USA. 1171 pages.
[4]
Domenico Amalfitano, Nicola Amatucci, Anna Rita Fasolino, Ugo Gentile, Gianluca Mele, Roberto Nardone, Valeria Vittorini, and Stefano Marrone. 2014. Improving Code Coverage in Android Apps Testing by Exploiting Patterns and Automatic Test Case Generation. In International workshop on Long-term industrial collaboration on software engineering (WISE 2014). ACM, V"a sterás, Sweden, 29--34.
[5]
Domenico Amalfitano, Anna Rita Fasolino, and Porfirio Tramontana. 2009. Experimenting a reverse engineering technique for modelling the behaviour of rich internet applications. In 2009 IEEE International Conference on Software Maintenance. IEEE, 571--574.
[6]
Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Salvatore De Carmine, and Atif M. Memon. 2012. Using GUI ripping for automated testing of Android applications. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM Press, New York, New York, USA, 258.
[7]
Domenico Amalfitano, Vincenzo Riccio, Ana C. R. Paiva, and Anna Rita Fasolino. 2018. Why does the orientation change mess up my Android application? From GUI failures to code faults. Softw. Test., Verif. Reliab., Vol. 28, 1 (2018).
[8]
Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering - FSE '12. ACM Press, New York, New York, USA, 1.
[9]
Google Android. 2008. Announcing the Android 1.0 SDK, release 1. http://goo.gl/5PSQHj
[10]
Google Android. 2015. Android - What To Test. http://goo.gl/AL22tJ
[11]
Google Android. 2015. Android Navigation Drawer. http://goo.gl/nnJOoj
[12]
Google Android. 2015. Up and running with material design. https://goo.gl/GmsJSJ
[13]
Google Android. 2016. Pure Android. http://goo.gl/LqNPyS
[14]
Google Android. 2016. Tabs. https://www.google.com/design/spec/components/tabs.html
[15]
Apple. 2007. Apple Reinvents the Phone with iPhone. https://goo.gl/AoFdyx
[16]
Stephan Arlt, Cristiano Bertolini, and Martin Schaf. 2011. Behind the Scenes: An Approach to Incorporate Context in GUI Test Case Generation. In IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops (ICSTW 2011). Washington, DC, USA, 222--231.
[17]
Andrea Avancini and Mariano Ceccato. 2013. Security testing of the communication among Android applications. In 8th International Workshop on Automation of Software Test (AST 2013). IEEE Press, 57--63. http://dl.acm.org/citation.cfm?id=2662413.2662427
[18]
Leonid Batyuk, Markus Herpich, Seyit Ahmet Camtepe, Karsten Raddatz, Aubrey-Derrick Schmidt, and Sahin Albayrak. 2011. Using static analysis for automatic assessment and mitigation of unwanted and malicious activities within Android applications. In 2011 6th International Conference on Malicious and Unwanted Software. IEEE, 66--72.
[19]
Nick Butcher and Android Developers. 2016. Android Design in Action: Navigation Anti-Patterns. https://www.youtube.com/watch?v=Sww4omntVjs
[20]
Nick Butcher and Roman Nurik. 2016. Android Design in Action - Navigation anti - Patterns. http://www.readable.com/ff647Z8N
[21]
Capgemini, HP, and Sogeti. 2015. World Quality Report 2015--16. Technical Report. https://goo.gl/SVoKtl
[22]
Capgemini, Hp, Sogeti, and Hp. 2014. World Quality Report 2014--15. Technical Report. 1--64 pages. http://goo.gl/jzN2aA
[23]
E.J. Chikofsky and J.H. Cross. 1990. Reverse Engineering and Design Recovery: a Taxonomy. IEEE Software, Vol. 7, 1 (1990), 13--17.
[24]
Inês Coimbra Morgado and Ana C. R. Paiva. 2015. Test Patterns for Android Mobile Applications. In 20th European Conference on Pattern Languages of Programs (Europlop 2015). Irsee, Germany. http://dl.acm.org/citation.cfm?id=2855354
[25]
Inês Coimbra Morgado and Ana C. R. Paiva. 2015. Testing approach for mobile applications through reverse engineering of UI patterns. In Sixth International Workshop on Testing Techniques for Event BasED Software .
[26]
Inês Coimbra Morgado and Ana C. R. Paiva. 2015. The iMPAcT Tool: Testing UI Patterns on Mobile Applications. In 30th IEEE/ACM International Conference on Automated Software Engineering (ASE 2015). Lincoln, NE, USA. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7372083
[27]
Inês Coimbra Morgado and Ana C. R. Paiva. 2016. Impact of execution modes on finding Android failures. The 7th International Conference on Ambient Systems, Networks and Technologies, Vol. 83 (2016), 284--291.
[28]
Inês Coimbra Morgado, Ana C. R. Paiva, and Joao Pascoal Faria. 2011. Reverse Engineering of Graphical User Interfaces. In The Sixth International Conference on Software Engineering Advances (ICSEA '11). Barcelona, 293--298.
[29]
Inês Coimbra Morgado, Ana C. R. Paiva, and Joao Pascoal Faria. 2012. Dynamic Reverse Engineering of Graphical User Interfaces. International Journal On Advances in Software, Vol. 5, 3 and 4 (2012), 224--236. http://goo.gl/yRoIKF
[30]
Pedro Costa, Ana C. R. Paiva, and Miguel Nabuco. 2014. Pattern Based GUI Testing for Mobile Applications. In 9th International Conference on the Quality of Information and Communications Technology (QUATIC 2014). IEEE, Guimar a es, Portugal, 66--74.
[31]
Marco Cunha, Ana C R Paiva, Hugo Sereno Ferreira, and Rui Abreu. 2010. PETTool: A pattern-based GUI testing tool. In Software Technology and Engineering (ICSTE), 2010 2nd International Conference on, Vol. 1. IEEE, San Juan, PR, V1--202 -- VI--206.
[32]
Muneer Ahmad Dar and Javed Parvez. 2014. Enhancing security of Android & IOS by implementing need-based security (NBS). In 2014 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT). IEEE, 728--733.
[33]
Apple Developer. 2016. iOS Human Interface Guidilines. https://goo.gl/kUhwJE
[34]
Dominik Franke, Corinna Elsemann, Stefan Kowalewski, and Carsten Weise. 2011. Reverse Engineering of Mobile Application Lifecycles. In 18th Working Conference on Reverse Engineering (WCRE '11). IEEE, 283--292.
[35]
Dominik Franke, Stefan Kowalewski, Carsten Weise, and Nath Prakobkosol. 2012. Testing Conformance of Life Cycle Dependent Properties of Mobile Applications. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. IEEE, 241--250.
[36]
Yanick Fratantonio, Aravind Machiry, Antonio Bianchi, Christopher Kruegel, and Giovanni Vigna. 2015. CLAPP: characterizing loops in Android applications. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering - ESEC/FSE 2015. ACM Press, New York, New York, USA, 687--697.
[37]
Alessandra Gorla, Ilaria Tavecchia, Florian Gross, and Andreas Zeller. 2014. Checking app behavior against app descriptions. In Proceedings of the 36th International Conference on Software Engineering - ICSE 2014. ACM Press, New York, New York, USA, 1025--1035.
[38]
A. M. P. Grilo, A. C. R. Paiva, and J. P. Faria. 2010. Reverse engineering of GUI models for testing. In The 5th Iberian Conference on Information Systems and Technologies (CISTI '10). IEEE, 1--6. http://goo.gl/bXcIy
[39]
Daniel R. Hackner and Atif M. Memon. 2008. Test case generator for GUITAR. In Companion of the 13th international conference on Software engineering (ICSE Companion '08) (ICSE Companion '08). ACM Press, New York, New York, USA, 959.
[40]
Konstantin Holl and Frank Elberzhager. 2014. A Mobile-Specific Failure Classification and Its Usage to Focus Quality Assurance. In 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications. IEEE, 385--388.
[41]
Cuixiong Hu and Iulian Neamtiu. 2011. Automating GUI testing for android applications. In 6th International Workshop on Automation of Software Test (AST 2011). ACM, 77--83.
[42]
Gennaro Imparato. 2015. A combined technique of GUI ripping and input perturbation testing for Android apps. In 37th International Conference on Software Engineering - Volume 2 (ICSE '15). IEEE Press, 760--762. http://dl.acm.org/citation.cfm?id=2819009.2819159
[43]
Nathan Ingraham. 2013. Apple announces 1 million apps in the App Store, more than 1 billion songs played on iTunes radio. http://goo.gl/z3RprB
[44]
ISO/IEC. 2011. ISO/IEC 25010:2011 - Systems and software engineering Systems and software Quality Requirements and Evaluation (SQuaRE) System and software quality models. Technical Report. https://www.iso.org/obp/ui/#iso:std:iso-iec:25010
[45]
Casper S. Jensen, Mukul R. Prasad, and Anders Møller. 2013. Automated testing with targeted event sequence generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis - ISSTA 2013. ACM Press, New York, New York, USA, 67.
[46]
Mona Erfani Joorabchi and Ali Mesbah. 2012. Reverse Engineering iOS Mobile Applications. In 2012 19th Working Conference on Reverse Engineering. IEEE, 177--186.
[47]
Antti Kervinen, Mika Maunumaa, Tuula Pakkönen, Mika Katara, Wolfgang Grieskamp, and Carsten Weise. 2005. Model-Based Testing Through a GUI. In 5th International Workshop on Formal Approaches to Testing of Software (FATES 2005) (Lecture Notes in Computer Science), Wolfgang Grieskamp and Carsten Weise (Eds.), Vol. 3997. Springer-Verlag Berlin, Berlin, Heidelberg, 16--31.
[48]
Yuta Maezawa, Hironori Washizaki, and Shinichi Honiden. 2012. Extracting Interaction-Based Stateful Behavior in Rich Internet Applications. In 2012 16th European Conference on Software Maintenance and Reengineering. IEEE, 423--428.
[49]
Riyadh Mahmood, Naeem Esfahani, Thabet Kacem, Nariman Mirzaei, Sam Malek, and Angelos Stavrou. 2012. A whitebox approach for automated security testing of Android applications on the cloud. In 7th International Workshop on Automation of Software Test (AST 2012). IEEE, 22--28.
[50]
A Marchetto, P Tonella, and F Ricca. 2010. Under and Over Approximation of State Models Recovered for Ajax Applications. In 2010 14th European Conference on Software Maintenance and Reengineering. IEEE, 236--239. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5714441
[51]
Tiago Monteiro and Ana C. R. Paiva. 2013. Pattern Based GUI Testing Modeling Environment. In Sixth IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW 2013). IEEE, Luxembourg, Luxembourg, 140--143.
[52]
Rodrigo M. L. M. Moreira and Ana C. R. Paiva. 2014. A GUI Modeling DSL for Pattern-Based GUI Testing PARADIGM. In 9th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE'2014). Lisbon, Portugal.
[53]
Rodrigo M. L. M. Moreira and Ana C. R. Paiva. 2014. PBGT tool: an integrated modeling and testing environment for pattern-based GUI testing. In 29th ACM/IEEE international conference on Automated software engineering (ASE 2014). ACM Press, New York, New York, USA, 863--866.
[54]
Rodrigo M. L. M. Moreira, Ana C. R. Paiva, and Atif Memon. 2013. A pattern-based approach for GUI modeling and testing. In 24th IEEE International Symposium on Software Reliability Engineering (ISSRE 2013). IEEE, Passadena, CA, 288--297.
[55]
Rodrigo M. L. M. Moreira, Ana C. R. Paiva, Miguel Nabuco, and Atif Memon. 2017. Pattern-based GUI testing: Bridging the gap between design and quality assurance. Softw. Test., Verif. Reliab., Vol. 27, 3 (2017).
[56]
Henry Muccini, Antonio di Francesco, and Patrizio Esposito. 2012. Software testing of mobile applications: Challenges and future research directions. In 7th International Workshop on Automation of Software Test (AST 2012). IEEE, Zurich, Switzerland, 29--35.
[57]
Miguel Nabuco, Ana C.R. Paiva, Rui Camacho, and Joao Pascoal Faria. 2013. Inferring UI patterns with Inductive Logic Programming. In 8th Iberian Conference on Information Systems and Technologies (CISTI '13). Lisbon, Portugal, 1--5.
[58]
Miguel Nabuco and Ana C. R. Paiva. 2014. Model-Based Test Case Generation for Web Applications. In 14th International Conference on Computational Science and Applications (ICCSA 2014) .
[59]
Theresa Neil. 2014. Mobile Design Pattern Gallery: UI Patterns for Smartphone Apps 2nd ed.). O'Reilly Media, Inc., Sebastopol, Canada.
[60]
Cu D. Nguyen, Alessandro Marchetto, and Paolo Tonella. 2012. Combining model-based and combinatorial testing for effective test case generation. In Proceedings of the 2012 International Symposium on Software Testing and Analysis - ISSTA 2012. ACM Press, New York, New York, USA, 100.
[61]
Erik G. Nilsson. 2009. Design patterns for user interface for mobile applications. Advances in Engineering Software, Vol. 40, 12 (dec 2009), 1318--1328.
[62]
Ana C. R. Paiva, Joao C. P. Faria, and Pedro M. C. Mendes. 2007. Reverse engineered formal models for GUI testing. In The 12th international conference on Formal methods for industrial critical systems. Springer-Verlag, 218--233.
[63]
Tristan Ravitch, E. Rogan Creswick, Aaron Tomb, Adam Foltzer, Trevor Elliott, and Ledah Casburn. 2014. Multi-App Security Analysis with FUSE. In Proceedings of the 4th Program Protection and Reverse Engineering Workshop on 4th Program Protection and Reverse Engineering Workshop - PPREW-4. ACM Press, New York, New York, USA, 1--10.
[64]
A. Rohatgi, A. Hamou-Lhadj, and J. Rilling. 2008. An Approach for Mapping Features to Code Based on Static and Dynamic Analysis. In 2008 16th IEEE International Conference on Program Comprehension. IEEE, 236--241.
[65]
Clara Sacramento and Ana C. R. Paiva. 2014. Web Application Model Generation through Reverse Engineering and UI Pattern Inferring. In 9th International Conference on the Quality of Information and Communications Technology (QUATIC 2014). IEEE, Guimar a es, Portugal, 105--115.
[66]
Alireza Sahami Shirazi, Niels Henze, Albrecht Schmidt, Robin Goldberg, Benjamin Schmidt, and Hansjörg Schmauder. 2013. Insights into layout patterns of mobile user interfaces by an automatic analysis of android apps. In 5th ACM SIGCHI symposium on Engineering interactive computing systems. ACM, 275--284.
[67]
Hossain Shahriar, Sarah North, and Edward Mawangi. 2014. Testing of Memory Leak in Android Applications. In 2014 IEEE 15th International Symposium on High-Assurance Systems Engineering. IEEE, 176--183.
[68]
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools Approach 1 ed.). Morgan Kaufmann Publishers, San Francisco, CA, USA.
[69]
Heila van der Merwe, Brink van der Merwe, and Willem Visse. 2012. Verifying android applications using Java PathFinder. ACM SIGSOFT Software Engineering Notes, Vol. 37, 6 (nov 2012), 1.
[70]
Mark Wilson. 2008. T-Mobile G1: Full Details of the HTC Dream Android Phone. http://goo.gl/6vqI4E
[71]
Qing Xie. 2006. Developing cost-effective model-based techniques for GUI testing. In 28th international conference on Software engineering - (ICSE 2006). ACM Press, New York, New York, USA, 997--1000.
[72]
Wei Yang, Mukul R. Prasad, and Tao Xie. 2013. A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications. In 16th International Conference on Fundamental Approaches to Software Engineering (FASE'13). Rome, Italy, 250--265.
[73]
Siena Yu and Shingo Takada. 2015. External Event-Based Test Cases for Mobile Application. In Proceedings of the Eighth International C* Conference on Computer Science & Software Engineering - C3S2E '15. ACM Press, New York, New York, USA, 148--149.

Cited By

View all
  • (2023)A Systematic Review on Pattern-based GUI Testing of Android and Web Apps: State-of-the-Art, Taxonomy, Challenges and Future Directions2023 25th International Multitopic Conference (INMIC)10.1109/INMIC60434.2023.10465949(1-7)Online publication date: 17-Nov-2023
  • (2021)Evaluating Sensor Interaction Failures in Mobile ApplicationsQuality of Information and Communications Technology10.1007/978-3-030-85347-1_5(49-63)Online publication date: 25-Aug-2021
  • (2021)A mapping study on mutation testing for mobile applicationsSoftware Testing, Verification and Reliability10.1002/stvr.180132:8Online publication date: 20-Dec-2021
  • Show More Cited By

Index Terms

  1. The iMPAcT Tool for Android Testing

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Human-Computer Interaction
    Proceedings of the ACM on Human-Computer Interaction  Volume 3, Issue EICS
    June 2019
    553 pages
    EISSN:2573-0142
    DOI:10.1145/3340630
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 June 2019
    Published in PACMHCI Volume 3, Issue EICS

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. UI patterns
    2. android testing
    3. mobile crawler
    4. mobile testing
    5. reverse engineering
    6. software test automation
    7. software testing

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 28 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)A Systematic Review on Pattern-based GUI Testing of Android and Web Apps: State-of-the-Art, Taxonomy, Challenges and Future Directions2023 25th International Multitopic Conference (INMIC)10.1109/INMIC60434.2023.10465949(1-7)Online publication date: 17-Nov-2023
    • (2021)Evaluating Sensor Interaction Failures in Mobile ApplicationsQuality of Information and Communications Technology10.1007/978-3-030-85347-1_5(49-63)Online publication date: 25-Aug-2021
    • (2021)A mapping study on mutation testing for mobile applicationsSoftware Testing, Verification and Reliability10.1002/stvr.180132:8Online publication date: 20-Dec-2021
    • (2020)Reverse Engineering of Android Applications: REiMPAcTQuality of Information and Communications Technology10.1007/978-3-030-58793-2_30(369-382)Online publication date: 31-Aug-2020
    • (2019)Model-Based Testing of Post-WIMP Interactions Using Object Oriented Petri-NetsFormal Methods. FM 2019 International Workshops10.1007/978-3-030-54994-7_35(486-502)Online publication date: 7-Oct-2019

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media