research-article

Predicting Crashing Releases of Mobile Applications

Authors:

Yasutaka Kamei,

Xinyu WangAuthors Info & Claims

ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Article No.: 29, Pages 1 - 10

https://doi.org/10.1145/2961111.2962606

Published: 08 September 2016 Publication History

Abstract

Context: The quality of mobile applications has a vital impact on their user's experience, ratings and ultimately overall success. Given the high competition in the mobile application market, i.e., many mobile applications perform the same or similar functionality, users of mobile apps tend to be less tolerant to quality issues.

Goal: Therefore, identifying these crashing releases early on so that they can be avoided will help mobile app developers keep their user base and ensure the overall success of their apps.

Method: To help mobile developers, we use machine learning techniques to effectively predict mobile app releases that are more likely to cause crashes, i.e., crashing releases. To perform our prediction, we mine and use a number of factors about the mobile releases, that are grouped into six unique dimensions: complexity, time, code, diffusion, commit, and text, and use a Naive Bayes classified to perform our prediction.

Results: We perform an empirical study on 10 open source mobile applications containing a total of 2,638 releases from the F-Droid repository. On average, our approach can achieve F1 and AUC scores that improve over a baseline (random) predictor by 50% and 28%, respectively. We also find that factors related to text extracted from the commit logs prior to a release are the best predictors of crashing releases and have the largest effect.

Conclusions: Our proposed approach could help to identify crash releases for mobile apps.

References

[1]

K. Allix, Q. Jerome, T. F. Bissyande, J. Klein, R. State, and Y. le Traon. A forensic analysis of android malware-how is malware written and how it could be detected? In COMPSAC, 2014.

Digital Library

[2]

Android, Apple, Google, Microsoft, AppBrain, BlackBerry, V. sources (WindowsCentral.com), and A. (n.d.). Number of apps available in leading app stores as of july 2015.

[3]

P. Bhattacharya, M. Iliofotou, I. Neamtiu, and M. Faloutsos. Graph-based analysis and prediction for software evolution. In ICSE, 2012.

Digital Library

[4]

D. Bowes, T. Hall, M. Harman, Y. Jia, F. Sarro, and F. Wu. Mutation-aware fault prediction. In ISSTA 2016.

Digital Library

[5]

L. Breiman. Random forests. Machine learning, 2001.

Digital Library

[6]

N. Cliff. Ordinal methods for behavioral data analysis. 2014.

[7]

T. Dhaliwal, F. Khomh, and Y. Zou. Classifying field crash reports for fixing bugs: A case study of mozilla firefox. In ICSM 2011.

Digital Library

[8]

A. Estabrooks and N. Japkowicz. A mixture-of-experts framework for learning from imbalanced data sets. In Advances in Intelligent Data Analysis. 2001.

Digital Library

[9]

F. Ferrucci, C. Gravino, P. Salza, and F. Sarro. Investigating functional and code size measures for mobile applications: A replicated study. In Product-Focused Software Process Improvement.

Digital Library

[10]

A. Finkelstein, M. Harman, Y. Jia, F. Sarro, and Y. Zhang. Mining app stores: Extracting technical, business and customer rating information for analysis and prediction. RN, 13:21, 2013.

[11]

A. S. Foulkes. Applied statistical genetics with R. 2009.

[12]

R. Francese, C. Gravino, M. Risi, G. Scanniello, and G. Tortora. On the use of requirements measures to predict software project and product measures in the context of android mobile apps: a preliminary study. In EUROMICRO-SEAA 2015.

Digital Library

[13]

L. Guerrouj, S. Azad, and P. C. Rigby. The influence of app churn on app success and stackoverflow discussions. In SANER 2015.

[14]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The weka data mining software: an update. ACM SIGKDD explorations newsletter, 2009.

Digital Library

[15]

J. Han and M. Kamber. Data Mining, Southeast Asia Edition: Concepts and Techniques. 2006.

Digital Library

[16]

M. Harman, Y. Jia, and Y. Zhang. App store mining and analysis: Msr for app stores. In MSR, 2012.

Digital Library

[17]

A. E. Hassan. Predicting faults using the complexity of code changes. In ICSE 2009.

Digital Library

[18]

N. Japkowicz and S. Stephen. The class imbalance problem: A systematic study. Intelligent data analysis, 2002.

Digital Library

[19]

T. Jiang, L. Tan, and S. Kim. Personalized defect prediction. In ASE 2013.

Digital Library

[20]

M. E. Joorabchi, A. Mesbah, and P. Kruchten. Real challenges in mobile app development. In ESEM, 2013.

[21]

Y. Kamei, E. Shihab, B. Adams, A. E. Hassan, A. Mockus, A. Sinha, and N. Ubayashi. A large-scale empirical study of just-in-time quality assurance. TSE, 2013.

Digital Library

[22]

H. Khalid. On identifying user complaints of ios apps. In ICSE, 2013.

Digital Library

[23]

H. Khalid, E. Shihab, M. Nagappan, and A. E. Hassan. What do mobile app users complain about? a study on free ios apps. IEEE Software, 2015.

[24]

S. Kim, E. J. Whitehead, and Y. Zhang. Classifying software changes: Clean or buggy? TSE.

Digital Library

[25]

S. Kim, T. Zimmermann, E. J. Whitehead Jr, and A. Zeller. Predicting faults from cached history. In ICSE, 2007.

Digital Library

[26]

D. E. Krutz, M. Mirakhorli, S. A. Malachowsky, A. Ruiz, J. Peterson, A. Filipski, and J. Smith. A dataset of open-source android applications. In MSR.

Digital Library

[27]

A. Lamkanfi, S. Demeyer, E. Giger, and B. Goethals. Predicting the severity of a reported bug. In MSR, 2010.

[28]

S. Lessmann, B. Baesens, C. Mues, and S. Pietsch. Benchmarking classification models for software defect prediction: A proposed framework and novel findings. TSE.

Digital Library

[29]

A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for android apps. In FSE, 2013.

Digital Library

[30]

W. Martin, M. Harman, Y. Jia, F. Sarro, and Y. Zhang. The app sampling problem for app Store mining. In MSR 2015.

Digital Library

[31]

W. Martin, F. Sarro, and M. Harman. Causal impact analysis for app releases in google play. In FSE 2016.

Digital Library

[32]

W. Martin, F. Sarro, and M. Harman. Causal impact analysis applied to app releases in google play and windows phone store. RN, 15:07, 2015.

[33]

W. Martin, F. Sarro, Y. Jia, Y. Zhang, and M. Harman. A survey of app store analysis for software engineering. RN, 16:02,2016.

[34]

A. McCallum, K. Nigam, et al. A comparison of event models for naive bayes text classification. In AAAI-98 workshop.

[35]

T. Menzies, Z. Milton, B. Turban, B. Cukic, Y. Jiang, and A. Bener. Defect prediction from static code features: current results, limitations, new approaches. Automated Software Engineering, 2010.

Digital Library

[36]

R. Minelli and M. Lanza. Software analytics for mobile applications--insights & lessons learned. In CSMR, 2013.

Digital Library

[37]

A. Mockus. Organizational volatility and its effects on software defects. In FSE, 2010.

Digital Library

[38]

A. Mockus and D. M. Weiss. Predicting risk of software changes. Bell Labs Technical Journal, 2000.

[39]

R. Moser, W. Pedrycz, and G. Succi. A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In ICSE, 2008.

Digital Library

[40]

H. Muccini, A. Di Francesco, and P. Esposito. Software testing of mobile applications: Challenges and future research directions. In AST, 2012.

Digital Library

[41]

N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In ICSE, 2005.

Digital Library

[42]

N. Nagappan, T. Ball, and A. Zeller. Mining metrics to predict component failures. In ICSE, 2006.

Digital Library

[43]

J. Nam, S. J. Pan, and S. Kim. Transfer defect learning. In ICSE 2013.

Digital Library

[44]

M. Nayebi, B. Adams, and G. Ruhe. Release practices in mobile apps? users and developers perception. In SANER 2016.

[45]

F. Rahman, D. Posnett, A. Hindle, E. Barr, and P. Devanbu. Bugcache for inspections: hit or miss? In ESEC/FSE, 2011.

Digital Library

[46]

J. D. Rennie, L. Shih, J. Teevan, D. R. Karger, et al. Tackling the poor assumptions of naive bayes text classifiers. In ICML, 2003.

[47]

B. Robinson and P. Francis. Improving industrial adoption of software engineering research: a comparison of open and closed source software. In ESEM 2010, ESEM '10.

Digital Library

[48]

D. Romano and M. Pinzger. Using source code metrics to predict change-prone java interfaces. In ICSM, 2011.

Digital Library

[49]

C. Rosen, B. Grawi, and E. Shihab. Commit guru: Analytics and risk prediction of software commits. In ESEC/FSE 2015.

Digital Library

[50]

C. Rosen and E. Shihab. What are mobile developers asking about? a large scale study using stack overflow. EMSE.

Digital Library

[51]

E. Shihab, A. E. Hassan, B. Adams, and Z. M. Jiang. An industrial study on the risk of software changes. In FSE, 2012.

Digital Library

[52]

J. Śliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? SIGSOFT Softw. Eng. Notes, 2005.

Digital Library

[53]

J. Su, H. Zhang, C. X. Ling, and S. Matwin. Discriminative parameter learning for bayesian networks. In ICML, 2008.

Digital Library

[54]

R. Subramanyam and M. S. Krishnan. Empirical analysis of ck metrics for object-oriented design complexity: Implications for software defects. TSE, 2003.

Digital Library

[55]

F. Thung, D. Lo, M. H. Osman, and M. R. Chaudron. Condensing class diagrams by analyzing design and network metrics using optimistic classification. In ICPC, 2014.

Digital Library

[56]

Y. Tian, J. Lawall, and D. Lo. Identifying linux bug fixing patches. In ICSE, 2012.

Digital Library

[57]

Y. Tian, D. Lo, and C. Sun. Drone: Predicting priority of reported bugs by multi-factor analysis. In ICSM, 2013.

Digital Library

[58]

B. Turhan, T. Menzies, A. B. Bener, and J. Di Stefano. On the relative value of cross-company and within-company data far defect prediction. EMSE, 2009.

Digital Library

[59]

H. Valdivia and E. Shihab. Characterizing and predicting blocking bugs in open source projects. In MSR, 2014.

Digital Library

[60]

H. Valdivia Garcia and E. Shihab. Characterizing and predicting blocking bugs in open source projects. In MSR, 2014.

Digital Library

[61]

F. Wilcoxon. Some rapid approximate statistical procedures. Annals of the New York Academy of Sciences, 52(6):808--814,1950.

[62]

R. Wu, H. Zhang, S. Kim, and S.-C. Cheung. Relink: recovering links between bugs and changes. In ESEC/FSE, 2011.

Digital Library

[63]

X. Xia, D. Lo, E. Shihab, X. Wang, and X. Yang. Elblocker: Predicting blocking bugs with ensemble imbalance learning. Information and Software Technology, 61:93--106, 2015.

Digital Library

[64]

P. Yu and C.-m. Au Yeung. App mining: finding the real value of mobile applications. In WWW, 2014.

Digital Library

[65]

H. Zimmermann. Fuzzy Set Theory and Its Applications Second, Revised Edition. 1992.

Digital Library

Cited By

Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3439002
Wang HGao ZHu XLo DGrundy JWang X(2024)Just-In-Time TODO-Missed Commits DetectionIEEE Transactions on Software Engineering10.1109/TSE.2024.340500550:11(2732-2752)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3405005
Tao HNiu XXu LFu LCao QChen HShang SXian Y(2024)A comparative study of software defect binomial classification prediction models based on machine learningSoftware Quality Journal10.1007/s11219-024-09683-332:3(1203-1237)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09683-3
Show More Cited By

Recommendations

Evaluating Mobile Applications: A Spreadsheet Case Study

The power of mobile devices has increased dramatically in the last few years. These devices are becoming more sophisticated and allow users to accomplish a wide variety of tasks while on the move. The ease with which mobile apps can be created and ...
User Perceptions of the Security of Mobile Applications

With the exponential increase in the use of mobile devices across the globe, there is a concomitant need to understand how mobile users perceive the security of mobile application, and the potential risks involved in accessing and downloading them. ...
Mobile Clinical Decision Support Systems and Applications: A Literature and Commercial Review

The latest advances in eHealth and mHealth have propitiated the rapidly creation and expansion of mobile applications for health care. One of these types of applications are the clinical decision support systems, which nowadays are being implemented in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEM '16: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

September 2016

457 pages

ISBN:9781450344272

DOI:10.1145/2961111

General Chair:
Marcela Genero
University of Castilla-La Mancha, Spain
,
Program Chairs:
Andreas Jedlitschka
Fraunhofer IESE, Germany
,
Magne Jørgensen
Simula Research Laboratory, Norway
,
Giuseppe Scanniello
University of Basilicata, Italy
,
Sreedevi Sampath
University of Maryland Baltimore County, USA
,
Danilo Caivano
SER&Practices, Italy
,
Daniel Port
University of Hawaii, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ESEM '16

Sponsor:

SIGSOFT

ESEM '16: ACM/IEEE 9th International Symposium on Empirical Software Engineering and Measurement

September 8 - 9, 2016

Ciudad Real, Spain

Acceptance Rates

ESEM '16 Paper Acceptance Rate 27 of 122 submissions, 22%;

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
471
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)4

Reflects downloads up to 14 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Clerissi DDenaro GMobilio MMariani L(2024)DBInputs: Exploiting Persistent Data to Improve Automated GUI TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.343900250:9(2412-2436)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3439002
Wang HGao ZHu XLo DGrundy JWang X(2024)Just-In-Time TODO-Missed Commits DetectionIEEE Transactions on Software Engineering10.1109/TSE.2024.340500550:11(2732-2752)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3405005
Tao HNiu XXu LFu LCao QChen HShang SXian Y(2024)A comparative study of software defect binomial classification prediction models based on machine learningSoftware Quality Journal10.1007/s11219-024-09683-332:3(1203-1237)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09683-3
Wimalasooriya CLicorish Sda Costa DMacDonell S(2024)Just-in-Time crash prediction for mobile appsEmpirical Software Engineering10.1007/s10664-024-10455-729:3Online publication date: 8-May-2024
https://dl.acm.org/doi/10.1007/s10664-024-10455-7
Zhu XQiu TWang JLai X(2024)A novel instance‐based method for cross‐project just‐in‐time defect predictionSoftware: Practice and Experience10.1002/spe.331654:6(1087-1117)Online publication date: 24-Jan-2024
https://doi.org/10.1002/spe.3316
Sun WYan MLiu ZXia XLei YLo D(2023)Revisiting the Identification of the Co-evolution of Production and Test CodeACM Transactions on Software Engineering and Methodology10.1145/360718332:6(1-37)Online publication date: 30-Sep-2023
https://dl.acm.org/doi/10.1145/3607183
C. SMenzies T(2023)Assessing the Early Bird Heuristic (for Predicting Project Quality)ACM Transactions on Software Engineering and Methodology10.1145/358356532:5(1-39)Online publication date: 24-Jul-2023
https://dl.acm.org/doi/10.1145/3583565
Yin GWang WLi H(2023)OdegVul: An Approach for Statement-Level Defect PredictionInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402350061434:04(569-595)Online publication date: 20-Nov-2023
https://doi.org/10.1142/S0218194023500614
Yu DLi SChen XSun T(2023)Identifying the severity of technical debt issues based on semantic and structural informationSoftware Quality Journal10.1007/s11219-023-09651-331:4(1499-1526)Online publication date: 10-Oct-2023
https://doi.org/10.1007/s11219-023-09651-3
Jorayeva MAkbulut ACatal CMishra A(2022)Deep Learning-Based Defect Prediction for Mobile ApplicationsSensors10.3390/s2213473422:13(4734)Online publication date: 23-Jun-2022
https://doi.org/10.3390/s22134734
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten