research-article

Public Access

The impact of tool configuration spaces on the evaluation of configurable taint analysis for Android

Authors:

Austin Mordahl,

Shiyi WeiAuthors Info & Claims

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 466 - 477

https://doi.org/10.1145/3460319.3464823

Published: 11 July 2021 Publication History

Abstract

The most popular static taint analysis tools for Android allow users to change the underlying analysis algorithms through configuration options. However, the large configuration spaces make it difficult for developers and users alike to understand the full capabilities of these tools, and studies to-date have only focused on individual configurations. In this work, we present the first study that evaluates the configurations in Android taint analysis tools, focusing on the two most popular tools, FlowDroid and DroidSafe. First, we perform a manual code investigation to better understand how configurations are implemented in both tools. We formalize the expected effects of configuration option settings in terms of precision and soundness partial orders which we use to systematically test the configuration space. Second, we create a new dataset of 756 manually classified flows across 18 open-source real-world apps and conduct large-scale experiments on this dataset and micro-benchmarks. We observe that configurations make significant tradeoffs on the performance, precision, and soundness of both tools. The studies to-date would reach different conclusions on the tools' capabilities were they to consider configurations or use real-world datasets. In addition, we study the individual options through a statistical analysis and make actionable recommendations for users to tune the tools to their own ends. Finally, we use the partial orders to test the tool configuration spaces and detect 21 instances where options behaved in unexpected and incorrect ways, demonstrating the need for rigorous testing of configuration spaces.

References

[1]

2019. FlowDroid. https://github.com/secure-software-engineering/FlowDroid 72734bd629dfae2aacaf6e6973abfe73d035c979.

[2]

2020. APKMirror. §MALL. https://www.apkmirror.com Accessed 2020-02-10.

[3]

2021. Apktool. §MALL. https://ibotpeaches.github.io/Apktool/

[4]

2021. Automated Combinatorial Testing for Software (ACTS). §MALL. https://www.nist.gov/programs-projects/automated-combinatorial-testing-software-acts

[5]

2021. DroidBench 3.0. §MALL. https://github.com/FoelliX/ReproDroid

[6]

2021. Fortify Static Code Analyzer. §MALL. https://www.microfocus.com/en-us/solutions/application-security

[7]

2021. FossDroid. §MALL. https://fossdroid.com

[8]

2021. HCL AppScan on Cloud. §MALL. https://www.hcltechsw.com/wps/portal/products/appscan/home

[9]

Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. 2014. FlowDroid: Precise Context, Flow, Field, Object-Sensitive and Lifecycle-Aware Taint Analysis for Android Apps. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). Association for Computing Machinery, New York, NY, USA. 259–269. isbn:9781450327848 https://doi.org/10.1145/2594291.2594299

Digital Library

[10]

Hamid Bagheri, Alireza Sadeghi, Reyhaneh Jabbarvand, and Sam Malek. 2016. Practical, formal synthesis and automatic enforcement of security policies for android. In 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 514–525.

[11]

Amiangshu Bosu, Fang Liu, Danfeng Yao, and Gang Wang. 2017. Collusive data leak and more: Large-scale threat analysis of inter-app communications. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. 71–85.

Digital Library

[12]

Dan Boxler and Kristen R Walcott. 2018. Static Taint Analysis Tools to Detect Information Flows. In Proceedings of the International Conference on Software Engineering Research and Practice (SERP). 46–52.

[13]

Stefano Calzavara, Ilya Grishchenko, and Matteo Maffei. 2016. HornDroid: Practical and sound static analysis of Android applications by SMT solving. In 2016 IEEE European Symposium on Security and Privacy (EuroS&P). 47–62.

[14]

Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, and Charles J. Colbourn. 2003. Constructing Test Suites for Interaction Testing. In Proceedings of the 25th International Conference on Software Engineering (ICSE ’03). IEEE Computer Society, USA. 38–48. isbn:076951877X

[15]

Xingmin Cui, Jingxuan Wang, Lucas CK Hui, Zhongwei Xie, Tian Zeng, and Siu-Ming Yiu. 2015. Wechecker: efficient and precise detection of privilege escalation vulnerabilities in android apps. In Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks. 1–12.

Digital Library

[16]

Yu Feng, Saswat Anand, Isil Dillig, and Alex Aiken. 2014. Apposcopy: Semantics-based detection of android malware through static analysis. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 576–587.

Digital Library

[17]

Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2010. Regularization Paths for Generalized Linear Models via Coordinate Descent. Journal of Statistical Software, 33, 1 (2010), 1–22. http://www.jstatsoft.org/v33/i01/

[18]

Michael I Gordon, Deokhwan Kim, Jeff H Perkins, Limei Gilham, Nguyen Nguyen, and Martin C Rinard. 2015. Information flow analysis of android applications in droidsafe. In NDSS. 15, 110.

[19]

Jianmei Guo, Krzysztof Czarnecki, Sven Apel, Norbert Siegmund, and Andrzej Wąsowski. 2013. Variability-aware performance prediction: A statistical learning approach. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). 301–311.

Digital Library

[20]

Jianmei Guo, Dingyu Yang, Norbert Siegmund, Sven Apel, Atrisha Sarkar, Pavel Valov, Krzysztof Czarnecki, Andrzej Wasowski, and Huiqun Yu. 2018. Data-efficient performance learning for configurable systems. Empirical Software Engineering, 23, 3 (2018), 1826–1867.

Digital Library

[21]

H. Ha and H. Zhang. 2019. DeepPerf: Performance Prediction for Configurable Software with Deep Sparse Neural Network. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 1095–1106. https://doi.org/10.1109/ICSE.2019.00113

Digital Library

[22]

Wei Huang, Yao Dong, Ana Milanova, and Julian Dolby. 2015. Scalable and precise taint analysis for android. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. 106–117.

Digital Library

[23]

Dave King, Boniface Hicks, Michael Hicks, and Trent Jaeger. 2008. Implicit Flows: Can’t Live with ’Em, Can’t Live without ’Em. In Information Systems Security, 4th International Conference, ICISS 2008, Hyderabad, India, December 16-20, 2008. Proceedings, R. Sekar and Arun K. Pujari (Eds.) (Lecture Notes in Computer Science, Vol. 5352). Springer, 56–70. https://doi.org/10.1007/978-3-540-89862-7_4

Digital Library

[24]

William Klieber, Lori Flynn, Amar Bhosale, Limin Jia, and Lujo Bauer. 2014. Android taint flow analysis for app sets. In Proceedings of the 3rd ACM SIGPLAN International Workshop on the State of the Art in Java Program Analysis. 1–6.

Digital Library

[25]

Ondřej Lhoták. 2006. Program analysis using binary decision diagrams. 68.

[26]

Ondřej Lhoták and Laurie Hendren. 2003. Scaling Java Points-to Analysis Using SPARK. In Proceedings of the 12th International Conference on Compiler Construction (CC’03). Springer-Verlag, Berlin, Heidelberg. 153–169. isbn:3540009043

Digital Library

[27]

Ondřej Lhoták and Laurie Hendren. 2008. Evaluating the Benefits of Context-Sensitive Points-to Analysis Using a BDD-Based Implementation. ACM Trans. Softw. Eng. Methodol., 18, 1 (2008), Article 3, Oct., 53 pages. issn:1049-331X https://doi.org/10.1145/1391984.1391987

Digital Library

[28]

Li Li, Alexandre Bartel, Tegawendé F Bissyandé, Jacques Klein, Yves Le Traon, Steven Arzt, Siegfried Rasthofer, Eric Bodden, Damien Octeau, and Patrick McDaniel. 2015. Iccta: Detecting inter-component privacy leaks in android apps. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. 1, 280–291.

[29]

Li Li, Tegawendé F. Bissyandé, Damien Octeau, and Jacques Klein. 2016. DroidRA: Taming Reflection to Support Whole-Program Analysis of Android Apps. ISSTA 2016. Association for Computing Machinery, New York, NY, USA. 318–329. isbn:9781450343909 https://doi.org/10.1145/2931037.2931044

Digital Library

[30]

L. Luo, E. Bodden, and J. Späth. 2019. A Qualitative Analysis of Android Taint-Analysis Results. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 102–114.

[31]

Ana Milanova, Atanas Rountev, and Barbara G. Ryder. 2005. Parameterized Object Sensitivity for Points-to Analysis for Java. ACM Trans. Softw. Eng. Methodol., 14, 1 (2005), Jan., 1–41. issn:1049-331X https://doi.org/10.1145/1044834.1044835

Digital Library

[32]

Austin Mordahl and Shiyi Wei. 2021. The Impact of Tool Configuration Spaces on the Evaluation of Configurable Taint Analysis for Android. https://doi.org/10.5281/zenodo.4729325

[33]

Changhai Nie and Hareton Leung. 2011. A Survey of Combinatorial Testing. ACM Comput. Surv., 43, 2 (2011), Article 11, Feb., 29 pages. issn:0360-0300 https://doi.org/10.1145/1883612.1883618

Digital Library

[34]

Felix Pauck, Eric Bodden, and Heike Wehrheim. 2018. Do android taint analysis tools keep their promises? In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 331–341.

Digital Library

[35]

Lina Qiu, Yingying Wang, and Julia Rubin. 2018. Analyzing the analyzers: Flowdroid/iccta, amandroid, and droidsafe. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 176–186.

Digital Library

[36]

Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise Interprocedural Dataflow Analysis via Graph Reachability. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’95). Association for Computing Machinery, New York, NY, USA. 49–61. isbn:0897916921 https://doi.org/10.1145/199448.199462

Digital Library

[37]

Norbert Siegmund, Alexander Grebhahn, Sven Apel, and Christian Kästner. 2015. Performance-influence models for highly configurable systems. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. 284–294.

Digital Library

[38]

Norbert Siegmund, Marko Rosenmüller, Martin Kuhlemann, Christian Kästner, Sven Apel, and Gunter Saake. 2012. SPL Conqueror: Toward optimization of non-functional properties in software product lines. Software Quality Journal, 20, 3 (2012), 487–517.

Digital Library

[39]

Yannis Smaragdakis, Martin Bravenboer, and Ondřej Lhoták. 2011. Pick Your Contexts Well: Understanding Object-Sensitivity. SIGPLAN Not., 46, 1 (2011), Jan., 17–30. issn:0362-1340 https://doi.org/10.1145/1925844.1926390

Digital Library

[40]

Ole Tange. 2020. GNU Parallel 20200522 (’Kraftwerk’). https://doi.org/10.5281/zenodo.3841377 GNU Parallel is a general parallelizer to run multiple serial command line programs in parallel without changing them.

[41]

Thomas Thüm, Sven Apel, Christian Kästner, Ina Schaefer, and Gunter Saake. 2014. A Classification and Survey of Analysis Strategies for Software Product Lines. ACM Comput. Surv., 47, 1 (2014), Article 6, June, 45 pages. issn:0360-0300 https://doi.org/10.1145/2580950

Digital Library

[42]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58, 1 (1996), 267–288.

[43]

Fengguo Wei, Sankardas Roy, and Xinming Ou. 2014. Amandroid: A precise and general inter-component data flow analysis framework for security vetting of android apps. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. 1329–1341.

Digital Library

[44]

Shiyi Wei, Piotr Mardziel, Andrew Ruef, Jeffrey S. Foster, and Michael Hicks. 2018. Evaluating Design Tradeoffs in Numeric Static Analysis for Java. In Programming Languages and Systems - 27th European Symposium on Programming, ESOP 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2018, Thessaloniki, Greece, April 14-20, 2018, Proceedings. 653–682. https://doi.org/10.1007/978-3-319-89884-1_23

[45]

Shengqian Yang, Dacong Yan, Haowei Wu, Yan Wang, and Atanas Rountev. 2015. Static Control-Flow Analysis of User-Driven Callbacks in Android Applications. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE ’15). IEEE Press, 89–99. isbn:9781479919345

[46]

Y. Zhang, J. Guo, E. Blais, and K. Czarnecki. 2015. Performance Prediction of Configurable Software Systems by Fourier Learning (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 365–373. https://doi.org/10.1109/ASE.2015.15

Digital Library

Cited By

Zhang ZKlees GWang EHicks MWei S(2023)Fuzzing Configurations of Program OptionsACM Transactions on Software Engineering and Methodology10.1145/358059732:2(1-21)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3580597
Luo LPiskachev GKrishnamurthy RDolby JBodden ESchäf M(2023)Model Generation For Java Frameworks2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00024(165-175)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00024
Inayoshi HKakei SSaito S(2023)Execution Recording and Reconstruction for Detecting Information Flows in Android AppsIEEE Access10.1109/ACCESS.2023.324072411(10730-10750)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3240724
Show More Cited By

Index Terms

The impact of tool configuration spaces on the evaluation of configurable taint analysis for Android
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
2. Software and its engineering
  1. Software organization and properties
    1. Software functional properties
      1. Formal methods
        Automated static analysis

Recommendations

Scalable and precise taint analysis for Android
ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and Analysis

We propose a type-based taint analysis for Android. Concretely, we present DFlow, a context-sensitive information flow type system, and DroidInfer, the corresponding type inference analysis for detecting privacy leaks in Android apps. We present novel ...
Do Android taint analysis tools keep their promises?
ESEC/FSE 2018: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

In recent years, researchers have developed a number of tools to conduct taint analysis of Android applications. While all the respective papers aim at providing a thorough empirical evaluation, comparability is hindered by varying or unclear evaluation ...
ConfTainter: Static Taint Analysis For Configuration Options
ASE '23: Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering

The prevalence and severity of software configuration-induced issues have driven the design and development of a number of detection and diagnosis techniques. Many of these techniques need to perform static taint analysis on configuration-related ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2021

685 pages

ISBN:9781450384599

DOI:10.1145/3460319

General Chair:
Cristian Cadar
Imperial College London, UK
,
Program Chair:
Xiangyu Zhang
Purdue University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

ISSTA '21

Sponsor:

SIGSOFT

ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

July 11 - 17, 2021

Virtual, Denmark

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)13

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZKlees GWang EHicks MWei S(2023)Fuzzing Configurations of Program OptionsACM Transactions on Software Engineering and Methodology10.1145/358059732:2(1-21)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3580597
Luo LPiskachev GKrishnamurthy RDolby JBodden ESchäf M(2023)Model Generation For Java Frameworks2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00024(165-175)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00024
Inayoshi HKakei SSaito S(2023)Execution Recording and Reconstruction for Detecting Information Flows in Android AppsIEEE Access10.1109/ACCESS.2023.324072411(10730-10750)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3240724
Schott SPauck F(2022)Benchmark Fuzzing for Android Taint Analyses2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM55253.2022.00007(12-23)Online publication date: Oct-2022
https://doi.org/10.1109/SCAM55253.2022.00007

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents