Abstract
Current intrusion detection approaches based on control flow integrity (CFI) can detect the majority of control flow hijacking attacks, but few of them take into account the impact of environment on CFI, so there may exist false alarms. In this paper, we have investigated systematically the impact of environment on branch transfer from time, space and mechanisms of Linux operating system. Moreover, we have presented finite state automata (FSA) to describe difference patterns caused by these environmental factors, and have exploited FSA-Stack model to detect these impacts. Finally, for some common applications (gzip, grep, tesseract, bzip2 etc.), we have leveraged a dynamic binary instrumentation tool Pin to record direct and indirect branch transfers produced by them and the shared libraries they depend on. The experimental results demonstrate that impact of environment on branch transfer exists universally and normally among usual applications, and the difference patterns of impacts can be beneficial to understand and mitigate the false alarms of CFI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Forrest, S., Hofmeyr, S., Somayaji, A.: The evolution of system-call monitoring. In: Proceeding of the 24th Annual Computer Security Applications Conference, California, USA, pp. 418–430 (2008)
Wee, K., Moon, B.: Automatic generation of finite state automata for detecting intrusions using system call sequences. In: Gorodetsky, V., Popyack, L., Skormin, V. (eds.) MMM-ACNS 2003. LNCS, vol. 2776, pp. 206–216. Springer, Heidelberg (2003). doi:10.1007/978-3-540-45215-7_17
Kruegel, C., Mutz, D., Valeur, F., Vigna, G.: On the detection of anomalous system call arguments. In: Snekkenes, E., Gollmann, D. (eds.) ESORICS 2003. LNCS, vol. 2808, pp. 326–343. Springer, Heidelberg (2003). doi:10.1007/978-3-540-39650-5_19
Maggi, F., Matteucci, M., Zanero, S.: Detecting intrusions through system call sequence and argument analysis. IEEE Trans. Dependable Secure Comput. 7(4), 381–395 (2010)
Shacham, H.: The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In: Proceeding of the 14th ACM Conference on Computer and Communications Security, Alexandria, USA, pp. 552–561. ACM (2007)
Bletsch, T., Jiang, X., Freeh, V.W., Liang, Z.: Jump-oriented programming: a new class of code-reuse attack. In: Proceeding of the 6th ACM Symposium on Information, Computer and Communications Security, Hong Kong, China, pp. 30–40. ACM (2011)
Abadi, M., Budiu, M., Erlingsson, U., Ligatti, J.: Control-flow integrity. In: Proceeding of the 12th ACM Conference on Computer and Communications Security, Alexandria, USA, pp. 340–353. ACM (2005)
Abadi, M., Budiu, M., Erlingsson, U., Ligatti, J.: Control-flow integrity principles, implementations, and applications. J. ACM Trans. Inf. Syst. Secur. 13(1), 1–41 (2009)
Xia, Y., Liu, Y., Chen, H., Zang, B.: CFIMon: detecting violation of control flow integrity using performance counters. In: IEEE/IFIP International Conference on Dependable Systems and Networks, Boston, USA, pp. 1–12. IEEE/IFTP (2012)
Yuan, P., Zeng, Q., Ding, X.: Hardware-assisted fine-grained code-reuse attack detection. In: Bos, H., Monrose, F., Blanc, G. (eds.) RAID 2015. LNCS, vol. 9404, pp. 66–85. Springer, Cham (2015). doi:10.1007/978-3-319-26362-5_4
Cheng, Y., Zhou, Z., Yu, M., Ding, X., Deng, R.H.: ROPecker: a generic and practical approach for defending against ROP attacks. In: Proceeding of Symposium on Network and Distributed System Security, San Diego, USA. ISOC (2014)
Intel Manual. Intel 64 and IA-32 architecture software developers manual, vol. 3
Zhong, S.: Certified software. Commun. ACM 53(12), 56–66 (2010)
Murali A, Rao M. A survey on intrusion detection approaches. In: Proceeding of the First International Conference on Information and Communication Technologies, Karachi, Pakistan, pp. 233–240. IEEE (2005)
Garcia-Teodoro, P., Diaz-Verdejo, J., Maci-Fernndez, G., Vazque, Z.: Anomaly-based network intrusion detection: techniques, systems and challenges. Comput. Secur. 28(1), 18–28 (2009). Elsevier
Mytkowicz, T., Diwan, A., Hauswirth, M., Sweenry, P.F.: Producing wrong data without doing anything obviously wrong! ACM Sigplan Not. 44(3), 265–276 (2009)
Yeung, D.Y., Ding, Y.: Host-based intrusion detection using dynamic and static behavioral models. Pattern Recogn. 36(1), 229–243 (2003). Elsevier
Li, P., Park, H., Gao, D., Fu, J.: Bridging the gap between data-flow and control-flow analysis for anomaly detection. In: Proceeding of the 24th Annual Computer Security Application Conference, California, USA. IEEE (2008)
Giffin, J.T., Dagon, D., Jha, S., Lee, W., Miller, B.P.: Environment-sensitive intrusion detection. In: Proceeding of Recent Advances in Intrusion Detection, Hamburg, Germany, pp. 185–206. Springer (2006)
Zhang, C., Wei, T., Chen, Z,, Duan, L.: Practical control flow integrity and randomization for binary executables. In: Proceeding of IEEE Symposium on Security and Privacy, San Francisco, USA, pp. 559–573. IEEE (2013)
Zhang, M., Sekar, R.: Control flow integrity for COTS binaries. In: Proceeding of the 22nd USENIX Security Symposium, pp. 337–352. IEEE, Washington, D.C. (2013)
Wang, M., Yin, H., Bhaskar, A.V., Continent, B.C., et al.: Finer-grained control flow integrity for stripped binaries. In: Proceedings of the 31st Annual Computer Security Applications Conference, Los Angeles, USA, pp. 331–340. IEEE (2015)
Lin, Y., Tang, X., Gao, D., Fu, J.: Control flow integrity enforcement with dynamic code optimization. In: Bishop, M., Nascimento, A.C.A. (eds.) ISC 2016. LNCS, vol. 9866, pp. 366–385. Springer, Cham (2016). doi:10.1007/978-3-319-45871-7_22
Pappas, V.: kBouncer: efficient and transparent ROP mitigation. Technical report, Columbia University (2012)
Zhao, T., Tang, Y., Xu, W., Fu, G., Qi, S., Jia, X., et al.: Exactly reproducible program execution and its application in computer architecture simulation. Chin. J. Comput. 34(11), 2073–2083 (2011)
Shacham, H., Page, M., Pfaff, B.: On the effectiveness of address-space randomization. In: Proceeding of the 11th ACM Conference on Computer and Communications Security 2004, pp. 298–307 (2004)
Luk, C.K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., et al.: Pin: building customized program analysis tools with dynamic instrumentation. ACM Sigplan Not. 40(6), 190–200 (2005)
Fu, J., Lin, Y., Zhang, X., Li, P.: Computation integrity measurement based on branch transfer. In: Proceeding of the 13th International Conference on Trust, Security and Privacy in Computing and Communications, Beijing, China, pp. 590–597. IEEE (2014)
Acknowledgements
Supported by the National Natural Science Foundation of China (61373168), and Doctoral Fund of Ministry of Education of China (20120141110002).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Fu, J., Lin, Y., Zhang, X. (2017). Impact of Environment on Branch Transfer of Software. In: Deng, R., Weng, J., Ren, K., Yegneswaran, V. (eds) Security and Privacy in Communication Networks. SecureComm 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 198. Springer, Cham. https://doi.org/10.1007/978-3-319-59608-2_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-59608-2_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59607-5
Online ISBN: 978-3-319-59608-2
eBook Packages: Computer ScienceComputer Science (R0)