research-article

Scanning the Internet for Liveness

Authors:

Philipp Richter,

Srikanth Sundaresan,

Zakir Durumeric,

Steven J. Murdoch,

Richard Mortier,

Vern PaxsonAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 48, Issue 2

Pages 2 - 9

https://doi.org/10.1145/3213232.3213234

Published: 01 May 2018 Publication History

Abstract

Internet-wide scanning depends on a notion of liveness: does a target IP address respond to a probe packet? However, the interpretation of such responses, or lack of them, is nuanced and depends on multiple factors, including: how we probed, how different protocols in the network stack interact, the presence of filtering policies near the target, and temporal churn in IP responsiveness. Although often neglected, these factors can significantly affect the results of active measurement studies. We develop a taxonomy of liveness which we employ to develop a method to perform concurrent IPv4 scans using ICMP, five TCP-based, and two UDP-based protocols, comprehensively capturing all responses to our probes, including negative and cross-layer responses. Leveraging our methodology, we present a systematic analysis of liveness and how it manifests in active scanning campaigns, yielding practical insights and methodological improvements for the design and the execution of active Internet measurement studies.

References

[1]

Lance Alt, Robert Beverly, and Alberto Dainotti. Uncovering Network Tarpits with Degreaser. In Proceedings of the 30th Annual Computer Security Applications Conference, ACSAC '14, New Orleans, Louisiana, USA, 2014.

Digital Library

[2]

Genevieve Bartlett, John Heidemann, and Christos Papadopoulos. Understanding Passive and Active Service Discovery. In Proceedings of ACM IMC 2007, San Diego, California, USA, 2007.

Digital Library

[3]

John Blackford and Mike Digdon. CPE WAN Management Protocol. Technical Report TR-069, Broadband Forum, November 2013. Issue 1 Amendment 5. CWMP v1.4.

[4]

Randy Bush, Olaf Maennel, Matthew Roughan, and Steve Uhlig. Internet Optometry: Assessing the Broken Glasses in Internet Reachability. In Proceedings of ACM IMC 2009, Chicago, Illinois, USA, 2009.

Digital Library

[5]

Xue Cai and John Heidemann. Understanding Block-level Address Usage in the Visible Internet. In Proceedings of ACM SIGCOMM 2010, New Delhi, India, 2010.

Digital Library

[6]

k. claffy, Y. Hyun, K. Keys, M. Fomenkov, and D. Krioukov. Internet Mapping: from Art to Science. In IEEE DHS Cybersecurity Applications and Technologies Conference for Homeland Security (CATCH), pages 205–211, Waltham, MA, Mar 2009.

Digital Library

[7]

TR-069 CPE WAN Management Protocol. https://www.broadband-forum.org/technical/download/TR-069_Amendment-5.pdf.

[8]

A. Dainotti, K. Benson, A. King, k. claffy, M. Kallitsis, E. Glatz, and X. Dimitropoulos. Estimating Internet address space usage through passive measurements. ACM CCR, 44(1):42–49, Jan 2014.

Digital Library

[9]

A. Dainotti, K. Benson, A. King, B. Huffaker, E. Glatz, X. Dimitropoulos, P. Richter, A. Finamore, and A. Snoeren. Lost in Space: Improving Inference of IPv4 Address Space Utilization. IEEE Journal on Selected Areas in Communications (JSAC), 34(6):1862–1876, Jun 2016.

[10]

Zakir Durumeric, David Adrian, Ariana Mirian, Michael Bailey, and J. Alex Halderman. A Search Engine Backed by Internet-Wide Scanning. In Proceedings of the 22nd ACM Conference on Computer and Communications Security, October 2015.

Digital Library

[11]

Zakir Durumeric, James Kasten, Michael Bailey, and J. Alex Halderman. Analysis of the HTTPS Certificate Ecosystem. In Proceedings of ACM IMC 2013, Barcelona, Spain, 2013. ACM.

Digital Library

[12]

Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. ZMap: Fast Internet-wide Scanning and Its Security Applications. In Proceedings of the 22Nd USENIX Conference on Security, SEC'13, pages 605–620, Berkeley, CA, USA, 2013. USENIX Association.

Digital Library

[13]

Xun Fan and John Heidemann. Selecting Representative IP Addresses for Internet Topology Studies. In Proceedings of ACM IMC 2010, Melbourne, Australia, 2010.

Digital Library

[14]

Ramesh Govindan and Hongsuda Tangmunarunkit. Heuristics for Internet map discovery. In Proceedings of INFOCOM 2000, Tel Aviv, Israel, 2000.

[15]

M. H. Gunes and K. Saracc. Analyzing router responsiveness to active measurement probes. In Proceedings of PAM 2009, 2009.

Digital Library

[16]

John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, and Joseph Bannister. Exploring Visible Internet Hosts through Census and Survey. Technical Report ISI-TR-2007-640, USC/Information Sciences Institute, May 2007.

[17]

Nadia Heninger, Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. Mining Your Ps and Qs: Detection of Widespread Weak Keys in Network Devices. In Proceedings of the 21st USENIX Conference on Security Symposium, Security'12, Berkeley, CA, USA, 2012.

Digital Library

[18]

B. Huffaker, M. Fomenkov, D. Moore, and k. claffy. Macroscopic analyses of the infrastructure: measurement and visualization of Internet connectivity and performance. In PAM 2001, Amsterdam, Netherlands, 2001.

[19]

Sheharbano Khattak, David Fifield, Sadia Afroz, Mobin Javed, Srikanth Sundaresan, Vern Paxson, Steven J. Murdoch, and Damon McCoy. Do You See What I See?: Differential Treatment of Anonymous Users. In Proceedings of NDSS 2016, San Diego, CA, United States, 2016.

[20]

Marc Kührer, Thomas Hupperich, Jonas Bushart, Christian Rossow, and Thorsten Holz. Going Wild: Large-Scale Classification of Open DNS Resolvers. In Proceedings of ACM IMC 2015, Tokyo, Japan, 2015.

Digital Library

[21]

Derek Leonard and Dmitri Loguinov. Demystifying Service Discovery: Implementing an Internet-wide Scanner. In Proceedings of ACM IMC 2010, Melbourne, Australia, 2010.

Digital Library

[22]

M. Luckie, Y. Hyun, and B. Huffaker. Traceroute Probe Method and Forward IP Path Inference. In Proceedings of ACM IMC 2008, Vouliagmeni, Greece, 2008.

Digital Library

[23]

Antonio Nappa, Zhaoyan Xu, Juan Caballero, and Guofei Gu. CyberProbe: Towards Internet-Scale Active Detection of Malicious Servers. In Proceedings of NDSS 2014, San Diego, CA, USA, 2014.

[24]

Ramakrishna Padmanabhan, Amogh Dhamdhere, Emile Aben, kc claffy, and Neil Spring. Reasons Dynamic Addresses Change. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.

Digital Library

[25]

Jeffrey Pang, James Hendricks, Aditya Akella, Roberto De Prisco, Bruce Maggs, and Srinivasan Seshan. Availability, Usage, and Deployment Characteristics of the Domain Name System. In Proceedings of ACM IMC 2004, Taormina, Sicily, Italy, 2004.

Digital Library

[26]

Jean-Jacques Pansiot and Dominique Grad. On Routes and Multicast Trees in the Internet. ACM CCR, 28(1):41–50, January 1998.

Digital Library

[27]

J. Postel. Internet Control Message Protocol. RFC 792, September 1981. https://tools.ietf.org/html/rfc792.

Digital Library

[28]

J. Postel. Transmission Control Protocol. RFC 793, September 1981. https://tools.ietf.org/html/rfc793.

[29]

N. Provos and P. Honeyman. ScanSSH - Scanning the Internet for SSH Servers. In 16th USENIX Systems Administration Conference (LISA), New York, NY, USA, 2001.

[30]

Lin Quan and John Heidemann. Detecting Internet Outages with Active Probing (extended). Technical Report ISI-TR-2011-672, USC/Information Sciences Institute, May 2010.

[31]

Lin Quan, John Heidemann, and Yuri Pradkin. When the Internet Sleeps: Correlating Diurnal Networks With External Factors (extended). Technical Report ISI-TR-2014-691b, USC/Information Sciences Institute, May 2014. (updated August 2014).

Digital Library

[32]

Philipp Richter, Georgios Smaragdakis, David Plonka, and Arthur Berger. Beyond Counting: New Perspectives on the Active IPv4 Address Space. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.

Digital Library

[33]

Philipp Richter, Florian Wohlfart, Narseo Vallina-Rodriguez, Mark Allman, Randy Bush, Anja Feldmann, Christian Kreibich, Nicholas Weaver, and Vern Paxson. A Multi-perspective Analysis of Carrier-Grade NAT Deployment. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.

Digital Library

[34]

Matthew Roughan, Walter Willinger, Olaf Maennel, Debbie Perouli, and Randy Bush. 10 Lessons from 10 Years of Measuring and Modeling the Internet's Autonomous Systems. IEEE Journal on Selected Areas in Communications, 29(9):1810–1821, 2011.

[35]

Yuval Shavitt and Eran Shir. DIMES: Let the Internet Measure Itself. ACM CCR, 35(5):71–74, October 2005.

Digital Library

[36]

Neil Spring, Ratul Mahajan, and David Wetherall. Measuring ISP Topologies with Rocketfuel. In Proceedings of ACM SIGCOMM 2002, New York, NY, USA, 2002.

Digital Library

[37]

P. Srisuresh, B. Ford, S. Sivakumar, and S. Guha. NAT Behavioral Requirements for ICMP. RFC 5508 (Best Current Practice), April 2009. Updated by RFC 7857.

[38]

Mark Thomas, Leigh Metcalf, Jonathan M. Spring, Paul Krystosek, and Katherine Prevost. SiLK: A tool suite for unsampled network flow analysis at scale. In IEEE BigData Congress, pages 184–191, Anchorage, Jul 2014.

Digital Library

[39]

Feng Wang, Zhuoqing Morley Mao, Jia Wang, Lixin Gao, and Randy Bush. A Measurement Study on the Impact of Routing Events on End-to-end Internet Path Performance. In Proceedings of ACM SIGCOMM 2006, Pisa, Italy, 2006.

Digital Library

[40]

ZMap. https://github.com/zmap/zmap/.

Cited By

Luo YLi CWang ZYang J(2024)IPREDS: Efficient Prediction System for Internet-wide Port and Service ScanningProceedings of the ACM on Networking10.1145/36494702:CoNEXT1(1-24)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3649470
Holzbauer FMaier MUllrich JVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Destination Reachable: What ICMPv6 Error Messages Reveal About Their SourcesProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688420(280-294)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688420
Griffioen HKoursiounis GSmaragdakis GDoerr CVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Have you SYN me? Characterizing Ten Years of Internet ScanningProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688409(149-164)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688409
Show More Cited By

Index Terms

Scanning the Internet for Liveness
1. Networks
  1. Network properties
    1. Network dynamics
  2. Network protocols

Recommendations

Target generation for internet-wide IPv6 scanning
IMC '17: Proceedings of the 2017 Internet Measurement Conference

Fast IPv4 scanning has enabled researchers to answer a wealth of new security and measurement questions. However, while increased network speeds and computational power have enabled comprehensive scans of the IPv4 address space, a brute-force approach ...
Towards Better Internet Citizenship: Reducing the Footprint of Internet-wide Scans by Topology Aware Prefix Selection
IMC '16: Proceedings of the 2016 Internet Measurement Conference

Internet service discovery is an emerging topic to study the deployment of protocols. Towards this end, our community periodically scans the entire advertised IPv4 address space. In this paper, we question this principle. Being good Internet citizens ...
Seeds of Scanning: Exploring the Effects of Datasets, Methods, and Metrics on IPv6 Internet Scanning
IMC '24: Proceedings of the 2024 ACM on Internet Measurement Conference

Large-scale Internet scanning is a vital research tool. While IPv4 can be exhaustively probed, the size of IPv6 precludes complete enumeration, limiting large-scale measurement. Target Generation Algorithms (TGAs)---algorithms which ingest lists of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 48, Issue 2

April 2018

51 pages

ISSN:0146-4833

DOI:10.1145/3213232

Issue’s Table of Contents

Copyright © 2018 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 2018

Published in SIGCOMM-CCR Volume 48, Issue 2

Check for updates

Badges

Artifacts Available

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
500
Total Downloads

Downloads (Last 12 months)73
Downloads (Last 6 weeks)12

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Luo YLi CWang ZYang J(2024)IPREDS: Efficient Prediction System for Internet-wide Port and Service ScanningProceedings of the ACM on Networking10.1145/36494702:CoNEXT1(1-24)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3649470
Holzbauer FMaier MUllrich JVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Destination Reachable: What ICMPv6 Error Messages Reveal About Their SourcesProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688420(280-294)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688420
Griffioen HKoursiounis GSmaragdakis GDoerr CVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Have you SYN me? Characterizing Ten Years of Internet ScanningProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688409(149-164)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688409
Everson DCheng L(2024)A Survey on Network Attack Surface MappingDigital Threats: Research and Practice10.1145/36400195:2(1-25)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3640019
Anghel RZhauniarovich YGañán C(2024)Who's Got My Back? Measuring the Adoption of an Internet-wide BGP RTBH ServiceProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/36390298:1(1-25)Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.1145/3639029
Fainchtein RSherr M(2024)You Can Find Me Here: A Study of the Early Adoption of GeofeedsPassive and Active Measurement10.1007/978-3-031-56252-5_11(228-245)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56252-5_11
Sethuraman MBischof ZDainotti A(2024)Towards Improving Outage Detection with Multiple Probing ProtocolsPassive and Active Measurement10.1007/978-3-031-56249-5_8(189-205)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56249-5_8
Sarabi AYin TLiu MMontpetit MLeivadeas AUhlig SJaved M(2023)An LLM-based Framework for Fingerprinting Internet-connected DevicesProceedings of the 2023 ACM on Internet Measurement Conference10.1145/3618257.3624845(478-484)Online publication date: 24-Oct-2023
https://dl.acm.org/doi/10.1145/3618257.3624845
Izhikevich LTran MKallitsis MFass ADurumeric ZMontpetit MLeivadeas AUhlig SJaved M(2023)Cloud Watching: Understanding Attacks Against Cloud-Hosted ServicesProceedings of the 2023 ACM on Internet Measurement Conference10.1145/3618257.3624818(313-327)Online publication date: 24-Oct-2023
https://dl.acm.org/doi/10.1145/3618257.3624818
Song GHe LZhao TLuo YWu YFan LLi CWang ZYang J(2023)Which Doors Are Open: Reinforcement Learning-based Internet-wide Port Scanning2023 IEEE/ACM 31st International Symposium on Quality of Service (IWQoS)10.1109/IWQoS57198.2023.10188692(1-10)Online publication date: 19-Jun-2023
https://doi.org/10.1109/IWQoS57198.2023.10188692
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents