research-article

A programmable architecture for scalable and real-time network traffic measurements

Authors:

Chen-Nee Chuah,

Soheil GhiasiAuthors Info & Claims

ANCS '08: Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

Pages 109 - 118

https://doi.org/10.1145/1477942.1477958

Published: 06 November 2008 Publication History

Abstract

Accurate and real-time traffic measurement is becoming increasingly critical for large variety of applications including accounting, bandwidth provisioning and security analysis. Existing network measurement techniques, however, have major difficulty dealing with large number of flows in today's high-speed networks and offer limited scalability with increasing link speeds. Consequently, the current state of the art solutions have to resort to conservative sampling of the traffic stream and/or accounting for only a few frequent flows that often fail to provide accurate estimates of traffic features.

In this paper, we present a novel hardware-software co-designed solution that is programmable and adaptable to runtime situations offering high-throughputs that can easily match current link-speeds. The key to our design is orthogonalization of memory lookups from traffic measurements through our query-driven measurement scheme. We have prototyped our approach on a Xilinx platform using Microblaze soft-core processors integrated with Virtex-II Pro FPGA fabric. We demonstrate the scalability of our architecture and also compare it with a recent offline (non real-time) sampling-based software alternative. The comparison shows that our architecture performs orders better in terms of speed and throughput even while being used as an offline solution.

References

[1]

Cisco NetFlow. http://www.cisco.com/warp/public/732/Tech/netflow.

[2]

Sampled NetFlow. http://www.cisco.com/en/US/docs/ios/12\_0s/feature/guide/12s\_sanf.html.

[3]

CAIDA: Cooperative Association for Internet Data Analysis. http://www.caida.org/home/.

[4]

B. Amrutur and M. Horowtiz. Speed and power scaling of srams. In IEEE J. Solid-State Circuits, volume 35(2), 2000.

[5]

N. Brownlee, C. Mills, and G. Ruth. Traffic Flow Measurement: Architecture. RFC 2722, 1999. http://www.ietf.org/rfc/rfc2722.txt.

Digital Library

[6]

K. Cho, R. Kaizaki, and A. Kato. Aguri: An Aggregation-based Traffic Profiler. In Proc. Quality of Future Internet Services, 2001.

Digital Library

[7]

N. Duffield, C. Lund, and M. Thorup. Estimating Flow Distributions from Sampled Flow Statistics. In SIGCOMM, 2003.

Digital Library

[8]

N. G. Duffield, C. Lund, and M. Thorup. Flow Sampling Under Hard Resource Constraints. In SIGMETRICS, 2004.

Digital Library

[9]

C. Estan, K. Keys, D. Moore, and G. Varghese. Building a Better Netflow. In SIGCOMM, 2004.

Digital Library

[10]

C. Estan, S. Savage, and G. Varghese. Automatically inferring patterns of resource consumption in network traffic. In SIGCOMM, 2003.

Digital Library

[11]

C. Estan and G. Varghese. New Directions in Traffic Measurement and Accounting: Focusing on the elephants, ignoring the mice. ACM Transactions on Computer Systems, 21(3):270--313, 2003.

Digital Library

[12]

C. Estan, G. Varghese, and M. Fisk. Bitmap Algorithms for Counting Active Flows on High Speed Links. In Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, 2003.

Digital Library

[13]

W. Fang and L. Peterson. Inter-as: Traffic Patterns and their Implications. In Proceedings of IEEE GLOBECOM, 1999.

[14]

M. Kodialam, T. Lakshman, and S. Mohanty. Runs bAsed Traffic Estimator (RATE): A simple, memory efficient scheme for per-flow rate estimation. In INFOCOM, 2004.

[15]

A. Kumar and J. Xu. Sketch Guided Sampling -- Using On-Line Estimates of Flow Size for Adaptive Data Collection. In INFOCOM, 2006.

[16]

J. Mudigonda, H. M. Vin, and S. W. Keckler. Reconciling performance and programmability in networking systems. In SIGCOMM, pages 73--84, 2007.

Digital Library

[17]

V. Paxson, K. Asanovic, S. Dharmapurikar, J. Lockwood, R. Pang, R. Sommer, and N. Weaver. Rethinking hardware support for network analysis and intrusion prevention. In USENIX Hot Security, August-2006.

Digital Library

[18]

P. Phaal, S. Panchen, and N. McKee. InMon corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks, 2001. RFC 3176.

Digital Library

[19]

D. Plonka. FlowScan: A Network Traffic Flow Reporting and Visualization Tool. In USENIX LISA, pages 305--317, 2000.

Digital Library

[20]

G. Varghese. Network Algorithmics. Morgan Kaufmann, 2005.

[21]

T. Wolf, R. Ramaswamy, S. Bunga, and N. Yang. An architecture for distributed real-time passive network measurement. In MASCOTS, pages 335--344, 2006.

Digital Library

[22]

L. Yuan, C.-N. Chuah, and P. Mohapatra. ProgME: towards programmable network measurement. In SIGCOMM, pages 97--108, 2007.

Digital Library

[23]

Y. Zhang, S. Singh, S. Sen, N. Duffield, and C. Lund. Online identification of hierarchical heavy hitters: algorithms, evaluation, and applications. In IMC, 2004.

Digital Library

Cited By

Nimkar SMankar C(2017)Review on network traffic measurements and analysis2017 International Conference on Inventive Systems and Control (ICISC)10.1109/ICISC.2017.8068643(1-6)Online publication date: Jan-2017
https://doi.org/10.1109/ICISC.2017.8068643
Puš VBenáček PBetz VConstantinides G(2014)Application specific processor with high level synthesized instructions (abstract only)Proceedings of the 2014 ACM/SIGDA international symposium on Field-programmable gate arrays10.1145/2554688.2554754(246-246)Online publication date: 26-Feb-2014
https://dl.acm.org/doi/10.1145/2554688.2554754
Khan FHosein NGhiasi SChuah CSharma P(2014)Streaming solutions for fine-grained network traffic measurements and analysisIEEE/ACM Transactions on Networking (TON)10.1109/TNET.2013.226322822:2(377-390)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1109/TNET.2013.2263228
Show More Cited By

Index Terms

A programmable architecture for scalable and real-time network traffic measurements

Recommendations

A Scalable and Programmable Modular Traffic Manager Architecture

A key issue in the design of next-generation Internet routers and switches will be provision of Traffic Manager (TM) functionality in the datapaths of their high-speed switching fabrics. A new architecture that allows dynamic deployment of different TM ...
Network-on-Chip Programmable Platform in Versal^TM ACAP Architecture
FPGA '19: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

This paper outlines the Network-on-Chip (NoC) on Xilinx's next generation Versal-architecture. It is a hardened NoC that is present in Xilinx's next-generation 7nm architecture devices. These devices include many other new hardened features that make up ...
A real-time embedded architecture for SIFT

SIFT has shown a great success in various computer vision applications. However, its large computational complexity has been a challenge to most embedded implementations. This paper presents a low-cost embedded system based on a new architecture that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '08: Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems

November 2008

191 pages

ISBN:9781605583464

DOI:10.1145/1477942

General Chair:
Mark Franklin
Washington University in St. Louis
,
Program Chairs:
D. K. Panda
Ohio State University
,
Dimitri Stiliadis
Bell Laboratory

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

ANCS '08

Sponsor:

ANCS '08: Symposium on Architecture for Networking and Communications Systems

November 6 - 7, 2008

California, San Jose

Acceptance Rates

ANCS '08 Paper Acceptance Rate 17 of 67 submissions, 25%;

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
333
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nimkar SMankar C(2017)Review on network traffic measurements and analysis2017 International Conference on Inventive Systems and Control (ICISC)10.1109/ICISC.2017.8068643(1-6)Online publication date: Jan-2017
https://doi.org/10.1109/ICISC.2017.8068643
Puš VBenáček PBetz VConstantinides G(2014)Application specific processor with high level synthesized instructions (abstract only)Proceedings of the 2014 ACM/SIGDA international symposium on Field-programmable gate arrays10.1145/2554688.2554754(246-246)Online publication date: 26-Feb-2014
https://dl.acm.org/doi/10.1145/2554688.2554754
Khan FHosein NGhiasi SChuah CSharma P(2014)Streaming solutions for fine-grained network traffic measurements and analysisIEEE/ACM Transactions on Networking (TON)10.1109/TNET.2013.226322822:2(377-390)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1109/TNET.2013.2263228
Khan FGhiasi SChuah C(2014)A Dynamically Reconfigurable System for Closed-Loop Measurements of Network TrafficIEEE Transactions on Computers10.1109/TC.2012.22863:2(263-275)Online publication date: 1-Feb-2014
https://dl.acm.org/doi/10.1109/TC.2012.228
Yuan LChuah CMohapatra P(2011)ProgMEIEEE/ACM Transactions on Networking (TON)10.1109/TNET.2010.206698719:1(115-128)Online publication date: 1-Feb-2011
https://dl.acm.org/doi/10.1109/TNET.2010.2066987
Khan FHosein NChuah CGhiasi S(2011)Streaming Solutions for Fine-Grained Network Traffic Measurements and AnalysisProceedings of the 2011 ACM/IEEE Seventh Symposium on Architectures for Networking and Communications Systems10.1109/ANCS.2011.45(227-238)Online publication date: 3-Oct-2011
https://dl.acm.org/doi/10.1109/ANCS.2011.45
Khan FGokhale MChuah C(2010)FPGA Based Network Traffic Analysis Using Traffic Dispersion PatternsProceedings of the 2010 International Conference on Field Programmable Logic and Applications10.1109/FPL.2010.103(519-524)Online publication date: 31-Aug-2010
https://dl.acm.org/doi/10.1109/FPL.2010.103
Khan FHosein NVernon SGhiasi S(2010)BURAQProceedings of the 2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines10.1109/FCCM.2010.36(185-192)Online publication date: 2-May-2010
https://dl.acm.org/doi/10.1109/FCCM.2010.36
Sommers JBarford PCrovella MCrowley PMaltz DVan der Merwe K(2009)Router primitives for programmable active measurementProceedings of the 2nd ACM SIGCOMM workshop on Programmable routers for extensible services of tomorrow10.1145/1592631.1592635(13-18)Online publication date: 21-Aug-2009
https://dl.acm.org/doi/10.1145/1592631.1592635
Hu CLiu B(2009)Self-Tuning the Parameter of Adaptive Non-linear Sampling Method for Flow StatisticsProceedings of the 2009 International Conference on Computational Science and Engineering - Volume 0110.1109/CSE.2009.19(16-21)Online publication date: 29-Aug-2009
https://dl.acm.org/doi/10.1109/CSE.2009.19

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents