research-article

Going viral: flash crowds in an open CDN

Authors:

Patrick Wendell,

Michael J. FreedmanAuthors Info & Claims

IMC '11: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference

Pages 549 - 558

https://doi.org/10.1145/2068816.2068867

Published: 02 November 2011 Publication History

Get Access

Abstract

Handling flash crowds poses a difficult task for web services. Content distribution networks (CDNs), hierarchical web caches, and peer-to-peer networks have all been proposed as mechanisms for mitigating the effects of these sudden spikes in traffic to under-provisioned origin sites. Other than a few anecdotal examples of isolated events to a single server, however, no large-scale analysis of flash-crowd behavior has been published to date.

In this paper, we characterize and quantify the behavior of thousands of flash crowds on CoralCDN, an open content distribution network running at several hundred POPs. Our analysis considers over four years of CDN traffic, comprising more than 33 billion HTTP requests. We draw conclusions in several areas, including (i) the potential benefits of cooperative vs. independent caching by CDN nodes, (ii) the efficacy of elastic redirection and resource provisioning, and (iii) the ecosystem of portals, aggregators, and social networks that drive traffic to third-party websites.

References

[1]

P. Barford and D. Plonka. Characteristics of network traffic flow anomalies. In Proc. Internet Measurement Workshop (IMW), Nov. 2001.

Digital Library

Google Scholar

[2]

A. Chankhunthod, P. Danzig, C. Neerdaels, M. Schwartz, and K. Worrell. A hierarchical Internet object cache. In Proc. USENIX Annual Technical Conference, Jan. 1996.

Digital Library

Google Scholar

[3]

A. Davis, J. Parikh, and W. E. Weihl. EdgeComputing: Extending enterprise applications to the edge of the Internet. In Proc. World Wide Web Conference (WWW), May 2004.

Digital Library

Google Scholar

[4]

M. J. Freedman. Experiences with CoralCDN: A five-year operational view. In Proc. Networked Systems Design and Implementation (NSDI), Apr. 2010.

Digital Library

Google Scholar

[5]

M. J. Freedman, E. Freudenthal, and D. Mazières. Democratizing content publication with Coral. In Proc. Networked Systems Design and Implementation (NSDI), Mar. 2004.

Digital Library

Google Scholar

[6]

J. Jung, B. Krishnamurthy, and M. Rabinovich. Flash crowds and Denial of Service attacks: Characterization and implications for CDNs and web sites. In Proc. World Wide Web Conference (WWW), May 2002.

Digital Library

Google Scholar

[7]

B. Maggs. Personal communication, 2009.

Google Scholar

[8]

R. Mahajan, S. M. Bellovin, S. Floyd, J. Ioannidis, V. Paxson, and S. Shenker. Controlling high bandwidth aggregates in the network. Computer Communications Review, 32 (3), July 2002.

Digital Library

Google Scholar

[9]

E. Nygren, R. K. Sitaraman, and J. Sun. The Akamai network: A platform for high-performance Internet applications. SIGOPS Operating Systems Review, 44, Aug. 2010.

Digital Library

Google Scholar

[10]

V. N. Padmanabhan and K. Sripanidkulchai. The case for cooperative networking. In Proc. Intl. Workshop on Peer-to-Peer Systems (IPTPS), Mar. 2002.

Digital Library

Google Scholar

[11]

S. Scellato, C. Mascolo, M. Musolesi, and J. Crowcroft. Track globally, deliver locally: Improving content delivery networks by tracking geographic social cascades. In Proc. World Wide Web Conference (WWW), Mar. 2011.

Digital Library

Google Scholar

[12]

S. Shakkottai and R. Johari. Demand-aware content distribution on the Internet. IEEE ACM Trans. Networking, 18 (2), Apr. 2010.

Digital Library

Google Scholar

[13]

T. Stading, P. Maniatis, and M. Baker. Peer-to-peer caching schemes to address flash crowds. In Proc. Intl. Workshop on Peer-to-Peer Systems (IPTPS), Mar. 2002.

Digital Library

Google Scholar

[14]

L. Wang, V. Pai, and L. Peterson. The effectiveness of request redirection on CDN robustness. SIGOPS Operating Systems Review, 36, Dec. 2002.

Digital Library

Google Scholar

Cited By

View all

Gessert FWingerath WRitter NGessert FWingerath WRitter N(2024)Caching in Forschung und IndustrieSchnelles und skalierbares Cloud-Datenmanagement10.1007/978-3-031-54388-3_5(91-140)Online publication date: 3-May-2024
https://doi.org/10.1007/978-3-031-54388-3_5
Srivastava SAgrahari SSingh A(2023)Early Spam Detection Using Time-Based Cache in Graph databaseNew Generation Computing10.1007/s00354-023-00223-441:3(607-634)Online publication date: 13-Jun-2023
https://doi.org/10.1007/s00354-023-00223-4
Zhang CTan HLi GHan ZJiang SLi X(2022)Online File Caching in Latency-Sensitive Systems with Delayed Hits and BypassingIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796969(1059-1068)Online publication date: 2-May-2022
https://doi.org/10.1109/INFOCOM48880.2022.9796969
Show More Cited By

Index Terms

Going viral: flash crowds in an open CDN
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
    2. Metrics

Recommendations

Autonomic multi-server distribution in flash crowds alleviation network
EUC'07: Proceedings of the 2007 conference on Emerging direction in embedded and ubiquitous computing

The Flash crowds are rapid increase in access to contents of web sites, which makes the web sites inaccessible, leaving the clients with unsatisfied requests. The major shortcoming of flash crowds researches is that they do not assist vital resizing ...
Autonomic Multi-server Distribution in Flash Crowds Alleviation Network
Emerging Directions in Embedded and Ubiquitous Computing
Abstract
The Flash crowds are rapid increase in access to contents of web sites, which makes the web sites inaccessible, leaving the clients with unsatisfied requests. The major shortcoming of flash crowds researches is that they do not assist vital ...
Using mini-flash crowds to infer resource constraints in remote web servers
INM '07: Proceedings of the 2007 SIGCOMM workshop on Internet network management

Unexpected surges in request traffic (e.g., flash crowds) can exercise server-side resources such as access bandwidth, CPU, and disks in unanticipated ways. Administrators today do not have the requisite tools to fully understand the effect that flash ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IMC '11: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference

November 2011

612 pages

ISBN:9781450310130

DOI:10.1145/2068816

Program Chairs:
Patrick Thiran
EPFL, Switzerland
,
Walter Willinger
AT&T Labs-Research, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

USENIX Assoc: USENIX Assoc

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IMC '11

Sponsor:

IMC '11: Internet Measurement Conference

November 2 - 4, 2011

Berlin, Germany

Acceptance Rates

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Gessert FWingerath WRitter NGessert FWingerath WRitter N(2024)Caching in Forschung und IndustrieSchnelles und skalierbares Cloud-Datenmanagement10.1007/978-3-031-54388-3_5(91-140)Online publication date: 3-May-2024
https://doi.org/10.1007/978-3-031-54388-3_5
Srivastava SAgrahari SSingh A(2023)Early Spam Detection Using Time-Based Cache in Graph databaseNew Generation Computing10.1007/s00354-023-00223-441:3(607-634)Online publication date: 13-Jun-2023
https://doi.org/10.1007/s00354-023-00223-4
Zhang CTan HLi GHan ZJiang SLi X(2022)Online File Caching in Latency-Sensitive Systems with Delayed Hits and BypassingIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796969(1059-1068)Online publication date: 2-May-2022
https://doi.org/10.1109/INFOCOM48880.2022.9796969
Akther NDhar KIqbal SHuda MAsaduzzaman (2022) Interest forwarding strategy in Named Data Networks (NDN) using Thompson SamplingJournal of Network and Computer Applications10.1016/j.jnca.2022.103458205:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.jnca.2022.103458
Karaata MDabees AAlazemi F(2022)A reliable concurrent multicast algorithm for content distributionThe Journal of Supercomputing10.1007/s11227-021-04291-578:8(10542-10574)Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1007/s11227-021-04291-5
Li GZhang CNi HTan H(2022)Online File Caching on Multiple Caches in Latency-Sensitive SystemsComputational Data and Social Networks 10.1007/978-3-031-26303-3_26(292-304)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1007/978-3-031-26303-3_26
TADA HMURATA MAIDA M(2021)Mitigation of Flash Crowd in Web Services By Providing Feedback Information to UsersIEICE Transactions on Information and Systems10.1587/transinf.2020MPP0003E104.D:1(63-75)Online publication date: 1-Jan-2021
https://doi.org/10.1587/transinf.2020MPP0003
Yang JYue YRashmi K(2021)A Large-scale Analysis of Hundreds of In-memory Key-value Cache Clusters at TwitterACM Transactions on Storage10.1145/346852117:3(1-35)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3468521
McCarthy JChaudhry SKuppuudaiyar PLoomba RClarke S(2021)QoSA-ICN: An information-centric approach to QoS in vehicular environmentVehicular Communications10.1016/j.vehcom.2021.100351(100351)Online publication date: Mar-2021
https://doi.org/10.1016/j.vehcom.2021.100351
Salunke KRagavendran U(2021)Shield Techniques for Application Layer DDoS Attack in MANET: A Methodological ReviewWireless Personal Communications10.1007/s11277-021-08556-3Online publication date: 16-Jun-2021
https://doi.org/10.1007/s11277-021-08556-3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Autonomic multi-server distribution in flash crowds alleviation network

Autonomic Multi-server Distribution in Flash Crowds Alleviation Network

Using mini-flash crowds to infer resource constraints in remote web servers