Article

System design issues for internet middleware services: deductions from a large client trace

Authors:

Steven D. Gribble,

Eric A. BrewerAuthors Info & Claims

USITS'97: Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems

Page 19

Published: 08 December 1997 Publication History

Abstract

In this paper, we present the analysis of a large client-side web trace gathered from the Home IP service at the University of California at Berkeley. Specifically, we demonstrate the heterogeneity of web clients, the existence of a strong and very predictable diurnal cycle in the clients' web activity, the burstiness of clients' requests at small time scales (but not large time scales, implying a lack of self-similarity), the presence of locality of reference in the clients' requests that is a strong function of the client population size, and the high latency that services encounter when delivering data to clients, implying that services will need to maintain a very large number of simultaneously active requests. We then present system design issues for Internet middleware services that were drawn both from our trace analysis and our implementation experience of the TranSend transformation proxy.

References

[1]

{1} Allpen Software Home Page. http://www.allpen.com.

[2]

{2} Thomas E. Anderson, Henry M. Levy, Brian N. Bershad, and Edward D. Lazowska. The interaction of architecture and operating system design. In Proceedings of the Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, April 1991.

Digital Library

[3]

{3} Rob Barrett, Paul P. Maglio, and Daniel C. Kellem. How to personalize the web. In Proceedings of the 1997 Conference on Human Factors in Compuer Systems (CHI 1997), Atlanta, Georgia, USA, March 1997.

Digital Library

[4]

{4} Jan Beran, Robert Sherman, Murad S. Taqqu, and Walter Willinger. Long-range dependence in variable-bit-rate video trafic. IEEE Transactions on Communications, 43:1566-79, March 1995.

[5]

{5} Tim Berners-Lee, R. Fielding, and H. Frystyk. Hypertext transfer protocol - http/1.0. RFC 1945, May 1996.

Digital Library

[6]

{6} Timothy C. Bickmore and Bill N. Schilit. Digestor: Device-independent access to the world wide web. In Proceedings for the Sixth International World Wide Web Conference, 1997. available at http://www.fxpal.xerox.com/papers/bic97/.

[7]

{7} C. Brooks, M. S. Mazer, S. Meeks, and J. Miller. Application-specific proxy servers as http stream transducers. In Proceedings of the 4th International World Wide Web Conference, May 1996.

[8]

{8} A. Chankhunthod, P. B. Danzig, C. Neerdaels, M. F. Schwartz, and K. J. Worrell. A hierarchical internet object cache. In Proceedings of the 1996 Usenix Annual Technical Conference, January 1996.

Digital Library

[9]

{9} Mark E. Crovella and Azer Bestavros. Explaining world wide web trafic self-similarity. Technical Report TR-95-015, Computer Science Department, Boston University, Oct. 1995.

[10]

{10} Carlos R. Cunha, Azer Bestavros, and Mark E. Crovella. Characteristics of www client-traces. Technical Report BU-CS-95-010, Computer Science Department, Boston University, July 1995.

Digital Library

[11]

{11} P. B. Danzig, R. S. Hall, and M. F. Schwartz. A case for caching file objects inside internetworks. In Proceedings of SIGCOMM '93, September 1993.

Digital Library

[12]

{12} Fred Douglis, Anja Feldmann, Balachander Krishnamurthy, and Jeffrey Mogul. Rate of change and other metrics: a live study of the world wide web. In Proceedings of the 1997 USENIX Symposium on Internet Technolgoies and Systems (USITS), Monterey, CA, USA, December 1997.

[13]

{13} Armando Fox and Eric A. Brewer. Reducing WWW Latency and Bandwidth Requirements via Real-Time Distillation. In Proceedings of the 5th International World Wide Web Conference, May 1996.

[14]

{14} Armando Fox, Steven D. Gribble, Eric A. Brewer, and Elan Amir. Adapting to network and client variation via on-demand dynamic distillation. In Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-V), October 1997.

[15]

{15} Armando Fox, Steven D. Gribble, Yatin Chawathe, Eric A. Brewer, and Paul Gauthier. Scalable cluster-based network services. In To Appear in the Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP-16), October 1997.

Digital Library

[16]

{16} Nicolas D. Georganas. Self-similar ("fractal") trafic in atm networks. In Proceedings of the 2nd International Workshop on Advanced Teleservices and High-Speed Communications Architectures (IWACA '94), pages 1-7, Heidelberg, Germany, September 1994.

[17]

{17} Ian Goldberg. The internet protocol scanning engine. Personal communications.

[18]

{18} Graphic, Visualization, & Usability Center. 6th www user survey. Summary available at http://www.cc.gatech.edu/gvu/user_surveys/survey-10-1996/.

[19]

{19} M. Frans Kaashoek, Dawson R. Engler, Gregory R. Ganger, and Deborah A. Wallach. Server operating systems. In Proceedings of the SIGOPS European Workshop, September 1996.

[20]

{20} Tom M. Kroeger, Jeff Mogul, and Carlos Maltzahn. Digital's web proxy traces. Online at ftp://ftp.digital.com/pub/DEC/traces/proxy/ webtraces.html, aug 1996.

[21]

{21} Will E. Leland, Murad S. Taqqu, Walter Willinger, and Daniel V. Wilson. On the self-similar nature of Ethernet trafic (extended version). IEEE/ACM Transactions on Networking, 2, February 1994.

Digital Library

[22]

{22} Nikolai Likhanov, Boris Tsybakov, and Nicolas D. Georganas. Analysis of an atm buffer with self-similar ("fractal") input trafic. In Proceedings of IEEE INFOCOM '95, Boston, MA, April 1995. IEEE.

[23]

{23} Bruce Mah. An empirical model of http network trafic. In Proceedings of INFOCOM '97, Kobe, Japan, apr. 1997.

[24]

{24} Benoit Mandelbrot. Self-similar error clusters in communication systems and the concept of conditional stationarity. IEEE Transactions on Communication Technology, COM-13, 1965.

[25]

{25} David Marwood and Brad Duska. Squid proxy analysis (spa). Report available at http://www.cs.ubc.ca/spider/marwood/, 1996.

[26]

{26} Microsoft Corporation Home Page. http://www.microsoft.com.

[27]

{27} Jeffrey C. Mogul. Operating systems support for busy internet servers. In Proceedings HotOS-V, Orcas Island, Washington, May 1995.

[28]

{28} Netscape Corporation Home Page. http://www.netscape.com.

[29]

{29} John K. Ousterhout. Why aren't operating systems getting faster as fast as hardware? In Proceedings of USENIX Technical Conference, June 1990.

[30]

{30} Vern Paxson and Sally Floyd. Wide-area traffic: the failure of Poisson modeling. In ACM SIGCOMM '94 Conference on Communications Architectures, Protocols and Applications, London, UK, August 1994.

Digital Library

[31]

{31} Margo I. Selzter. Issues and challenges facing the world wide web. Talk presented to Lotus Corporation, available at http://www.eecs.harvard.edu/margo/slides/ lotus.html, March 1997.

[32]

{32} Stephen Williams, Marc Abrams, Charles R. Standridge, Ghaleb Abdulla, and Edward A. Fox. Removal policies in network caches for world-wide web documents. In Proceedings of ACM SIGCOMM, Stanford, CA, USA, August 1997.

Cited By

Sundaresan SFeamster NTeixeira RMagharei NPapagiannaki KGummadi KPartridge C(2013)Community contribution award -- Measuring and mitigating web performance bottlenecks in broadband access networksProceedings of the 2013 conference on Internet measurement conference10.1145/2504730.2504741(213-226)Online publication date: 23-Oct-2013
https://dl.acm.org/doi/10.1145/2504730.2504741
Ihm SPai VThiran PWillinger W(2011)Towards understanding modern web trafficProceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference10.1145/2068816.2068845(295-312)Online publication date: 2-Nov-2011
https://dl.acm.org/doi/10.1145/2068816.2068845
Shifrin MCidon I(2010)C3Proceedings of the 7th international conference on Wireless on-demand network systems and services10.5555/1834182.1834189(31-38)Online publication date: 3-Feb-2010
https://dl.acm.org/doi/10.5555/1834182.1834189
Show More Cited By

Index Terms

System design issues for internet middleware services: deductions from a large client trace
1. Information systems
  1. Information retrieval
  2. Information storage systems
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Organizing principles for web applications

Recommendations

Design and Implementation of the NetTraveler Middleware System based on Web Services
AICT-ICIW '06: Proceedings of the Advanced Int'l Conference on Telecommunications and Int'l Conference on Internet and Web Applications and Services

We present NetTraveler, a database middleware system for WANs that is designed to efficiently run queries over sites that are either mobile clients or enterprise servers. NetTraveler is designed to cope with dynamic WANs where data sources go off-line, ...
Design of Session State Migration Middleware for Disruption-Free Services
NGMAST '12: Proceedings of the 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies

Various services, such as streaming, network games and thin clients, which keep a session state of user equipments (UEs) for a long period, are provided from servers. This paper proposes a session state migration which relocates session states from one ...
System support for large-scale internet services

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

USITS'97: Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems

December 1997

266 pages

Publisher

USENIX Association

United States

Publication History

Published: 08 December 1997

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

50
Total Citations
View Citations
19
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sundaresan SFeamster NTeixeira RMagharei NPapagiannaki KGummadi KPartridge C(2013)Community contribution award -- Measuring and mitigating web performance bottlenecks in broadband access networksProceedings of the 2013 conference on Internet measurement conference10.1145/2504730.2504741(213-226)Online publication date: 23-Oct-2013
https://dl.acm.org/doi/10.1145/2504730.2504741
Ihm SPai VThiran PWillinger W(2011)Towards understanding modern web trafficProceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference10.1145/2068816.2068845(295-312)Online publication date: 2-Nov-2011
https://dl.acm.org/doi/10.1145/2068816.2068845
Shifrin MCidon I(2010)C3Proceedings of the 7th international conference on Wireless on-demand network systems and services10.5555/1834182.1834189(31-38)Online publication date: 3-Feb-2010
https://dl.acm.org/doi/10.5555/1834182.1834189
Fox AGoldberg IGribble SLee DPolito ABrewer EBlair GDavies NRaymond K(2009)Experience with Top Gun WingmanProceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing10.5555/1659232.1659266(407-424)Online publication date: 25-Nov-2009
https://dl.acm.org/doi/10.5555/1659232.1659266
Chawathe YBrewer EBlair GDavies NRaymond K(2009)System support for scalable and fault tolerant internet servicesProceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing10.5555/1659232.1659239(71-88)Online publication date: 25-Nov-2009
https://dl.acm.org/doi/10.5555/1659232.1659239
Badam APark KPai VPeterson LRexford JSirer E(2009)HashCacheProceedings of the 6th USENIX symposium on Networked systems design and implementation10.5555/1558977.1558986(123-136)Online publication date: 22-Apr-2009
https://dl.acm.org/doi/10.5555/1558977.1558986
Gill PArlitt MLi ZMahanti ADovrolis CRoughan M(2007)Youtube traffic characterizationProceedings of the 7th ACM SIGCOMM conference on Internet measurement10.1145/1298306.1298310(15-28)Online publication date: 24-Oct-2007
https://dl.acm.org/doi/10.1145/1298306.1298310
Du BDemmer MBrewer ECarr LDe Roure DIyengar AGoble CDahlin M(2006)Analysis of WWW traffic in Cambodia and GhanaProceedings of the 15th international conference on World Wide Web10.1145/1135777.1135894(771-780)Online publication date: 23-May-2006
https://dl.acm.org/doi/10.1145/1135777.1135894
Kim BKim I(2006)Content distribution strategy using web-cached multicast techniqueProceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part IV10.1007/11751632_123(1146-1155)Online publication date: 8-May-2006
https://dl.acm.org/doi/10.1007/11751632_123
Laoutaris NZissimopoulos VStavrakakis I(2005)On the optimization of storage capacity allocation for content distributionComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2004.07.02047:3(409-428)Online publication date: 1-Feb-2005
https://dl.acm.org/doi/10.1016/j.comnet.2004.07.020
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents