Towards realistic benchmarks for virtual infrastructure resource allocators
Authors: 
Qin Yin, 
Timothy RoscoeAuthors Info & Claims
Article No.: 5, Pages 1 - 6
Abstract
The rise of virtual infrastructures has renewed research interest in the design and implementation of resource allocators which allow users to reserve or use combinations of (virtual) nodes, switches, and network links on a variety of virtual infrastructures ranging from network testbeds to cloud computing facilities. However, since the area is still relatively new, work on resource allocation lacks realistic benchmarks. In this paper, we suggest how to generate realistic benchmarks for virtual infrastructure, and present our workload generator as an example, which is based on a 5-year trace of experiments submitted to the popular Emulab testbed. We demonstrate the importance as well as potential applications of such benchmarks by using the generated test workload in various evaluation scenarios.
References
[1]
Arlitt, M. F., and Williamson, C. L. Internet web servers: Workload characterization and performance implications. IEEE/ACM Transactions on Networking 5 (1997), 631--645.
[2]
Barford, P., and Crovella, M. Generating representative web workloads for network and server performance evaluation. SIGMETRICS Perform, Eval. Rev. 26, 1 (June 1998), 151--160.
[3]
Calzarossa, M., Massari, L., and Tessera, D. Workload characterization issues and methodologies. In Proc. of Performance Evaluation: Origins and Directions (2000), pp. 459--481.
[4]
Calzarossa, M., and Serazzi, G. Workload characterization: A survey. In Proc. of the IEEE (1993), pp. 1136--1150.
[5]
Chapin, S. J., Cirne, W., Feitelson, D. G., Jones, J. P., Leutenegger, S. T., Schwiegelshohn, U., Smith, W., and Talby, D. Benchmarks and standards for the evaluation of parallel job schedulers. In Proc. of JSSPP'99, pp. 67--90.
[6]
Chen, P. M., and Patterson, D. A. A new approach to I/O performance evaluation: self-scaling I/O benchmarks, predicted I/O performance. ACM Trans. Comput. Syst. (1994), 308--339.
[7]
Chen, Y., Ganapathi, A. S., Griffith, R., and Katz, R. H. Analysis and lessons from a publicly available google cluster trace. Tech. Rep. UCB/EECS-2010-95, EECS Department, University of California, Berkeley, Jun 2010.
[8]
Cirne, W., and Berman, F. A comprehensive model of the supercomputer workload. In Proc. of WWC'01, pp. 140--148.
[9]
Condor, http://www.cs.wisc.edu/condor/.
[10]
Amazon EC2. http://aws.amazon.com/ec2/.
[11]
Emulab. http://www.emulab.net/.
[12]
Emulab trace. http://www.emulab.net/downloads/topfiles/topfiles-anonymized-2007-06-18.tar.gz.
[13]
Eucalyputus Cloud, http://www.eucalyptus.com/.
[14]
Fu, Y., Chase, J., Chun, B., Schwab, S., and Vahdat, A. SHARP: an architecture for secure resource peering. In Proc. of SOSP'03, pp. 133--148.
[15]
Ganapathi, A. S., Chen, Y, Fox, A., Katz, R. H., and Patterson, D. A. Statistics-driven workload modeling for the cloud. Tech. Rep. UCB/EECS-2009-160, EECS Department, University of California, Berkeley, Nov 2009.
[16]
GENI. http://www.geni.net/.
[17]
Globus, http://www.globus.org.
[18]
Jain, R. K. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley, New York, NY, USA, April 1991.
[19]
Khan, A., Yan, X., Tao, S., and Anerousis, N. Workload characterization and prediction in the cloud: A multiple time series approach. In Proc. of Cloudman'12.
[20]
Mirkovic, J., Hussain, A., and Shi, H. A comparative study of network testbed usage characteristics. Under submission, USC/Information Sciences Institute, 2012.
[21]
Mishra, A. K., Hellerstein, J. L., Cirne, W., and Das, C. R. Towards characterizing cloud backend workloads: insights from google compute clusters. SIGMETRICS Perform, Eval. Rev. 37, 4 (Mar. 2010), 34--41.
[22]
ORCA/BEN: A prototype GENI control plane for a metro-scale optical testbed. https://geni-orca.renci.org/trac/.
[23]
PlanetLab. http://www.planet-lab.org/.
[24]
ProtoGENI. http://www.protogeni.net/trac/protogeni.
[25]
Sirius calendar service, http://www.planet-lab.org/node/114.
[26]
Smith, J., Golden, P., and Appleton, B. Airline: a strategic management simulation. Prentice Hall, 1991.
[27]
Song, B., Ernemann, C, and Yahyapour, R. Parallel computer workload modeling with markov chains. In Proc. of JSSPP'04, pp. 47--62.
[28]
Sotomayor, B., Keahey, K., and Foster, I. Combining batch execution and leasing using virtual machines. In Proc. of HPDC'08, pp. 87--96.
[29]
TPC - Transaction Processing Performance Council. http://www.tpc.org/.
[30]
VTNI. http://www.vini-veritas.net/.
[31]
Wang, G., Butt, A. R., Monti, H., and Gupta, K. Towards synthesizing realistic workload traces for studying the hadoop ecosystem. In Proc. of MASCOTS'11, pp. 400--408.
[32]
Yin, Q., and Roscoe, T. A better way to negotiate for testbed resources. In Proc. of APSys'11, pp. 19:1--19:5.
[33]
Yin, Q., and Roscoe, T. Vf2x: Fast, efficient virtual network mapping for real testbed workloads. In Proc. of TridentCom'12.
Recommendations
Towards realistic benchmarks for virtual infrastructure resource allocators
APSys '12: Proceedings of the Third ACM SIGOPS Asia-Pacific conference on SystemsThe rise of virtual infrastructures has renewed research interest in the design and implementation of resource allocators which allow users to reserve or use combinations of (virtual) nodes, switches, and network links on a variety of virtual ...
Conservative garbage collection for general memory allocators
This paper explains a technique that integrates conservative garbage collection on top of general memory allocators. This is possible by using two data structures named malloc-tables and jump-tables that are computed at garbage collection time to map ...
A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment
This study presents a 3D virtual reality (VR) keyboard system with realistic haptic feedback. The system uses two five-fingered data gloves to track finger positions and postures, uses micro-speakers to create simulated vibrations, and uses a head-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2012
101 pages
ISBN:9781450316699
DOI:10.1145/2349896
Copyright © 2012 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- Kaist
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 23 July 2012
Check for updates
Qualifiers
- Research-article
Conference
APSys '12
Sponsor:
Acceptance Rates
Overall Acceptance Rate 169 of 430 submissions, 39%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 99Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in