research-article

Live gang migration of virtual machines

Authors:

Umesh Deshpande,

Xiaoshuang Wang,

Kartik GopalanAuthors Info & Claims

HPDC '11: Proceedings of the 20th international symposium on High performance distributed computing

Pages 135 - 146

https://doi.org/10.1145/1996130.1996151

Published: 08 June 2011 Publication History

Abstract

This paper addresses the problem of simultaneously migrating a group of co-located and live virtual machines (VMs), i.e, VMs executing on the same physical machine. We refer to such a mass simultaneous migration of active VMs as "live gang migration". Cluster administrators may often need to perform live gang migration for load balancing, system maintenance, or power savings. Application performance requirements may dictate that the total migration time, network traffic overhead, and service downtime, be kept minimal when migrating multiple VMs. State-of-the-art live migration techniques optimize the migration of a single VM. In this paper, we optimize the simultaneous live migration of multiple co-located VMs. We present the design, implementation, and evaluation of a de-duplication based approach to perform concurrent live migration of co-located VMs. Our approach transmits memory content that is identical across VMs only once during migration to significantly reduce both the total migration time and network traffic. Using the QEMU/KVM platform, we detail a proof-of-concept prototype implementation of two types of de-duplication strategies (at page level and sub-page level) and a differential compression approach to exploit content similarity across VMs. Evaluations over Gigabit Ethernet with various types of VM workloads demonstrate that our prototype for live gang migration can achieve significant reductions in both network traffic and total migration time.

References

[1]

Superfasthash www.azillionmonkeys.com/qed/hash.html.

[2]

A. Arcangeli, I. Eidus, and C. Wright. Increasing memory density by using ksm. In Proc. of Linux Symposium, July 2009.

[3]

F. Bellard. Qemu, a fast and portable dynamic translator. In Proc. of USENIX Annual Technical Conference, April 2005.

Digital Library

[4]

N. Bobroff, A. Kochut, and K. Beaty. Dynamic placement of virtual machines for managing sla violations. In Proc. of Integrated Network Management, page 119--128, May 2007.

[5]

C. Clark, K. Fraser, S. Hand, J.G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In Proc. of Network System Design and Implementation, May 2005.

Digital Library

[6]

Dbench. http://samba.org/ftp/tridge/dbench.

[7]

D. Gupta, S. Lee, M. Vrable, S. Savage, A. C Snoeren, G. Varghese, G. M Voelker, and A. Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. In Proc. of Operating Systems Design and Implementation, December 2010.

Digital Library

[8]

J.G. Hansen and E. Jul. Self-migration of operating systems. In Proc. of ACM SIGOPS European Workshop, September 2004.

Digital Library

[9]

M. Hines, U. Deshpande, and K. Gopalan. Post-copy live migration of virtual machines. SIGOPS Operating Syst. Review, 43(3):14--26, July 2009.

Digital Library

[10]

W. Huang, M. Koop, Q. Gao, and D.K. Panda. Virtual machine aware communication libraries for high performance computing. In Proc. of SuperComputing, November 2007.

Digital Library

[11]

H. Jin, L. Deng, S. Wu, X. Shi, and X. Pan. Live virtual machine migration with adaptive, memory compression. In Proc. of Cluster Computing and Workshops, August 2009.

[12]

K. Kim, C. Kim, S-I. Jung, H Shin, and J-S. Kim. Inter-domain socket communications supporting high performance and full binary compatibility on xen. In Proc. of Virtual Execution Environments, March 2008.

Digital Library

[13]

A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori. Kvm: The linux virtual machine monitor. In Proc. of Linux Symposium, June 2007.

[14]

J. MacDonald. Xdelta http://www.xdelta.org/.

[15]

V. Makhija, B. Herndon, P. Smith, L. Roderick, E. Zamost, and J. Anderson. VMmark: A scalable benchmark for virtualized systems. Technical Report 2006-002, 2002.

[16]

G. Milos, D.G. Murray, S. Hand, and M.A. Fetterman. Satori: Enlightened page sharing. In Proc. of USENIX Annual Technical Conference, June 2009.

Digital Library

[17]

MySQL. MySQL Cluster http://www.mysql.com/products/database/cluster.

[18]

M. Nelson, B. H Lim, and G. Hutchins. Fast transparent migration for virtual machines. In Proc. of USENIX Annual Technical Conference, April 2005.

Digital Library

[19]

P. Riteau, C. Morin, and T. Priol. Shrinker: Efficient wide area live virtual machine migration using distributed content-based addressing. In http://hal.inria.fr/inria-00454727/en/, February 2009.

[20]

C. P Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M. S Lam, and M. Rosenblum. Optimizing the migration of virtual computers. In Proc. of Operating Systems Design and Implementation, December 2002.

Digital Library

[21]

VMWare Distributed Resource Scheduler. http://www.vmware.com/products/vi/vc/drs.html.

[22]

Sysbench. Sysbench benchmark http://sysbench.sourceforge.net/index.html.

[23]

TCPDUMP. http://www.tcpdump.org.

[24]

TPC-H Benchmark. http://www.tpc.org/tpch/.

[25]

A. Verma, P. Ahuja, and A. Neogi. pMapper: power and migration cost aware application placement in virtualized systems. In International Conference on Middleware, December 2008.

Digital Library

[26]

C. A. Waldspurger. Memory resource management in VMware ESX server. In Proc. of Operating Systems Design and Implementation, December 2002.

Digital Library

[27]

J. Wang, K. L Wright, and K. Gopalan. XenLoop: a transparent high performance inter-vm network loopback. In Proc. of High performance distributed computing, June 2008.

Digital Library

[28]

T. Wood, K. Ramakrishnan, J. van der Merwe, and P. Shenoy. CloudNet: a platform for optimized WAN migration of virtual machines. University of Massachusetts Technical Report TR-2010, 2, 2010.

[29]

T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif. Sandpiper: Black-box and gray-box resource management for virtual machines. The International Journal of Computer and Telecommunications Networking, 53(17), December 2009.

Digital Library

[30]

X. Zhang, Z. Huo, J. Ma, and D. Meng. Exploiting data deduplication to accelerate live virtual machine migration. In Proc. of International Conference on Cluster Computing, September 2010.

Digital Library

[31]

X. Zhang, S. McIntosh, P. Rohatgi, and J. L. Griffin. Xensocket: a high-throughput interdomain transport for virtual machines. In Proc. of International Conference on Middleware, November 2007.

Digital Library

Cited By

Singh SSingh D(2023)Comprehensive Analysis of VM Migration Trends in Cloud Data CentersRecent Patents on Engineering10.2174/187221211766622112916072617:6Online publication date: Nov-2023
https://doi.org/10.2174/1872212117666221129160726
He TBuyya R(2023)A Taxonomy of Live Migration Management in Cloud ComputingACM Computing Surveys10.1145/361535356:3(1-33)Online publication date: 5-Oct-2023
https://dl.acm.org/doi/10.1145/3615353
Eswaran RYan MGopalan KSha KBanerjee SChen J(2023)Template-Aware Live Migration of Virtual MachinesProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3626812(336-340)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3626812
Show More Cited By

Index Terms

Live gang migration of virtual machines
1. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems

Recommendations

Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning
VEE '09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments

We present the design, implementation, and evaluation of post-copy based live migration for virtual machines (VMs) across a Gigabit LAN. Live migration is an indispensable feature in today's virtualization technologies. Post-copy migration defers the ...
Post-copy live migration of virtual machines

We present the design, implementation, and evaluation of post-copy based live migration for virtual machines (VMs) across a Gigabit LAN. Post-copy migration defers the transfer of a VM's memory contents until after its processor state has been sent to ...
Inter-rack live migration of multiple virtual machines
VTDC '12: Proceedings of the 6th international workshop on Virtualization Technologies in Distributed Computing Date

Within datacenters, often multiple virtual machines (VMs) need to be live migrated simultaneously for various reasons such as maintenance, power savings, and load balancing. Such mass simultaneous live migration of multiple VMs can trigger large data ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HPDC '11: Proceedings of the 20th international symposium on High performance distributed computing

June 2011

296 pages

ISBN:9781450305525

DOI:10.1145/1996130

General Chair:
Arthur "Barney" Maccabe
Oak Ridge National Lab, USA
,
Program Chair:
Douglas Thain
University of Notre Dame, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HPDC '11

Sponsor:

University of Arizona
SIGARCH

HPDC '11: The 20th International Symposium on High-Performance Parallel and Distributed Computing

June 8 - 11, 2011

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 166 of 966 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
1,221
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)4

Reflects downloads up to 30 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Singh SSingh D(2023)Comprehensive Analysis of VM Migration Trends in Cloud Data CentersRecent Patents on Engineering10.2174/187221211766622112916072617:6Online publication date: Nov-2023
https://doi.org/10.2174/1872212117666221129160726
He TBuyya R(2023)A Taxonomy of Live Migration Management in Cloud ComputingACM Computing Surveys10.1145/361535356:3(1-33)Online publication date: 5-Oct-2023
https://dl.acm.org/doi/10.1145/3615353
Eswaran RYan MGopalan KSha KBanerjee SChen J(2023)Template-Aware Live Migration of Virtual MachinesProceedings of the Eighth ACM/IEEE Symposium on Edge Computing10.1145/3583740.3626812(336-340)Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1145/3583740.3626812
Vatsal SVerma S(2023)Virtual machine migration based algorithmic approach for safeguarding environmental sustainability by renewable energy usage maximization in Cloud data centresInternational Journal of Information Technology10.1007/s41870-023-01478-2Online publication date: 16-Sep-2023
https://doi.org/10.1007/s41870-023-01478-2
Singh GSingh A(2022)Optimization of SLA aware live migration of multiple virtual machines using Lagrange multiplierFuture Generation Computer Systems10.1016/j.future.2022.01.001130:C(279-291)Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.future.2022.01.001
Feng DFeng D(2022)Overview of Data DeduplicationData Deduplication for High Performance Storage System10.1007/978-981-19-0112-6_2(9-23)Online publication date: 3-Jun-2022
https://doi.org/10.1007/978-981-19-0112-6_2
Vatsal SAgarwal S(2022)Safeguarding Cloud Services Sustainability by Dynamic Virtual Machine Migration with Re-allocation Oriented Algorithmic ApproachSmart Trends in Computing and Communications10.1007/978-981-16-9967-2_40(425-435)Online publication date: 6-Jul-2022
https://doi.org/10.1007/978-981-16-9967-2_40
Singh GTailwal RKathayat MSingh A(2022)IPAM: An Improved Penalty Aware VM Migration TechniqueComputational Science and Its Applications – ICCSA 202210.1007/978-3-031-10522-7_23(324-336)Online publication date: 15-Jul-2022
https://doi.org/10.1007/978-3-031-10522-7_23
Ozawa YShinagawa T(2021)Exploiting Sub-page Write Protection for VM Live Migration2021 IEEE 14th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD53861.2021.00063(484-490)Online publication date: Sep-2021
https://doi.org/10.1109/CLOUD53861.2021.00063
Nicolae BWozniak JDorier MCappello F(2020)DeepClone: Lightweight State Replication of Deep Learning Models for Data Parallel Training2020 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER49012.2020.00033(226-236)Online publication date: Sep-2020
https://doi.org/10.1109/CLUSTER49012.2020.00033
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents