research-article

Empowering automatic data-center management with machine learning

Authors:

Josep Ll. Berral,

Ricard Gavaldà,

Jordi TorresAuthors Info & Claims

SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied Computing

Pages 170 - 172

https://doi.org/10.1145/2480362.2480397

Published: 18 March 2013 Publication History

Get Access

Abstract

The Cloud as computing paradigm has become nowadays crucial for most Internet business models. Managing and optimizing its performance on a moment-by-moment basis is not easy given as the amount and diversity of elements involved (hardware, applications, workloads, customer needs...). Here we show how a combination of scheduling algorithms and data mining techniques helps improving the performance and profitability of a data-center running virtualized web-services. We model the data-center's main resources (CPU, memory, IO), quality of service (viewed as response time), and workloads (incoming streams of requests) from past executions. We show how these models to help scheduling algorithms make better decisions about job and resource allocation, aiming for a balance between throughput, quality of service, and power consumption.

References

[1]

J. Berral, R. Gavaldà, and J. Torres. Li-BCN Workload 2010, 2011. http://www.lsi.upc.edu/dept/techreps/llistat_detallat.php?id=1099.

Google Scholar

[2]

J. S. Chase, D. C. Anderson, P. N. Thakar, and A. M. Vahdat. Managing energy and server resources in hosting centers. In 18th ACM SOSP 2001.

Digital Library

Google Scholar

[3]

Í. Goiri, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres. Energy-aware Scheduling in Virtualized Datacenters. In 12th IEEE Cluster 2010.

Digital Library

Google Scholar

[4]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The WEKA data mining software: an update. SIGKDD Explor. Newsl., 11(1):10--18, 2009.

Digital Library

Google Scholar

[5]

F. Julià, J. Roldàn, R. Nou, O. Fitó, Vaquè, G. Í., and J. Berral. EEFSim: Energy Efficency Simulator, 2010.

Google Scholar

[6]

B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing, 13(5):14--22, Sept. 2009.

Digital Library

Google Scholar

Cited By

View all

Yoo S(2023)New trends in photonic switching and optical networking architectures for data centers and computing systems [Invited]Journal of Optical Communications and Networking10.1364/JOCN.48457715:8(C288)Online publication date: 27-Jul-2023
https://doi.org/10.1364/JOCN.484577
Wang TLi BChen MYu SWang TLi BChen MYu S(2022)Machine Learning Empowered Intelligent Data Center NetworkingMachine Learning Empowered Intelligent Data Center Networking10.1007/978-981-19-7395-6_3(15-99)Online publication date: 25-Oct-2022
https://doi.org/10.1007/978-981-19-7395-6_3
Silva GKlôh VYokoyama AGritz MSchulze BFerro M(2020)SMCis: Scientific Applications Monitoring and Prediction for HPC EnvironmentsHigh Performance Computing Systems10.1007/978-3-030-41050-6_5(69-84)Online publication date: 14-Feb-2020
https://doi.org/10.1007/978-3-030-41050-6_5
Show More Cited By

Index Terms

Empowering automatic data-center management with machine learning

Recommendations

Machine Learning Approach for Live Migration Cost Prediction in VMware Environments
CLOSER 2019: Proceedings of the 9th International Conference on Cloud Computing and Services Science

Virtualization became a commonly used technology in datacenters during the last decade. Live migration is an essential feature in most of the clusters hypervisors. Live migration process has a cost that includes the migration time, downtime, IP network ...
Embedding individualized machine learning prediction models for energy efficient VM consolidation within Cloud data centers
Abstract
The fast growth in demand for utility-based IT services has lead to the formation of large scale Cloud data centers. The electrical energy consumption of these data centers results in high operational costs and carbon dioxide ...
Highlights
- A Machine Learning (ML) based method is provided for predicting over-utilization in hosts.
Adaptive Virtual Machine Management in the Cloud: A Performance-Counter-Driven Approach

The success of cloud computing technologies heavily depends on both the underlying hardware and system software support for virtualization. In this study, we propose to elevate the capability of the hypervisor to monitor and manage co-running virtual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied Computing

March 2013

2124 pages

ISBN:9781450316569

DOI:10.1145/2480362

Conference Chairs:
Sung Y. Shin
South Dakota State University, United States
,
José Carlos Maldonado
ICMC - University of São Paulo, Brazil

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

SAC '13

Sponsor:

SIGAPP

SAC '13: SAC '13

March 18 - 22, 2013

Coimbra, Portugal

Acceptance Rates

SAC '13 Paper Acceptance Rate 255 of 1,063 submissions, 24%;

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
211
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yoo S(2023)New trends in photonic switching and optical networking architectures for data centers and computing systems [Invited]Journal of Optical Communications and Networking10.1364/JOCN.48457715:8(C288)Online publication date: 27-Jul-2023
https://doi.org/10.1364/JOCN.484577
Wang TLi BChen MYu SWang TLi BChen MYu S(2022)Machine Learning Empowered Intelligent Data Center NetworkingMachine Learning Empowered Intelligent Data Center Networking10.1007/978-981-19-7395-6_3(15-99)Online publication date: 25-Oct-2022
https://doi.org/10.1007/978-981-19-7395-6_3
Silva GKlôh VYokoyama AGritz MSchulze BFerro M(2020)SMCis: Scientific Applications Monitoring and Prediction for HPC EnvironmentsHigh Performance Computing Systems10.1007/978-3-030-41050-6_5(69-84)Online publication date: 14-Feb-2020
https://doi.org/10.1007/978-3-030-41050-6_5
Dayarathna MWen YFan R(2016)Data Center Energy Consumption Modeling: A SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2015.248118318:1(732-794)Online publication date: Sep-2017
https://doi.org/10.1109/COMST.2015.2481183
Zhang JHuang HWang X(2016)Resource provision algorithms in cloud computingJournal of Network and Computer Applications10.1016/j.jnca.2015.12.01864:C(23-42)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.jnca.2015.12.018
Nascimento DPires CMestre D(2016)Applying machine learning techniques for scaling out data quality algorithms in cloud computing environmentsApplied Intelligence10.1007/s10489-016-0774-245:2(530-548)Online publication date: 2-Apr-2016
https://doi.org/10.1007/s10489-016-0774-2
Nascimento DPires CMestre DWainwright RCorchado JBechini AHong J(2015)A data quality-aware cloud service based on metaheuristic and machine learning provisioning algorithmsProceedings of the 30th Annual ACM Symposium on Applied Computing10.1145/2695664.2695753(1696-1703)Online publication date: 13-Apr-2015
https://dl.acm.org/doi/10.1145/2695664.2695753
Berral JGavaldà RTorres J(2013)Power-Aware Multi-data Center Management Using Machine LearningProceedings of the 2013 42nd International Conference on Parallel Processing10.1109/ICPP.2013.102(858-867)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1109/ICPP.2013.102

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Machine Learning Approach for Live Migration Cost Prediction in VMware Environments

Embedding individualized machine learning prediction models for energy efficient VM consolidation within Cloud data centers

Adaptive Virtual Machine Management in the Cloud: A Performance-Counter-Driven Approach

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations