Modelling and simulation of ElasticSearch using CloudSim
Authors: 
Malika Bendechache, Sergej Svorobej, Patricia Takako Endo, Manuel Noya Mario, M. Eduardo Ares, James Byrne, Theo LynnAuthors Info & Claims
Pages 223 - 230
Abstract
Simulation can be a powerful technique for evaluating the performance of large-scale cloud computing services in a relatively low cost, low risk and time-sensitive manner. Large-scale data indexing, distribution and management is complex to analyse in a timely manner. In this paper, we extend the CloudSim cloud simulation framework to model and simulate a distributed search engine architecture and its workload characteristics. To test the simulation framework, we develop a model based on a real-world ElasticSearch deployment on Linknovate.com. An experimental evaluation of the framework, comparing simulated and actual query response time, precision and resource utilisation, suggests that the proposed framework is capable of predicting performance at different scales in a precise, accurate and efficient manner. The results can assist ElasticSearch users to manage their scalability and infrastructure requirements.
References
[1]
"Adults: Media use and attitudes report 2019," https://www.ofcom.org.uk, accessed: 2019-06-07.
[2]
"Number of explicit core search queries powered by search engines in the united states as of January 2019 (in billions)," https://www.statista.com/statistics/265796/us-search-engines-ranked-by-number-of-core-searches, accessed: 2019-06-07.
[3]
A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel, "The cost of a cloud: Research problems in data center networks," SIGCOMM Comput. Commun. Rev., vol. 39, no. 1, pp. 68--73, Dec. 2008. [Online]. Available
[4]
B. B. Cambazoglu and R. Baeza-Yates, "Scalability and efficiency challenges in large-scale web search engines," in Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR '16. New York, NY, USA: ACM, 2016, pp. 1223--1226. [Online]. Available: http://doi.acm.org.dcu.idm.oclc.org/10.1145/2911451.2914808
[5]
Elasticsearch B.V, "Open Source Search Analytics - ElasticSearch," 2019. [Online]. Available: https://www.elastic.co/
[6]
O. Kononenko, O. Baysal, R. Holmes, and M. W. Godfrey, "Mining modern repositories with elasticsearch," in Proceedings of the 11th Working Conference on Mining Software Repositories, ser. MSR 2014. New York, NY, USA: ACM, 2014, pp. 328--331. [Online]. Available: http://doi.acm.org.dcu.idm.oclc.org/10.1145/2597073.2597091
[7]
R. Buyya, R. Ranjan, and R. N. Calheiros, "Modeling and simulation of scalable cloud computing environments and the cloudsim toolkit: Challenges and opportunities," in 2009 international conference on high performance computing & simulation. IEEE, 2009, pp. 1--11.
[8]
S. Svorobej, P. Takako Endo, M. Bendechache, C. Filelis-Papadopoulos, K. M. Giannoutakis, G. A. Gravvanis, D. Tzovaras, J. Byrne, and T. Lynn, "Simulating fog and edge computing scenarios: An overview and research challenges," Future Internet, vol. 11, no. 3, p. 55, 2019.
[9]
S. Mehmi, H. K. Verma, and A. Sangal, "Simulation modeling of cloud computing for smart grid using cloudsim," Journal of Electrical Systems and Information Technology, vol. 4, no. 1, pp. 159--172, 2017.
[10]
G. T. Hicham and E. A. Chaker, "Cloud computing cpu allocation and scheduling algorithms using cloudsim simulator." International Journal of Electrical & Computer Engineering (2088--8708), vol. 6, no. 4, 2016.
[11]
A. M. Law, W. D. Kelton, and W. D. Kelton, Simulation modeling and analysis. McGraw-Hill New York, 2000, vol. 3.
[12]
J. Idziorek, "Discrete event simulation model for analysis of horizontal scaling in the cloud computing model," in Proceedings of the 2010 Winter Simulation Conference. IEEE, 2010, pp. 3004--3014.
[13]
J. Byrne, S. Svorobej, K. M. Giannoutakis, D. Tzovaras, P. J. Byrne, P. stberg, A. Gourinovitch, and T. Lynn, "A review of cloud computing simulation platforms and related environments," in Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER., INSTICC. SciTePress, 2017, pp. 679--691.
[14]
T. Lynn, A. Gourinovitch, J. Byrne, P. J. Byrne, S. Svorobej, K. Giannoutakis, D. Kenny, and J. Morrison, "A preliminary systematic review of computer science literature on cloud computing research using open source simulation platforms," in Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER., INSTICC. SciTePress, 2017, pp. 565--573.
[15]
R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, and R. Buyya, "Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms," Software: Practice and experience, vol. 41, no. 1, pp. 23--50, 2011.
[16]
B. Wickremasinghe, R. N. Calheiros, and R. Buyya, "Cloudanalyst: A cloudsim-based visual modeller for analysing cloud computing environments and applications," in 2010 24th IEEE international conference on advanced information networking and applications. IEEE, 2010, pp. 446--452.
[17]
S. K. Garg and R. Buyya, "Networkcloudsim: Modelling parallel applications in cloud simulations," in 2011 Fourth IEEE International Conference on Utility and Cloud Computing. IEEE, 2011, pp. 105--113.
[18]
M. Barika, S. Garg, A. Chan, R. N. Calheiros, and R. Ranjan, "Iotsimstream: Modelling stream graph application in cloud simulation," Future Generation Computer Systems, vol. 99, pp. 86--105, 2019.
[19]
A. Siavashi and M. Momtazpour, "Gpucloudsim: an extension of cloudsim for modeling and simulation of gpus in cloud data centers," The Journal of Supercomputing, vol. 75, no. 5, pp. 2535--2561, 2019.
[20]
M. C. Silva Filho, R. L. Oliveira, C. C. Monteiro, P. R. Mcio, and M. M. Freire, "Cloudsim plus: a cloud computing simulation framework pursuing software engineering principles for improved modularity, extensibility and correctness," in 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM). IEEE, 2017, pp. 400--406.
[21]
M. McCandless, E. Hatcher, and O. Gospodnetic, Lucene in action: covers Apache Lucene 3.0. Manning Publications Co., 2010.
[22]
R. Kuc and M. Rogozinski, Elasticsearch server. Packt Publishing Ltd, 2013.
[23]
J. Benesty, J. Chen, Y. Huang, and I. Cohen, "Pearson correlation coefficient," in Noise reduction in speech processing. Springer, 2009, pp. 1--4.
[24]
A. Inostrosa-Psijas, G. Wainer, V. Gil-Costa, and M. Marin, "Devs modeling of large scale web search engines," in Proceedings of the Winter Simulation Conference 2014. IEEE, 2014, pp. 3060--3071.
[25]
P. Kathiravelu and L. Veiga, "Concurrent and distributed cloudsim simulations," in 2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems. IEEE, 2014, pp. 490--493.
[26]
V. Moysiadis, P. Sarigiannidis, and I. Moscholios, "Towards distributed data management in fog computing," Wireless Communications and Mobile Computing, vol. 2018, 2018.
[27]
M. K. Nasution, "Modelling and simulation of search engine," in Journal of Physics: Conference Series, vol. 801, no. 1. IOP Publishing, 2017, p. 012078.
[28]
M. Marin, V. Gil-Costa, C. Bonacic, and A. Inostrosa, "Simulating search engines," Computing in Science & Engineering, vol. 19, no. 1, p. 62, 2017.
[29]
R. Kumar and G. Sahoo, "Cloud computing simulation using cloudsim," arXiv preprint arXiv:1403.3253, 2014.
[30]
W. Long, L. Yuqing, and X. Qingxin, "Using cloudsim to model and simulate cloud computing environment," in 2013 Ninth International Conference on Computational Intelligence and Security. IEEE, 2013, pp. 323--328.
Recommendations
Learning cloud computing and security through CloudSim simulation
InfoSec '14: Proceedings of the 2014 Information Security Curriculum Development ConferenceCloud computing is an emerging parallel and distributed service-oriented computing paradigm that provides platform service, software service, and infrastructure service through computing resource virtualization. Cloud computing is fast evolving and ...
Storage CloudSim
CLOSER 2014: Proceedings of the 4th International Conference on Cloud Computing and Services ScienceSince Cloud services are billed by the pay-as-you-go principle, organizations can save huge investment costs. Hence, they want to know, what costs will arise by the usage of those services. On the other hand, Cloud providers want to provide the best-...
Using CloudSim to Model and Simulate Cloud Computing Environment
CIS '13: Proceedings of the 2013 Ninth International Conference on Computational Intelligence and SecurityCloud computing will be a major technology in the development of the future Internet of Services. Service providers want to remove the bottle neck of the cloud computing system in order to satisfy user requirement. And in order to save energy consumption ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2019
344 pages
ISBN:9781728129235
- General Chair:
- Floriano De Rango,
- Program Chairs:
- Carlos Tavares Calafate,
- Miroslav Voznak
Sponsors
Publisher
IEEE Press
Publication History
Published: 17 April 2020
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
DS-RT '19
Sponsor:
DS-RT '19: International Symposium on Distributed Simulation and Real-Time Applications
October 7 - 9, 2019
Cosenza, Italy
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 67Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in