poster

Benchmarking microservice systems for software engineering research

Authors:

Xiang Zhou,

Xin Peng,

Tao Xie,

Jun Sun,

Chenjie Xu,

Chao Ji,

Wenyun ZhaoAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Pages 323 - 324

https://doi.org/10.1145/3183440.3194991

Published: 27 May 2018 Publication History

Get Access

Abstract

Despite the prevalence and importance of microservices in industry, there exists limited research on microservices, partly due to lacking a benchmark system that reflects the characteristics of industrial microservice systems. To fill this gap, we conduct a review of literature and open source systems to identify the gap between existing benchmark systems and industrial microservice systems. Based on the results of the gap analysis, we then develop and release a medium-size benchmark system of microservice architecture.

References

[1]

Acme.Com. 2015. Acme Air. (2015). Retrieved August 16, 2017 from https://github.com/acmeair/acmeair

Google Scholar

[2]

Carlos M. Aderaldo, Nabor C. Mendonça, Claus Pahl, and Pooyan Jamshidi. 2017. Benchmark Requirements for Microservices Architecture Research. In ECASE@ICSE 2017. 8--13.

Crossref

Google Scholar

[3]

MicroService System Benchmark. 2017. (2017). https://github.com/microcosmx/train_ticket

Google Scholar

[4]

Bifrost.Com. 2017. bifrost. (2017). Retrieved August 16, 2017 from https://github.com/sealuzh/bifrost-microservices-sample-application

Google Scholar

[5]

EShopOnContainers.Com. 2017. eShopOnContainers. (2017). Retrieved August 16, 2017 from https://github.com/dotnet-architecture/eShopOnContainers

Google Scholar

[6]

P. D. Francesco, I. Malavolta, and P. Lago. 2017. Research on Architecting Microservices: Trends, Focus, and Potential for Industrial Adoption. In ICSA 2017. 21--30.

Google Scholar

[7]

MusicStore.Com. 2017. Music Store. (2017). Retrieved August 16, 2017 from https://github.com/aspnet/MusicStore

Google Scholar

[8]

SocksShopGit.Com. 2017. Socks Shop Git. (2017). Retrieved August 16, 2017 from https://github.com/microservices-demo/microservices-demo

Google Scholar

[9]

SpringDemo.Com. 2017. Spring Cloud Demo Apps. (2017). Retrieved August 16, 2017 from https://github.com/kbastani/spring-cloud-microservice-example

Google Scholar

[10]

Staffjoy.Com. 2017. Staffjoy. (2017). Retrieved August 16, 2017 from https://github.com/Staffjoy/v2

Google Scholar

Cited By

View all

Laigner RZhang ZLiu YGomes LZhou Y(2025)Online Marketplace: A Benchmark for Data Management in MicroservicesProceedings of the ACM on Management of Data10.1145/37096533:1(1-26)Online publication date: 11-Feb-2025
https://dl.acm.org/doi/10.1145/3709653
Zhong CLi SZhang HHuang HYang LCai Y(2025)Refactoring Microservices to Microservices in Support of Evolutionary DesignIEEE Transactions on Software Engineering10.1109/TSE.2024.352348751:2(484-502)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1109/TSE.2024.3523487
Wang SYu DJin BYan WHan Q(2025) ASRMG : Business Topic Clustering‐Based Architecture Smell Refactoring for Microservices Granularity Software: Practice and Experience10.1002/spe.3404Online publication date: 4-Feb-2025
https://doi.org/10.1002/spe.3404
Show More Cited By

Index Terms

Benchmarking microservice systems for software engineering research
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Benchmarking microservice performance: a pattern-based approach
SAC '20: Proceedings of the 35th Annual ACM Symposium on Applied Computing

Benchmarking microservices serves to understand and check their non-functional properties for relevant workloads and over time. Performing benchmarks, however, can be costly: each microservice requires the design and implementation of a benchmark, ...
Graph-based trace analysis for microservice architecture understanding and problem diagnosis
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Microservice systems are highly dynamic and complex. For such systems, operation engineers and developers highly rely on trace analysis to understand architectures and diagnose various problems such as service failures and quality degradation. However, ...
Benchmarking the performance of microservice applications

Application performance is crucial for end user satisfaction. It has therefore been proposed to benchmark new software releases as part of the build process. While this can easily be done for system classes which come with a standard interface, e.g., ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

May 2018

231 pages

ISBN:9781450356633

DOI:10.1145/3183440

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

101
Total Citations
View Citations
1,119
Total Downloads

Downloads (Last 12 months)191
Downloads (Last 6 weeks)25

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Laigner RZhang ZLiu YGomes LZhou Y(2025)Online Marketplace: A Benchmark for Data Management in MicroservicesProceedings of the ACM on Management of Data10.1145/37096533:1(1-26)Online publication date: 11-Feb-2025
https://dl.acm.org/doi/10.1145/3709653
Zhong CLi SZhang HHuang HYang LCai Y(2025)Refactoring Microservices to Microservices in Support of Evolutionary DesignIEEE Transactions on Software Engineering10.1109/TSE.2024.352348751:2(484-502)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1109/TSE.2024.3523487
Wang SYu DJin BYan WHan Q(2025) ASRMG : Business Topic Clustering‐Based Architecture Smell Refactoring for Microservices Granularity Software: Practice and Experience10.1002/spe.3404Online publication date: 4-Feb-2025
https://doi.org/10.1002/spe.3404
Abdelfattah ACerny TYero Salazar JLi XTaibi DSong E(2024)Assessing Evolution of Microservices Using Static AnalysisApplied Sciences10.3390/app14221072514:22(10725)Online publication date: 20-Nov-2024
https://doi.org/10.3390/app142210725
Shetty MChen YSomashekar GMa MSimmhan YZhang XMace JVandevoorde DLas-Casas PGupta SNath SBansal CRajmohan S(2024)Building AI Agents for Autonomous Clouds: Challenges and Design PrinciplesProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698525(99-110)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698525
Cerny TAbdelfattah AGajewski DHarris PGortney MFilkov VRay BZhou M(2024)Is the 3D model the way to go when presenting microservice architecture?Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops10.1145/3691621.3694954(223-227)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691621.3694954
Park JLaddad SBali DZhang WShenker SZaharia MFilkov VRay BZhou M(2024)Everything Everywhere All At Once: Efficient Cross-Service Program Analysis with OverSeerProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops10.1145/3691621.3694937(82-87)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691621.3694937
Sooksatra KChy MArju MCerny TRivas PFilkov VRay BZhou M(2024)Using Static Analysis to Aid Monolith to Microservice System Transformation: Tuning Fuzzy c-Means in a VAE-Based GNN ApproachProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops10.1145/3691621.3694933(43-53)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691621.3694933
Liao YXu MLin YTeoh XXie XFeng RLiaw FZhang HDong JFilkov VRay BZhou M(2024)Detecting and Explaining Anomalies Caused by Web Tamper Attacks via Building Consistency-based NormalityProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695024(531-543)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695024
Zhu ZLee CTang XHe P(2024)HeMiRCA: Fine-Grained Root Cause Analysis for Microservices with Heterogeneous Data SourcesACM Transactions on Software Engineering and Methodology10.1145/367472633:8(1-25)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3674726
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Benchmarking microservice performance: a pattern-based approach

Graph-based trace analysis for microservice architecture understanding and problem diagnosis

Benchmarking the performance of microservice applications

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations