Nothing Special   »   [go: up one dir, main page]

Skip to main content

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15220))

Included in the following conference series:

  • 93 Accesses

Abstract

Ensemble-based component systems have been used for many years to develop collective adaptive systems (CAS). The DEECo component model offers a framework for modeling and implementing ensemble-based component systems. Being expressive enough and having semantics specifically tailored towards dynamically evolving systems, DEECo has proven to be fairly powerful in modeling complex and dynamic architectures. At the same time, its specific semantics turned out to be a hurdle for newcomers when expressing their design intention and understanding its implications and side effects. We see quite a potential in employing large language models (LLMs) to simplify creating and refining the DEECo architectures. Since this constitutes a large research scope, we focus in this paper on initial experiments demonstrating how well generic LLMs understand the advanced concepts of ensemble-based CAS embodied in DEECo and asses what expectation from LLMs is realistic in this context. Our results indicate that LLMs can indeed understand ensemble-based architectures, but it depends on the form in which the architecture is presented in the textual form. Using external DSL, which is very self-explanatory gave good results out of the box. Specifications embedded in existing programming languages needed prior explanation of how to interpret them.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.ascens-ist.eu/.

  2. 2.

    http://www.afarcloud.eu/.

  3. 3.

    https://trust40.ipd.kit.edu/.

  4. 4.

    https://www.scala-lang.org/.

  5. 5.

    https://platform.openai.com/docs/models.

  6. 6.

    https://github.com/smartarch/llm-deeco.

  7. 7.

    https://www.tiobe.com/tiobe-index/.

  8. 8.

    https://spectrum.ieee.org/the-top-programming-languages-2023.

References

  1. Ahmad, A., Waseem, M., Liang, P., et al.: Towards human-bot collaborative software architecting with ChatGPT. In: Proceedings of EASE 2023, Oulu, Finland, pp. 279–285 (2023). https://doi.org/10.1145/3593434.3593468

  2. Al Ali, R., Bures, T., Hnetynka, P., et al.: Dynamic security specification through autonomic component ensemble. In: Proceedings of ISoLA 2018, Limassol, Cyprus. LNCS. Springer (2018). https://doi.org/10.1007/978-3-030-03424-5_12

  3. Al Ali, R., Bures, T., Hnetynka, P., et al.: Toward autonomically composable and context-dependent access control specification through ensembles. Int. J. Softw. Tools Technol. Transfer 22(4), 511–522 (2020). https://doi.org/10.1007/s10009-020-00556-1

  4. Brown, T.B., Mann, B., Ryder, N., et al.: Language models are few-shot learners. In: Proceedings NIPS 2020, Vancouver, Canada, pp. 1877–1901 (2020) https://doi.org/10.1007/s10009-020-00556-1

  5. Bures, T., Gerostathopoulos, I., Hnetynka, P., et al.: DEECO: an ensemble-based component system. In: Proceedings of CBSE 2013, Vancouver, Canada, pp. 81–90 (2013). https://doi.org/10.1145/2465449.2465462

  6. Clark, P., Cowhey, I., Etzioni, O., et al.: Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge, (2018). https://doi.org/10.48550/arxiv1803.05457. arXiv: 1803.05457 [cs.AI]

  7. Cobbe, K., Kosaraju, V., Bavarian, M., et al.: Training Verifiers to Solve Math Word Problems, (2021). https://doi.org/10.48550/arXiv.2110.14168arXiv: 2110.14168 [cs.LG]

  8. Dhar, R., Vaidhyanathan, K., Varma, V.: Can LLMs Generate Architectural Design Decisions? -An Exploratory Empirical study (2024). https://doi.org/10.48550/arxiv.2403.01709. arXiv: 2403.01709 [cs.SE]. (accepted to ICSA 2024)

  9. Donakanti, R., Jain, P., Kulkarni, S., et al.: Reimagining Self-Adaptation in the Age of Large Language Models (2024). https://doi.org/10.48550/arxiv.2404.09866arXiv: 2404.09866 [cs.SE]

  10. Hendrycks, D., Burns, C., Basart, S., et al.: Measuring Massive Multitask Language Understanding, (2021). https://doi.org/10.48550/arxiv.2009.03300arXiv: 2009.03300 [cs.CY]

  11. Hnetynka, P., Bures, T., Gerostathopoulos, I., et al.: Using component ensembles for modeling autonomic component collaboration in smart farming. In: Proceedings of SEAMS 2020, Seoul, Korea, pp. 156–162. ACM (2020). https://doi.org/10.1145/3387939.3391599

  12. Krijt, F., Jiracek, Z., Bures, T., et al.: Intelligent ensembles - a declarative group description language and java framework. In: Proceedings of SEAMS 2017, Buenos Aires, Argentina, pp. 116–122 (2017). https://doi.org/10.1109/SEAMS.2017.17

  13. Li, B., Wu, W., Tang, Z., et al.: DevBench: a comprehensive benchmark for software development (2024). https://doi.org/10.48550/arxiv.2403.08604arXiv: 2403.08604 [cs.CL]

  14. Liang, P., Bommasani, R., Lee, T., et al.: Holistic Evaluation of Language Models (2023). https://doi.org/10.48550/arxiv.2211.09110. arXiv:2211. 09110 [cs.CL]

  15. Mernik, M., Heering, J., Sloane, A.M.: When and how to develop domain-specific languages. ACM Comput. Surv. 37(4), 316–344 (2005). https://doi.org/10.1145/1118890.1118892

  16. Sakaguchi, K., Bras, R.L., Bhagavatula, C., et al.: WinoGrande: an adversarial winograd schema challenge at scale (2019). https://doi.org/10.48550/arxiv.1907.10641.arXiv: 1907.10641 [cs.CL]

  17. Sutawika, L., Schoelkopf, H., Gao, L., et al.: EleutherAI/lm-evaluation-harness: v0.4.2, version v0.4.2 (2024). https://doi.org/10.5281/zenodo.10829972

  18. Topfer, M., Abdullah, M., Krulis, M., et al.: ML-DEECo: a machine-learning- enabled framework for self-organizing components. In: Companion proceedings of ACSOS 2022, Virtual event, pp. 66–69. IEEE (2022). https://doi.org/10.1109/ACSOSC56246.2022.00033

  19. Wirsing, M., H–lzl, M., Koch, N., et al.: Software Engineering for Collective Autonomic Systems (The ASCENS Approach). Springer (2015)

    Google Scholar 

  20. Zellers, R., Holtzman, A., Bisk, Y., et al.: HellaSwag: Can a Machine Really Finish Your Sentence? (2019). https://doi.org/10.48550/arxiv.1905.07830arXiv:1905.07830 [cs.CL]

Download references

Acknowledgment

This work has been partially supported by the EU project ExtremeXP grant agreement 101093164, partially by the Charles University Grant Agency projects 408622 and 269723, and partially by Charles University institutional funding SVV 260698.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Petr Hnětynka .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Töpfer, M., Khalyeyev, D., Bureš, T., Hnětynka, P., Plášil, F. (2025). How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Rigorous Engineering of Collective Adaptive Systems. ISoLA 2024. Lecture Notes in Computer Science, vol 15220. Springer, Cham. https://doi.org/10.1007/978-3-031-75107-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-75107-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-75106-6

  • Online ISBN: 978-3-031-75107-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics