Extending parallel programming patterns with adaptability features
Abstract
Today, all computers have some degree of usable parallelism. Modern computers are explicitly equipped with hardware support for parallelism, such as multiple nodes, multicores, multiple CPUs, and accelerators. At the same time, the Cloud Continuum has become a viable platform for running parallel applications. Building software for these parallel and distributed platforms can be challenging due to the numerous considerations programmers must make during the development process. With this in mind, the high-performance computing literature proposed the concept of parallel patterns to hide some complexities. However, there are no patterns that address the design and creation of adaptive applications. Taking the compute continuum era in mind, we present how adaptability features can be explored within each parallel programming pattern, providing technical details on managing dynamic resources and handling changes in application behavior. In addition to this contribution, we also address practical implications by presenting some frameworks that can be used to implement adaptive applications and examples of using them with the proposed patterns.
References
[1]
Risco S, Moltó G, Naranjo DM, and Blanquer I Serverless workflows for containerised applications in the cloud continuum J. Grid Comput. 2021
[2]
Gill SS, Wu H, Patros P, Ottaviani C, Arora P, Pujol VC, Haunschild D, Parlikad AK, Cetinkaya O, Lutfiyya H, Stankovski V, Li R, Ding Y, Qadir J, Abraham A, Ghosh SK, Song HH, Sakellariou R, Rana O, Rodrigues JJ, Kanhere SS, Dustdar S, Uhlig S, Ramamohanarao K, and Buyya R Modern computing: vision and challenges Telemat. Inform. Rep. 2024 13 100116
[3]
Martín-Álvarez I, Aliaga JI, Castillo M, Iserte S, and Mayo R Dynamic spawning of mpi processes applied to malleability Int. J. High Perform. Comput. Appl. 2023
[4]
Sena, A. C., Ribeiro, F. S., Rebello, V. E., Nascimento, A. P., Boeres, C.: Autonomic malleability in iterative mpi applications. In: 2013 25th International Symposium on Computer Architecture and High Performance Computing. pp. 192–199 (2013)
[5]
Kehrer S and Blochinger W Migrating parallel applications to the cloud: assessing cloud readiness based on parallel design decisions SICS Softw.-Intensive Cyber-Phys. Syst. 2019 34 2–3 73-84
[6]
Wan L, Cui X, Li Y, Zheng W, and Yuan X Heteropp: a directive-based heterogeneous cooperative parallel programming framework Concurr. Comput.: Pract. Exp. 2024
[7]
Galante G and da Rosa Righi R Adaptive parallel applications: from shared memory architectures to fog computing (2002–2022) Clust. Comput. 2022 25 6 4439-4461
[8]
Galante G and da Rosa Righi R Antonopoulos N and Gillam L Exploring cloud elasticity in scientific applications Cloud computing - principles, systems and applications, second edition, computer communications and networks 2017 Springer Cham 101-125
[9]
Czappa F, Calotoiu A, Höhl T, Mantel H, Nguyen T, and Wolf F Design-time performance modeling of compositional parallel programs Parallel Comput. 2021 108 102839
[10]
Mattson T, Sanders B, and Massingill B Patterns for parallel programming 2004 1 Boston Addison-Wesley Professional
[11]
McCool MD, Robison AD, and Reinders J Structured parallel programming patterns for efficient computation 2012 Waltham Elsevier
[12]
Dooley, J.F.: Parallel Design Patterns, Apress, Berkeley, CA. Ch. vol. 6, pp. 191–209 (2017).
[13]
Danelutto M, Mencagli G, Torquati M, González-Vélez H, and Kilpatrick P Algorithmic skeletons and parallel design patterns in mainstream parallel programming Int. J. Parallel Program. 2021 49 2 177-198
[14]
Keutzer, K., Mattson, T.: Our pattern language, (2023)https://patterns.eecs.berkeley.edu/ Accessed June 2023
[15]
Gamma E, Helm R, Johnson R, and Vlissides JM Design patterns: elements of reusable object-oriented software 1994 Boston Addison-Wesley Professional
[16]
Rinaldi, L., Torquati, M., Mencagli, G., Danelutto, M., Menga, T.: Accelerating actor-based applications with parallel patterns. In: 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, PDP 2019, IEEE, pp. 140–147 (2019)
[17]
Wilder B Cloud Architecture Patterns 2012 Sebastopol O’Reilly Media Inc
[18]
Erl T, Cope R, and Naserpour A Cloud computing design patterns 2015 1 Hoboken Prentice Hall Press
[19]
Abuseta Y and Swesi K Design patterns for self adaptive systems engineering Int. J. Softw. Eng. Appl. 2015 6 4 11-28
[20]
Kehrer S and Blochinger W Elastic parallel systems for high performance cloud computing: State-of-the-art and future directions Parallel Processing Letters 2019 29 02 1950006
[21]
Al-Dhuraibi Y, Paraiso F, Djarallah N, and Merle P Elasticity in cloud computing: state of the art and research challenges IEEE Trans. Serv. Comput. 2018 11 2 430-447
[22]
Straesser, M., Grohmann, J., von Kistowski, J., Eismann, S., Bauer, A., Kounev, S.: Why is it not solved yet?: Challenges for production-ready autoscaling. In: Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering, ICPE ’22, ACM, (2022)
[23]
Asanovic K, Bodik R, Demmel J, Keaveny T, Keutzer K, Kubiatowicz J, Morgan N, Patterson D, Sen K, Wawrzynek J, Wessel D, and Yelick K A view of the parallel computing landscape Commun. ACM 2009 52 10 56-67
[24]
Martín G, Singh DE, Marinescu M-C, and Carretero J Enhancing the performance of malleable mpi applications by using performance-aware dynamic reconfiguration Parallel Comput. 2015 46 60-77
[25]
Aldinucci M, Campa S, Danelutto M, Kilpatrick P, and Torquati M Design patterns percolating to parallel programming framework implementation Int. J. Parallel Program. 2014 42 6 1012-1031
[26]
Sterling T, Anderson M, and Brodowicz M High Performance Computing: Modern Systems and Practices 2017 1 San Francisco Morgan Kaufmann Publishers Inc.
[27]
Senger H and da Silva FAB Bounds on the scalability of bag-of-tasks applications running on master-slave platforms Parallel Processing Letters 2012 22 02 1250004
[28]
Danelutto, M., De Matteis, T., Mencagli, G., Torquati, M.: A divide-and-conquer parallel pattern implementation for multicores, in: Proceedings of the 3rd International Workshop on Software Engineering for Parallel Systems, SEPS 2016, Association for Computing Machinery, New York, NY, USA, 2016, p. 10-19.
[29]
Ghit B, Yigitbasi N, Iosup A, and Epema D Balanced resource allocations across multiple dynamic mapreduce clusters SIGMETRICS Perform. Eval. Rev. 2014 42 1 329-341
[30]
Czarnul P Parallel programming for modern high performance computing systems 2018 Boca Raton CRC Press
[31]
Rinaldi L, Torquati M, Sensi DD, Mencagli G, and Danelutto M Improving the performance of actors on multi-cores with parallel patterns Int. J. Parallel Program. 2020 48 4 692-712
[32]
Tasharofi, S., Dinges, P., Johnson, R. E.: Why do scala developers mix the actor model with other concurrency models?, in: G. Castagna (Ed.), ECOOP 2013 - Object-Oriented Programming - 27th European Conference, Montpellier, France, July 1-5, 2013. Proceedings, Vol. 7920 of Lecture Notes in Computer Science, Springer, 2013, pp. 302–326.
[33]
Aubanel E Elements of parallel computing 2016 1 Boca Raton Chapman & Hall/CRC
[34]
Prasad SK, Gupta A, Rosenberg AL, Sussman A, and Weems CC Topics in parallel and distributed computing: introducing concurrency in undergraduate courses 2015 1 San Francisco Morgan Kaufmann Publishers Inc.
[35]
Aliaga JI, Castillo M, Iserte S, Martín-Álvarez I, and Mayo R A survey on malleability solutions for high-performance distributed computing Appl. Sci. 2022
[36]
Baldo, L., Brenner, L., Fernandes, L. G., Fernandes, P., Sales, A.: Performance models for master/slave parallel programs, Electronic Notes in Theoretical Computer Science 128 (4), 101–121 (2005) proceedings of the First International Workshop on Practical Applications of Stochastic Modelling (PASM 2004).
[37]
Wang, X. W., Zhang, J. Liao, H. M., Zha, L. Dynamic split model of resource utilization in mapreduce, in: Proceedings of the Second International Workshop on Data Intensive Computing in the Clouds, DataCloud-SC ’11, Association for Computing Machinery, New York, NY, USA, 2011, p. 21-30.
[38]
Liu Z, Zhang Q, Zhani MF, Boutaba R, Liu Y, and Gong Z Dreams: Dynamic resource allocation for mapreduce with data skew, in IFIP/IEEE International Symposium on Integrated Network Management (IM) 2015 2015 18-26
[39]
Moreno A, Sikora A, César E, Sorribes J, and Margalef T Hedpm: Load balancing of linear pipeline applications on heterogeneous systems J. Supercomput. 2017 73 9 3738-3760
[40]
Francesquini, E., Goldman, A., Méhaut, J.-F.: Improving the performance of actor model runtime environments on multicore and manycore platforms. In: Proceedings of the 2013 Workshop on Programming Based on Actors, Agents, and Decentralized Control, AGERE! 2013, Association for Computing Machinery, pp. 109-114. New York, NY, USA (2013)
[41]
Mo-Hellenbrand, A.: Resource-aware and elastic parallel software development for distributed-memory hpc systems, Ph.D. thesis, Technischen Universität München (2019). https://mediatum.ub.tum.de/doc/1471007/1471007.pdf
[42]
Rajan, D., Thrasher, A., Abdul-Wahid, B., Izaguirre, J.A., Emrich, S., Thain, D., Case studies in designing elastic applications. In: 13th IEEE/ACM International Symposium on Cluster. Cloud, and Grid Computing vol. 2013, pp. 466–473 (2013).
[43]
da Rosa Righi R, Rodrigues VF, da Costa CA, Galante G, Bona LCED, and Ferreto TC Autoelastic: automatic resource elasticity for high performance applications in the cloud IEEE Trans. Cloud Comput. 2016 4 1 6-19
[44]
Houzeaux G, Badia R, Borrell R, Dosimont D, Ejarque J, Garcia-Gasulla M, and López V Dynamic resource allocation for efficient parallel cfd simulations Comput. Fluids 2022 245 105577
[45]
Munhoz, V., Castro, M., Mendizabal, O.: Strategies for fault-tolerant tightly-coupled hpc workloads running on low-budget spot cloud infrastructures. In: 2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD). pp. 263–272 (2022).
[46]
Galante, G., Bona, L. C. E.: Supporting elasticity in openmp applications. In: Proceedings of the 2014 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP ’14, IEEE Computer Society. pp. 188-195. USA (2014).
[47]
Zhao, J., Gao, X., Li, Y.: Research on elastic extension of multi type resources for openmp program. In: 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). pp. 971–978. (2022).
[48]
Wrzesinska, G., van Nieuwpoort, R., Maassen, J., Bal, H.: Fault-tolerance, malleability and migration for divide-and-conquer applications on the grid. In: 19th IEEE International Parallel and Distributed Processing Symposium, pp. 10-pp. (2005).
[49]
Aubin, M. R., da Rosa Righi, R., Valiati, V. H., da Costa, C. A., Antunes, R. S., Galante, G.: Helastic: On combining threshold-based and serverless elasticity approaches for optimizing the execution of bioinformatics applications. J. Comput. Sci. vol. 53, p. 101407. (2021).
[50]
Fadika, Z., Govindaraju, M., Delma: Dynamically elastic mapreduce framework for cpu-intensive applications. In: 11th IEEE/ACM International Symposium on Cluster. Cloud and Grid Computing 2011. pp. 454–463 (2011).
[51]
Gordon, A. W., Lu, P.: Elastic phoenix: Malleable mapreduce for shared-memory systems In: E. R. Altman, W. Shi (eds.), Network and Parallel Computing - 8th IFIP International Conference, NPC 2011, Vol. 6985 of Lecture Notes in Computer Science, pp. 1–16. Springer (2011).
[52]
Chalvantzis, N., Konstantinou, I., Kozyris, N., Bbq: Elastic mapreduce over cloud platforms. In: 17th IEEE/ACM International Symposium on Cluster. Cloud and Grid Computing (CCGRID) 2017, pp. 766–771 (2017).
[53]
Meyer V, Rodrigues VF, Righi RDR, Costa CAD, Galante G, and Both CB Pipel: exploiting resource reorganisation to optimise performance of pipeline-structured applications in the cloud Int. J. Comput. Syst. Eng. 2019
[54]
Vogel, A., Griebler, D., De Sensi, D., Danelutto, M., Fernandes, L. G.: Autonomic and latency-aware degree of parallelism management in spar, in: G. Mencagli, D. B. Heras, V. Cardellini, E. Casalicchio, E. Jeannot, F. Wolf, A. Salis, C. Schifanella, R. R. Manumachu, L. Ricci, M. Beccuti, L. Antonelli, J. D. Garcia Sanchez, S. L. Scott (eds.), Euro-Par 2018: Parallel Processing Workshops, pp. 28–39 Springer International Publishing, Cham (2019)
[55]
Sang, B., Roman, P.-L., Eugster, P., Lu, H., Ravi, S., Petri, G.: Plasma: Programmable elasticity for stateful cloud computing applications. In: Proceedings of the Fifteenth European Conference on Computer Systems, EuroSys ’20, Association for Computing Machinery, New York, NY, USA, pp. 1–15. (2020).
[56]
Galante G and Bona LCED A programming-level approach for elasticizing parallel scientific applications J. Syst. Softw. 2015 110 239-252
[57]
De Sensi D, De Matteis T, and Danelutto M Simplifying self-adaptive and power-aware computing with nornir Future Gener. Comput. Syst. 2018 87 136-151
[58]
Mishra, N., Imes, C., Lafferty, J. D., Hoffmann, H.: Caloree: Learning control for predictable latency and low energy. In: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’18, Association for Computing Machinery, New York, NY, USA, pp. 184-198. (2018).
[59]
Pervaiz, A., Yang, Y. H., Duracz, A., Bartha, F., Sai, R., Imes, C., Cartwright, R., Palem, K., Lu, S., Hoffmann, H.: Goal: Supporting general and dynamic adaptation in computing systems. In: Proceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2022, Association for Computing Machinery, New York, NY, USA, pp. 16-32. (2022).
[60]
Rubio, J., Bilbao, C., Saez, J. C., Prieto-Matias, M.: Exploiting elasticity via os-runtime cooperation to improve cpu utilization in multicore systems. In: 2024 32nd Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp. 35–43. (2024).
[61]
Eberius, D., Rahman, M. W.-U., Ozog, D.: Evaluating the potential of elastic jobs in hpc systems, in: Proceedings of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, SC-W ’23, Association for Computing Machinery, New York, NY, USA, p. 1324-1333. (2023).
[62]
Galante, G., Bona, L. C. E. D., Schepke, C.: Improving OLAM with cloud elasticity. In: Computational Science and Its Applications – ICCSA 2014, Springer, pp. 46–60. (2014).
[63]
Moretti C, Thrasher A, Yu L, Olson M, Emrich S, and Thain D A framework for scalable genome assembly on clusters, clouds, and grids IEEE Trans. Parallel Distrib. Syst. 2012 23 12 2189-2197
Index Terms
- Extending parallel programming patterns with adaptability features
Index terms have been assigned to the content through auto-classification.
Recommendations
Teaching parallel design patterns to undergraduates in computer science
SIGCSE '14: Proceedings of the 45th ACM technical symposium on Computer science educationThe industry shift to emerging forms of parallel and distributed computing (PDC), including multi-core CPUs, cloud computing, and general-purpose use of GPUs, have naturally led to increased presence of PDC elements undergraduate Computer Science ...
A programming-level approach for elasticizing parallel scientific applications
We present an approach for exploring cloud elasticity in scientific applications.Elasticity control is embedded in the application source code.Elastic applications can adapt their execution environment according to demands.This approach creates a new ...
Using generative design patterns to generate parallel code for a distributed memory environment
Proceedings of the ACM SIGPLAN symposium on principles and practice of parallel programming (PPoPP 2003) and workshop on partial evaluation and semantics-based program manipulation (PEPM 2003)A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 15 June 2024
Accepted: 04 June 2024
Revision received: 29 May 2024
Received: 05 March 2024
Author Tags
Qualifiers
- Review-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025