Parallel virtual machines with RPython
Abstract
The RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support parallel execution since the framework relies on a Global Interpreter Lock (GIL): PyPy serialises the execution of multi-threaded Python programs.
We describe the rationale and design of a new parallel execution model for RPython that allows the generation of parallel virtual machines while leaving the language semantics unchanged. This model then allows different implementations of concurrency control, and we discuss an implementation based on a GIL and an implementation based on Software Transactional Memory (STM).
To evaluate the benefits of either choice, we adapt PyPy to work with both implementations (GIL and STM). The evaluation shows that PyPy with STM improves the runtime of a set of multi-threaded Python programs over PyPy with a GIL by factors in the range of 1.87x up to 5.96x when executing on a processor with 8 cores.
References
[1]
“The IronPython Project,” http://ironpython.net/.
[2]
“The Jython Project,” http://www.jython.org/.
[3]
Nicholas Riley and Craig Zilles, “Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution,” OOPSLA ’06, pp. 998–1008, ACM.
[4]
Rei Odaira, Jose G. Castanos, and Hisanobu Tomari, “Eliminating Global Interpreter Locks in Ruby Through Hardware Transactional Memory,” PPoPP ’14, pp. 131–142, ACM.
[5]
Fuad Tabba, “Adding Concurrency in Python Using a Commercial Processor’s Hardware Transactional Memory Support,” SIGARCH Comput. Archit. News, vol. 38, no. 5, pp. 12–19, Apr. 2010.
[6]
Yu David Liu, Xiaoqi Lu, and Scott F. Smith, “Coqa: Concurrent Objects with Quantized Atomicity,” CC’08/ETAPS’08, pp. 260–275, Springer-Verlag.
[7]
Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, and Armin Rigo, “Tracing the Meta-level: PyPy’s Tracing JIT Compiler,” ICOOOLPS ’09, pp. 18–25, ACM.
[8]
“CPython,” https://www.python.org/.
[9]
“The JRuby Project,” http://jruby.org/.
[10]
Colin Blundell, E. Lewis, and Milo Martin, “Unrestricted Transactional Memory: Supporting I/O and System Calls Within Transactions,” Technical Reports (CIS), May 2006.
[11]
“PyPy Source on Bitbucket,” https://bitbucket.org/ pypy/pypy.
[12]
Remigius Meier, Armin Rigo, and Thomas Gross, “A Transactional Memory System for Parallel Python,” https://bitbucket.org/pypy/extradoc/raw/ 12bda771698925b8296808959e7b830aef8b78b8/ talk/dls2014/paper/paper.pdf, Aug. 2014.
[13]
“PyPy Benchmarks on Bitbucket,” https://bitbucket. org/pypy/benchmarks.
[14]
Christian Wimmer and Thomas Würthinger, “Truffle: A Selfoptimizing Runtime System,” SPLASH ’12, pp. 13–14, ACM.
Index Terms
- Parallel virtual machines with RPython
Recommendations
Virtual machine design for parallel dynamic programming languages
To leverage the benefits of modern hardware, dynamic languages must support parallelism, and parallelism requires a virtual machine (VM) capable of parallel execution — a parallel VM. However, unrestricted concurrency and the dynamism of dynamic ...
Parallel virtual machines with RPython
DLS 2016: Proceedings of the 12th Symposium on Dynamic LanguagesThe RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support ...
A way forward in parallelising dynamic languages
ICOOOLPS '14: Proceedings of the 9th International Workshop on Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems PLEDynamic languages became very popular in recent years. At some point, the need for concurrency arose, and many of them made the choice to use a single global interpreter lock (GIL) to synchronise the interpreter in a multithreading scenario. This choice,...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
February 2017
131 pages
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 November 2016
Published in SIGPLAN Volume 52, Issue 2
Check for updates
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations9Total Citations
- 187Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)2
Reflects downloads up to 19 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in