-
Notifications
You must be signed in to change notification settings - Fork 162
Open
Description
Describe the bug
A fatal JVM error occurs I try to run multiple SubInterpreters in parallel (each in its own thread) when OWN_GIL is set to true.
The error trace I see is:
[thread 219 also had an error][thread 218 also had an error]
[thread 214 also had an error]
[thread 212 also had an error]
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb)[thread 220 also had an error]
at pc=0x00000041a4cc14c5, pid=64, tid=222
#
# JRE version: OpenJDK Runtime Environment Corretto-21.0.5.11.1 (21.0.5+11) (build 21.0.5+11-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-21.0.5.11.1 (21.0.5+11-LTS, mixed mode, emulated-client, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# C [libpython3.12.so.1.0+0x1db4c5][thread 215 also had an error]
[thread 216 also had an error]
[thread 213 also had an error]
_PyObject_Malloc+0x75
#
# Core dump will be written. Default location: /app/core
#
[thread 221 also had an error]
# An error report file with more information is saved as:
# /app/hs_err_pid64.log
[thread 211 also had an error]
[thread 217 also had an error]
[12.100s][warning][os] Loading hsdis library failed
#
# If you would like to submit a bug report, please visit:
# https://github.com/corretto/corretto-21/issues/
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
To Reproduce
The error can be consistently be reproduced from the following code snippet in a linux/amd64
architecture. Couldn't reproduce on linux/aarch64
.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import jep.Interpreter;
import jep.JepConfig;
import jep.MainInterpreter;
import jep.SubInterpreter;
import jep.SubInterpreterOptions;
public class Application {
private static final String JEP_PATH_ENV_VAR = "JEP_PATH";
/**
* @param args command line arguments for the application.
*/
public static void main(String... args) {
ExecutorService executor = Executors.newFixedThreadPool(12);
MainInterpreter.setJepLibraryPath(System.getProperty("tp_lib.python.jep-path", System.getenv(JEP_PATH_ENV_VAR)));
JepProvider provider = new JepProvider();
List<CompletableFuture<?>> futures = new ArrayList<>();
for (int i = 0; i < 1000; i++) {
futures.add(CompletableFuture.supplyAsync(() -> {
System.out.println(provider.getInterpreter().getValue("1 + 2"));
return null;
}, executor));
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
}
public static class JepProvider {
private static final JepConfig JEP_CONFIG = new JepConfig()
.setSubInterpreterOptions(SubInterpreterOptions.legacy().setOwnGIL(true));
private final ThreadLocal<Interpreter> subInterpreter = ThreadLocal.withInitial(() -> new SubInterpreter(JEP_CONFIG));
public Interpreter getInterpreter() {
return this.subInterpreter.get();
}
}
}
Expected behavior
Expect all executions to finish successfully without any errors.
Environment (please complete the following information):
- OS Platform, Distribution, and Version: Linux, Wolfi distro
bash-5.2# uname -a
Linux cce8f5bd761b 6.8.0-50-generic #51-Ubuntu SMP PREEMPT_DYNAMIC Sat Nov 9 18:03:35 UTC 2024 x86_64 Linux
- Python Distribution and Version: Python 3.12.7 (compiler
GCC 14.2.0
, github tagtags/v3.12.7-1-g9c9fef5-dirty:9c9fef5
). Also could reproduce on 3.13.1 - Java Distribution and Version:
OpenJDK 64-Bit Server VM Corretto-21.0.5.11.1 (21.0.5+11-LTS, mixed mode, emulated-client, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
- Jep Version:
4.2.2
- Python packages used (e.g. numpy, pandas, tensorflow): None
Additional context
Please don't hesitate to reach out
- If can provide more context about the error.
- If can help to reproduce the problem (could create a repo with a docker image to make this very straightforward).
- If can personally contribute to JEP to somehow fix this issue.
or if I can help in any other way.
Metadata
Metadata
Assignees
Labels
No labels