Repository navigation
Advise a pinned jep install that works, also for externally managed Pythons - #67
Merged
Merged
Conversation
runProcess wrote the whole input to the process before reading its output, so a process writing more than the pipe buffer (about 64 KB) to stdout before consuming all of its stdin blocked both sides forever. PythonDependencies pipes the script metadata and the imported module names, tens of KB, into the PLAN process this way. The input is now written, and the output read, on their own threads, while the calling thread waits for the process, so it can still be interrupted, destroying the process. A broken pipe of the input is no longer an error: when the process ends without reading its input, e.g. PLAN failing on an import, the caller gets its exit code and output, and so a PythonDependencyException with Python's traceback, instead of a bare IOException: Broken pipe. The test of destroying the process on a broken pipe is replaced by one destroying it when interrupted, the remaining way to leave runProcess while the process is alive. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
A failure of the output reader other than an IOException, e.g. an OutOfMemoryError on a huge output, is now rethrown on the calling thread, instead of surfacing as a NullPointerException of the missing output. Without input, the standard input is just closed, with no thread to write it. The interrupt test stops waiting for the pid file when the runner has already ended, so that it fails instead of hanging the suite, and the echo test asserts the length and content of the 10 MB output as counts, so that power-assert does not render it whole on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
The environment probe, the PLAN and the pip fallback started Python with -c and -m pip, which put the JVM's working directory first on sys.path. A json.py, tomllib.py, jep/ or pip/ there - likely in an agent's working directory full of generated files - replaced the module the installer imports, running unintended code, or made the probe report the wrong jep. -P would fix it, but it is unknown before Python 3.11, and 3.10 is supported. -I would ignore every PYTHON* variable, also the PYTHONUTF8 the process streams depend on, PYTHONPATH and the user site-packages, which the embedded interpreter still sees. So the code started with -c first removes the '' entry of the working directory from sys.path, before any import, and pip is run with runpy instead of -m. PIP_* variables and pip config files keep working, as nothing about the environment changes. uv is not changed: it queries the target interpreter itself, in isolated mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
The KDoc of pythonCommand now says precisely that only the implicit working directory entry of -c is removed, while the entries the user configured, e.g. with PYTHONPATH, are kept, as the embedded interpreter sees them too. The tests of PLAN without tomllib, the only ones of its Python < 3.11 behavior, now run it with the same prologue as production. The probe shadowing test runs its reference probe before creating the temporary directory, so that a failure of it does not leak the directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
PLAN ran with its standard error merged into its standard output, which was then parsed line by line as the install plan. Python block-buffers stdout in a pipe, so anything written to stderr during the run, e.g. a warning without a trailing line break, lands before or inside a plan line in the merged stream, corrupting or dropping a requirement, the same class of bug as #4 fixed for the environment probe. PLAN now runs with mergeErrorStream = false, only stdout is parsed, and stderr, where sys.exit writes PLAN's error messages, is included in the PythonDependencyException on failure. The uv / pip install runs keep the merged output, as it is only shown in the error message, never parsed. plan() takes the command as a defaulted parameter, a test seam for a PLAN run which writes to stderr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…ms (#14) The PLAN failure and the environment probe failure both combined stderr and stdout into the error message, in opposite orders; ProcessResult.failureOutput now does it once, stderr first, as it tells why the process failed, e.g. the message of sys.exit. The tests of PLAN without tomllib, or without any TOML parser, ran it with stderr merged into stdout, so their assertions on the sys.exit message passed only because of the merging, while plan() reads the streams separately. They now run through plan(), with the command as the test seam, so they check the message of the exception the user gets. The tests in a working directory with shadowing modules read stdout separately as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
When the stdout reader thread failed, it stored the failure and ended, while the calling thread kept waiting for the process. Closing the output pipe usually ends a writing process with a broken pipe, but a process which survives it, or never writes again, made runProcess wait forever and never rethrow the failure, where reading the output on the calling thread, before #13, would have thrown at once. The reader now destroys the process on a failure, so that the wait ends and the failure is rethrown. The output reading is an internal parameter of runProcess, a seam for the test to make it fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
The probe printed its values one per line and Kotlin took them by line position, so anything else writing to stdout - a sitecustomize module or a .pth file - shifted every value, and an empty first value, e.g. an empty sys.executable, was trimmed away, leaving too few lines and an IndexOutOfBoundsException instead of the documented IllegalStateException. The probe now writes its values separated by NUL, which no path can contain, between unique begin and end sentinel lines, preceded by an empty line in case the noise before does not end with a newline, and as bytes, so without newline translation. Kotlin takes the values only from between the last begin and the following end sentinel, and fails with an IllegalStateException showing the output when there are not exactly four. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
) An empty sys.executable was parsed correctly but then accepted silently: the embedded CPython got an empty program name, so it ignored the pyvenv.cfg of a virtual environment, and installing dependencies failed later with a bare IOException of running "". The probe now fails right away with an IllegalStateException telling that sys.executable is empty. The probe wrote to sys.stdout.buffer, which a sitecustomize replacing sys.stdout with a wrapper does not have, so it now writes to the original sys.__stdout__, after flushing both, which also bypasses any redirection of sys.stdout. When the output has no probe values, the message now names the Python executable and shows its stderr as well, as it tells why; the parsing returns null for the caller to report it, and is unit-tested on its own for output around the values, repeated markers, an empty value, a missing end marker, and a wrong number of values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…bytes (#21) The embedded interpreter used the encoding of the JVM's locale, so under LC_ALL=C with PYTHONUTF8=0, or a cp1252 Windows locale, printing non-ASCII text or reading a UTF-8 file failed. Jep's PyConfig has no UTF-8 mode setter, but CPython parses `-X utf8` from the argv set with PyConfig.setArgv before decoding the other values, which enables it, also over PYTHONUTF8=0. Under a non-UTF-8 POSIX locale, e.g. ISO-8859-1 with a home like /home/josé, a path is the locale's bytes, surrogate-escaped by Python in UTF-8 mode: - the environment probe failed encoding such a path to UTF-8; it now writes each value as the hex of its os.fsencode bytes, which Kotlin decodes with sun.jnu.encoding (UTF-8 on Windows), so it names the same file; - the dependency directory, a Java String passed by Jep as UTF-8, was another path in the embedded sys.path; it is now appended as os.fsdecode of its bytes. In argv, to PLAN and pip, it already round-trips unchanged. A non-ASCII path of the Python executable cannot be fixed here, as Jep passes the program name as UTF-8 too; the README documents it. PythonTest compared sys.path of a raw ProcessBuilder decoded as UTF-8; it now uses runProcess and compares ascii() of the paths on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Review findings of the UTF-8 mode fix: - The os.fsdecode conversion of the dependency directory had no regression test; PythonLocaleTest now forks a JVM in an ISO-8859-1 locale, created with localedef and LOCPATH (skipped without localedef), importing a module from a non-ASCII dependency directory. The forked JVM is awaited with a timeout, its output going to a file, so a stuck JVM cannot hang the build. - A probe value which is not valid in the file system encoding of the JVM was replaced with U+FFFD, naming another path, and a sign was accepted in its hex digits; it is now decoded strictly, and the probe fails clearly. - The UTF-8 mode depends on Jep setting argv before the other PyConfig values, so a new interpreter now checks sys.flags.utf8_mode, failing clearly if a future Jep breaks it. - The README documents that an inherited PYTHONIOENCODING still wins for the embedded streams, and that the ASCII limitation of the executable's path applies only to non-UTF-8 POSIX locales; CLAUDE.md keeps only the gotchas. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…err (#19) Any non-zero exit of the environment probe was reported as a missing jep package with a pip install remedy, also when Python itself could not run, e.g. a pyenv shim of a version which is not installed. The probe now exits with a dedicated code (83) when it cannot find jep, and only this one gets the jep headline; any other exit says Python cannot be run, with its exit code and output. The stderr of a successful probe was dropped, so a warning like "Could not find platform libraries" of a wrong PYTHONHOME left no trace when the embedded interpreter, configured the same way, failed later. It is kept in PythonProbe.error, and added to the failure of creating the first interpreter. The main code has no logging, so it is not logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…on (#19) Jep 4.3 rethrows the failure of initializing CPython as a plain java.lang.Error, and every later attempt as another Error ("previously failed to initialize"), which the catch of JepException and LinkageError missed, so the probe's stderr was never attached where it matters, e.g. a broken encodings module. EmbeddedPythonStarter now creates the interpreters of Python: any failure before the first interpreter exists, and any Error of Jep at any time, is a startup failure, always wrapped in an IllegalStateException with the original as cause, so its type does not depend on whether the probe wrote anything; the probe's stderr is offered as a possible hint. A later, unrelated failure of an interpreter is thrown as is, instead of blaming an old warning. The UTF-8 mode check now runs before the interpreter is kept, closing it on failure, as a later call skipped the check of the kept interpreter. The probe names a jep module shadowing the package, as "pip install jep" would only answer "Requirement already satisfied"; the failure messages no longer end with an empty line without output; the kept stderr is limited to its first 8 KiB, where a startup warning of Python is, as it lives as long as the JVM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
The advice of a missing jep package, and of a missing native Jep library, was `python -m pip install jep`, which fails on an externally managed Python (PEP 668, e.g. Homebrew or most Linux distributions), installs the latest jep instead of the one matching the org.ninia:jep library, and does not tell that building jep from its source needs a C compiler and a JDK. Both messages now advise creating a virtual environment, installing jep==<version> into it, and pointing to it with VIRTUAL_ENV or PythonEnvironment.pythonExecutable, with the platform's venv layout and the value of JAVA_HOME. The version is read at runtime from the pom.properties of the org.ninia:jep jar on the classpath, so that it matches the library actually in use, also when an application resolves another version than this build; without it, the advice falls back to an unpinned jep. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…19) Nothing tested the wiring in Python.call, so a forked JVM now starts the embedded CPython with a fake encodings package, which warns and fails only when not run with -c, as the probe is, and the test checks that the IllegalStateException carries the probe's warning; it fails when Python creates the SharedInterpreter directly. Jep's MainInterpreter is a singleton, so once an interpreter is created any failure, also an Error like NoClassDefFoundError, is not a startup failure, and is thrown as is. Jep wraps the failure of initializing CPython in an Error whose message is the toString() of its cause, so the headline takes the message of the cause instead of repeating its class. The kept stderr of the probe is limited including the note of truncation, without splitting a surrogate pair. The message of a probe without values no longer ends with an empty line without output, and a jep package directory which cannot be listed is reported naming it, instead of a raw NIO exception. The Python KDoc no longer promises a cause for the failures of resolving the environment, which have none. The probe's warning is attached only to a failure of the embedded CPython to start, and it is not logged, as the main code has no logging, so a later failure caused by the same misconfiguration carries no hint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Keeps the one jepLibrary function of #19, with its wrapping of a directory which cannot be listed, giving the pinned virtual environment advice of #37 in both of its messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…naged Python (#37) The advice of a missing jep sent every user to a new ~/.venvs/jep, also when the Python already was a virtual environment, e.g. of VIRTUAL_ENV, holding the other packages of the user, where a pip install of jep just works. The probe now tells whether the Python is externally managed (PEP 668): a base Python with an EXTERNALLY-MANAGED file in its standard library; also when jep is missing, with its own exit code 84, as it exits before writing its values. Only then is a virtual environment advised, otherwise the pinned pip install into the Python itself. The virtual environment is an absolute path of user.home, quoted when it has a space, as ~ and %USERPROFILE% are expanded only by a shell, not by an IDE run configuration or by PowerShell. As PyPI has no wheels of jep, it is always built from its source, so the advice tells about the Python headers, and that JAVA_HOME is optional where javac on the PATH is found. A broken jep is reinstalled with --force-reinstall --no-cache-dir, as a plain install would only tell that the requirement is already satisfied. The jep version is read lazily, a malformed pom.properties giving no version instead of an ExceptionInInitializerError, and the tests read the version catalog themselves, so that they also run outside Gradle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
…vice (#37) Following the advice still failed in a few common setups: - on stock Debian/Ubuntu, creating the virtual environment fails without the python3-venv package, so it is named next to python3-dev, - a virtual environment of uv has no pip, so uv pip install is advised too, - PowerShell takes a quoted command for a string, so on Windows the advice tells to prefix it with &, keeping the quoting which cmd needs. The README no longer gives the virtual environment as an example of PythonEnvironment.pythonExecutable, which must be its Python, and tells that the version of the jep package must match the library. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Base automatically changed from
issue-19-probe-failure-message
to
issue-21-non-utf8-locale
November 10, 2026 18:04
Base automatically changed from
issue-21-non-utf8-locale
to
issue-15-probe-parsing
November 10, 2026 18:05
Base automatically changed from
issue-15-probe-parsing
to
issue-14-plan-stderr
November 10, 2026 21:18
Base automatically changed from
issue-14-plan-stderr
to
issue-24-installer-sys-path
November 10, 2026 21:19
Base automatically changed from
issue-24-installer-sys-path
to
issue-13-runprocess-deadlock
November 10, 2026 21:20
* Do not import from the working directory in installer Pythons (#24) The environment probe, the PLAN and the pip fallback started Python with -c and -m pip, which put the JVM's working directory first on sys.path. A json.py, tomllib.py, jep/ or pip/ there - likely in an agent's working directory full of generated files - replaced the module the installer imports, running unintended code, or made the probe report the wrong jep. -P would fix it, but it is unknown before Python 3.11, and 3.10 is supported. -I would ignore every PYTHON* variable, also the PYTHONUTF8 the process streams depend on, PYTHONPATH and the user site-packages, which the embedded interpreter still sees. So the code started with -c first removes the '' entry of the working directory from sys.path, before any import, and pip is run with runpy instead of -m. PIP_* variables and pip config files keep working, as nothing about the environment changes. uv is not changed: it queries the target interpreter itself, in isolated mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Apply the review of the installer sys.path fix (#24) The KDoc of pythonCommand now says precisely that only the implicit working directory entry of -c is removed, while the entries the user configured, e.g. with PYTHONPATH, are kept, as the embedded interpreter sees them too. The tests of PLAN without tomllib, the only ones of its Python < 3.11 behavior, now run it with the same prologue as production. The probe shadowing test runs its reference probe before creating the temporary directory, so that a failure of it does not leak the directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Parse only the standard output of PLAN (#50) * Parse only the standard output of PLAN, not merged with stderr (#14) PLAN ran with its standard error merged into its standard output, which was then parsed line by line as the install plan. Python block-buffers stdout in a pipe, so anything written to stderr during the run, e.g. a warning without a trailing line break, lands before or inside a plan line in the merged stream, corrupting or dropping a requirement, the same class of bug as #4 fixed for the environment probe. PLAN now runs with mergeErrorStream = false, only stdout is parsed, and stderr, where sys.exit writes PLAN's error messages, is included in the PythonDependencyException on failure. The uv / pip install runs keep the merged output, as it is only shown in the error message, never parsed. plan() takes the command as a defaulted parameter, a test seam for a PLAN run which writes to stderr. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Share the failure output of a process, test PLAN without merged streams (#14) The PLAN failure and the environment probe failure both combined stderr and stdout into the error message, in opposite orders; ProcessResult.failureOutput now does it once, stderr first, as it tells why the process failed, e.g. the message of sys.exit. The tests of PLAN without tomllib, or without any TOML parser, ran it with stderr merged into stdout, so their assertions on the sys.exit message passed only because of the merging, while plan() reads the streams separately. They now run through plan(), with the command as the test seam, so they check the message of the exception the user gets. The tests in a working directory with shadowing modules read stdout separately as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Frame the environment probe's values instead of parsing stdout by line position (#61) * Delimit the values of the Python environment probe (#15) The probe printed its values one per line and Kotlin took them by line position, so anything else writing to stdout - a sitecustomize module or a .pth file - shifted every value, and an empty first value, e.g. an empty sys.executable, was trimmed away, leaving too few lines and an IndexOutOfBoundsException instead of the documented IllegalStateException. The probe now writes its values separated by NUL, which no path can contain, between unique begin and end sentinel lines, preceded by an empty line in case the noise before does not end with a newline, and as bytes, so without newline translation. Kotlin takes the values only from between the last begin and the following end sentinel, and fails with an IllegalStateException showing the output when there are not exactly four. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Reject an empty executable, read the probe from the original stdout (#15) An empty sys.executable was parsed correctly but then accepted silently: the embedded CPython got an empty program name, so it ignored the pyvenv.cfg of a virtual environment, and installing dependencies failed later with a bare IOException of running "". The probe now fails right away with an IllegalStateException telling that sys.executable is empty. The probe wrote to sys.stdout.buffer, which a sitecustomize replacing sys.stdout with a wrapper does not have, so it now writes to the original sys.__stdout__, after flushing both, which also bypasses any redirection of sys.stdout. When the output has no probe values, the message now names the Python executable and shows its stderr as well, as it tells why; the parsing returns null for the caller to report it, and is unit-tested on its own for output around the values, repeated markers, an empty value, a missing end marker, and a wrong number of values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Run the embedded Python in UTF-8 mode and pass paths losslessly (#64) * Run the embedded Python in UTF-8 mode, transfer paths as file system bytes (#21) The embedded interpreter used the encoding of the JVM's locale, so under LC_ALL=C with PYTHONUTF8=0, or a cp1252 Windows locale, printing non-ASCII text or reading a UTF-8 file failed. Jep's PyConfig has no UTF-8 mode setter, but CPython parses `-X utf8` from the argv set with PyConfig.setArgv before decoding the other values, which enables it, also over PYTHONUTF8=0. Under a non-UTF-8 POSIX locale, e.g. ISO-8859-1 with a home like /home/josé, a path is the locale's bytes, surrogate-escaped by Python in UTF-8 mode: - the environment probe failed encoding such a path to UTF-8; it now writes each value as the hex of its os.fsencode bytes, which Kotlin decodes with sun.jnu.encoding (UTF-8 on Windows), so it names the same file; - the dependency directory, a Java String passed by Jep as UTF-8, was another path in the embedded sys.path; it is now appended as os.fsdecode of its bytes. In argv, to PLAN and pip, it already round-trips unchanged. A non-ASCII path of the Python executable cannot be fixed here, as Jep passes the program name as UTF-8 too; the README documents it. PythonTest compared sys.path of a raw ProcessBuilder decoded as UTF-8; it now uses runProcess and compares ascii() of the paths on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Test the Latin-1 dependency directory, decode probe paths strictly (#21) Review findings of the UTF-8 mode fix: - The os.fsdecode conversion of the dependency directory had no regression test; PythonLocaleTest now forks a JVM in an ISO-8859-1 locale, created with localedef and LOCPATH (skipped without localedef), importing a module from a non-ASCII dependency directory. The forked JVM is awaited with a timeout, its output going to a file, so a stuck JVM cannot hang the build. - A probe value which is not valid in the file system encoding of the JVM was replaced with U+FFFD, naming another path, and a sign was accepted in its hex digits; it is now decoded strictly, and the probe fails clearly. - The UTF-8 mode depends on Jep setting argv before the other PyConfig values, so a new interpreter now checks sys.flags.utf8_mode, failing clearly if a future Jep breaks it. - The README documents that an inherited PYTHONIOENCODING still wins for the embedded streams, and that the ASCII limitation of the executable's path applies only to non-UTF-8 POSIX locales; CLAUDE.md keeps only the gotchas. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Tell a missing jep from a Python failing to run, keep the probe's stderr for startup failures (#65) * Tell a missing jep from a Python failing to run, keep the probe's stderr (#19) Any non-zero exit of the environment probe was reported as a missing jep package with a pip install remedy, also when Python itself could not run, e.g. a pyenv shim of a version which is not installed. The probe now exits with a dedicated code (83) when it cannot find jep, and only this one gets the jep headline; any other exit says Python cannot be run, with its exit code and output. The stderr of a successful probe was dropped, so a warning like "Could not find platform libraries" of a wrong PYTHONHOME left no trace when the embedded interpreter, configured the same way, failed later. It is kept in PythonProbe.error, and added to the failure of creating the first interpreter. The main code has no logging, so it is not logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Report the probe's stderr on any startup failure of the embedded Python (#19) Jep 4.3 rethrows the failure of initializing CPython as a plain java.lang.Error, and every later attempt as another Error ("previously failed to initialize"), which the catch of JepException and LinkageError missed, so the probe's stderr was never attached where it matters, e.g. a broken encodings module. EmbeddedPythonStarter now creates the interpreters of Python: any failure before the first interpreter exists, and any Error of Jep at any time, is a startup failure, always wrapped in an IllegalStateException with the original as cause, so its type does not depend on whether the probe wrote anything; the probe's stderr is offered as a possible hint. A later, unrelated failure of an interpreter is thrown as is, instead of blaming an old warning. The UTF-8 mode check now runs before the interpreter is kept, closing it on failure, as a later call skipped the check of the kept interpreter. The probe names a jep module shadowing the package, as "pip install jep" would only answer "Requirement already satisfied"; the failure messages no longer end with an empty line without output; the kept stderr is limited to its first 8 KiB, where a startup warning of Python is, as it lives as long as the JVM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG * Test the startup failure end to end, wrap only failures before start (#19) Nothing tested the wiring in Python.call, so a forked JVM now starts the embedded CPython with a fake encodings package, which warns and fails only when not run with -c, as the probe is, and the test checks that the IllegalStateException carries the probe's warning; it fails when Python creates the SharedInterpreter directly. Jep's MainInterpreter is a singleton, so once an interpreter is created any failure, also an Error like NoClassDefFoundError, is not a startup failure, and is thrown as is. Jep wraps the failure of initializing CPython in an Error whose message is the toString() of its cause, so the headline takes the message of the cause instead of repeating its class. The kept stderr of the probe is limited including the note of truncation, without splitting a surrogate pair. The message of a probe without values no longer ends with an empty line without output, and a jep package directory which cannot be listed is reported naming it, instead of a raw NIO exception. The Python KDoc no longer promises a cause for the failures of resolving the environment, which have none. The probe's warning is attached only to a failure of the embedded CPython to start, and it is not logged, as the main code has no logging, so a later failure caused by the same misconfiguration carries no hint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Recovered onto main after the stacked PR was merged into its parent branch (#46). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Takes the recovery branch for everything which is not the change of #37, whose original commits of #13 to #19 it carries squashed, along with the newer work of main, and reapplies the own change of #37 on top of it: the jep install advice, the externally managed detection, the version of jep from pom.properties, the README install section, and their tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
morisil
changed the base branch from
issue-13-runprocess-deadlock
to
recover-environment-fixes
November 10, 2026 21:29
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG # Conflicts: # src/main/kotlin/PythonEnvironment.kt # src/test/kotlin/PythonEnvironmentTest.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #37
Fix
The advice for a missing or broken jep (
python3 -m pip install jep) failed on externally managed Pythons (PEP 668), installed an unpinned version, and didn't tell what building jep needs. Now:META-INF/maven/org.ninia/jep/pom.properties(its manifest has no version) — so the advice matches theorg.ninia:jeplibrary actually on the classpath; without it, the advice falls back to an unpinnedjep.EXTERNALLY-MANAGEDmarker in its stdlib, as pip checks) — by exit code when jep is missing, or a probe value otherwise.<python> -m pip install jep==<version>, oruv pip install --python <python> jep==<version>;VIRTUAL_ENVorPythonEnvironment.pythonExecutableto it;--force-reinstall --no-cache-dir.apt install python3-dev, pluspython3-venvfor the venv recipe, on Debian/Ubuntu); on Windows the Microsoft C++ Build Tools andJAVA_HOME.~/%USERPROFILE%, which only a shell expands), quoted when they contain spaces, with a PowerShell¬e on Windows.Tests (
PythonEnvironmentTest)The advice for a venv, an externally managed Python, Windows paths with spaces, an unknown version, a broken jep; the probe reporting externally managed Pythons (simulated with a
sitecustomize); reading the version from malformed properties; the version read at runtime matches the version catalog.Review rounds
pip installworked. Fixed in 0575a58 (externally-managed detection), with absolute paths, the always-from-source build requirements, a lazy version read,--force-reinstall./usr/bin/python3, and a broken jep — all worked. Minor findings applied in 14b6efa (python3-venv, README examples, PowerShell,uv pipfor a venv without pip).Follow-ups filed
jeppackage version not matching theorg.ninia:jeplibrary passes the probe (now checkable with the version read here).🤖 Generated with Claude Code
https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG