Skip to content

Make libpython symbols global also without its development symlink (#77) - #94

Open
morisil wants to merge 2 commits into
issue-54-libpython-pathfrom
issue-77-global-symbols
Open

morisil wants to merge 2 commits into
issue-54-libpython-pathfrom
issue-77-global-symbols

Conversation

@morisil

@morisil morisil commented Nov 13, 2026

Copy link
Copy Markdown
Member

Fixes #77

Stacked on #78 (fix of #54) — merge #78 first, then this one (bottom-up, see CLAUDE.md).

When the dynamic loader finds libpython3.X.so.1.0 by itself (a system Python, or LD_LIBRARY_PATH), its symbols became global only by Jep's own dlopen("libpython3.X.so", RTLD_GLOBAL) of the development symlink — without it (e.g. libpython3.12-dev not installed) a C extension not linked to libpython (numpy, …) failed with undefined symbol.

Now #54's RTLD_GLOBAL preload also runs unless the loader finds both the soname and the development symlink, as the same file (jepLoadsGlobally) — so a system Python with its dev package stays untouched (no new foreign-linker warning on Java 22/23).

  • The probe also reports LDLIBRARY (the development symlink name).
  • /etc/ld.so.cache is read once for both lookups.
  • On Java 17–21 (no foreign linker) the symbols become global only after Python has started: a C extension imported during startup (.pth, sitecustomize) still needs the development symlink — README says so.
  • The warning when the symbols cannot be made global now also suggests the development package.

Tests compile a C extension not linked to libpython and import it in a forked JVM whose LD_LIBRARY_PATH holds only the soname: they fail without the fix on a uv CPython 3.13 base (undefined symbol: PyModule_AddStringConstant), and pass with it on Java 17, 22 and 25. With this machine's system Python they cannot reproduce the bug (its dev symlink is in ld.so.cache), see DEVELOPMENT.md.

Review

2 rounds:

  1. No blocking findings; polish (warning advice, one cache read, test helpers) applied.
  2. No blocking findings.

Left as polish: the warning names libpython3.X-dev as a placeholder; the soname-only tests pass silently (not skipped) where the loader finds the dev symlink.

Follow-ups

🤖 Generated with Claude Code

https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG

morisil and others added 2 commits November 13, 2026 09:47
Jep makes libpython global with dlopen of its development symlink
(LDLIBRARY, e.g. libpython3.12.so). When the dynamic loader finds
libpython by its soname but not that symlink, e.g. a system Python without
libpython3.12-dev, the symbols stayed local, and a C extension not linked
to libpython, like numpy's, failed with an undefined symbol.

The probe now also reports LDLIBRARY, and libpython is preloaded with
RTLD_GLOBAL (Java 22+) or made global in the first interpreter (Java
17-21) unless the loader finds both its soname and its development
symlink as the same file (jepLoadsGlobally). A system Python with its
development package is still left alone, so it gets no new restricted
method warning on Java 22/23.

PythonLibraryTest reproduces it with an LD_LIBRARY_PATH holding only the
soname of the venv's libpython, failing before with a uv base Python.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
#77)

- The warning of makePythonLibraryGlobal no longer only advises
  LD_LIBRARY_PATH, which cannot help a Python of the system without its
  development package, now reaching the Java 17-21 path: it also advises
  installing the package providing LDLIBRARY (pythonLibraryGlobalAdvice).
- jepLoadsGlobally reads /etc/ld.so.cache at most once for both names,
  and its finds seam takes only the name, the library being always the same.
- PythonLibraryTest shares compiling the extension and running the JVM
  in withExtension and importExtension.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant