Skip to content

Predict the dynamic loader conservatively: Jep's libpython name, LD_LIBRARY_PATH, hwcaps, cache (#93) - #97

Open
morisil wants to merge 2 commits into
issue-77-global-symbolsfrom
issue-93-loader-finds
Open

morisil wants to merge 2 commits into
issue-77-global-symbolsfrom
issue-93-loader-finds

Conversation

@morisil

@morisil morisil commented Nov 13, 2026

Copy link
Copy Markdown
Member

Fixes #93

Stacked on #94 (fix of #77) → #78 (fix of #54) — merge #78, then #94, then this one (bottom-up, see CLAUDE.md).

loaderFinds / jepLoadsGlobally decide whether libpython must be preloaded with RTLD_GLOBAL; a wrong "found" skips the preload and C extensions fail with undefined symbol, while a wrong "not found" only costs an unneeded preload (and the restricted-method warning on Java 22+). The prediction now errs only towards "not found":

  • The name Jep opens is read from libjep.so itself (its only NUL-terminated libpython… .so/.a name without a version suffix — PYTHON_LDLIBRARY of Jep's build), instead of LDLIBRARY; checked on every libjep build on the machine (cp310–cp314) and against Jep 4.3.2's sources. A static (.a), missing or ambiguous name means preload.
  • LD_LIBRARY_PATH: an empty entry (the working directory to glibc) holding the name, a ;, or an entry with $ ($ORIGIN, $LIB, $PLATFORM, expanded by glibc) gives "not found"; relative entries resolve against the working directory; a path invalid under the locale gives "not found" instead of an exception.
  • glibc-hwcaps subdirectories holding the name give "not found".
  • ld.so.cache listing different files for the name (its order is not the loader's priority) gives "not found".
  • Default directories (/lib64, /usr/lib64, …) no longer count: they differ by distribution and architecture (Ubuntu never searches /usr/lib64).
  • For a static or empty Jep name, the advice is LD_PRELOAD, as LD_LIBRARY_PATH cannot help.

A system Python with its development symlink still gets no preload (both names found through ld.so.cache), and the uv Python cases of #94 behave the same.

Review

2 rounds:

  1. No blocking findings, but more wrong-"found" cases (cache order, $ entries, /usr/lib64, InvalidPathException, a .a removed after Jep's build) → fixed.
  2. No blocking findings.

Not handled (documented on loaderFinds): musl's default directories, legacy hwcaps subdirectories before glibc 2.37, the RUNPATH of libjep.

Follow-ups

🤖 Generated with Claude Code

https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG

morisil and others added 2 commits November 13, 2026 15:04
…ens (#93)

loaderFinds tells "not found", costing only a harmless preload, where the
dynamic loader might take another file: an empty LD_LIBRARY_PATH entry with
a file of the name in the working directory (glibc takes it for the working
directory, musl skips it), a ';' in LD_LIBRARY_PATH (a separator to glibc,
not to musl), and a file of the name in a glibc-hwcaps subdirectory, also in
ld.so.cache. Relative entries are resolved in the working directory.

jepLibraryName mirrors get_libpython() of Jep's build: with a static
LDLIBRARY, e.g. libpython3.12.a of conda-forge, Jep opens the libpython3.12.so
of the same stem when the .a is missing from LIBDIR and LIBDIR/MULTIARCH,
so the probe also reports MULTIARCH. A static name is never taken as loaded
globally, nor named by pythonLibraryGlobalAdvice.

Fixes #93

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
… more conservatively (#93)

jepLibraryName reads PYTHON_LDLIBRARY from the native library of Jep, the only
null-terminated libpython string without a version suffix, instead of guessing
it from today's files of the Python, which could differ from the ones of the
build, e.g. a libpython3.12.a removed since. The probe drops LDLIBRARY and
MULTIARCH, no longer needed.

loaderFinds tells "not found" also for a '$' in LD_LIBRARY_PATH, expanded by
glibc, for different files of the name in ld.so.cache, whose string table is
not in the order of their priority, for a path invalid in the JVM, and no
longer searches the default directories, which differ between distributions.

pythonLibraryGlobalAdvice advises LD_PRELOAD when Jep opens a static library,
or none, as LD_LIBRARY_PATH cannot help there.

Fixes #93

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FJmHQ3jHRbVdpwUiNUkApG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant