Skip to content

Fix JIT stub veneers being reused after an OPcache restart on AArch64 - #24273

Open
pavimus wants to merge 3 commits into
php:PHP-8.4from
pavimus:fix-jit-aarch64-restart-veneers
Open

pavimus wants to merge 3 commits into
php:PHP-8.4from
pavimus:fix-jit-aarch64-restart-veneers

Conversation

@pavimus

@pavimus pavimus commented Oct 12, 2026 •

Copy link
Copy Markdown

Prepared with an AI assistant (Claude), which also ran the reproduction and the tests on my machine; I reviewed the result. The text below is AI-generated (quoted, as asked in CONTRIBUTING.md).

While testing OPcache restarts with tracing JIT on AArch64 we found crashes caused by a JIT issue that is independent of threading.

On AArch64 a branch from JIT code to a stub can be out of range (conditional branches reach
±1 MB), and IR emits a veneer next to the new code. zend_jit_set_veneer() /
zend_jit_commit_veneers() then replace the address of the stub in zend_jit_stub_handlers[]
by the address of that veneer, so that later code reuses it.

These veneers live after the JIT restart point (dasm_ptr[1]). zend_jit_restart() rewinds the
buffer to that point but leaves zend_jit_stub_handlers[] pointing to the old veneers. Code
compiled after the restart branches to those addresses, which are overwritten by new traces as the
buffer fills up again, and jumps into the middle of unrelated code when the stub is taken
(exception handling, leaving a function, ...).

Observed with opcache.jit=tracing, more than 1 MB of JIT code and repeated OOM restarts under
FrankenPHP (ZTS): crash in JIT code in 8 of 8 runs. At the crash, gdb shows e.g.
zend_jit_stub_handlers[JIT$$leave_function_handler] == $pc, and that address holds an ldr
of a trace, not the b of a veneer. In a single-process NTS server the same stale entries are
present at zend_jit_restart() (JIT$$exception_handler, JIT$$leave_function_handler,
JIT$$icall_throw, JIT$$leave_throw pointing past the restart point). @austinderrick found a
deterministic NTS reproducer (#24383, see the comment below); the test
ext/opcache/tests/jit/restart_stub_veneers.phpt is based on it: more than 1 MB of tracing JIT
code, opcache_reset(), then different code. Without the fix the CLI server crashes after the reset
(3/3, AArch64 NTS debug), with the fix it passes (3/3).

The fix keeps the original stub addresses in a third block after the pending veneers (allocated in
the same place as the existing two) and restores them in zend_jit_restart(). x86 and Windows
are not affected (no veneers).

With this change (and the GH-8739 fix, which is needed to have restarts at all under ZTS): 0 crashes
in 40 runs of the FrankenPHP scenario, and 9 of 10 runs without a crash on Apache event (the remaining
crash has a different signature: a separate tracing JIT bug, also without restarts, fixed in #24333). make test for ext/opcache and
Zend/tests passes.

🤖 Generated with Claude Code

On AArch64 a branch from JIT code to a stub may be out of range, and IR
emits a veneer next to the new code. zend_jit_commit_veneers() then
replaces the address of the stub in zend_jit_stub_handlers[] by the
address of the veneer, so that later code reuses it. These veneers live
after the JIT restart point: zend_jit_restart() rewinds the buffer below
them, but zend_jit_stub_handlers[] keeps pointing to them. Code compiled
after the restart branches to the old addresses, which are overwritten
by new traces, and crashes (e.g. on an exception or when leaving a
function through a stub). This needs more than 1MB of JIT code (the
range of conditional branches) before the restart.

Keep the original stub addresses after the pending veneers and restore
them in zend_jit_restart().
@austinderrick

austinderrick commented Oct 17, 2026 •

Copy link
Copy Markdown

Here is a deterministic NTS reproduction for this bug, from #24383, which I am closing as a duplicate of this PR.
I tested the same code against this PR and was able to validate that the fix resolved the issue.

With plain PHP-FPM (NTS, pm = static, 4 children) on the official php:8.4-fpm arm64 image, warming up with one workload, calling opcache_reset(), and then running a different workload hangs every request at 100% CPU. It reproduces on 8.4.23 and 8.4.26. The code after the reset has to differ from the code before it; otherwise IR rebuilds the same veneers at the same addresses.

Run ab.php after the reset
8.4.23 and 8.4.26, arm64 504 (hangs)
opcache.jit=disable 200
x86_64 200
8.4.23 with this PR's patch 200

In a Laravel application on the same image, the same sequence crashes every FPM worker with SIGSEGV (in zend_jit_trace_exit, or in JIT code calling zend_error_unchecked). A gdb dump of zend_jit_stub_handlers[] showed exception_handler, exception_handler_undef, interrupt_handler, leave_function_handler, icall_throw and leave_throw pointing about 1 MB past the restart point after the reset. With this PR's patch applied to 8.4.23, repeated resets in that application return 200 with no crashes.

To run it, put the files below in one directory and run ./run.sh on an arm64 host with Docker. JIT=disable ./run.sh and ./run.sh php:8.4-fpm --platform linux/amd64 are the controls.

Output on arm64 (8.4.23 and 8.4.26 behave the same):

Without reset:
  b.php: 200 in 1.533545s
  b.php: 200 in 0.149145s
  ab.php: 200 in 0.778559s
  ab.php: 200 in 0.313916s
With reset:
  b.php: 200 in 1.135747s
  b.php: 200 in 0.151503s
  reset.php
  ab.php: 504 in 60.047946s
  ab.php: 504 in 60.067511s
run.sh
#!/bin/bash
# Usage: ./run.sh [image] [extra docker run args...]
#   ./run.sh                                     # php:8.4-fpm, native arch
#   ./run.sh php:8.4-fpm --platform linux/amd64  # x86_64 control
#   JIT=disable ./run.sh                         # JIT-off control
set -u
cd "$(dirname "$0")"; d=$PWD
IMG=${1:-php:8.4-fpm}; shift || true
ini=conf/jit.ini
if [ "${JIT:-}" = disable ]; then sed 's/^opcache.jit=tracing/opcache.jit=disable/' conf/jit.ini > conf/jit-off.ini; ini=conf/jit-off.ini; fi
[ -d app/a ] || docker run --rm -v "$d":/w -w /w "$@" "$IMG" sh -c 'php gen.php && php gen2.php'
docker network create jitrepro >/dev/null 2>&1
up() {
    docker rm -f jitrepro-php jitrepro-web >/dev/null 2>&1
    docker run -d --name jitrepro-php --network jitrepro --network-alias php "$@" \
        -v "$d/app":/app -v "$d/$ini":/usr/local/etc/php/conf.d/zz-jit.ini:ro \
        -v "$d/conf/www.conf":/usr/local/etc/php-fpm.d/zz-www.conf:ro "$IMG" >/dev/null
    docker run -d --name jitrepro-web --network jitrepro -p 8099:80 \
        -v "$d/conf/nginx.conf":/etc/nginx/conf.d/default.conf:ro nginx:1.27-alpine >/dev/null
    for i in $(seq 1 30); do curl -s -o /dev/null http://localhost:8099/reset.php && break; sleep 1; done
    docker exec jitrepro-php php -r 'echo PHP_VERSION, " ", php_uname("m"), "\n";'
}
t() { curl -s -o /dev/null -w "  $1: %{http_code} in %{time_total}s\n" --max-time 120 http://localhost:8099/$1; }
up "$@"
echo "Without reset:"; curl -s -o /dev/null http://localhost:8099/reset.php; t b.php; t b.php; t ab.php; t ab.php
up "$@"
echo "With reset:"; t b.php; t b.php; curl -s -o /dev/null http://localhost:8099/reset.php; echo "  reset.php"; t ab.php; t ab.php
echo "Worker crashes: $(docker logs jitrepro-php 2>&1 | grep -c 'signal 11')"
docker rm -f jitrepro-php jitrepro-web >/dev/null 2>&1; docker network rm jitrepro >/dev/null 2>&1
gen.php
<?php
// Workload A: 1,500 classes with hot loops.
@mkdir(__DIR__ . '/app/a');
for ($i = 0; $i < 1500; $i++) {
    $src = "<?php\nclass C$i {\n";
    for ($m = 0; $m < 6; $m++) {
        $src .= "  public static function m$m(array \$a, int \$k): int {\n"
            . "    \$s = $i;\n"
            . "    foreach (\$a as \$j => \$v) { if ((\$v + \$k) % " . ($m + 2) . " === 0) { \$s += \$v * $m; } else { \$s ^= \$j; } \$s .= ''; \$s = (int)\$s; }\n"
            . "    \$str = str_repeat('x', \$k % 7); \$s += strlen(\$str);\n"
            . "    return \$s;\n  }\n";
    }
    file_put_contents(__DIR__ . "/app/a/C$i.php", $src . "}\n");
}
gen2.php
<?php
// Workload B: 3,000 classes with polymorphic branches, closures and caught exceptions.
@mkdir(__DIR__ . '/app/b');
for ($i = 0; $i < 3000; $i++) {
    $src = "<?php\nclass P$i implements ArrayAccess {\n  private array \$d = [];\n"
        . "  public function offsetExists(mixed \$o): bool { return isset(\$this->d[\$o]); }\n"
        . "  public function offsetGet(mixed \$o): mixed { return \$this->d[\$o] ?? null; }\n"
        . "  public function offsetSet(mixed \$o, mixed \$v): void { \$this->d[\$o] = \$v; }\n"
        . "  public function offsetUnset(mixed \$o): void { unset(\$this->d[\$o]); }\n"
        . "  public function run(array \$a): mixed { \$r = 0; foreach (\$a as \$k => \$v) {\n"
        . "    if (is_int(\$v)) { \$r += \$v * $i; } elseif (is_string(\$v)) { \$r .= substr(\$v, 0, 2); } elseif (is_array(\$v)) { \$r = count(\$v) + (int)\$r; } else { \$r = \$v ?? \$r; }\n"
        . "    \$this[\$k % 17] = \$r; \$x = fn(\$y) => \$y . \$k; \$r = is_string(\$r) ? strlen(\$x(\$r)) : \$r; }\n"
        . "    try { if (\$r % " . (($i % 9) + 2) . " === 0) { throw new RuntimeException('x'); } } catch (Throwable \$e) { \$r++; }\n"
        . "    return \$r; }\n}\n";
    file_put_contents(__DIR__ . "/app/b/P$i.php", $src);
}
conf/jit.ini
opcache.enable=1
opcache.jit=tracing
opcache.jit_buffer_size=64M
opcache.validate_timestamps=0
opcache.jit_max_root_traces=100000
opcache.jit_max_side_traces=100000
opcache.jit_max_exit_counters=65536
max_execution_time=0
error_reporting=E_ALL & ~E_DEPRECATED
display_errors=0
conf/www.conf
[www]
pm = static
pm.max_children = 4
conf/nginx.conf
server {
  listen 80;
  root /app;
  location ~ \.php$ {
    fastcgi_pass php:9000;
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME /app$fastcgi_script_name;
  }
}
app/a.inc
<?php
$a = range(1, 200);
for ($i = 0; $i < 1500; $i++) {
    require_once __DIR__ . "/a/C$i.php";
    $c = "C$i";
    for ($r = 0; $r < 4; $r++) {
        for ($m = 0; $m < 6; $m++) { $c::{"m$m"}($a, $r + $m); }
    }
}
app/b.inc
<?php
$mix = [];
for ($j = 0; $j < 120; $j++) { $mix[] = [$j, "s$j", [$j], null, 1.5][$j % 5]; }
for ($i = 0; $i < 3000; $i++) {
    require_once __DIR__ . "/b/P$i.php";
    $c = "P$i";
    (new $c())->run($mix);
}
app/b.php
<?php
require __DIR__ . "/b.inc";
$s = opcache_get_status(false);
echo "ok jit_used=", $s["jit"]["buffer_size"] - $s["jit"]["buffer_free"], "\n";
app/ab.php
<?php
require __DIR__ . "/a.inc";
require __DIR__ . "/b.inc";
echo "ok\n";
app/reset.php
<?php
var_dump(opcache_reset());

Based on the reproducer by Derrick Austin from php#24383: more than 1 MB of
tracing JIT code, opcache_reset(), then different code. Without the fix
the server crashes on AArch64 after the reset.

Co-authored-by: Derrick Austin <derrick.austin@interworks.com>
@pavimus

pavimus commented Oct 17, 2026

Copy link
Copy Markdown
Author

Prepared with an AI assistant (Claude); the text below is AI-generated (quoted, as asked in CONTRIBUTING.md).

Thanks a lot, @austinderrick! Your reproducer works here too, also with a single-process php -S
(NTS debug, AArch64): without the fix the server crashes on the first request after
opcache_reset(), with it every request passes, 3/3 each. A reduced version (200 + 400 classes,
about 1 s) is now a test in this PR: 46b03d6, with you as co-author.

Use php_opcache.dll on Windows and pass extension_dir explicitly, like
file_cache_frameless_handler.phpt.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants