[quangvdao]

Summary

This fixes the disk-persisted setup race in #24. That issue is closed, but the race is still on main.

Two problems, both in src/setup.rs and src/lib.rs:

1. Torn reads. save_setup truncates the setup file and rewrites it in place. A concurrent load_setup can read a partial file and panic.
2. Mismatched setups. When setup() finds no file, it generates a new random setup and saves it, overwriting any file another process saved in the meantime. Processes that start together each keep their own setup. A later load_setup then returns another process's setup, which doesn't match. Prover/verifier pairs built this way fail to verify.

Jolt works around both problems by holding its own lock around setup (urs_lock::lock_urs_cache). Other users of disk-persistence have no such lock, and neither do dory's own tests. Under plain cargo test, test threads share one setup file, so tests fail intermittently. nextest gives each test its own process, which makes failures rarer. Cold-start processes still race on the shared file: in ten cold nextest runs on this sandbox, one failed two integration tests that don't touch the cache. That failure did not reproduce, and I didn't capture its output, so treat this as likely rather than proven.

Changes

• save_setup writes a temporary sibling file (<name>.<32 random hex digits>.tmp), calls sync_all, then renames it into place. A reader sees either the old file or the new one, never a partial one.
  • The temporary file is opened with create_new (retrying on a name clash), so two writers never share one, even when processes in different containers or on different hosts share the cache directory and reuse PIDs.
  • A drop guard removes the temporary file unless it was renamed into place, so a failed or panicking write leaves nothing behind. A process killed mid-write still leaves its .tmp file.
  • On Unix, the parent directory is synced after publishing, so the published file survives a power loss. Windows has no directory sync, so this is skipped there.
• New pub(crate) save_setup_if_absent publishes the temporary file with hard_link, which fails if the destination exists. So the first process to save wins, and the others learn they lost. If the filesystem doesn't support hard links and no file exists yet, it falls back to rename.
• setup() uses save_setup_if_absent. If another process saved first, setup() loads and returns that saved setup, so every caller ends up with the same setup. If that file is removed before setup() can load it, setup() tries to save its own setup again (up to 8 attempts) instead of panicking.
• load_setup reports a file removed between its existence check and its open as "not found", so setup() handles it like a missing file.
• tests/setup_persistence.rs (its own test binary, arkworks + disk-persistence) points HOME at an empty temp dir. It runs eight setup::<BN254>(6) calls at once behind a barrier, then checks that each returned setup equals the saved one and that no .tmp file is left behind.
• CHANGELOG.md: three entries under Unreleased → Fixed.

No public API changes and no new dependencies. The random suffix comes from rand_core::OsRng, already a dependency. I did not use the tempfile crate, to keep the dependency set and the MSRV 1.75 lockfile unchanged.

Validation

On a Linux sandbox, against main at 5246da5:

• The new test fails on main, both with parallel threads and with --test-threads=1. It passes with this change.
• All CI steps pass on this branch:
  • cargo fmt --check;
  • clippy with the transparent features, with zk, and with --no-default-features;
  • cargo doc with -D warnings;
  • cargo nextest for both feature sets (86 and 143 passed);
  • all five examples.
• After the follow-up commit (exclusive random temp names, cleanup, directory sync, retry), on macOS: cargo fmt --check, both clippy configurations with -D warnings, cargo doc with -D warnings, tests/setup_persistence.rs, and the setup tests in test_arkworks pass. Windows is not built in CI or locally; the Unix-only directory sync is behind cfg(unix).
• Plain cargo test --features backends,parallel,cache,disk-persistence,zk still fails on this branch. The cause is a separate problem: the prepared-point cache returns points from an earlier setup (fixed in fix(cache): serve prepared points only for the points they came from #35). With both PRs applied, plain cargo test passed:
  • 3 runs out of 3 with parallel threads;
  • 1 run with --test-threads=1;
  • 1 run with the transparent features.

🤖 Generated with Claude Code