Skip to content

feat(cache): observe native P3 filesystem operation lifetimes - #786

Merged
lthibault merged 3 commits into
masterfrom
lthibault/p3-operation-lifecycle-780
Nov 1, 2026
Merged

lthibault merged 3 commits into
masterfrom
lthibault/p3-operation-lifecycle-780

Conversation

@lthibault

@lthibault lthibault commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Guest cancellation can precede native filesystem completion. This change observes native CID-directory materialization during P3 open-at, through the owner-local snapshot mechanism from #783.

Reconciled onto master after #783 merged as a85e4c4bf074a1d4f399de9daea92d5476bb9e2e, including #784. Base and former stacked-base trees are identical. Both implementation commits retained identical patches; the diff contains only #780 and its trace-level logging adjustment. An empty CI-trigger commit started PR checks after retargeting.

Qualified head: 39e675a9270fe31ab23b6d13d5b004c595a10900. Candidate tree: e28fcf2053909b0aeb20b2bb4d9af521210d915f, identical to the previously published candidate.

Refs #780, #753, #785. The first corrective cancellation implementation remains separately tracked in #787.

Guest cancellation currently does not cancel this selected native backend operation. #785 owns the cancellation contract; #787 owns the first corrective implementation. This PR observes existing behavior. Continued execution does not establish an independent Host need for the work.

Behavior and bounds

  • ForegroundTransaction::run_directory_build wraps the existing retained blocking operation. Records distinguish waiter lifetime, backend closure entry/outcome, and actual transaction receipt release. Backend return, retained-result disposal, cleanup completion, and capacity release remain separate facts.
  • Each root retains eight fixed slots under its existing lifecycle mutex. Active records cannot be overwritten. Fully retired records can be replaced; checked IDs and root generations distinguish reuse. An eight-bit mask associates records with the existing receipt. Weak observation handles add no receipt, permit, or guest authority.
  • Full slots or exhausted IDs omit observations without rejecting filesystem work. A saturating omission count makes incomplete coverage explicit. Records are neither a complete operation census nor an event history.
  • FsCompletion::try_snapshot() retains one immediate try_lock and explicit Busy, Unavailable, and Poisoned outcomes. No registry, polling loop, exporter, public endpoint, or fuel/scheduler instrumentation is added. Records contain no paths, CIDs, capability identifiers, payloads, or error text.
  • Both normal filesystem lifecycle events use TRACE, consistent with trace-level or on-demand observation. Disabled callsites do not evaluate the snapshot read. Existing Cell correlation and cleanup-certification error logging remain intact.

Opaque Wasmtime operations remain unsupported for individual backend phases. The pinned hook exposes admission and final opaque-owner release; a retained result or stream can outlive backend return. NativeFsLease cannot establish backend outcome or guest cancellation. Running means the selected closure entered and has not returned, not continuous syscall execution. The waiter represents this wrapper, not the entire P3 call.

Validation

The deterministic compiled-P3 scenario gates native publication, cancels the Cell, and witnesses Store destruction through bootstrap release. Native file contents independently prove later publication. A separate cleanup gate holds cleanup pending; deletion and completion witnesses establish final cleanup. Tests do not infer correctness solely from new diagnostic fields or use sleeps to establish ordering.

  • Final master-based qualification: 115 cache tests, 43 filesystem-interception tests, six pinned native P3 fixture tests, and 21 launcher tests passed. The native P3 fixture was rebuilt/validated. cargo build --workspace --tests --locked and cargo clippy --workspace --all-targets --locked -- -D warnings passed. Heavy checks used the shared build lock.
  • Trace adjustment: regression tests first failed on the old DEBUG events, then passed. All 21 launcher tests passed, including trace visibility, DEBUG suppression, correlation, and real P3 cancellation. cargo clippy -p ww --all-targets --locked -- -D warnings passed.
  • Dependency reconciliation: both commits retain identical patches under git range-diff; Rust source, tests, and filesystem documentation retain the reviewed hashes. All 21 launcher tests and the inherited release-workflow regression check passed on the reconciled base. Formatting and diff checks passed.
  • Independent final-candidate correctness/architecture and security reviews found no blockers, including the preserved P3 publication/cleanup witnesses and exact tree identity. Publication-time optional Claude review supplied no coverage because its OAuth session expired.
  • Seven mocked/static workflow regressions, nine installer shell cases, Bash syntax, and ShellCheck passed. Full canonical packaging and installed-boot validation passed in PR CI; local canonical kernel/status payloads are absent. Installed actionlint 1.7.12 reports the same two diagnostics on unchanged baseline and candidate workflow constructs (concurrency.queue and the intentional false publication condition); no workflow edits were made.

The original qualification report, raw logs, review hashes, and smoke evidence remain preserved locally under .context/780/ and .context/validation/; publication evidence is additive under .context/780/publish/, and final integration/test/review evidence is under .context/780/landing/ and .context/validation/land-*. These private workspace artifacts are not repository attachments.

Performance limits

A debug-only smoke compared the existing and observed wrappers in one binary: 20 warmup pairs and 100 alternating measured pairs, each creating 16 native files, setting lengths, renaming, deleting, and disposing results/receipts. All nine power observations reported AC. Median times were 1.222417 ms and 1.245167 ms (+1.86%); P95 times were 1.505209 ms and 1.521792 ms.

This smoke is not a release-build benchmark, production-overhead bound, or whole-system before/after comparison. Both paths contain the new root storage. It measures boundary observation work, not root-memory cost. No new timing claim is made for the logging-level adjustment.

Landing gates

Local master reconciliation, qualification, and independent review are complete. All five required checks passed on 39e675a9270fe31ab23b6d13d5b004c595a10900: Test, Check formatting, Clippy, Require changelog entry, and Solidity (Foundry). Rust CI also passed full host/standard-library tests, pinned P3 and first-party components, canonical packaging/release assembly, installed-package startup, fuzz regressions, and the Chess authority smoke. No review requests or unresolved review threads were present at final readback. Final reviewed-head and merged-tree verification remain merge gates. After merge verification, update and close #780; update #753 and #749 while preserving their remaining scope. Keep #785 and #787 open. No production deployment or manual release publication is requested.

Documentation

Status: current. Documentation matches the native P3 directory-build diagnostic and TRACE logging change after reconciliation onto merged master; authored documentation and source bytes are unchanged.

Documentation health:

Coverage debt: no zero-coverage or reference-only gaps found for this internal surface. No standalone newcomer tutorial was added. No affected diagram drift found. The debug performance smoke is qualification evidence, not a production-overhead bound.

@lthibault
lthibault force-pushed the lthibault/p3-operation-lifecycle-780 branch from 236b5fb to bf6a88c Compare November 1, 2026 09:52
@lthibault
lthibault changed the base branch from lthibault/pasted-text-v32 to master November 1, 2026 09:52
@lthibault
lthibault marked this pull request as ready for review November 1, 2026 09:54
@lthibault
lthibault merged commit 24a2ff0 into master Nov 1, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Observability delivery: P3 host-operation ownership and cancellation lifecycle (#753)

1 participant