Skip to content

fix: normalize new Ghidra PIE imports - #310

Merged
branover merged 2 commits into
mainfrom
fix/ghidra-address-consistency
Aug 15, 2026
Merged

branover merged 2 commits into
mainfrom
fix/ghidra-address-consistency

Conversation

@branover

Copy link
Copy Markdown
Owner

Summary

  • normalize newly imported PIE/ET_DYN programs to their ELF preferred image base before Ghidra analysis
  • preserve existing warm Ghidra projects at their current base and surface their image base/load bias as an explicit warning
  • expose the ELF-to-Ghidra address map and file-offset translation helpers to managed probes and re_script
  • keep address-bearing Observations isolated by mapping schema/base/bias so legacy and normalized coordinates cannot be silently mixed

Why

Ghidra 12.x commonly imports a PIE whose ELF load base is zero at 0x100000. That made otherwise identical virtual addresses disagree across Ghidra, radare2, and debugger output. New imports now use the ELF coordinate system, while expensive existing projects remain usable and clearly identify their historical coordinate system.

Verification

  • 1868 passed, 47 skipped, 14 deselected in the offline non-slow gate
  • 213 passed, 5 skipped in the affected offline suites
  • 4 passed in live Ghidra decompilation, emulation, and taint tests
  • just test reached the unrelated live HTTP fixture but the local host could not connect to its Docker bridge IP; the focused reproduction timed out identically at the fixture readiness check

@branover branover left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST-CHANGES

I completed both merge-gate passes against PR head 8b9371a and base c1b4ce7, using both gh pr diff 310 and git diff origin/main...HEAD.

Findings

  1. HIGH — src/hexgraph/sandbox/probes/pyghidra_lib.py:300: an unverified cold ELF mapping is still committed. When _elf_load_map() returns an unverified report (for example a valid ELF using the explicitly unsupported extended program-header count, or a truncated/otherwise unsupported ELF header), validate_address_mapping(..., allow_existing_base=False) returns normally. open_target() then completes analysis and commits the warm marker even though normalization was skipped and the promised ELF coordinate system was never validated. The report has no warning on the cold path, so a PIE can retain Ghidra's legacy bias silently despite the PR's contract. Impact: a newly created project can publish wrong addresses as if the normalized import succeeded. Fix: raise AddressMappingMismatch for an unverified ELF on the cold path (while continuing to preserve and warn for warm projects), and add a cold-path regression test for unsupported/truncated ELF metadata.

  2. HIGH — src/hexgraph/engine/re/ghidra_bridge.py:227: managed decompile responses discard the new mapping and warning. The server now attaches address_mapping/warning, but _ManagedOps.decompile() rebuilds out from selected fields and omits both. This affects decompile/list-functions and enrich_target() calls routed through a resident managed bridge. Impact: an existing warm project with a legacy load bias produces user-visible addresses without the required warning, and its Observations lack the coordinate fingerprint, so they can deduplicate against a different coordinate system. Fix: copy validated address_mapping and warning into the managed decompile result and cover the client reconstruction path, not only _serve_one().

  3. HIGH — src/hexgraph/sandbox/decompiler.py:50: radare2 fallback addresses are recorded under the legacy Ghidra mapping. When Ghidra misses a requested focus, GhidraDecompiler.decompile() substitutes the radare2 focus and tags it with focus_fallback=True; focus_only_payload() now unconditionally copies the original Ghidra address_mapping while dropping the fallback provenance. For a preserved legacy project, the focus address is the normalized radare2 address, but the Observation is fingerprinted as legacy-biased Ghidra output. Impact: the payload mixes coordinate systems, warns that its displayed address includes a bias when it does not, and a later real Ghidra result with the same mapping can incorrectly deduplicate to the fallback Observation. Fix: preserve the focus engine/coordinate provenance and only attach the Ghidra mapping to a focus that actually came from Ghidra (or model inventory and focus mappings separately); add a legacy-mapping fallback regression test.

Pass results

  • Correctness/test quality: the three blockers above prevent the coordinate and warning contracts from holding on cold-parser failures, managed bridge decompiles, and fallback decompiles. The focused affected suite passed: 108 passed.
  • Security/repository invariants: no loopback, sandbox, secret-handling, or policy-seam regression found. ELF parsing remains inside the sandbox probe and is bounded to fixed-size headers with a 4096-entry cap; no execution, egress, or rehosting gate changed. No public-repository confidential target details were found.
  • Docs/migrations/UX: MCP docs and generated VR-skill guidance ship with the behavior change. There is no DB model or frontend behavior change, so no migration or UX-contract update is required.
  • Broad test note: just test completed the suite except for the known environment-only Docker bridge readiness error in test_live_session_auth_flow_across_calls (172.17.x.x:8080 unreachable from the host); this is not attributed to this PR.

Comment thread src/hexgraph/sandbox/probes/pyghidra_lib.py
Comment thread src/hexgraph/sandbox/probes/pyghidra_lib.py
Comment thread src/hexgraph/sandbox/decompiler.py Outdated
Addresses review comments r3789589252, r3789589270, and r3789589291 on PR #310.

@branover branover left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

Follow-up review at PR head c9229f3: all three blocking findings from the REQUEST-CHANGES review are fixed.

  • Cold ELF mappings that cannot be verified now raise before a new warm marker can be committed. Existing warm projects remain preserve-and-warn, and relocatable ELF files without PT_LOAD ranges remain explicitly supported as not_applicable.
  • Managed bridge decompile reconstruction now propagates address_mapping and warning, so user-facing decompile/list/enrichment behavior and Observation identity retain the actual coordinate system.
  • A radare2 fallback focus now retains its engine/fallback provenance, is not fingerprinted with a legacy Ghidra mapping, and does not receive a misleading Ghidra-bias warning.

Verification: 110 passed in the affected address-mapping, managed-bridge, fallback, Observation, re_script, and tool-contract suites. The worktree is clean and commit c9229f3 was pushed. The three inline review threads include fix replies and are resolved.

The security/repository-invariant pass remains clean: hostile ELF parsing stays in the sandbox; loopback binding, secrets, and the execution/network/rehosting policy seams are unchanged; no confidential target details are present. Docs are current, and this PR has no model, migration, frontend, or UX-contract obligation.

CI must complete successfully on the new head before merge; the dispatching owner retains merge responsibility.

@branover
branover merged commit 2970458 into main Aug 15, 2026
7 checks passed
@branover
branover deleted the fix/ghidra-address-consistency branch August 15, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant