Simulation Internals

Simulation and Verification Internals

How FINN+ executes models and checks them. For configuring verification in a build, see Functional Verification.

Node-by-node execution

All verification goes through finn.core.onnx_exec, which walks the graph and dispatches each node to a handler. That is what allows a graph mixing standard ONNX nodes, QONNX custom nodes and HLS/RTL hardware nodes to execute at all: standard nodes go to ONNX Runtime, custom nodes to their CustomOp.execute_node().

Because tensors are stored as float32 carriers for integer values (see Implementation Details), execution can accumulate rounding error; qonnx.util.basic.sanitize_quant_values cleans integer tensors during the custom execution flow.

Execution modes

A hardware node's exec_mode attribute selects its handler, set in bulk by SetExecMode:

exec_mode	Path
"" (unset)	Python implementation on the abstract HW op
"cppsim"	Compiled C++ from finn-hlslib
"rtlsim"	XSI RTL simulation

Important

RTL components do not support cppsim. If cppsim is selected for an RTL component, execution falls back to the abstract HW-op parent. A cppsim pass over a design containing RTL variants therefore does not exercise those variants' implementations — worth remembering when a design passes cppsim and fails rtlsim.

cppsim

Preparation is PrepareCppSim (generate C++) then CompileCppSim (compile it). Both are NodeLocalTransformations, so they parallelize across nodes — see Writing Custom Transformations.

CompileCppSim requires code_gen_dir_cppsim to be set and raises FINNUserError telling you to run PrepareCppSim first otherwise. The generated code lives under FINN_BUILD_DIR; executing a node is literally running that binary on text files, so it can be debugged by hand — see Custom Operations.

rtlsim via XSI

finn.xsi wraps Vivado's XSI. Its module docstring documents the contract precisely:

• Importing it is always safe — it touches neither FINN settings nor the filesystem, so it can be imported from multiprocessing workers spawned with forkserver/spawn.
• The availability check and any Vivado-version-triggered rebuild happen only in ensure_available().
• That call is made once, early, by run_finn.py's prepare_finn() for the CLI and by tests/conftest.py's pytest_configure() for tests — both before any parallel work is dispatched, so the build is never raced.

If XILINX_VIVADO is unset, is_available() raises OSError: XILINX_VIVADO environment variable not set. In pytest this surfaces as an INTERNALERROR during collection, which is an environment problem, not a test failure.

rtlsim_exec.py drives stitched-IP simulation.

The verify_step mechanism

verify_step implements every entry in verify_steps. Its shape:

1. Normalize the graph — SortGraph, GiveUniqueNodeNamesRecursive, GiveReadableTensorNames — so that dumps are readable and stable.
2. Resolve the verification IO pair via cfg._resolve_verification_io_pair(), which validates that both .npy files exist and asserts matching batch sizes.
3. For each sample: execute, compare, record.
4. Write verification_output/verify_<step>_<batch>_<SUCCESS|FAIL>.npz when verify_save_full_context is on, plus per-tensor statistics in topological order.

Input shapes that do not match the model are reshaped with a warning rather than rejected.

need_parent

After step_create_dataflow_partition the model is a parent graph containing a StreamingDataflowPartition node plus a child graph. Verification of those steps passes need_parent=True, which:

• asserts save_intermediate_models is enabled,
• loads intermediate_models/dataflow_parent.onnx,
• saves the current model as verify_<step>.onnx,
• runs execute_parent(...) over both.

Tensor names in the dump are then prefixed with the SDP node name.

The comparison

Three checks, all of which must pass:

res1 = np.isclose(out, exp, atol=cfg.verification_atol, rtol=cfg.verification_rtol).all()
res2 = np.mean(np.abs(out - exp)) <= cfg.verification_mean_atol
res3 = np.mean(np.abs((out - exp) / exp_safe)) <= cfg.verification_mean_rtol

exp_safe substitutes np.finfo(float).eps for zeros to avoid division by zero. The mean checks exist because element-wise closeness alone is brittle — one near-zero output element can fail isclose on an otherwise correct model.

rtlsim pre-hooks

verify_step accepts an optional rtlsim_pre_hook, passed through to execute_onnx. This is how MLO drives a rolled design via verify_mlo. Note that context saving is skipped whenever a pre-hook is present.

Waveforms

Tracing is controlled by the rtlsim_trace node attribute (node-by-node) or graph metadata property (stitched IP), and RTLSIM_TRACE_DEPTH:

Depth	Shows
1 (default)	Top-level input/output streams
2	Per-layer input/output streams
3	Per full-layer I/O including FIFO count signals

step_measure_rtlsim_performance temporarily forces depth 3 when verify_save_rtlsim_waveforms is set, restoring the original afterwards.

Shared infrastructure

The layer-parallel simulation used for FIFO sizing is the same machinery: simulation_build.py compiles per-node backends and simulation_connected.py runs them as communicating processes, optionally across MPI. step_measure_rtlsim_performance constructs a NodeConnectedSimulation directly with performance_sim=True.

See also

• Functional Verification — the user-facing guide
• FIFO Sizing Internals — same simulation infrastructure
• Custom Operations — how a node provides its execution paths
• Multi-Layer Offload Internals