Repository navigation
Add optional extract function to variables - #3
Merged
Merged
Conversation
A variable can set `extract`, a function that receives the whole unpacked frame and returns the variable frame, or `nothing` when the variable is not in that packet. Absent variables are `missing` in every view and their transfer functions are not called. This supports databases where a parameter's location depends on the packet content (APID, frame format, subcommutation counter), which a fixed `position` and `size` cannot express. The field defaults to `nothing`, the 11-argument positional constructor is kept, and variables without `extract` follow the same code path as before.
Adding a type parameter to ExecutionNode made plan building about 0.4 us slower per variable, even for variables without an extract callback. Those variables now use the unchanged ExecutionNode, and only extract-backed ones are wrapped in an ExtractExecutionNode. Nodes are pushed from each branch so plan building does not test which node type it holds.
RianKoja
marked this pull request as ready for review
October 21, 2026 01:40
Member
|
Thanks @RianKoja ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional
extractfield toTelemetryVariableDescription(and keyword toadd_variable!) for variables whose location in the frame depends on the packet content.Motivation
In many telemetry formats, the same parameter can sit at different offsets depending on a selector in the packet, such as the APID, a frame format code, or a subcommutation counter. A fixed
positionandsizecan describe only one of those placements, andbtfonly receives that slice, so it cannot check the selector.Behavior
extract(frame)receives the whole unpacked frame and returns the variable frame passed tobtf, already in the variable byte order (endiannessis not applied).nothingmeans the variable is not in this packet: every view of it ismissing, its transfer functions are not called, and dependents see(raw = missing, processed = missing)in the context.position = 0andsize = 0.Backward compatibility
The change is intended as a minor release (3.1.0):
nothing, so the keyword constructor is unchanged, and the previous 11-argument positional constructor is kept as an outer constructor (the test suite itself uses it).add_variable!gains one keyword with a default. Existing calls are unchanged.ExecutionNodeis unchanged. Variables withextractare wrapped in a new internalExtractExecutionNode, so variables without it follow the same path as before.save_telemetryserializes packets, not variable descriptions.Checked:
extract(present and absent, every view, dependents, validation, the 11-argument constructor).isequalper column) with 3.0.1 and with this branch.benchmark/benchmarks.jl(best of two runs each, 1 thread): every scenario is within 4% ofmainexceptpackets-128, which is 15% faster, and allocation counts are equal or lower. Plan building for variables withoutextractcosts the same as onmain.extract, decoding 2000 packets x 32 variables (half of them absent) takes about 6% longer than the same variables read by position.The version in
Project.tomlis left for the release commit.