Repository navigation
[fix](bench): encode FP8 fixtures as E4M3FN bytes - #1
Open
FranklinNexus wants to merge 1 commit into
Open
FranklinNexus wants to merge 1 commit into
FranklinNexus wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Follow-up to kvcache-ai#2267.
The FP8 benchmark fixture rounds scaled FP32 weights and numerically casts them to
uint8, butAVX2FP8_MOEinterprets those bytes as E4M3 values. For a block containing448and1, the scale is1: the existing fixture emits0x01for1, which decodes to2^-9, instead of the E4M3FN encoding0x38.Extract the blockwise fixture into a CPU-only helper and convert the normalized values to
torch.float8_e4m3fnbefore viewing their raw bytes astorch.uint8. Preserve the block layout, per-block scales, and NumPy output types.Add a CPU CI regression test with an independent scalar format reference. It covers all 254 finite encodings, signed zero, rounding at and around subnormal/exponent boundaries, per-block scales, partial blocks, and noncontiguous input. The test has a script entry point for the repository's CI runner.
Apply the repository's Black configuration to all three changed Python files. The remaining benchmark changes are formatting only; its RAWINT4 and timing code are AST-identical to the base revision.
Validation on Windows, Python 3.13.13, and the pinned PyTorch 2.9.1 (
2.9.1+cpu), with third-party pytest plugin autoload disabled:python -m pytest kt-kernel/test/per_commit/test_bench_fp8_fixture.py -q --tb=short: 20 passed.python kt-kernel/test/per_commit/test_bench_fp8_fixture.py: 20 passed, confirming the script-style CI entry point runs the tests.git diff --check: passed.defaultsuite.The native extension and full performance benchmark were not run. This change corrects fixture encoding and makes no kernel-performance claim.
Before submitting