Skip to content

[fix](bench): encode FP8 fixtures as E4M3FN bytes - #1

Open
FranklinNexus wants to merge 1 commit into
YuvrajSinghBhadoria2:feat/anchor-mechanism-kvcachefrom
FranklinNexus:fix/fp8-benchmark-fixture
Open

FranklinNexus wants to merge 1 commit into
YuvrajSinghBhadoria2:feat/anchor-mechanism-kvcachefrom
FranklinNexus:fix/fp8-benchmark-fixture

Conversation

@FranklinNexus

Copy link
Copy Markdown

What does this PR do?

Follow-up to kvcache-ai#2267.

The FP8 benchmark fixture rounds scaled FP32 weights and numerically casts them to uint8, but AVX2FP8_MOE interprets those bytes as E4M3 values. For a block containing 448 and 1, the scale is 1: the existing fixture emits 0x01 for 1, which decodes to 2^-9, instead of the E4M3FN encoding 0x38.

Extract the blockwise fixture into a CPU-only helper and convert the normalized values to torch.float8_e4m3fn before viewing their raw bytes as torch.uint8. Preserve the block layout, per-block scales, and NumPy output types.

Add a CPU CI regression test with an independent scalar format reference. It covers all 254 finite encodings, signed zero, rounding at and around subnormal/exponent boundaries, per-block scales, partial blocks, and noncontiguous input. The test has a script entry point for the repository's CI runner.

Apply the repository's Black configuration to all three changed Python files. The remaining benchmark changes are formatting only; its RAWINT4 and timing code are AST-identical to the base revision.

Validation on Windows, Python 3.13.13, and the pinned PyTorch 2.9.1 (2.9.1+cpu), with third-party pytest plugin autoload disabled:

  • python -m pytest kt-kernel/test/per_commit/test_bench_fp8_fixture.py -q --tb=short: 20 passed.
  • python kt-kernel/test/per_commit/test_bench_fp8_fixture.py: 20 passed, confirming the script-style CI entry point runs the tests.
  • The same integer-cast regression fails against the original helper, as expected.
  • Black check on all three changed files and git diff --check: passed.
  • CI registration discovery: CPU default suite.

The native extension and full performance benchmark were not run. This change corrects fixture encoding and makes no kernel-performance claim.

Before submitting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant