You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The proposed commit message follows Conventional Commits.
Bugs / Features
Related issue(s) linked. Reproduced directly on main.
Tests for the changes have been added.
Docs have been added or updated.
A patch changeset for @voltagent/rag has been added.
What is the current behavior?
When an oversized AST block is split by the token chunker, its output IDs add both the callback index and the current output length. Later AST blocks can reuse an ID already emitted by the split.
What is the new behavior?
Derive each split chunk ID from the number of chunks already emitted. The regression covers a large AST block and two subsequent small blocks.
Notes for reviewers
The upstream fixture produces four unique IDs for five chunks. The patched regression also checks the exact sequence from code-0 through code-4.
vitest run --typecheck passed 48 RAG tests across 20 files.
Scoped TypeScript, changed-file Biome, changeset Prettier and whitespace checks passed.
CommonJS, ESM and type declaration builds passed.
OpenAI Codex assisted with reproduction, implementation and local validation.
Summary by cubic
Fixes duplicate chunk IDs when token splitting expands an oversized AST block before other blocks. Split chunks now derive their ID from the number of chunks already emitted instead of combining the callback index with the output length, so later blocks can no longer reuse an ID from the split.
Adds a regression test for a large AST block followed by two smaller blocks, plus a patch changeset for @voltagent/rag.
Written for commit ca72ade. Summary will update on new commits.
Summary by CodeRabbit
Bug Fixes
Fixed duplicate code chunk IDs when a large syntax block is split into multiple chunks before other blocks. Chunk IDs now remain unique and sequential.
No actionable comments were generated in the recent review.
Walkthrough
The AST code chunker now assigns token-based chunk IDs from the current output length. A regression test checks that IDs remain sequential and unique when an oversized AST block is split before two smaller blocks.
Changes: AST chunk ID assignment
Layer / File(s)
Summary
Chunk ID assignment and regression coverage packages/rag/src/chunkers/code-chunker.ts, packages/rag/src/chunkers/code-chunker-ast-ids.spec.ts, .changeset/unique-code-ast-chunk-ids.md
The chunker assigns each token chunk an ID based on the current output length. The regression test checks five sequential, unique IDs. A changeset declares a patch release for @voltagent/rag.
This fixes duplicate chunk IDs when an oversized code block is split. The change is small and covered by a regression test, so merge risk is minimal.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: a fix in the rag package so that IDs for oversized AST chunks stay unique. The changeset, the regression test, and the one-line change in code-chunker.ts all match thi…
Description check
The description follows the repository template. It includes the PR checklist, the current and new behavior, and reviewer notes. The author checked the test and changeset items. Two items stay uncheck…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: a fix in the rag package so that IDs for oversized AST chunks stay unique. The changeset, the regression test, and the one-line change in code-chunker.ts all match this. The title follows the conventional commit style and is 45 characters long.
Full details: Description check
Explanation
The description follows the repository template. It includes the PR checklist, the current and new behavior, and reviewer notes. The author checked the test and changeset items. Two items stay unchecked: related issue linking (the author notes the bug was reproduced on main but gives no issue) and docs. The template also expects a 'fixes (issue)' line, which is missing. The description is mostly complete, so the check passes.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
Bugs / Features
@voltagent/raghas been added.What is the current behavior?
When an oversized AST block is split by the token chunker, its output IDs add both the callback index and the current output length. Later AST blocks can reuse an ID already emitted by the split.
What is the new behavior?
Derive each split chunk ID from the number of chunks already emitted. The regression covers a large AST block and two subsequent small blocks.
Notes for reviewers
code-0throughcode-4.vitest run --typecheckpassed 48 RAG tests across 20 files.OpenAI Codex assisted with reproduction, implementation and local validation.
Summary by cubic
Fixes duplicate chunk IDs when token splitting expands an oversized AST block before other blocks. Split chunks now derive their ID from the number of chunks already emitted instead of combining the callback index with the output length, so later blocks can no longer reuse an ID from the split.
Adds a regression test for a large AST block followed by two smaller blocks, plus a patch changeset for
@voltagent/rag.Written for commit ca72ade. Summary will update on new commits.
Summary by CodeRabbit