Repository navigation
fix(core): persist approved tool results immediately after execution #1541
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tomwangcn
wants to merge
17
commits into
VoltAgent:main
Choose a base branch
from
tomwangcn:fix/core-approved-tool-result-flush
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 7 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
ff2da6b
fix(core): persist client approval decisions on buffered tool parts
tomwangcn 359b9f4
fix(core): persist approved tool results immediately after execution
tomwangcn 1577ff6
fix(core): queue approval decisions and keep terminal evidence
tomwangcn 107e066
fix(core): seed explicit approval input when memory omits the call
tomwangcn 0f7eabe
Merge branch 'fix/core-approval-persistence' into fix/core-approved-t…
9183914
fix(core): serialize approval input saves with result persistence
tomwangcn 6724ebf
test(core): force concurrent input and result write ordering
tomwangcn 7eb35be
fix(core): hydrate complete approval owners before pending upserts
tomwangcn ca11774
fix(core): retain full omitted owners and test runtime context limits
tomwangcn 44c59d8
test(core): preserve stored sibling parts with partial approval input
tomwangcn 9f9b2ff
Merge branch 'fix/core-approval-persistence' into fix/core-approved-t…
2281c53
fix(core): retain incoming text and stored terminal approval evidence
tomwangcn 09fa531
fix(core): merge approval hydration corrections into result persistence
tomwangcn 1c0e2b1
fix(core): preserve omitted attachment types during approval hydration
tomwangcn fbbdf3f
Merge branch 'fix/core-approval-persistence' into fix/core-approved-t…
f81a3ef
fix(core): match hydrated attachments by file identity
tomwangcn 7103a38
Merge branch 'fix/core-approval-persistence' into fix/core-approved-t…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@voltagent/core": patch | ||
| --- | ||
|
|
||
| fix(core): persist client approval decisions onto buffered tool parts so restored conversations keep `approval-responded` state instead of re-validating stale parts and throwing `AI_TypeValidationError` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@voltagent/core": patch | ||
| --- | ||
|
|
||
| fix(core): flush approved `needsApproval` tool results to memory as soon as execution completes and preserve executed tool parts when a client resends an older message, so an interrupted turn no longer causes a resent approval to re-execute the tool without a new approval prompt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| /** | ||
| * Regression tests for https://github.com/VoltAgent/voltagent/issues/1535 | ||
| * | ||
| * After a `needsApproval` tool call is approved and runs, memory must store | ||
| * the tool part as `state: "output-available"` WITH `approval.approved` | ||
| * preserved. Previously the buffer was seeded only from stored memory (which | ||
| * still held the pre-approval part, `approval: { id }`), the approval | ||
| * response in the input never reached the buffer, and the flush overwrote | ||
| * the input message - so the stored part lost `approved: true` and every | ||
| * later `generateText` on the conversation threw AI_TypeValidationError. | ||
| */ | ||
| import { MockLanguageModelV3 } from "ai/test"; | ||
| import { describe, expect, it } from "vitest"; | ||
| import { z } from "zod"; | ||
| import { Memory } from "../memory"; | ||
| import { InMemoryStorageAdapter } from "../memory/adapters/storage/in-memory"; | ||
| import { createTool } from "../tool"; | ||
| import { Agent } from "./agent"; | ||
|
|
||
| const usage = { | ||
| inputTokens: 10, | ||
| outputTokens: 5, | ||
| totalTokens: 15, | ||
| inputTokenDetails: { | ||
| noCacheTokens: 10, | ||
| cacheReadTokens: 0, | ||
| cacheWriteTokens: 0, | ||
| }, | ||
| outputTokenDetails: { | ||
| textTokens: 5, | ||
| reasoningTokens: 0, | ||
| }, | ||
| }; | ||
|
|
||
| const makeAgent = (storage: InMemoryStorageAdapter, contextLimit?: number) => { | ||
| const model = new MockLanguageModelV3({ | ||
| modelId: "test-model", | ||
| doGenerate: async ({ prompt }) => { | ||
| await new Promise((r) => setTimeout(r, 1)); | ||
| return prompt.some((m: any) => m.role === "tool") | ||
| ? { | ||
| content: [{ type: "text" as const, text: "Refund done." }], | ||
| finishReason: { unified: "stop" as const, raw: "stop" }, | ||
| usage, | ||
| warnings: [], | ||
| } | ||
| : { | ||
| content: [ | ||
| { | ||
| type: "tool-call" as const, | ||
| toolCallId: "call_1", | ||
| toolName: "refund", | ||
| input: '{"amount":50}', | ||
| }, | ||
| ], | ||
| finishReason: { unified: "tool-calls" as const, raw: "tool_use" }, | ||
| usage, | ||
| warnings: [], | ||
| }; | ||
| }, | ||
| }); | ||
|
|
||
| return new Agent({ | ||
| name: "a", | ||
| instructions: "x", | ||
| model: model as any, | ||
| memory: new Memory({ storage, options: { contextLimit } }), | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| tools: [ | ||
| createTool({ | ||
| name: "refund", | ||
| description: "refund", | ||
| parameters: z.object({ amount: z.number() }), | ||
| needsApproval: true, | ||
| execute: async () => ({ refunded: true }), | ||
| }), | ||
| ], | ||
| }); | ||
| }; | ||
|
|
||
| const memory = { userId: "u", conversationId: "c" }; | ||
|
|
||
| const storedToolParts = async (storage: InMemoryStorageAdapter) => { | ||
| const stored = await storage.getMessages("u", "c"); | ||
| return stored.flatMap((m: any) => m.parts ?? []).filter((p: any) => p.type === "tool-refund"); | ||
| }; | ||
|
|
||
| describe("needsApproval persistence (issue #1535)", () => { | ||
| it("keeps approval.approved when an approved tool's result is persisted, and later turns work", async () => { | ||
| const storage = new InMemoryStorageAdapter(); | ||
| const agent = makeAgent(storage); | ||
|
|
||
| // Turn 1: model calls the tool, agent asks for approval. | ||
| await agent.generateText("refund 50", { memory }); | ||
|
|
||
| const stored1 = await storage.getMessages("u", "c"); | ||
| const msg = structuredClone(stored1.at(-1)) as any; | ||
| const part = msg.parts.find((p: any) => p.type === "tool-refund"); | ||
| expect(part.state).toBe("approval-requested"); | ||
|
|
||
| // Client approves (shape produced by the AI SDK's addToolApprovalResponse). | ||
| part.state = "approval-responded"; | ||
| part.approval = { id: part.approval.id, approved: true }; | ||
|
|
||
| // Turn 2: tool runs. | ||
| await agent.generateText([msg], { memory }); | ||
|
|
||
| const parts2 = await storedToolParts(storage); | ||
| expect(parts2).toHaveLength(1); | ||
| expect(parts2[0].state).toBe("output-available"); | ||
| expect(parts2[0].output).toBeDefined(); | ||
| expect(parts2[0].approval).toMatchObject({ approved: true }); | ||
|
|
||
| // Turn 3: a plain follow-up must not fail AI SDK UI message validation. | ||
| const result = await agent.generateText("did it go through?", { memory }); | ||
| expect(result.text).toBe("Refund done."); | ||
| }, 30000); | ||
|
|
||
| it("keeps approval.approved === false when the client denies the tool", async () => { | ||
| const storage = new InMemoryStorageAdapter(); | ||
| const agent = makeAgent(storage); | ||
|
|
||
| await agent.generateText("refund 50", { memory }); | ||
|
|
||
| const stored1 = await storage.getMessages("u", "c"); | ||
| const msg = structuredClone(stored1.at(-1)) as any; | ||
| const part = msg.parts.find((p: any) => p.type === "tool-refund"); | ||
|
|
||
| // Client denies. | ||
| part.state = "approval-responded"; | ||
| part.approval = { id: part.approval.id, approved: false }; | ||
|
|
||
| await agent.generateText([msg], { memory }); | ||
|
|
||
| const parts2 = await storedToolParts(storage); | ||
| expect(parts2).toHaveLength(1); | ||
| // The client's denial decision is preserved, never rewritten to an approval. | ||
| expect(parts2[0].approval?.approved).toBe(false); | ||
| }, 30000); | ||
| }); | ||
|
|
||
| it("keeps approval evidence when memory context is omitted", async () => { | ||
| const storage = new InMemoryStorageAdapter(); | ||
| const agent = makeAgent(storage, 0); | ||
| await agent.generateText("refund 50", { memory }); | ||
| const stored = await storage.getMessages("u", "c"); | ||
| const message = structuredClone(stored.at(-1)) as any; | ||
| const part = message.parts.find((p: any) => p.type === "tool-refund"); | ||
| part.state = "approval-responded"; | ||
| part.approval.approved = true; | ||
| await agent.generateText([message], { memory }); | ||
| const parts = await storedToolParts(storage); | ||
| expect(parts[0]).toMatchObject({ state: "output-available", approval: { approved: true } }); | ||
| }); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seed the approved call before merging its result.
With
memory.options.contextLimit: 0,prepareConversationContextreturns no messages.generateTextdoes not add the incoming approval message to the buffer. This tool-onlyaddModelMessagescall then reachesmergeAssistantMessagewithrequireExisting: trueand returns without adding the result. The flush has nothing to save.Seed the incoming approved tool part, including its input and approval, before merging the result. Cover an interruption before
onStepFinishwith an empty memory context.