Skip to content

fix(ai-sdk): log AI SDK v7 inline file data as attachments - #2751

Closed
Paulo Martins (pmartinsdev) wants to merge 1 commit into
braintrustdata:mainfrom
pmartinsdev:fix/ai-sdk-v7-inline-file-attachments
Closed

Paulo Martins (pmartinsdev) wants to merge 1 commit into
braintrustdata:mainfrom
pmartinsdev:fix/ai-sdk-v7-inline-file-attachments

Conversation

@pmartinsdev

Copy link
Copy Markdown

Fixes #2750.

Context

With AI SDK 7, file parts reach the model spans as { type: "file", mediaType, data: { type: "data", data: Uint8Array } } (SharedV4FileData). processContentPart passed part.data, the wrapper, to convertDataToAttachment, which only handles a string, Uint8Array, Buffer or Blob, so no attachment was made and deepCopyEvent copied the raw bytes as an object with one key per byte. 1 MB of image becomes about 120 MB of heap on every model call.

We call Claude through Vertex, which takes no image URLs, so the AI SDK downloads every image in the conversation and each model call carries all of them. A 38-image call (24.5 MB of images) peaked at 2,651 MB of heap with wrapAISDK and at 117 MB without it, and our 4 GB serverless instances were killed for memory.

Description

processContentPart now unwraps { type: "data", data } before calling convertDataToAttachment, so the bytes (or a base64 string) become an attachment the same way a plain Uint8Array already does. The other tagged variants (url, reference, text) are left as they were, and a Blob is never unwrapped since the check requires a data key.

Testing

  • A new test in ai-sdk-plugin.test.ts sends a tagged Uint8Array file part through processAISDKCallInput and checks that it is logged as an Attachment. It fails without the fix and passes with it.
  • vitest run src/instrumentation/plugins/ai-sdk-plugin.test.ts (134 passed), vitest run src/wrappers/ai-sdk (20 passed), pnpm run check:typings and prettier are clean. Changeset included (patch).
  • We applied the same change to 3.32.0 with pnpm patch, and the same 38-image call peaked at 130 MB of heap with tracing on.

@AbhiPrasad

Copy link
Copy Markdown
Member

hey Paulo Martins (@pmartinsdev) thanks for the PR! I opened #2752 on top of it to add some more tests. Gonna close this one in favor of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

wrapAISDK copies AI SDK v7 file bytes one key per byte instead of turning them into attachments

2 participants