Skip to content

[Automated] Draft docs (agentgateway): llm: add original model as first class concept - #1442

Open
github-actions[bot] wants to merge 3 commits into
mainfrom
pr-tracker-draft-agentgateway-4215
Open

github-actions[bot] wants to merge 3 commits into
mainfrom
pr-tracker-draft-agentgateway-4215

Conversation

@github-actions

@github-actions github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Rendered output was not compared with the base build: the site does not build at the base commit ed3fb85, so rendering outside the declared versions is unchecked.

agentgateway/agentgateway#4215 — llm: add original model as first class concept

Docs versions: standalone/main, kubernetes/main

No code-path rule covers 3 touched file(s). The planner could route these only by identifier search, so the candidate pages may miss what they changed, including in the other mode:

  • crates/agentgateway/src/proxy/httpproxy.rs
  • crates/llm/src/lib.rs
  • crates/llm/src/types/detect.rs

What changes for users: CEL expressions can read llm.originalModel, which records the model name that the client sent before virtual model routing, aliases, provider overrides, or request body rewrites. Model cost lookup now tries the response model, then the request model, then llm.originalModel, so catalogs can price requests whose model was rewritten.

What the docs now say: The standalone and Kubernetes CEL variables pages explain when to use llm.originalModel. The standalone and Kubernetes model-cost pages explain the fallback order that model cost lookup uses when the first model does not match a catalog entry.

Not verified: No cluster or proxy run; the configuration is unrun, and internal links are unchecked.

How agentgateway-4215 was drafted, and what was not verified

Plan, and what changed it

  • Planned: Document the new llm.originalModel CEL field and the model-cost lookup fallback in both declared documentation trees.
  • Learned: The generated CEL reference already lists llm.originalModel, and the trace attribute page already describes agw.ai.original_model, so the non-generated CEL variables and model-cost pages were the right authored surfaces.
  • Did: Updated the standalone and Kubernetes CEL variables pages, the Kubernetes model-cost page, and the shared standalone cost-catalog asset.

Why a documentation change is necessary

The pull request adds llm.originalModel to the CEL schema and records the client-sent model before model rewrites. It also changes model cost lookup to fall back from the response model to the request model and then to the original model. The existing authored CEL variables pages described llm.requestModel but did not explain how to read the original client model, and the model-cost pages did not describe the new fallback order.

What changed on disk

  • content/docs/standalone/main/reference/cel/variables.md: Adds llm.originalModel to the LLM CEL variable guidance.
  • content/docs/kubernetes/main/reference/cel/variables.md: Adds llm.originalModel to the LLM CEL variable guidance.
  • content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md: Documents the model-cost lookup fallback order.
  • assets/agw-docs/standalone/cost-catalog.md: Documents the model-cost lookup fallback order for standalone main, gated out of released trees.

How to verify this change

  1. Configure an LLM route that rewrites the requested model, such as a virtual model from the virtual models guide.
  2. Add an access log or metric CEL expression that reads llm.originalModel, then send a request through the rewritten model. The emitted field should show the model name that the client sent.
  3. Configure a model cost catalog entry for the original client model but not for the rewritten request or response model, then send the request again. The access log or metrics should show a priced request instead of a missing catalog lookup.

What was not verified

  • Internal links are unchecked because Hugo was not run.
  • The cost fallback behavior was read from the diff and tests, but no request was sent through a running proxy.

Target pages checked and left alone

  • assets/agw-docs/pages/agentgateway/llm/api-types/gemini.md: Native Gemini wire format page; not the owner for CEL variables or catalog lookup behavior.
  • assets/agw-docs/pages/agentgateway/llm/cost-tracking.md: Prometheus cost-estimation guide; the realized-cost catalog pages own this behavior.
  • assets/agw-docs/pages/observability/traces/attribute-reference.md: Already documents agw.ai.original_model; the new CEL field is generated and documented on the CEL variables pages.
  • assets/agw-docs/pages/reference/cel/custom-functions.md: Custom function authoring page; not the owner for built-in llm fields.
  • assets/agw-docs/pages/security/rate-limit-http.md: Local rate-limiting guide; the change does not add rate-limit configuration.
  • assets/agw-docs/pages/traffic-management/transformations/access-logs.md: Transformation/access-log example page; the CEL variables pages own field availability.
  • content/docs/standalone/main/documentation/configuration/resiliency/rate-limits.md: Standalone rate-limit page; the change does not add rate-limit configuration.
  • Standalone LLM API-type pages (completions.md, embeddings.md, gemini.md, messages.md, rerank.md, responses.md, and token-count.md): Wire-format pages; no one API type owns the cross-format original-model field.
  • Standalone LLM overview, playground, prompt-guards, and virtual-models pages: These pages already point readers to telemetry for rewritten models or cover other workflows.
  • Standalone and Kubernetes access-log index/view/export/database pages: The change adds a CEL field and cost lookup behavior, not a new logging workflow.
  • Standalone UI pages: The change has no UI-specific configuration.
  • Standalone and Kubernetes CEL section index, playground, and YAML example pages: They are index/example pages; the variables page owns field availability.
  • Kubernetes model pages (about.md, serve.md, virtual.md, and _index.md): These pages already cover model routing, and the new field is better documented in the CEL and model-cost surfaces.
  • content/docs/kubernetes/main/documentation/llm/guardrails/overview.md: Guardrails page; the change does not add guardrail configuration.
  • content/docs/kubernetes/main/documentation/observability/access-logs/_index.md and view.md: Access-log pages; the change does not add a new logging procedure.

Counterpart sections not drafted

  • kgateway-dev/kgateway.dev:content/docs/envoy/main/security/ratelimit/local.md: Not used. The counterpart covers local rate limiting, and this change documents LLM CEL and model-cost behavior instead.
How the pages were chosen (triage report)

New configuration: no page mentions it yet

agentgateway/agentgateway#4215 — llm: add original model as first class concept

  • Product: agentgateway (upstream) → agentgateway/website
  • Docs version: standalone/main, kubernetes/main
  • Summary (read from the body): 1. its exposed in CEL to users who want to use it 2. In cost lookup, we use response model > request model > original model.
  • Not tested: no cluster was available, so every command in this diff is unrun
  • No rule covers 3 touched file(s). The candidate pages below come from the files that DID match a rule, so they may scope this narrower than the change is. Read these before deciding the change is only about the pages listed:
    • crates/agentgateway/src/proxy/httpproxy.rs
    • crates/llm/src/lib.rs
    • crates/llm/src/types/detect.rs

A surface this change does not reach

them again: they describe a field on a deployment mode the code change
never reached, so a reader following them would configure something that
does not exist yet. The rest of the draft is untouched.

  • content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md

Either the control-plane work for that mode is still owed, or the topic
belongs only to the other mode. The run summary names which surface and
which identifiers.

Release notes

This pull request also adds its release-note entries:

  • content/docs/kubernetes/main/release-notes/release-notes.md: Original client model available in CEL and cost lookup (feature, in a new 1.8 section)
  • content/docs/standalone/main/release-notes/release-notes.md: Original client model available in CEL and cost lookup (feature, in a new 1.8 section)

Draft branch: pr-tracker-draft-agentgateway-4215

…st class concept

Drafts-Commit: 4cce188
Signed-off-by: GitHub Action <action@github.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Deploying agentproxy with  Cloudflare Pages  Cloudflare Pages

Latest commit: 72e3790
Status: ✅  Deploy successful!
Preview URL: https://0e1411f5.agentproxy.pages.dev
Branch Preview URL: https://pr-tracker-draft-agentgatewa-yzua.agentproxy.pages.dev

View logs

The release-notes entry for this change now lives in the combined
release-notes PR, #1482, so the draft PRs no longer conflict on
the release-notes pages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
@kristin-kronstain-brown

Copy link
Copy Markdown
Collaborator

Moved this PR's release-notes entry to the combined release-notes PR, #1482, so the draft PRs stop conflicting on the release-notes pages. Review or change the entry there.

Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants