Repository navigation
[Automated] Draft docs (agentgateway): llm: add original model as first class concept - #1442
Open
github-actions[bot] wants to merge 3 commits into
Open
github-actions[bot] wants to merge 3 commits into
github-actions[bot] wants to merge 3 commits into
Conversation
…st class concept Drafts-Commit: 4cce188 Signed-off-by: GitHub Action <action@github.com>
Deploying agentproxy with
|
| Latest commit: |
72e3790
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0e1411f5.agentproxy.pages.dev |
| Branch Preview URL: | https://pr-tracker-draft-agentgatewa-yzua.agentproxy.pages.dev |
The release-notes entry for this change now lives in the combined release-notes PR, #1482, so the draft PRs no longer conflict on the release-notes pages. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Kristin Brown <kristin.brown@solo.io>
Collaborator
|
Moved this PR's release-notes entry to the combined release-notes PR, #1482, so the draft PRs stop conflicting on the release-notes pages. Review or change the entry there. |
Signed-off-by: Kristin Brown <kristin.brown@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rendered output was not compared with the base build: the site does not build at the base commit ed3fb85, so rendering outside the declared versions is unchecked.
agentgateway/agentgateway#4215 — llm: add original model as first class concept
Docs versions:
standalone/main,kubernetes/mainNo code-path rule covers 3 touched file(s). The planner could route these only by identifier search, so the candidate pages may miss what they changed, including in the other mode:
crates/agentgateway/src/proxy/httpproxy.rscrates/llm/src/lib.rscrates/llm/src/types/detect.rsWhat changes for users: CEL expressions can read
llm.originalModel, which records the model name that the client sent before virtual model routing, aliases, provider overrides, or request body rewrites. Model cost lookup now tries the response model, then the request model, thenllm.originalModel, so catalogs can price requests whose model was rewritten.What the docs now say: The standalone and Kubernetes CEL variables pages explain when to use
llm.originalModel. The standalone and Kubernetes model-cost pages explain the fallback order that model cost lookup uses when the first model does not match a catalog entry.Not verified: No cluster or proxy run; the configuration is unrun, and internal links are unchecked.
How
agentgateway-4215was drafted, and what was not verifiedPlan, and what changed it
llm.originalModelCEL field and the model-cost lookup fallback in both declared documentation trees.llm.originalModel, and the trace attribute page already describesagw.ai.original_model, so the non-generated CEL variables and model-cost pages were the right authored surfaces.Why a documentation change is necessary
The pull request adds
llm.originalModelto the CEL schema and records the client-sent model before model rewrites. It also changes model cost lookup to fall back from the response model to the request model and then to the original model. The existing authored CEL variables pages describedllm.requestModelbut did not explain how to read the original client model, and the model-cost pages did not describe the new fallback order.What changed on disk
content/docs/standalone/main/reference/cel/variables.md: Addsllm.originalModelto the LLM CEL variable guidance.content/docs/kubernetes/main/reference/cel/variables.md: Addsllm.originalModelto the LLM CEL variable guidance.content/docs/kubernetes/main/documentation/llm/cost-controls/costs.md: Documents the model-cost lookup fallback order.assets/agw-docs/standalone/cost-catalog.md: Documents the model-cost lookup fallback order for standalone main, gated out of released trees.How to verify this change
llm.originalModel, then send a request through the rewritten model. The emitted field should show the model name that the client sent.What was not verified
Target pages checked and left alone
assets/agw-docs/pages/agentgateway/llm/api-types/gemini.md: Native Gemini wire format page; not the owner for CEL variables or catalog lookup behavior.assets/agw-docs/pages/agentgateway/llm/cost-tracking.md: Prometheus cost-estimation guide; the realized-cost catalog pages own this behavior.assets/agw-docs/pages/observability/traces/attribute-reference.md: Already documentsagw.ai.original_model; the new CEL field is generated and documented on the CEL variables pages.assets/agw-docs/pages/reference/cel/custom-functions.md: Custom function authoring page; not the owner for built-inllmfields.assets/agw-docs/pages/security/rate-limit-http.md: Local rate-limiting guide; the change does not add rate-limit configuration.assets/agw-docs/pages/traffic-management/transformations/access-logs.md: Transformation/access-log example page; the CEL variables pages own field availability.content/docs/standalone/main/documentation/configuration/resiliency/rate-limits.md: Standalone rate-limit page; the change does not add rate-limit configuration.completions.md,embeddings.md,gemini.md,messages.md,rerank.md,responses.md, andtoken-count.md): Wire-format pages; no one API type owns the cross-format original-model field.about.md,serve.md,virtual.md, and_index.md): These pages already cover model routing, and the new field is better documented in the CEL and model-cost surfaces.content/docs/kubernetes/main/documentation/llm/guardrails/overview.md: Guardrails page; the change does not add guardrail configuration.content/docs/kubernetes/main/documentation/observability/access-logs/_index.mdandview.md: Access-log pages; the change does not add a new logging procedure.Counterpart sections not drafted
kgateway-dev/kgateway.dev:content/docs/envoy/main/security/ratelimit/local.md: Not used. The counterpart covers local rate limiting, and this change documents LLM CEL and model-cost behavior instead.How the pages were chosen (triage report)
New configuration: no page mentions it yet
agentgateway/agentgateway#4215 — llm: add original model as first class concept
agentgateway/websitecrates/agentgateway/src/proxy/httpproxy.rscrates/llm/src/lib.rscrates/llm/src/types/detect.rsA surface this change does not reach
them again: they describe a field on a deployment mode the code change
never reached, so a reader following them would configure something that
does not exist yet. The rest of the draft is untouched.
content/docs/kubernetes/main/documentation/llm/cost-controls/costs.mdEither the control-plane work for that mode is still owed, or the topic
belongs only to the other mode. The run summary names which surface and
which identifiers.
Release notes
This pull request also adds its release-note entries:
content/docs/kubernetes/main/release-notes/release-notes.md: Original client model available in CEL and cost lookup (feature, in a new 1.8 section)content/docs/standalone/main/release-notes/release-notes.md: Original client model available in CEL and cost lookup (feature, in a new 1.8 section)Draft branch:
pr-tracker-draft-agentgateway-4215