Skip to content

docs: SDK guide with Go examples and LiteLLM SDK migration - #1484

Open
jamesilse-solo wants to merge 4 commits into
agentgateway:mainfrom
jamesilse-solo:sdk-guide
Open

jamesilse-solo wants to merge 4 commits into
agentgateway:mainfrom
jamesilse-solo:sdk-guide

Conversation

@jamesilse-solo

Copy link
Copy Markdown

Reframes the standalone and Kubernetes OpenAI SDK pages as guides to using SDKs with agentgateway, per the Slack thread on SDK golden paths.

Changes

  • Retitle the page to "SDKs (Python, Node.js, Go)". The URL stays openai-sdk/.
  • Intro explains that the OpenAI SDK reaches any configured provider; agentgateway translates where needed.
  • Add a Go example (openai-go/v3).
  • Add "Call more than one provider" with the openai/* and anthropic/* provider-prefix pattern.
  • Add "Migrate from the LiteLLM SDK": same provider/model strings, before/after code, and a call mapping table.
  • Add "Authenticate with a JWT": pass the IdP token as the SDK API key.
  • Link the LiteLLM config import guide (main, latest) to the new migration section.

Kubernetes page:

  • Fix the base URL note. It said to leave /v1 out of base_url; the SDK appends only /chat/completions, so requests returned 404 with the default route and with the /openai custom route. The page now says to end the base URL with /v1.
  • Add the same Go, multi-provider, LiteLLM, and JWT sections, using AgentgatewayModel and AgentgatewayPolicy.
  • Add a warning: after a listener allows AgentgatewayModel, it serves /v1/chat/completions from models, so an unprefixed model name that an HTTPRoute used to serve returns model not found. HTTPRoutes on other paths, such as /openai, keep working.

The multi-provider and LiteLLM sections render for 1.6.x and later only.

Testing

Ran against agentgateway 1.7.0 and 1.6.0-alpha.2 with the same results.

The page's Python, Node.js (openai 7.52.0), and Go (openai-go/v3 v3.96.0) snippets, run as written against the page's catch-all config: OpenAI returned responses for all three.

With jwtAuth (strict) and the provider-prefix config:

  • Python sync, streaming, and AsyncOpenAI; Go openai-go/v3: OpenAI returned responses for openai/gpt-4o-mini
  • anthropic/claude-sonnet-4-5: routed to api.anthropic.com with the prefix stripped (no Anthropic key, so it stopped at Anthropic auth)
  • JWT passed as api_key: accepted, jwt.sub in access logs; missing or bad token returns 401

Kubernetes: agentgateway 1.7.0 on kind.

  • The page's Python, Node.js, and Go snippets, run as written with INGRESS_GW_ADDRESS set to a port-forward: OpenAI returned responses for all three.
  • Base URL without /v1: 404 on the default route and the /openai route. With /v1 (or /openai/v1): 200.
  • AgentgatewayModel with openai/* and anthropic/*, plus a strict jwtAuthentication policy: same results as standalone for Python sync, streaming, async, and Go.
  • Models and the default HTTPRoute on one listener: prefixed models work, unprefixed gpt-4o-mini returns model not found, /openai/v1 still works.

Not covered: Kubernetes 1.6.x, and page titles for 1.6.x and older.

jamesilse-solo and others added 4 commits November 5, 2026 18:34
…gration

- Retitle the standalone OpenAI SDK page to cover Python, Node.js, and Go
- Explain that the OpenAI SDK reaches any provider through agentgateway
- Add a Go example with openai-go/v3
- Add multi-provider (provider prefix) and LiteLLM SDK migration sections (1.6.x+)
- Add a short JWT section: pass the IdP token as the SDK API key
- Link the LiteLLM config import guide to the SDK migration section
… note

- Fix the base URL guidance: the OpenAI SDK needs /v1 in the base URL; without it, requests 404
- Add Go, multi-provider (AgentgatewayModel), LiteLLM SDK migration, and JWT sections
- Warn that allowing AgentgatewayModel on a listener moves /v1 LLM paths from HTTPRoutes to models
@jamesilse-solo
jamesilse-solo marked this pull request as ready for review November 7, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant