Repository navigation
docs: add an offline custom LLM quick start - #8807
Felix Shi (shi-felix) wants to merge 3 commits into
Conversation
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
Strix Security ReviewWarning This pull request has 117 commits after the last Strix review ( ✅ No open security findings. PR summary: Adds an offline custom-LLM quick-start (a fixed-reply `BaseLLM` stub with no network or API key) to the English and Korean custom-LLM docs, and a pytest smoke test that runs the stub directly and through a one-task `Crew`. Review summaryThis PR is documentation plus a single test: it adds an "Offline Quick Start" section with a self-contained Review 1 on this PR · Reviewed by Strix |
Summary
The custom LLM guide opened with a
requests.postimplementation against a placeholder endpoint, and its "Testing Your Custom LLM" example also calledhttps://api.test.com. A new contributor could not run any of it without replacing the endpoint and getting credentials, so the interface was explained but never shown working.This adds a deterministic offline example ahead of the real-provider one.
Fixes #8003.
Changes
docs/edge/en/learn/custom-llm.mdx: a new "Offline Quick Start" section with a completeEchoLLM(BaseLLM)stub, the exact command to run it, and its expected output (direct: pong/crew: pong). It declaressupports_function_calling()andsupports_stop_words()asFalse, states that the stub is for learning and testing, and points at where to swap in a real request. The existing provider example stays as the next step, now labelled as such.docs/edge/ko/learn/custom-llm.mdx: the same section in Korean.lib/crewai/tests/test_custom_llm.py: one smoke check that calls the stub directly and then runs a single-agent, single-taskCrewwith it — no key, no network.docs/edge/ar/learn/custom-llm.mdxstops before the Quick Start section in this checkout, anddocs/edge/pt-BR/learn/custom-llm.mdxstill renders its code blocks as# (não traduzir blocos de código)placeholders, so there is no equivalent section to update in those two locales. I left them to the translation tool rather than inventing content there.Testing
The example runs end to end without a provider:
Python 3.13.14 on Windows. No credentials, no network calls.
AI assistance
Written with AI assistance (WorkBuddy) and reviewed before submission.
Note
Low Risk
Documentation and an offline smoke test only; no runtime or security-sensitive code paths change.
Overview
Adds an Offline Quick Start ahead of the provider-based custom LLM guide so readers can run
BaseLLMend-to-end without API keys or network calls.The new
EchoLLMstub returns a fixed reply, setssupports_function_calling()andsupports_stop_words()toFalse, and includes a runnable script plus expecteddirect: pong/crew: pongoutput. Prose explains extracallkwargs and that CrewAI handles stop words/tools when capabilities are declared honestly. The existing Quick Start is reframed as the next step for a real provider (English intro line updated; Korean page gets the same section).test_custom_llm.pyadds a documentedEchoLLMmirror andtest_offline_custom_llm_example_runs_without_a_provider, smoke-testing directcalland a one-agentCrew.kickoff()with no credentials.Reviewed by Cursor Bugbot for commit 2d38f60. Bugbot is set up for automated code reviews on this repo. Configure here.