Skip to content

docs: add an offline custom LLM quick start - #8807

Open
Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:docs/offline-custom-llm-example
Open

Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:docs/offline-custom-llm-example

Conversation

@shi-felix

@shi-felix Felix Shi (shi-felix) commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary

The custom LLM guide opened with a requests.post implementation against a placeholder endpoint, and its "Testing Your Custom LLM" example also called https://api.test.com. A new contributor could not run any of it without replacing the endpoint and getting credentials, so the interface was explained but never shown working.

This adds a deterministic offline example ahead of the real-provider one.

Fixes #8003.

Changes

  • docs/edge/en/learn/custom-llm.mdx: a new "Offline Quick Start" section with a complete EchoLLM(BaseLLM) stub, the exact command to run it, and its expected output (direct: pong / crew: pong). It declares supports_function_calling() and supports_stop_words() as False, states that the stub is for learning and testing, and points at where to swap in a real request. The existing provider example stays as the next step, now labelled as such.
  • docs/edge/ko/learn/custom-llm.mdx: the same section in Korean.
  • lib/crewai/tests/test_custom_llm.py: one smoke check that calls the stub directly and then runs a single-agent, single-task Crew with it — no key, no network.

docs/edge/ar/learn/custom-llm.mdx stops before the Quick Start section in this checkout, and docs/edge/pt-BR/learn/custom-llm.mdx still renders its code blocks as # (não traduzir blocos de código) placeholders, so there is no equivalent section to update in those two locales. I left them to the translation tool rather than inventing content there.

Testing

The example runs end to end without a provider:

$ python offline_custom_llm.py
direct: pong
crew: pong
python -m pytest lib/crewai/tests/test_custom_llm.py -k "offline_custom_llm"
1 passed, 8 deselected, 15 warnings in 19.13s

Python 3.13.14 on Windows. No credentials, no network calls.

AI assistance

Written with AI assistance (WorkBuddy) and reviewed before submission.


Note

Low Risk
Documentation and an offline smoke test only; no runtime or security-sensitive code paths change.

Overview
Adds an Offline Quick Start ahead of the provider-based custom LLM guide so readers can run BaseLLM end-to-end without API keys or network calls.

The new EchoLLM stub returns a fixed reply, sets supports_function_calling() and supports_stop_words() to False, and includes a runnable script plus expected direct: pong / crew: pong output. Prose explains extra call kwargs and that CrewAI handles stop words/tools when capabilities are declared honestly. The existing Quick Start is reframed as the next step for a real provider (English intro line updated; Korean page gets the same section).

test_custom_llm.py adds a documented EchoLLM mirror and test_offline_custom_llm_example_runs_without_a_provider, smoke-testing direct call and a one-agent Crew.kickoff() with no credentials.

Reviewed by Cursor Bugbot for commit 2d38f60. Bugbot is set up for automated code reviews on this repo. Configure here.

@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 43d8372.

Approve preview

@strix-security

strix-security Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 117 commits after the last Strix review (a7aa1d7). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds an offline custom-LLM quick-start (a fixed-reply `BaseLLM` stub with no network or API key) to the English and Korean custom-LLM docs, and a pytest smoke test that runs the stub directly and through a one-task `Crew`.

Review summary

This PR is documentation plus a single test: it adds an "Offline Quick Start" section with a self-contained EchoLLM(BaseLLM) stub to the English and Korean custom-LLM guides, relabels the existing provider example as the follow-on step, and adds a matching pytest smoke test that exercises the stub directly and through a one-agent, one-task Crew. The added example makes no network calls, holds no credentials, and is a fixed-string stub; the test only instantiates framework objects locally. No production code, configuration, dependency, or CI behavior changes, and no security issues were found in the changed lines.

Review 1 on this PR · a7aa1d7 · requested Oct 30 05:51 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[docs] Add a runnable offline custom LLM quick-start example

1 participant