Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/edge/ar/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1045,6 +1045,37 @@ mode: "wide"
[كتالوج النماذج المنسّق لـ CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)،
وهو عام ولا يتطلب مصادقة، للحصول على المعرّفات الحالية وأطوال السياق والأسعار.
</Accordion>

<Accordion title="Fireworks AI">
يتصل CrewAI بـ Fireworks AI عبر عميل OpenAI الأصلي، لذا لا حاجة لتثبيت LiteLLM.

عيّن متغيرات البيئة التالية في ملف `.env`:
```toml Code
# Required
FIREWORKS_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
```

مثال الاستخدام في مشروع CrewAI:
```python Code
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct",
temperature=0.7
)
```

<Info>
ميزات Fireworks AI:
- نماذج مفتوحة الأوزان خلف نقطة نهاية متوافقة مع OpenAI
- البث واستدعاء الأدوات والمخرجات المنظمة بمخطط JSON
- عمليات نشر بدون خادم أو عند الطلب
</Info>

تُزال بادئة `fireworks_ai/` فقط. أما المسار `accounts/<account>/models/<model>` الذي يليها
فيصل إلى SDK دون تغيير، لذا تظل المعرّفات المرتبطة بالحساب تعمل.
</Accordion>
</AccordionGroup>

## بث الاستجابات
Expand Down
31 changes: 31 additions & 0 deletions docs/edge/en/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1188,6 +1188,37 @@ In this section, you'll find detailed examples that help you select, configure,
[model catalog curated for CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
which is public and needs no authentication, for current ids, context lengths and pricing.
</Accordion>

<Accordion title="Fireworks AI">
CrewAI talks to Fireworks AI through its native OpenAI client, so no LiteLLM install is needed.

Set the following environment variables in your `.env` file:
```toml Code
# Required
FIREWORKS_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
```

Example usage in your CrewAI project:
```python Code
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct",
temperature=0.7
)
```

<Info>
Fireworks AI features:
- Open-weight models behind an OpenAI-compatible endpoint
- Streaming, tool calling and JSON schema structured outputs
- Serverless and on-demand deployments
</Info>

Only the `fireworks_ai/` prefix is removed. The `accounts/<account>/models/<model>`
path that follows reaches the SDK unchanged, so account-scoped model ids keep working.
</Accordion>
</AccordionGroup>

## Streaming Responses
Expand Down
31 changes: 31 additions & 0 deletions docs/edge/ko/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -788,6 +788,37 @@ CrewAI는 고유한 기능, 인증 방법, 모델 역량을 제공하는 다양
인증 없이 공개된 [CrewAI용으로 선별된 모델 카탈로그](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)를
참조하세요.
</Accordion>

<Accordion title="Fireworks AI">
CrewAI는 네이티브 OpenAI 클라이언트로 Fireworks AI에 연결하므로 LiteLLM 설치가 필요하지 않습니다.

`.env` 파일에 다음 환경 변수를 설정하십시오:
```toml Code
# Required
FIREWORKS_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
```

CrewAI 프로젝트에서의 예시 사용법:
```python Code
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct",
temperature=0.7
)
```

<Info>
Fireworks AI 특징:
- OpenAI 호환 엔드포인트로 제공되는 오픈 웨이트 모델
- 스트리밍, 도구 호출, JSON 스키마 구조화 출력
- 서버리스 및 온디맨드 배포
</Info>

`fireworks_ai/` 접두사만 제거됩니다. 뒤따르는 `accounts/<account>/models/<model>` 경로는
그대로 SDK에 전달되므로 계정 단위 모델 ID도 동작합니다.
</Accordion>
</AccordionGroup>

## 스트리밍 응답
Expand Down
31 changes: 31 additions & 0 deletions docs/edge/pt-BR/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -761,6 +761,37 @@ Nesta seção, você encontrará exemplos detalhados que ajudam a selecionar, co
[catálogo de modelos selecionado para o CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
que é público e não exige autenticação, para obter os IDs atuais, tamanhos de contexto e preços.
</Accordion>

<Accordion title="Fireworks AI">
O CrewAI se comunica com o Fireworks AI pelo cliente OpenAI nativo, então não é necessário instalar o LiteLLM.

Defina as seguintes variáveis de ambiente no seu arquivo `.env`:
```toml Code
# Required
FIREWORKS_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
FIREWORKS_BASE_URL=https://api.fireworks.ai/inference/v1
```

Exemplo de uso em seu projeto CrewAI:
```python Code
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct",
temperature=0.7
)
```

<Info>
Recursos do Fireworks AI:
- Modelos de pesos abertos por trás de um endpoint compatível com OpenAI
- Streaming, chamada de ferramentas e saídas estruturadas com JSON schema
- Implantações serverless e on-demand
</Info>

Apenas o prefixo `fireworks_ai/` é removido. O caminho `accounts/<account>/models/<model>`
que vem depois chega ao SDK sem alteração, então IDs de modelo por conta continuam funcionando.
</Accordion>
</AccordionGroup>

## Respostas em streaming
Expand Down
8 changes: 8 additions & 0 deletions lib/crewai/src/crewai/llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ def _ensure_litellm() -> bool:
"dashscope",
"snowflake",
"deepinfra",
"fireworks_ai",
]


Expand All @@ -242,6 +243,7 @@ def _ensure_litellm() -> bool:
"dashscope": "dashscope",
"snowflake": "snowflake",
"deepinfra": "deepinfra",
"fireworks_ai": "fireworks_ai",
}


Expand Down Expand Up @@ -473,6 +475,11 @@ def _matches_provider_pattern(cls, model: str, provider: str) -> bool:
if provider == "snowflake":
return True

if provider == "fireworks_ai":
# Fireworks ids are accounts/<account>/models/<model>; only the
# provider prefix is stripped.
return True

return False

@classmethod
Expand Down Expand Up @@ -698,6 +705,7 @@ def _get_native_provider(cls, provider: str) -> type[BaseLLM] | None:
"cerebras",
"dashscope",
"deepinfra",
"fireworks_ai",
}
if provider in openai_compatible_providers:
from crewai.llms.providers.openai_compatible.completion import (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ class ProviderConfig:
base_url_env="DEEPINFRA_BASE_URL",
api_key_required=True,
),
"fireworks_ai": ProviderConfig(
base_url="https://api.fireworks.ai/inference/v1",
api_key_env="FIREWORKS_API_KEY",
base_url_env="FIREWORKS_BASE_URL",
api_key_required=True,
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fireworks context windows miss catalog

Medium Severity

Native fireworks_ai routing inherits OpenAICompletion.get_context_window_size, which queries OPENAI_CONTEXT_WINDOWS only. Catalogued Fireworks limits never match, so those models compact against the 8192 default instead of their published windows.

Additional Locations (1)
Fix in Cursor Fix in Web

Triggered by learned rule: Context-window catalog matching

Reviewed by Cursor Bugbot for commit f5d5ffa. Configure here.

}

_OLLAMA_DEFAULT_PORT = 11434
Expand Down Expand Up @@ -150,6 +156,7 @@ class OpenAICompatibleCompletion(OpenAICompletion):
- cerebras: Cerebras (https://cerebras.ai)
- dashscope: Alibaba Dashscope/Qwen (https://dashscope.aliyun.com)
- deepinfra: DeepInfra (https://deepinfra.com)
- fireworks_ai: Fireworks AI (https://fireworks.ai)

Example:
# Using provider prefix
Expand Down
102 changes: 102 additions & 0 deletions lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ def test_deepinfra_config(self):
assert config.base_url_env == "DEEPINFRA_BASE_URL"
assert config.api_key_required is True

def test_fireworks_ai_config(self):
"""Test Fireworks AI provider configuration."""
config = OPENAI_COMPATIBLE_PROVIDERS["fireworks_ai"]
assert config.base_url == "https://api.fireworks.ai/inference/v1"
assert config.api_key_env == "FIREWORKS_API_KEY"
assert config.base_url_env == "FIREWORKS_BASE_URL"
assert config.api_key_required is True


class TestNormalizeOllamaBaseUrl:
"""Tests for _normalize_ollama_base_url helper."""
Expand Down Expand Up @@ -180,6 +188,26 @@ def test_deepinfra_missing_api_key_names_env_var(self):
with pytest.raises(MissingAPIKeyError, match="DEEPINFRA_API_KEY"):
llm.call("Say hi")

def test_fireworks_ai_missing_api_key_names_env_var(self):
"""Fireworks requires a key; the error names FIREWORKS_API_KEY."""
with patch.dict(os.environ, {}, clear=True), pytest.raises(
ValueError, match="FIREWORKS_API_KEY"
):
OpenAICompatibleCompletion(
model="accounts/fireworks/models/llama-v3p1-8b-instruct",
provider="fireworks_ai",
)

def test_fireworks_ai_does_not_borrow_openai_credentials(self):
"""An OpenAI key alone must not satisfy Fireworks."""
with patch.dict(os.environ, {"OPENAI_API_KEY": "sk-openai"}, clear=True), pytest.raises(
ValueError, match="FIREWORKS_API_KEY"
):
OpenAICompatibleCompletion(
model="accounts/fireworks/models/llama-v3p1-8b-instruct",
provider="fireworks_ai",
)

def test_api_key_from_env(self):
"""Test API key is read from environment variable."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key-from-env"}):
Expand Down Expand Up @@ -419,6 +447,80 @@ def test_llm_with_explicit_provider(self):
assert llm.provider == "deepseek"
assert llm.model == "deepseek-chat"

def test_llm_creates_openai_compatible_for_fireworks_ai(self):
"""fireworks_ai/<id> routes natively instead of falling back to LiteLLM."""
with patch.dict(os.environ, {"FIREWORKS_API_KEY": "test-key"}):
llm = LLM(model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct")
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "fireworks_ai"
assert llm.model == "accounts/fireworks/models/llama-v3p1-8b-instruct"
assert llm.base_url == "https://api.fireworks.ai/inference/v1"

def test_fireworks_ai_route_names_the_native_class(self):
"""Routing resolves to the native class, so no LiteLLM fallback is built."""
route = LLM._resolve_route(
"fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct", {}
)
assert route.native_class is OpenAICompatibleCompletion
assert route.provider == "fireworks_ai"

def test_fireworks_ai_keeps_the_account_and_model_segments(self):
"""Only the provider prefix is removed; the rest of the path reaches the SDK."""
with patch.dict(os.environ, {"FIREWORKS_API_KEY": "test-key"}):
llm = LLM(
model="fireworks_ai/accounts/acme/models/qwen3-235b-a22b-instruct-2507"
)
assert llm.provider == "fireworks_ai"
assert llm.model == "accounts/acme/models/qwen3-235b-a22b-instruct-2507"

def test_llm_creates_openai_compatible_for_fireworks_ai_explicit_provider(self):
"""Explicit provider="fireworks_ai" routes natively without the model prefix."""
with patch.dict(os.environ, {"FIREWORKS_API_KEY": "test-key"}):
llm = LLM(
model="accounts/fireworks/models/llama-v3p1-8b-instruct",
provider="fireworks_ai",
)
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "fireworks_ai"
assert llm.model == "accounts/fireworks/models/llama-v3p1-8b-instruct"

def test_fireworks_ai_base_url_env_override(self):
"""FIREWORKS_BASE_URL redirects Fireworks to a proxy or gateway."""
with patch.dict(
os.environ,
{
"FIREWORKS_API_KEY": "test-key",
"FIREWORKS_BASE_URL": "https://fireworks-gateway.example.com/inference/v1",
},
):
llm = LLM(model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct")
assert llm.base_url == "https://fireworks-gateway.example.com/inference/v1"

def test_fireworks_ai_explicit_configuration_wins_over_env(self):
"""Explicit api_key/base_url beat FIREWORKS_API_KEY and FIREWORKS_BASE_URL."""
with patch.dict(
os.environ,
{
"FIREWORKS_API_KEY": "env-key",
"FIREWORKS_BASE_URL": "https://env.example.com/inference/v1",
},
):
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct",
api_key="explicit-key",
base_url="https://explicit.example.com/inference/v1",
)
assert llm.api_key == "explicit-key"
assert llm.base_url == "https://explicit.example.com/inference/v1"

def test_fireworks_ai_client_params_use_fireworks_endpoint_and_key(self):
"""The OpenAI client is built against Fireworks, not api.openai.com."""
with patch.dict(os.environ, {"FIREWORKS_API_KEY": "fw-test"}):
llm = LLM(model="fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct")
params = llm._get_client_params()
assert params["api_key"] == "fw-test"
assert params["base_url"] == "https://api.fireworks.ai/inference/v1"

def test_llm_passes_kwargs_to_completion(self):
"""Test LLM passes kwargs to OpenAICompatibleCompletion."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key"}):
Expand Down