Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/edge/ar/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1045,6 +1045,38 @@ mode: "wide"
[كتالوج النماذج المنسّق لـ CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)،
وهو عام ولا يتطلب مصادقة، للحصول على المعرّفات الحالية وأطوال السياق والأسعار.
</Accordion>

<Accordion title="Together AI">
يتصل CrewAI بـ Together AI عبر عميل OpenAI الأصلي، لذا لا حاجة لتثبيت LiteLLM.

عيّن متغيرات البيئة التالية في ملف `.env`:
```toml Code
# Required
TOGETHER_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
TOGETHER_BASE_URL=https://api.together.ai/v1
```

مثال الاستخدام في مشروع CrewAI:
```python Code
llm = LLM(
model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
temperature=0.7
)
```

<Info>
ميزات Together AI:
- نماذج مفتوحة الأوزان خلف نقطة نهاية متوافقة مع OpenAI
- البث واستدعاء الأدوات والمخرجات المنظمة بمخطط JSON
- نقاط نهاية بدون خادم ومخصّصة
</Info>

تُزال بادئة `together/` فقط. تحتفظ معرّفات النماذج بصيغة `organization/model`، لذا يصل
`together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput` إلى الـ API بالصيغة
`Qwen/Qwen3-235B-A22B-Instruct-2507-tput`.
</Accordion>
</AccordionGroup>

## بث الاستجابات
Expand Down
32 changes: 32 additions & 0 deletions docs/edge/en/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1188,6 +1188,38 @@ In this section, you'll find detailed examples that help you select, configure,
[model catalog curated for CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
which is public and needs no authentication, for current ids, context lengths and pricing.
</Accordion>

<Accordion title="Together AI">
CrewAI talks to Together AI through its native OpenAI client, so no LiteLLM install is needed.

Set the following environment variables in your `.env` file:
```toml Code
# Required
TOGETHER_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
TOGETHER_BASE_URL=https://api.together.ai/v1
```

Example usage in your CrewAI project:
```python Code
llm = LLM(
model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
temperature=0.7
)
```

<Info>
Together AI features:
- Open-weight models behind an OpenAI-compatible endpoint
- Streaming, tool calling and JSON schema structured outputs
- Serverless and dedicated endpoints
</Info>

Only the `together/` prefix is removed. Model ids keep their `organization/model`
form, so `together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput` reaches the API as
`Qwen/Qwen3-235B-A22B-Instruct-2507-tput`.
</Accordion>
</AccordionGroup>

## Streaming Responses
Expand Down
32 changes: 32 additions & 0 deletions docs/edge/ko/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -788,6 +788,38 @@ CrewAI는 고유한 기능, 인증 방법, 모델 역량을 제공하는 다양
인증 없이 공개된 [CrewAI용으로 선별된 모델 카탈로그](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)를
참조하세요.
</Accordion>

<Accordion title="Together AI">
CrewAI는 네이티브 OpenAI 클라이언트로 Together AI에 연결하므로 LiteLLM 설치가 필요하지 않습니다.

`.env` 파일에 다음 환경 변수를 설정하십시오:
```toml Code
# Required
TOGETHER_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
TOGETHER_BASE_URL=https://api.together.ai/v1
```

CrewAI 프로젝트에서의 예시 사용법:
```python Code
llm = LLM(
model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
temperature=0.7
)
```

<Info>
Together AI 특징:
- OpenAI 호환 엔드포인트로 제공되는 오픈 웨이트 모델
- 스트리밍, 도구 호출, JSON 스키마 구조화 출력
- 서버리스 및 전용 엔드포인트
</Info>

`together/` 접두사만 제거됩니다. 모델 ID는 `organization/model` 형식을 유지하므로
`together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput`은 API에
`Qwen/Qwen3-235B-A22B-Instruct-2507-tput`으로 전달됩니다.
</Accordion>
</AccordionGroup>

## 스트리밍 응답
Expand Down
32 changes: 32 additions & 0 deletions docs/edge/pt-BR/concepts/llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -761,6 +761,38 @@ Nesta seção, você encontrará exemplos detalhados que ajudam a selecionar, co
[catálogo de modelos selecionado para o CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
que é público e não exige autenticação, para obter os IDs atuais, tamanhos de contexto e preços.
</Accordion>

<Accordion title="Together AI">
O CrewAI se comunica com a Together AI pelo cliente OpenAI nativo, então não é necessário instalar o LiteLLM.

Defina as seguintes variáveis de ambiente no seu arquivo `.env`:
```toml Code
# Required
TOGETHER_API_KEY=<your-api-key>

# Optional, point at a proxy or a gateway
TOGETHER_BASE_URL=https://api.together.ai/v1
```

Exemplo de uso em seu projeto CrewAI:
```python Code
llm = LLM(
model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
temperature=0.7
)
```

<Info>
Recursos da Together AI:
- Modelos de pesos abertos por trás de um endpoint compatível com OpenAI
- Streaming, chamada de ferramentas e saídas estruturadas com JSON schema
- Endpoints serverless e dedicados
</Info>

Apenas o prefixo `together/` é removido. Os IDs dos modelos mantêm a forma
`organization/model`, então `together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput` chega
à API como `Qwen/Qwen3-235B-A22B-Instruct-2507-tput`.
</Accordion>
</AccordionGroup>

## Respostas em streaming
Expand Down
7 changes: 7 additions & 0 deletions lib/crewai/src/crewai/llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ def _ensure_litellm() -> bool:
"dashscope",
"snowflake",
"deepinfra",
"together",
]


Expand All @@ -242,6 +243,7 @@ def _ensure_litellm() -> bool:
"dashscope": "dashscope",
"snowflake": "snowflake",
"deepinfra": "deepinfra",
"together": "together",
}


Expand Down Expand Up @@ -473,6 +475,10 @@ def _matches_provider_pattern(cls, model: str, provider: str) -> bool:
if provider == "snowflake":
return True

if provider == "together":
# Together ids are org/model; only the provider prefix is stripped.
return True

return False

@classmethod
Expand Down Expand Up @@ -698,6 +704,7 @@ def _get_native_provider(cls, provider: str) -> type[BaseLLM] | None:
"cerebras",
"dashscope",
"deepinfra",
"together",
}
if provider in openai_compatible_providers:
from crewai.llms.providers.openai_compatible.completion import (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ class ProviderConfig:
base_url_env="DEEPINFRA_BASE_URL",
api_key_required=True,
),
"together": ProviderConfig(
base_url="https://api.together.ai/v1",
api_key_env="TOGETHER_API_KEY",
base_url_env="TOGETHER_BASE_URL",
api_key_required=True,
),
}

_OLLAMA_DEFAULT_PORT = 11434
Expand Down Expand Up @@ -150,6 +156,7 @@ class OpenAICompatibleCompletion(OpenAICompletion):
- cerebras: Cerebras (https://cerebras.ai)
- dashscope: Alibaba Dashscope/Qwen (https://dashscope.aliyun.com)
- deepinfra: DeepInfra (https://deepinfra.com)
- together: Together AI (https://together.ai)

Example:
# Using provider prefix
Expand Down
95 changes: 95 additions & 0 deletions lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ def test_deepinfra_config(self):
assert config.base_url_env == "DEEPINFRA_BASE_URL"
assert config.api_key_required is True

def test_together_config(self):
"""Test Together AI provider configuration."""
config = OPENAI_COMPATIBLE_PROVIDERS["together"]
assert config.base_url == "https://api.together.ai/v1"
assert config.api_key_env == "TOGETHER_API_KEY"
assert config.base_url_env == "TOGETHER_BASE_URL"
assert config.api_key_required is True


class TestNormalizeOllamaBaseUrl:
"""Tests for _normalize_ollama_base_url helper."""
Expand Down Expand Up @@ -180,6 +188,24 @@ def test_deepinfra_missing_api_key_names_env_var(self):
with pytest.raises(MissingAPIKeyError, match="DEEPINFRA_API_KEY"):
llm.call("Say hi")

def test_together_missing_api_key_names_env_var(self):
"""Together requires a key; the error names TOGETHER_API_KEY."""
with patch.dict(os.environ, {}, clear=True), pytest.raises(
ValueError, match="TOGETHER_API_KEY"
):
OpenAICompatibleCompletion(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo", provider="together"
)

def test_together_does_not_borrow_openai_credentials(self):
"""An OpenAI key alone must not satisfy Together."""
with patch.dict(os.environ, {"OPENAI_API_KEY": "sk-openai"}, clear=True), pytest.raises(
ValueError, match="TOGETHER_API_KEY"
):
OpenAICompatibleCompletion(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo", provider="together"
)

def test_api_key_from_env(self):
"""Test API key is read from environment variable."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key-from-env"}):
Expand Down Expand Up @@ -419,6 +445,75 @@ def test_llm_with_explicit_provider(self):
assert llm.provider == "deepseek"
assert llm.model == "deepseek-chat"

def test_llm_creates_openai_compatible_for_together(self):
"""together/<org>/<model> routes natively instead of falling back to LiteLLM."""
with patch.dict(os.environ, {"TOGETHER_API_KEY": "test-key"}):
llm = LLM(model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo")
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "together"
assert llm.model == "meta-llama/Llama-3.3-70B-Instruct-Turbo"
assert llm.base_url == "https://api.together.ai/v1"

def test_together_route_names_the_native_class(self):
"""Routing resolves to the native class, so no LiteLLM fallback is built."""
route = LLM._resolve_route("together/meta-llama/Llama-3.3-70B-Instruct-Turbo", {})
assert route.native_class is OpenAICompatibleCompletion
assert route.provider == "together"

def test_together_keeps_the_organization_and_model_segments(self):
"""Only the provider prefix is removed; the org/model path reaches the SDK."""
with patch.dict(os.environ, {"TOGETHER_API_KEY": "test-key"}):
llm = LLM(model="together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput")
assert llm.provider == "together"
assert llm.model == "Qwen/Qwen3-235B-A22B-Instruct-2507-tput"

def test_llm_creates_openai_compatible_for_together_explicit_provider(self):
"""Explicit provider="together" routes natively without the model prefix."""
with patch.dict(os.environ, {"TOGETHER_API_KEY": "test-key"}):
llm = LLM(
model="meta-llama/Llama-3.3-70B-Instruct-Turbo", provider="together"
)
assert isinstance(llm, OpenAICompatibleCompletion)
assert llm.provider == "together"
assert llm.model == "meta-llama/Llama-3.3-70B-Instruct-Turbo"

def test_together_base_url_env_override(self):
"""TOGETHER_BASE_URL redirects Together to a proxy or gateway."""
with patch.dict(
os.environ,
{
"TOGETHER_API_KEY": "test-key",
"TOGETHER_BASE_URL": "https://together-gateway.example.com/v1",
},
):
llm = LLM(model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo")
assert llm.base_url == "https://together-gateway.example.com/v1"

def test_together_explicit_configuration_wins_over_env(self):
"""Explicit api_key/base_url beat TOGETHER_API_KEY and TOGETHER_BASE_URL."""
with patch.dict(
os.environ,
{
"TOGETHER_API_KEY": "env-key",
"TOGETHER_BASE_URL": "https://env.example.com/v1",
},
):
llm = LLM(
model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo",
api_key="explicit-key",
base_url="https://explicit.example.com/v1",
)
assert llm.api_key == "explicit-key"
assert llm.base_url == "https://explicit.example.com/v1"

def test_together_client_params_use_together_endpoint_and_key(self):
"""The OpenAI client is built against Together, not api.openai.com."""
with patch.dict(os.environ, {"TOGETHER_API_KEY": "tg-test"}):
llm = LLM(model="together/meta-llama/Llama-3.3-70B-Instruct-Turbo")
params = llm._get_client_params()
assert params["api_key"] == "tg-test"
assert params["base_url"] == "https://api.together.ai/v1"

def test_llm_passes_kwargs_to_completion(self):
"""Test LLM passes kwargs to OpenAICompatibleCompletion."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key"}):
Expand Down