Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
ba5caeb
Add Monocle tracing
imohammedansari Jul 8, 2026
60559dd
Merge branch 'main' into monocle-instrumentation
imohammedansari Jul 9, 2026
39b4d49
Config-gate Monocle telemetry in the Gateway lifespan
imohammedansari Jul 10, 2026
a97ff2c
Merge branch 'main' into monocle-instrumentation
imohammedansari Jul 10, 2026
745409c
Clarify Monocle/Langfuse single-provider guidance
imohammedansari Jul 10, 2026
803be48
Address review: optional extra, exporter validation, off-box warning,…
imohammedansari Jul 11, 2026
c2910f2
docs: align Monocle README section with the other tracing providers
imohammedansari Jul 11, 2026
c1f8bff
Address review: verified Langfuse coexistence, lifespan test, scope docs
imohammedansari Jul 11, 2026
232fd03
Merge branch 'main' of https://github.com/bytedance/deer-flow into mo…
imohammedansari Jul 11, 2026
93f968d
docs: reword Monocle intro sentence
imohammedansari Jul 11, 2026
cb165f8
Merge branch 'main' into monocle-instrumentation
WillemJiang Jul 11, 2026
e0122c9
fix(tests): run the import-time regression in a subprocess
imohammedansari Jul 11, 2026
ec30867
Merge branch 'main' into monocle-instrumentation
imohammedansari Jul 11, 2026
6c36548
Address review: console warning scope, embedded hint, honest naming, …
imohammedansari Jul 13, 2026
96fd6ec
Merge branch 'main' into monocle-instrumentation
imohammedansari Jul 13, 2026
3ac4310
fix(tests): pin the factory logger level in the embedded-hint tests
imohammedansari Jul 14, 2026
2920b60
Address review: co-export disclosure, lifespan failure test, exporter…
imohammedansari Jul 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,6 @@ config.yaml.bak
/frontend/playwright-report/
.gstack/
.worktrees

# Monocle agent-observability trace output (local file exporter)
.monocle/
23 changes: 19 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ DeerFlow has newly integrated the intelligent search and crawling toolset indepe
- [IM Channels](#im-channels)
- [LangSmith Tracing](#langsmith-tracing)
- [Langfuse Tracing](#langfuse-tracing)
- [Using Both Providers](#using-both-providers)
- [Monocle Tracing](#monocle-tracing)
- [Using Multiple Providers](#using-multiple-providers)
- [From Deep Research to Super Agent Harness](#from-deep-research-to-super-agent-harness)
- [Core Features](#core-features)
- [Skills \& Tools](#skills--tools)
Expand Down Expand Up @@ -593,11 +594,25 @@ If you are using a self-hosted Langfuse instance, set `LANGFUSE_BASE_URL` to you

These are injected into `RunnableConfig.metadata` at the graph invocation root for both the gateway path (`runtime/runs/worker.py::run_agent`) and the embedded path (`client.py::DeerFlowClient.stream`), so any LangChain-compatible callback can read them. Set `DEER_FLOW_ENV` (or `ENVIRONMENT`) to tag traces by deployment environment.

#### Using Both Providers
#### Monocle Tracing

If both LangSmith and Langfuse are enabled, DeerFlow attaches both tracing callbacks and reports the same model activity to both systems.
DeerFlow also supports [Monocle](https://github.com/monocle2ai/monocle) telemetry for its LangGraph/LangChain runs.

If a provider is explicitly enabled but missing required credentials, or if its callback fails to initialize, DeerFlow fails fast when tracing is initialized during model creation and the error message names the provider that caused the failure.
Add the following to your `.env` file:

```bash
MONOCLE_TRACING=true
MONOCLE_EXPORTERS=file # file, console, okahu, s3, blob, gcs (default: file)
OKAHU_API_KEY=okh_xxxxxxxx # required only for the `okahu` exporter
```

Monocle is off by default and is initialized at Gateway startup when enabled. The `file` exporter writes one trace per run to `.monocle/`; open those in the [Monocle VS Code extension](https://marketplace.visualstudio.com/items?itemName=OkahuAI.monocle-apptrace) to inspect the span timeline and token counts.

Send your Monocle traces to [Okahu](https://www.okahu.ai), an agent-observability platform, to analyze them across many runs.

#### Using Multiple Providers

LangSmith and Langfuse attach as LangChain callbacks, so you can enable both and DeerFlow reports each run to both. If an enabled provider is missing required credentials or fails to initialize, DeerFlow fails fast and names it. Monocle uses a global OpenTelemetry provider rather than a callback, so it runs alongside LangSmith but not Langfuse.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User-facing version of the same Langfuse-conflict claim (see inline on monocle.py).

"...so it runs alongside LangSmith but not Langfuse" is stated as a flat rule, but rests on the same unverified assumption that Langfuse's callback path owns the global OTel provider. Worth softening ("may conflict; verify") or removing until the interaction is confirmed — otherwise Langfuse users are told they can't use Monocle when they likely can.


For Docker deployments, tracing is disabled by default. Set `LANGSMITH_TRACING=true` and `LANGSMITH_API_KEY` in your `.env` to enable it.

Expand Down
6 changes: 6 additions & 0 deletions backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,6 +705,12 @@ LangSmith and Langfuse are both supported. The wiring lives in two layers:

Returns `{}` when Langfuse is not in the enabled providers — LangSmith-only deployments are unaffected. Set `DEER_FLOW_ENV` (or `ENVIRONMENT`) to tag traces by deployment environment. Tests live in `tests/test_tracing_factory.py`, `tests/test_tracing_metadata.py`, `tests/test_worker_langfuse_metadata.py`, `tests/test_client_langfuse_metadata.py`, and `tests/test_subagent_executor.py::TestSubagentTracingWiring`.

**Monocle telemetry** is a third provider, structurally unlike LangSmith/Langfuse. It is **not** a LangChain callback: `tracing/monocle.py::setup_monocle_tracing_if_enabled()` calls `monocle_apptrace.setup_monocle_telemetry()` once, which installs a **process-global OTel `TracerProvider`**, patches span serialization, and auto-instruments the openai/langchain/langgraph clients. Because that is a one-time, process-global side effect (not a per-run callback), it is initialized from the **Gateway lifespan** (`app/gateway/app.py`) — never from `build_tracing_callbacks()` — and it is **off by default**. The setup call was deliberately moved out of `agents/__init__.py`, so `import deerflow.agents` must never start tracing (pinned by `tests/test_monocle_tracing.py::test_no_import_time_setup`).

Unlike the Langfuse metadata above, DeerFlow injects **no** per-run fields into Monocle traces — the only attribute it sets is `workflow_name="deer-flow"`; every span attribute (`span.type`, `entity.*`, token usage, span inputs/outputs, `scope.agentic.session`) is produced by Monocle's own metamodel and auto-instrumentation, so there is no DeerFlow trace-attribute layer to maintain here.

Config is env-driven like the others — `MonocleTracingConfig`, built in `get_tracing_config()` and gated by `is_monocle_tracing_enabled()`. `MONOCLE_TRACING` enables it; `MONOCLE_EXPORTERS` selects exporters (default `file` → trace JSON in `.monocle/`; also `console`, `okahu`, `s3`, `blob`, `gcs`, where `okahu` requires `OKAHU_API_KEY`). `setup_monocle_tracing_if_enabled()` stays a thin wrapper on purpose: `monocle_apptrace` already guards duplicate setup (`instrumentor.py::check_duplicate_setup`) and never force-overrides an existing global provider, so the wrapper only gates on config. It also **warns** when Langfuse (v4) is enabled: only one library can own the single global OpenTelemetry provider, and Monocle initializes here at startup (before Langfuse's per-run handler), so with both on Langfuse loses its spans — enable one OTel tracer. (LangSmith is a plain callback and coexists fine.) Tests: `tests/test_monocle_tracing.py`.

### Config Schema

**`config.yaml`** key sections:
Expand Down
9 changes: 9 additions & 0 deletions backend/app/gateway/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
from app.gateway.trace_middleware import TraceMiddleware, resolve_trace_enabled
from deerflow.config import app_config as deerflow_app_config
from deerflow.logging_config import DEFAULT_LOG_DATE_FORMAT, DEFAULT_LOG_FORMAT, configure_logging
from deerflow.tracing.monocle import setup_monocle_tracing_if_enabled
from deerflow.uploads.manager import cleanup_stale_upload_staging_files

AppConfig = deerflow_app_config.AppConfig
Expand Down Expand Up @@ -189,6 +190,14 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
config = get_gateway_config()
logger.info(f"Starting API Gateway on {config.host}:{config.port}")

# Agent observability (Monocle). Off by default; enabled with
# MONOCLE_TRACING. Initialized here at startup — not at import time — so a
# plain `import deerflow.agents` never installs a process-global tracer.
try:
setup_monocle_tracing_if_enabled()
Comment thread
WillemJiang marked this conversation as resolved.
Resolved

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Embedded / TUI / CLI path gets no Monocle tracing.

LangSmith and Langfuse are attached at the graph invocation root for both runtime/runs/worker.py::run_agent (this gateway path) and client.py::DeerFlowClient.stream (embedded). Monocle is initialized only here in the Gateway lifespan, so DeerFlowClient (programmatic library use) and the deerflow TUI get no traces even with MONOCLE_TRACING=true (confirmed: no monocle wiring under client.py or deerflow/tui/).

That's a real asymmetry vs the other two providers — at minimum call it out in AGENTS.md (embedded/TUI users must call setup_monocle_tracing_if_enabled() themselves, or accept that only the Gateway is instrumented).

Also: this lifespan wiring — the one line that makes the feature work — has no test asserting it's invoked during startup. A focused test monkeypatching the helper and asserting it's called from the lifespan would close that gap.

except Exception: # pragma: no cover - observability must never break startup

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This except is # pragma: no cover, and test_gateway_lifespan_initializes_monocle patches setup with a spy returning False, so the "setup raises -> Gateway keeps serving" guarantee isn't exercised. A test driving the lifespan with a setup that raises (e.g., monkeypatch setup_monocle_tracing_if_enabled to raise ValueError) and asserting the lifespan still completes would close it.

Side note: a Monocle misconfiguration is silently ignored after this one startup log (per-run build_tracing_callbacks() re-validates only langsmith/langfuse, not Monocle), so a user who fixes the config won't get tracing without a restart - worth a line in the docs.

logger.exception("Monocle tracing setup failed; continuing without it")
Comment on lines +198 to +201

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graceful-degrade vs. fail-fast asymmetry — worth documenting. This except Exception swallows the ValueError from monocle.validate() (bad exporter / missing OKAHU_API_KEY) and the RuntimeError for a missing extra, so a misconfigured Monocle setup logs at ERROR and continues without tracing — the Gateway stays up. That's a deliberate contrast with LangSmith/Langfuse, where build_tracing_callbacks() -> validate_enabled_tracing_providers() raises and aborts the agent run.

The behavior is internally consistent with "cannot break startup", but the asymmetry isn't called out in backend/AGENTS.md. A one-line note there (Monocle misconfiguration degrades gracefully rather than failing fast) would make the intent explicit instead of discovered.


# Pre-warm tiktoken encoding cache so the first memory-injection request
# never blocks on the BPE data download (which hits an OpenAI/Azure URL
# that may be unreachable in restricted networks — see issue #3402).
Expand Down
2 changes: 2 additions & 0 deletions backend/packages/harness/deerflow/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
get_enabled_tracing_providers,
get_explicitly_enabled_tracing_providers,
get_tracing_config,
is_monocle_tracing_enabled,
is_tracing_enabled,
validate_enabled_tracing_providers,
)
Expand All @@ -27,6 +28,7 @@
"get_tracing_config",
"get_explicitly_enabled_tracing_providers",
"get_enabled_tracing_providers",
"is_monocle_tracing_enabled",
"is_tracing_enabled",
"validate_enabled_tracing_providers",
]
30 changes: 30 additions & 0 deletions backend/packages/harness/deerflow/config/tracing_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,27 @@ def validate(self) -> None:
raise ValueError(f"Langfuse tracing is enabled but required settings are missing: {', '.join(missing)}")


class MonocleTracingConfig(BaseModel):
"""Configuration for Monocle telemetry."""

enabled: bool = Field(...)
exporters: str = Field(...)

@property
def is_configured(self) -> bool:
return self.enabled

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_configured diverges from the LangSmith/Langfuse pattern.

LangSmith/Langfuse fold credential presence into is_configured (self.enabled and bool(self.api_key)); here it's just self.enabled, so is_monocle_tracing_enabled() returns True even for a config validate() will reject (e.g. okahu without OKAHU_API_KEY). Validation is deferred to setup, where it's caught+logged rather than surfaced as "not enabled."

Defensible as a composite-validation choice, but a one-line comment explaining why Monocle's is_configured is intentionally coarser would stop a future reader from "fixing" it to match the others.


def validate(self) -> None:
# No external credentials are required; the "file" exporter writes locally.
return None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MONOCLE_EXPORTERS is unvalidated, and this validate() is dead code. TracingConfig.validate_enabled() only calls langsmith.validate() / langfuse.validate() — it never calls self.monocle.validate() — so this no-op never runs. Concretely: (a) a typo in MONOCLE_EXPORTERS (e.g. fle) is passed straight to setup_monocle_telemetry and fails inside monocle at runtime with an obscure error; (b) MONOCLE_EXPORTERS=okahu without OKAHU_API_KEY isn't caught early, unlike Langfuse which validates required settings and names them.

Recommend an exporter allowlist (file/console/okahu/s3/blob/gcs) + require OKAHU_API_KEY when okahu is selected, mirroring the Langfuse pattern — and actually wire self.monocle.validate() into validate_enabled() (or drop this method).



class TracingConfig(BaseModel):
"""Tracing configuration for supported providers."""

langsmith: LangSmithTracingConfig = Field(...)
langfuse: LangfuseTracingConfig = Field(...)
monocle: MonocleTracingConfig = Field(...)

@property
def is_configured(self) -> bool:
Expand Down Expand Up @@ -125,6 +141,10 @@ def get_tracing_config() -> TracingConfig:
secret_key=_first_env_value("LANGFUSE_SECRET_KEY"),
host=_first_env_value("LANGFUSE_BASE_URL") or "https://cloud.langfuse.com",
),
monocle=MonocleTracingConfig(
enabled=_env_flag_preferred("MONOCLE_TRACING"),
exporters=_first_env_value("MONOCLE_EXPORTERS") or "file",
),
)
return _tracing_config

Expand All @@ -149,6 +169,16 @@ def is_tracing_enabled() -> bool:
return get_tracing_config().is_configured


def is_monocle_tracing_enabled() -> bool:
"""Whether Monocle OTel observability is enabled (via ``MONOCLE_TRACING``).

Kept separate from :func:`get_enabled_tracing_providers` because Monocle is a
process-global instrumentor activated at startup, not a per-run LangChain
callback.
"""
return get_tracing_config().monocle.is_configured


def reset_tracing_config() -> None:
"""Discard the cached :class:`TracingConfig` so the next call rebuilds it.

Expand Down
36 changes: 36 additions & 0 deletions backend/packages/harness/deerflow/tracing/monocle.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
"""Monocle telemetry: initialized once from the Gateway lifespan when ``MONOCLE_TRACING`` is set."""

from __future__ import annotations

import logging

from deerflow.config import (
get_enabled_tracing_providers,
get_tracing_config,
is_monocle_tracing_enabled,
)

logger = logging.getLogger(__name__)


def setup_monocle_tracing_if_enabled() -> bool:
"""Initialize Monocle telemetry when ``MONOCLE_TRACING`` is enabled; a no-op otherwise.

``monocle_apptrace.setup_monocle_telemetry()`` is idempotent, so this stays a thin,
config-gated wrapper. Returns ``True`` when enabled.
"""
if not is_monocle_tracing_enabled():
return False

# Only one OpenTelemetry provider can own the process. Monocle initializes here (at
# startup), before Langfuse's per-run handler, so enabling both means Langfuse loses
# its spans — warn so the operator turns one off.
if "langfuse" in get_enabled_tracing_providers():
logger.warning("MONOCLE_TRACING is enabled alongside Langfuse; both need the global OpenTelemetry provider and only one can win. Enable only one of them.")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Langfuse-conflict warning is the key logic branch in the module and has no test. Add a caplog test asserting the warning fires when MONOCLE_TRACING and Langfuse are both enabled (and stays silent when only one is). It's the one piece of real branching here and it's currently uncovered.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: the Langfuse/OTel conflict is asserted but not verified.

In this repo, Langfuse is wired as a LangChain CallbackHandler (tracing/factory.py::_create_langfuse_handler → from langfuse.langchain import CallbackHandler as LangfuseCallbackHandler), attached at the graph root — DeerFlow never calls langfuse.start_tracing() or trace.set_tracer_provider() for it. So the claim "both need the global OpenTelemetry provider and only one can win" isn't obviously true for this codebase's Langfuse integration.

Two concerns:

  • If the v4 callback doesn't actually contend for the global OTel provider, this warning is noise and the README's "not Langfuse" rule is misleading (Monocle would coexist with Langfuse just as it does with LangSmith).
  • test_warns_when_langfuse_also_enabled only asserts the warning is logged, not that spans are actually lost — so the conflict is claimed in prose/tests but never demonstrated.

Suggestion: verify against the installed langfuse version whether its callback installs/owns the global OTel provider at per-run time. If yes, add a test proving Monocle's provider survives (or Langfuse drops); if no, drop the warning and correct README/AGENTS.md.


exporters = get_tracing_config().monocle.exporters
from monocle_apptrace import setup_monocle_telemetry

setup_monocle_telemetry(workflow_name="deer-flow", monocle_exporters_list=exporters)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security: spans capture prompts + tool I/O verbatim, with no DeerFlow-controlled scrubbing layer. Auto-instrumentation records span inputs/outputs — prompts, tool arguments (write_file body, bash command strings), and model responses. DeerFlow's existing tracing (tracing/metadata.py "never copies context") is careful about what it sends; Monocle captures at the SDK layer, which DeerFlow can't scrub. Request-scoped secrets are safe from bash args (they go through env=), but write_file/read_file/web_fetch I/O carries sensitive data and is captured unscrubbed.

Two suggestions: (1) log a loud startup warning whenever a non-file exporter is configured (e.g. okahu ships all of this off-box to an external collector — operators shouldn't be able to miss that); (2) document which DeerFlow data surfaces are/aren't captured. file-by-default + the config.example.yaml disclosure is a good baseline.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call installs a process-global OTel TracerProvider and patches span serialization for the whole process, and it shares that provider with Langfuse v4. Coexistence is verified by test_coexists_with_langfuse, but that test reaches into OTel SDK privates (provider._active_span_processor._span_processors) and depends on Langfuse reusing the existing provider rather than replacing it. A Langfuse major bump or an OTel-SDK change could silently break span export for both providers.

The mitigation (off by default, optional extra) is sound - just flagging this as the one coupling to weigh against the risk:high label. If a public API to list a provider's span processors ever lands in the OTel SDK, swapping the private introspection for it would harden the test.

logger.info("Monocle telemetry enabled (exporters=%s)", exporters)
return True
1 change: 1 addition & 0 deletions backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ dependencies = [
"pyjwt>=2.13.0",
"email-validator>=2.0.0",
"e2b-code-interpreter>=2.8.1",
"monocle_apptrace>=0.8.8",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an optional extra, not a hard runtime dependency. Monocle is off by default, yet monocle_apptrace>=0.8.8 is in core dependencies, so every install pulls the full OTel stack (opentelemetry-api/-sdk/-exporter-otlp-proto-http/-instrumentation, wrapt, rfc3986, requests) whether tracing is used or not. The repo already has precedent: boxlite and tui are [project.optional-dependencies]. The lazy from monocle_apptrace import ... inside setup_monocle_tracing_if_enabled means converting to an optional extra is low-risk. Also worth weighing: pinning a pre-1.0 (0.8.x) vendor-affiliated auto-instrumentation library as a core dep — breaking releases are possible before 1.0.

]

[project.optional-dependencies]
Expand Down
76 changes: 76 additions & 0 deletions backend/tests/test_monocle_tracing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
"""Tests for Monocle telemetry setup.

Covers the config gate (``MONOCLE_TRACING`` default off / toggle on), the setup
helper's behavior, and the regression that importing ``deerflow.agents`` no
longer sets up telemetry at import time.
"""

from __future__ import annotations

import pytest

from deerflow.config import is_monocle_tracing_enabled
from deerflow.config.tracing_config import get_tracing_config, reset_tracing_config
from deerflow.tracing.monocle import setup_monocle_tracing_if_enabled


@pytest.fixture(autouse=True)
def clear_monocle_env(monkeypatch):
for name in ("MONOCLE_TRACING", "MONOCLE_EXPORTERS"):
monkeypatch.delenv(name, raising=False)
reset_tracing_config()
yield
reset_tracing_config()


def test_disabled_by_default():
assert is_monocle_tracing_enabled() is False
assert get_tracing_config().monocle.enabled is False


def test_setup_noop_when_disabled(monkeypatch):
called = False

def _fail(*args, **kwargs):
nonlocal called
called = True

monkeypatch.setattr("monocle_apptrace.setup_monocle_telemetry", _fail)
assert setup_monocle_tracing_if_enabled() is False
assert called is False


def test_toggles_on_and_sets_up(monkeypatch):
monkeypatch.setenv("MONOCLE_TRACING", "true")
reset_tracing_config()

captured: dict = {}
monkeypatch.setattr("monocle_apptrace.setup_monocle_telemetry", lambda **kw: captured.update(kw))

assert is_monocle_tracing_enabled() is True
assert setup_monocle_tracing_if_enabled() is True
assert captured == {"workflow_name": "deer-flow", "monocle_exporters_list": "file"}


def test_custom_exporters(monkeypatch):
monkeypatch.setenv("MONOCLE_TRACING", "true")
monkeypatch.setenv("MONOCLE_EXPORTERS", "file,console")
reset_tracing_config()

captured: dict = {}
monkeypatch.setattr("monocle_apptrace.setup_monocle_telemetry", lambda **kw: captured.update(kw))

assert setup_monocle_tracing_if_enabled() is True
assert captured["monocle_exporters_list"] == "file,console"


def test_no_import_time_setup():
"""Regression: importing deerflow.agents must not install telemetry.

The setup call used to live at module import in ``deerflow/agents/__init__``.
It now happens only via the gateway lifespan, so the package namespace must
not even carry ``setup_monocle_telemetry``.
"""
import deerflow.agents as agents

assert not hasattr(agents, "setup_monocle_telemetry")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regression check could be stronger, and idempotency is untested. not hasattr(agents, "setup_monocle_telemetry") only proves the name isn't on the agents namespace — it doesn't prove a global OTel TracerProvider wasn't installed. A more rigorous regression would assert the global provider wasn't replaced. Separately, the wrapper's idempotency claim (the docstring cites upstream check_duplicate_setup) is trusted, not exercised — consider a double-invoke test so a future caller can't silently double-instrument.

45 changes: 45 additions & 0 deletions backend/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

25 changes: 25 additions & 0 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,31 @@ logging:
enabled: false
format: text

# ============================================================================
# Monocle telemetry
# ============================================================================
# Optional agent tracing via Monocle. Configured through environment variables
# (like LangSmith/Langfuse), not a config.yaml key, and OFF by default. When
# enabled, Monocle is initialized once at Gateway startup and auto-instruments
# the LLM/agent clients; a plain library import does not start tracing.
#
# MONOCLE_TRACING=true # enable (default: off)
# MONOCLE_EXPORTERS=file # file, console, okahu, s3, blob, gcs (default: file)
# OKAHU_API_KEY=okh_... # required only for the `okahu` exporter
#
# The `file` exporter writes one trace JSON per run to `.monocle/` under the
# process working directory (git-ignored). Each trace captures the workflow and
# per-agent/LLM/tool spans, including span inputs and outputs (prompts, tool
# arguments, and model responses), token usage, and timings.
#
# Data-exfiltration note: `file` persists prompts and model outputs to local

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: this block sits under the logging: section, but Monocle is telemetry/tracing, not logging. Consider placing it under a dedicated tracing header for discoverability. (Separately — the data-exfiltration disclosure here is well done; the actionable ask is in monocle.py: a startup warning when a non-file exporter sends data off-box.)

# disk; non-file exporters (e.g. `okahu`) send that same span data off-box to an
# external collector. Enable only exporters whose data handling you trust, and
# be aware that prompts/outputs may contain sensitive content.
#
# Note: Monocle and Langfuse both install a global OpenTelemetry TracerProvider;
# enable only one OTel-based tracer at a time (see backend/AGENTS.md → Tracing).

# ============================================================================
# Token Usage
# ============================================================================
Expand Down
Loading