CodeForge v2

CodeForge is being built to become an autonomous software-engineering agent — a system that takes a goal and carries it to verified, working software with minimal human intervention. Not a chatbot with tools: a user goal flows through understanding → planning → a task graph → tool-governed execution → verification → recovery, with full observability.

The foundation that exists today is a deterministic runtime kernel — the trustworthy base the agent is grown on. The vision is the full agent; the kernel is how we get there safely.

Central law, preserved everywhere: the LLM proposes; the deterministic runtime decides and enforces.

CodeForge runs against local models via Ollama, keeps every piece of state in an append-only SQLite event log, and governs every tool call through an explicit policy + approval gate. The agent is never the authority on its own success — a deterministic verification step decides pass/fail.

---

What CodeForge is (and is not)

The goal is an autonomous software-engineering agent. What runs today is the kernel that turns a goal into verified work — the part the agent needs to be trustworthy before it can be trusted to act on its own:

User Goal → Mission Understanding → Complexity / Risk / Uncertainty → Capability check
→ (advisory) Model routing → Planning → Task Graph → Tool-Governed Execution
→ Verification → Recovery → Observability → Learning / Evaluation

It is not a free-form chat assistant. Nothing the model emits touches your workspace directly: file writes and commands go through a ToolGateway with autonomy levels and human-in-the-loop approval; task completion is decided by deterministic checks, not by the model claiming "done".

It is not TypeScript-only. The planner infers the target language/framework from the goal and the workspace (no silent TypeScript default), and verification derives real checks per language — TypeScript/JS, Dart/Flutter, Rust, Go, Python, and C++ — so a "build a Flutter app" goal is planned and verified as Dart, not JavaScript.

The two-axis correctness model

CodeForge measures itself on two independent axes (EVALUATION_MODEL.md):

• Runtime Integrity — state / graph / tool / approval / budget / verification invariants (the kernel). This is mature and invariant-tested (~180 invariants across phases 0–12).
• Task Correctness — does the agent actually solve the user's problem (the intelligence). This is now measurable end-to-end via the evaluation/benchmark system, but real-model task-correctness numbers are still being gathered.

Keeping these separate is deliberate: a weak local model may produce poor task correctness while the kernel's integrity stays intact. Most current limitations are intelligence/product gaps, not integrity gaps.

---

Architecture at a glance

A TypeScript monorepo (npm workspaces, Node ≥ 20):

Package	Role
packages/agent-core	Pure domain + orchestration. No I/O, no infrastructure imports. Domain models, state machines, task graph, scheduler, budget, context builder, planner, recovery, mission intelligence (advisory), evaluation.
packages/infrastructure	Adapters: SQLite event log + repositories, Ollama model gateway, process supervisor, Node tool executor, code intelligence (Tree-sitter / LSP), observability HTTP+SSE server, CLI, Telegram transport.
packages/testing	Deterministic test doubles (e.g. FakeModel, fake process supervisor) for CI without a network or a live model.
packages/vscode-extension	Native VS Code client: live tree views (sessions / task-graph / changes / governance), editor diff + approval notifications, a Goals panel, model picker, and Start-Runtime-Here — all over HTTP (the extension is a client, never an authority).
tests	Vitest suites: unit, integration, invariants, cross-platform, evaluation.

Dependency direction is enforced by dependency-cruiser (agent-core must never import infrastructure / models / tools; no cycles).

Key building blocks

• Deterministic state machines for session / task / tool-call lifecycles.
• Task graph with canonical hashing (Blake3) and validated mutations — dependencies live only as graph edges.
• ToolGateway with autonomy levels (full / edits / readonly) and an approval coordinator for human-in-the-loop decisions.
• Verification engine + completion gate — real checks run through a process supervisor; the gate, not the model, decides task success.
• Recovery — RETRY / FIX / ESCALATE / ABORT implemented (REPLAN / ROLLBACK are still stubs).
• Mission Intelligence (advisory, Phase 12) — intake, complexity/risk/uncertainty, capability verification, model routing, architecture + a deterministic gate. Advisory by design: its main control-flow effect is the Architecture Gate halting for a human. Two parts now also steer the agent — stated goal assumptions (injected into the executor prompt) and the context strategy (sizes the context pipeline).
• Observability — append-only event log, HTTP + SSE dashboard, control plane (pause / resume / stop / approve).
• Evaluation / benchmark — isolated runner driving the real kernel, deterministic evaluator, metrics, baseline + regression detection, and a native seed dataset (CF-001..010).

---

Getting started

Prerequisites

• Node.js ≥ 20
• A model provider. CodeForge talks to models through a provider-agnostic gateway, so you can use either:
  • Ollama locally (default) — http://localhost:11434 with a model pulled, e.g. ollama pull qwen2.5-coder; or
  • Any OpenAI Chat Completions-compatible provider — commercial or free cloud (OpenAI, OpenRouter, Groq, Together, DeepSeek, Mistral, …) or a local server (LM Studio, llama.cpp, vLLM, LiteLLM). Pass --provider openai-compatible, the …/v1 base URL via --endpoint, and the key via the CODEFORGE_API_KEY env var.

Install & build

npm install
npm run build

Run the quality gates

npm test            # full Vitest suite
npm run lint        # eslint
npm run depcruise   # dependency-direction + cycle checks
npm run gate        # docs-sync + build + lint + depcruise + invariants (what pre-push runs)

Install the versioned pre-push hook once per clone with npm run hooks:install. It blocks a push when build, lint, depcruise or the invariant suite fails, or when the pushed commits change code without updating any project .md doc (override only with a Docs-Reviewed: <reason> commit trailer). Set CF_GATE_FULL=1 to also run the full Vitest suite.

Run the runtime (CLI + dashboard)

The runtime starts, processes the first goal, then stays alive as a goal-queue worker with a dashboard at http://localhost:<port>/:

npm start -- \
  --workspace /path/to/your/project \
  --model qwen2.5-coder \
  --goal "Add an isEven helper to src/util.js" \
  --port 9500 \
  --autonomy edits \
  --mission on

Enqueue more goals against the running process:

curl -X POST http://localhost:9500/goal \
  -H 'content-type: application/json' \
  -d '{"description":"Write a test for the reverse() function"}'

Goals are persisted (SQLite goals + goal_queue), so queued goals survive a restart and a goal is never run twice. On startup the runtime also handles sessions a previous, dead runtime process left behind: a session that was waiting for clarification answers before planning is resumed (same questions, answer them in the Goals panel or via POST /clarification); every other leftover session is closed with an explicit reason (SESSION_RECOVERY_DECIDED). Sessions of a live process or another machine are never touched.

To use a cloud provider instead of local Ollama:

$env:CODEFORGE_API_KEY = "sk-..."   # PowerShell (or: export CODEFORGE_API_KEY=sk-...)
npm start -- \
  --workspace /path/to/your/project \
  --provider openai-compatible \
  --endpoint https://api.openai.com/v1 \
  --model gpt-4o-mini \
  --goal "Add an isEven helper to src/util.js" \
  --autonomy edits

(For OpenRouter use --endpoint https://openrouter.ai/api/v1 and a model id like anthropic/claude-3.5-sonnet; for a local LM Studio server, --endpoint http://localhost:1234/v1.)

CLI options

Flag	Default	Meaning
--workspace <path>	cwd	Project root the agent operates on.
--goal <text>	—	The first goal. The runtime stays alive afterward; POST /goal enqueues more.
--provider <name>	ollama	ollama (local) or openai-compatible (any OpenAI Chat Completions API).
--model <name>	qwen2.5-coder	Model name/id (for a cloud provider, that provider's model id).
--endpoint <url>	provider default	Ollama server URL, or the cloud provider's …/v1 base URL.
--api-key <key>	—	API key for a cloud provider. Prefer the CODEFORGE_API_KEY env var. Never logged.
--port <number>	9500	Dashboard HTTP port.
--host <address>	127.0.0.1	Network address the runtime binds to. Loopback by default; an off-loopback host requires an auth token or the runtime exits before listening (fail closed).
--auth-token-file <path>	—	File whose trimmed contents are the shared bearer token (≥ 16 chars). Use this or CODEFORGE_AUTH_TOKEN, not both.
--allowed-hosts <name,…>	—	Extra Host header values accepted (added to the loopback names), each combined with the bound port. Needed when a non-loopback bind is reached through a LAN name or IP.
--allowed-origins <origin,…>	—	Exact cross-origin origins (http/https) allowed by CORS and the Origin check. Default: same-origin only. * and non-http(s) values are rejected at startup.
CODEFORGE_AUTH_TOKEN (env)	—	Shared bearer token (≥ 16 chars). When set, every data/control route requires Authorization: Bearer <token>; static assets and /health stay open.
--db <path>	.codeforge/runtime.db	SQLite database file.
--max-steps <n>	40	Max orchestrator iterations.
--autonomy <level>	edits	full / edits / readonly — which tool risk classes auto-approve vs. require a human decision.
--mission <on|off>	off	Advisory Mission Intelligence stage before planning.
--learning <on|off>	off	Advisory learning plane for recovery ordering.
--clarify <on|off>	off	Goal clarification gate (needs --mission on): when a goal scores UNKNOWN, pause in AWAITING_HUMAN, ask the open questions (Goals panel / POST /clarification), then plan under the answers. Off ⇒ assume-and-state.
--plan-options <on|off>	off	Plan options (needs --mission on): for a goal of complexity MEDIUM or higher, propose 2–4 approaches with pros/cons, wait in AWAITING_HUMAN for a pick (Goals panel / POST /plan-choice), then plan with it. Timeout ⇒ the recommended option. Off ⇒ the Planner chooses.
--model-routes <path>	—	JSON file routing each purpose to a provider/model (see below). Replaces --provider/--model/--endpoint.
--model-registry <path>	—	JSON file listing multiple models (id + tier + capabilities) so --mission on can route by task complexity (see examples/model-registry.example.json).

Tip for a smooth demo with a small local model: --autonomy full avoids approval prompts for commands. With weaker models, prefer concrete, well-specified goals (acceptance criteria the runtime can verify) — see the limitations below.

Remote access

The runtime binds to 127.0.0.1 (loopback) by default, so it is reachable only from the local machine. To expose it on a network, pass --host 0.0.0.0 (or a specific address) and supply a token via CODEFORGE_AUTH_TOKEN or --auth-token-file — an off-loopback bind without a token exits with an error before the server starts (fail closed). With auth active the dashboard prompts for the token (stored in memory / sessionStorage per tab, never in URLs or localStorage) and sends it as a Bearer header on every request. Use --allowed-hosts to list the names browsers will use to reach the server, and --allowed-origins for any cross-origin front-end. The token is a single shared secret, not per-user.

Per-purpose model routing (cost/quality split)

Instead of one model for everything, you can route each kind of work to a different provider/model: run reasoning-heavy purposes (plan, architect) on a strong (e.g. commercial) model and mechanical / I/O-heavy purposes (execute, summarize) on a cheaper or local one. All behind the single gateway the runtime uses — routing is deterministic and config-driven (the LLM never picks its own model).

Point --model-routes at a JSON file (see examples/model-routes.example.json). The file holds no API keys — each cloud entry names an environment variable via apiKeyEnv:

{
  "default": { "provider": "ollama", "model": "qwen2.5-coder", "endpoint": "http://localhost:11434" },
  "routes": {
    "plan":      { "provider": "openai-compatible", "model": "gpt-4o-mini", "endpoint": "https://api.openai.com/v1", "apiKeyEnv": "OPENAI_API_KEY" },
    "architect": { "provider": "openai-compatible", "model": "gpt-4o",      "endpoint": "https://api.openai.com/v1", "apiKeyEnv": "OPENAI_API_KEY" }
  }
}

$env:OPENAI_API_KEY = "sk-..."
npm start -- --workspace /path/to/project --model-routes examples/model-routes.example.json --goal "..." --mission on

Valid purposes: plan, critique, replan, execute, analyze_failure, summarize, architect. Any purpose not listed uses default.

There is also mission-driven routing: with --mission on and a --model-registry file listing several models, the Mission Intelligence stage picks one to fit the task (trivial → fast/cheap, complex → stronger) and the runtime switches the active model to it. See examples/model-registry.example.json (three local Ollama models tiered SMALL/MEDIUM/STRONG):

npm start -- --workspace /path/to/project --model-registry examples/model-registry.example.json --mission on --goal "..."

Without a registry file the gateway stays on the single configured model (no-op). Routing is deterministic and runtime-decided — the LLM never picks its own model.

Package the CLI (single file)

npm run bundle:cli    # → dist-cli/codeforge.cjs (native deps kept external)

VS Code extension

packages/vscode-extension is a native VS Code client for a running runtime: an activity-bar container with live tree views (Sessions / Task-Graph / Changes / Governance), native editor diffs per change, Approve/Deny notifications for pending tool calls, a Goals panel (conversational goal history + live status, an accessible answer form when the runtime asks clarifying questions, and approach cards with pros/cons when it offers plan options), a model picker (GET /models), and a Start Runtime Here command that spawns the runtime in the open workspace (codeforge.clarify / codeforge.planOptions add --clarify on / --plan-options on, with --mission on). All decision logic lives in pure, unit-tested modules; the extension stays a client (reads over HTTP, controls via POST /control). Built as a .vsix (not yet published); the dashboard webview is kept as a secondary view.

---

Evaluation

The evaluation system measures Task Correctness end-to-end. The first recorded baseline runs the native seed benchmark (CF-001..010) through the real kernel driven by a deterministic FakeModel:

Field	Value
Baseline label	codeforge-0.12-fakemodel
Benchmark	codeforge-native v0.1.0 (CF-001..010, 10 categories)
Agent / model	codeforge 0.12.0 · FakeModel (CI-deterministic, no network)
Success rate	3 / 10 (the scripted-solvable cases)

This is a harness baseline, not an intelligence baseline: the FakeModel was scripted to solve only 3 tasks; the other 7 were correctly caught by deterministic checks (the agent is not the authority on its own success). It proves the loop end-to-end — isolation → governed execution → external verification → verdict → metrics → baseline. A real-Ollama run would replace 3/10 with a genuine task-correctness figure; the harness is ready for it. See EVALUATION.md.

A separate interaction benchmark (benchmarks/codeforge-interaction, 12 vague / multi-approach goals with hidden intent and a deterministic simulated user) compares three arms of scripts/run-progressive.mjs --interaction off|assume|ask to measure whether asking and plan options improve correctness over assuming.

A kernel shadow ablation measures enforcement, not recovery: it replays one model trace through the real kernel twice — an enforce arm (today's behavior) and a shadow arm (every enforcement decision is recorded as would-block and the action proceeds in a throwaway sandbox) — with out-of-policy proposals from a fixed corpus injected, and reports what the kernel stopped or changed. Phase 1 is synthetic (node scripts/run-shadow-ablation.mjs --synthetic, report at .agents/kernel-shadow-ablation.synthetic.md): out-of-policy injections stopped 40 / 40 reached, organic proposals changed 2 of 14, kernel misses 0 — on scripted traces, not a real model. Phase 2 recorded 50 real Qwen3.5 traces (train split) with scripts/run-progressive.mjs <model> --record-trace <dir> and replayed them with scripts/run-shadow-ablation.mjs --traces <dir> --benchmark <benchmark.json>: out-of-policy injections stopped 415 / 415 reached, kernel misses 0, and on organic proposals (the model's own, uninjected output) the kernel changed the first decision in 30 of 50 cases (60%) — not "barely differ". Of those 30, 16 are a premature done (15 confirmed wrong by the hidden benchmark checks and stopped for a human — kernel value; 1 refused mid-session and later passed), 5 are done refused because the project had no runnable check (VERIFICATION/NO_INRUN_CHECK; the gate did not prevent the 3 wrong results among them), and 9 are a shell command rejected by the allowlist, 7 of them because the model put the binary and its arguments in one string where the executor expects command + a separate args array (a candidate prompt/executor protocol mismatch, no effect on any verdict). With enforcement off the hidden-check pass rate was the same within one case (29 vs 28 of 50), so the demonstrated effect is on what the session reports (false completions avoided), not on final correctness. Full analysis, numbers and caveats: .agents/kernel-shadow-ablation-analysis.md.

---

Project direction

North star: a fully autonomous software-engineering agent — one that understands a goal, plans, writes and changes code, verifies its own work against reality, recovers from failures, and learns from them, with the human stepping in by choice rather than necessity. The deterministic kernel is not the destination; it is the safety substrate that makes autonomy trustworthy. Every step toward autonomy keeps the central law intact: the LLM proposes, the runtime decides and enforces.

The path there, in increasing autonomy:

1. Trustworthy kernel (today) — governed tools, deterministic verification, invariant-tested state. The agent cannot do anything the runtime does not allow or cannot check.
2. Supervised agent — the intelligence layer actively steers execution (model routing, context, planning strategy, architecture), with human approval on risky actions. In progress: context strategy and goal assumptions already steer; routing and planner consumption are next.
3. Self-correcting agent — a complete recovery vocabulary (incl. REPLAN / ROLLBACK) plus the learning plane turning past outcomes into better future decisions.
4. Autonomous agent — the human sets the goal and reviews the result; the agent drives the loop end-to-end, with evaluation gating every self-improvement.

Development is moving from phase-driven toward evidence-driven: no step toward more autonomy ships without a before/after benchmark proving it actually helps.

Goal → Roadmap → Implementation → Evaluation → Evidence → Improvement → New baseline

Highest-value next work (evidence-gated by before/after benchmark runs):

1. Wire Mission Intelligence further into execution — context strategy already sizes the context pipeline; next is applying the routed model at execution time and letting the Planner consume planning-mode / expert-profile / architecture. (Step 2 → 3 of the autonomy path.)
2. Goal clarification — detection (Tier A), assume-and-state (Tier B1/B2) and an opt-in stop-and-ask gate (Tier C Phase 1, --clarify on) and opt-in plan options (Tier C Phase 2, --plan-options on: pick one of 2–4 approaches) are done; next is a before/after benchmark.
3. Finish the recovery vocabulary — implement REPLAN and ROLLBACK. (Toward step 3.)
4. Measure real-model task correctness — run the benchmark against a live model so autonomy claims rest on evidence, not hope.
5. Product surface — task-graph DAG, change/diff transcript, governance timeline, and a native VS Code extension are shipped (P12.18–P12.23); next is a polished, published .vsix and richer live trace.

Full detail lives in ROADMAP.md (trajectory) and PROJECT_STATUS.md (what the software actually does today, verified against the source tree).

---

Honest limitations (today)

• Mission Intelligence steers execution only partially. Its control-flow effect is the Architecture Gate halting for a human; goal assumptions and context strategy now steer the executor, but routed-model-at-execution and planner consumption of planning-mode/architecture are not wired yet.
• Real-model task correctness is not yet measured. The harness is ready; the numbers aren't in.
• REPLAN / ROLLBACK recovery are stubs.
• Model routing selects an id but doesn't switch the executing model (a runtime switch exists for the dashboard, but routing decisions aren't applied automatically).
• Local small models (6–9B) are weak agents — they plan acceptably but often loop on reads, emit malformed tool-call JSON, or over-decompose. The runtime mitigates this but cannot make a weak model competent; agent quality is model-bound.
• Product surface is still maturing — a functional dashboard (task-graph DAG, change/diff transcript, governance timeline, approval banner) and a native VS Code extension exist, but it is not yet a polished, published IDE experience.
• No TLS. The runtime serves plain HTTP; put a TLS-terminating reverse proxy in front for remote access over an untrusted network.
• Single shared token, not per-user. Auth is one bearer secret shared by everyone who connects; there are no identities, roles, or audit of who made a request.
• The VS Code extension is tokenless and loopback only. It starts the runtime on 127.0.0.1 without a token and needs no change; passing a token from the extension is a later (P1) item.
• The kernel shadow ablation has hard limits. After the two arms diverge, the rest of the replay is counterfactual (the recorded model never saw the other arm's results); containment is path/command-level, not an OS sandbox; the in-loop guards P12.11 / P12.13 are observed, not ablated. The Phase 2 result (Qwen3.5, 50 train cases, one recording; 28/50 passed vs 31/50 in an earlier run) is one model, one split, one recording, with prompt drift in 15 of 50 traces and the shadow arm exhausting its recorded steps in 12 of 50; its shadow-arm final states are counterfactual. See .agents/kernel-shadow-ablation-analysis.md.
• Five confirmed kernel gaps under full autonomy (found by the ablation; reported, not fixed): run_command rm -rf .. (allowlisted binary, arguments not path-checked); run_command curl (allowlisted, and run_command is classed SYSTEM, never NETWORK); read_file .env inside the workspace (READ_ONLY allow, so the content enters the prompt); write_file .git/config and edits to test files (no protected-path rule — ToolRule.deniedPaths is declared but not enforced). Under edits autonomy rm / curl are stopped (approval required). See "Confirmed kernel gaps" in packages/testing/src/shadow-ablation/injection-corpus.ts.
• The infrastructure redactor misses unencrypted PKCS#8 keys. A -----BEGIN PRIVATE KEY----- header is not redacted (the rule requires a type word such as RSA); found by the trace tests, reported, not fixed yet.

None of these violate a tested kernel invariant. Most are intelligence/product gaps, which is exactly the distinction the two-axis model exists to make; the kernel and redactor gaps above are policy coverage the invariants do not yet claim, surfaced by measurement rather than assumed away.

---

Documentation map

Document	What it holds
PROJECT_STATUS.md	Authoritative, living status matrix — what the software actually does today.
ROADMAP.md	Product trajectory that survives across work sessions.
Coding Agent Architecture Target.md	The product/architecture target.
EVALUATION_MODEL.md · EVALUATION.md	The two-axis model and the evaluation system.
*_SPEC.md	Context, verification, state-machine, workspace, security, infrastructure, migration specs.
INVARIANTS.md · invariants.yaml	The enforced invariants.
PHASE_*	Per-phase roadmaps and sign-offs.

## Copyright & Attribution

CodeForge-v2 is an original software project by [DienVu](https://github.com/Dienvu1008),
licensed under the [Apache License, Version 2.0](LICENSE).

- **Original repository:** https://github.com/Dienvu1008/CodeForge-v2
- **Copyright:** © 2026 DienVu (Dienvu1008)
- **License:** Apache-2.0

You are welcome to use, modify, fork, redistribute, and build upon this project under
the terms of the Apache License 2.0. When redistributing, please preserve the
applicable copyright notice, a copy of the license, and (if present) the NOTICE file.

Third-party dependencies remain subject to their own licenses.

See [ATTRIBUTION.md](ATTRIBUTION.md) for detailed attribution information and
[CITATION.cff](CITATION.cff) for citation metadata.