Skip to content

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Tellico agentic setup

Portable OpenCode client setup for the two Qwen3.8-27B servers running on the Tellico cluster.

The repository contains no API key or SSH private key. The installer retrieves the model API key through your authenticated Tellico SSH connection and stores it only on the client device with mode 0600.

There are four ways in:

Who it is for What the user needs
Client, tunnel mode (this README) People with a cluster account A Tellico account, an authorized SSH key, and this repository
Client, gateway mode (this README) The same agentic setup, without a cluster account A URL, an API key, and this repository
Raw API (gateway/) Any OpenAI-compatible tool or script A URL and an API key. Nothing installed
Phone (relay/) Following and steering your own sessions, on any machine, from a phone Your API key and a browser, plus --remote-control on each computer

The two client modes give an identical session: the same two node-pinned workers, the same models, the same context. They differ only in how the requests travel. Tunnel mode forwards the private cluster endpoints over your own SSH connection. Gateway mode sends them to the API gateway instead, which needs no SSH, no cluster account and no tunnel at all -- just the URL and key whoever runs the gateway gave you.

The gateway itself runs on one always-on host in tunnel mode, so all three paths share the same tunnel, cluster key and per-slot context.

Architecture

OpenCode provider Cluster node GPUs Slots Context per slot Local endpoint
tellico-0/qwen3.8-27b-gsq-iq3s tellico-compute0 2 x V100 16 GB 2 131,072 127.0.0.1:18080
tellico-1/qwen3.8-27b-gsq-iq3s tellico-compute1 2 x V100 16 GB 2 131,072 127.0.0.1:18081

Each server divides one 262,144-token pool across two slots, so four concurrent requests fit cluster-wide; beyond that, requests queue. A single OpenCode session can issue several at once, because title, summary, compaction, and subagent calls all go to the same two servers.

The model is Qwen3.8-27B in ISTA-DASLab's GSQ-RCO IQ3_S quantisation (11.8 GB, publisher-reported lossless on LiveCodeBench v6), with a DFlash2 draft model for speculative decoding, chosen on 2026-10-09 for agentic coding at 128k. On a 12-task unit-tested coding suite it passed 11/12 against 4/12 for the Qwen3.6-35B-A3B it replaced, and its tool calls held up where the 35B's did not. Historical measurements before the engine work were 72-86 tok/s alone, 52-70 each with two requests and 430-650 tok/s cold prefill. The 2026-10-15 snapshot, one request at a time on tellico-compute0 over a 17-request replay of real agent traffic, is 105-216 tok/s with a median of 152 and ~1500 tok/s cold prefill. Time to first token was fixed separately on 2026-10-16: a returning session now answers in 0.9-1.1 s whether its next turn continues or branches from what the server holds, where a branching turn previously re-read the whole prompt (75.1 s for one sharing 97012 tokens). The model is hybrid -- 16 attention layers and 48 recurrent ones -- so a slot can only resume where a context checkpoint was saved, and the two the server kept were both at the very end of the prompt; tellico-v100 r6 changed which ones survive and raised the budget to 8 after a 61-minute fault soak. Which end of the tok/s range a request lands on is decided almost entirely by how well the draft model predicts it, not by context length: a round of generation takes 31-38 ms whatever the depth, while the number of tokens it yields ranges from 3.9 to 6.7. Tool-heavy editing of familiar code drafts well and runs near 200; unfamiliar prose-ish reasoning drafts badly and runs near 110. Measured on one build, one node, one replay (2026-10-16), the whole spread is just acceptance:

draft acceptance 0.42 0.45 0.50 0.64 0.73 0.84
tok/s 132 86 130 160 183 230

So a session that reads 86 tok/s is a session the drafter cannot predict -- long prose at depth, typically -- not a regression.

This is also why the sampler is a speed setting here and not only a quality one: a drafted token survives only if the target's sampler picks that token, so a flatter target rejects tokens the drafter got right. Measured over the same replay set, median acceptance is 0.643 at temperature 0.6 and 0.705 at 0.5, which the fit above puts at 166 against 181 tok/s. The default moved 0.6 -> 0.5 on 2026-10-16, after eight repeats of the 12-task coding suite came out level (90/96 at 0.6, 92/96 at 0.3, 90/96 at top_p 0.8) and a 1600-token thinking-mode degeneracy probe found no repetition at any temperature down to greedy -- repetition being the documented risk, since Qwen recommends 0.6 for this model. What the sampler cannot fix is free prose: that stays at acceptance ~0.26 whatever the setting, and a higher-precision drafter does not help either (DFlash2 Q8_0 measured -0.8% with acceptance unchanged), so a long prose answer is near 80 tok/s by nature. To tell the two apart, print_timing in /data/gclab/qwen38/logs/server-<job>-<node>.err gives that request's own rate and whether the other slot was busy, and tellico-v100/bench/shape_probe.py sends one deep prompt twice, once as tool-shaped work and once as long prose, printing acceptance beside tok/s. tellico-compute1 measures 7-8% below compute0 on identical builds: one of its two GPUs does not boost above its 1312 MHz application clock while the others reach 1530, and nvidia-smi -ac is root-only, so only an administrator can fix it.

Since 2026-10-13 the server reads prompts in 1024-token rather than 2048-token chunks, which halves the stall a prompt causes other streams on its node and costs about 20% of prefill speed.

Do not expect this to improve much further. The generation speed is bounded by two measured facts: the draft model emits a fixed 8-token block per round, and a round cannot be shorter than about 15 ms because one GPU has to read its half of the model's weights once. Together those put the practical ceiling near 250 tok/s on well-drafted requests. The analysis, and the five engine changes that were implemented and then rejected for measuring slower, are in the tellico-v100 repository (docs/report-2026-10-15-r5.md).

These are named measurement snapshots, not guarantees for every task or context length; draft acceptance, cache reuse and concurrency affect the result. Measure completed, verified work as well as token throughput.

Slots are not free in VRAM: a third 128k slot does not fit beside the drafter (it does without it, at 24-34 tok/s each, which is slower for everyone). Speculation makes the model evaluate a block of tokens per pass, so output is a valid sample but not byte-identical to a non-speculative run. The trade is set on the cluster side in service.env (QWEN38_MODEL, QWEN38_CTX, QWEN38_SLOTS, QWEN38_SPEC_*); limit.context in opencode.json has to match the per-slot figure that produces.

In gateway mode the local endpoints are replaced by the gateway's node-pinned paths -- https://HOST/v1/node0 and .../node1 -- which route to the same two servers. Everything else in the table, including the per-slot context, is identical.

An SSH connection forwards the two private cluster endpoints to localhost. OpenCode gets two primary agents per node and two node-pinned subagents:

Agent What it is
build-tellico-0, build-tellico-1 The default. One agent, one server, no delegation. Reads, edits and runs checks itself.
orchestrate-tellico-0, orchestrate-tellico-1 Started with --orchestrate. Implements, checks and integrates directly; delegates substantial independent work when it helps. Workers may remain idle.
frontier-tellico Started with --frontier, or tab. A hosted frontier model (GPT-6.1 Terra at high effort by default, or Sol) that plans, briefs the workers, checks their work and integrates it; cannot edit files.
tellico-worker-0, tellico-worker-1 Subagents the orchestrate and frontier leads dispatch to, one pinned to each node.

tab switches between the primaries inside a session, so the choice at launch is only where the session starts. See Orchestrate or solo.

The same two servers also back three more harnesses. pi-tellico runs the pi coding agent and crush-tellico runs Crush; both are single-agent, so each is the counterpart of the default opencode-tellico session in its own harness. hermes-tellico runs Hermes, which has subagents, so it is the counterpart of opencode-tellico --orchestrate instead: lead on one node, children on the other. See The pi harness, The Crush harness and The Hermes harness.

The cluster side of the service -- the Slurm job, the per-node llama.cpp server and the commands that start and inspect them -- lives in qwen38-cluster, and only the service owner needs it. The two repositories share the API key path, the port, the node names and the per-slot context, so a change to capacity on the cluster means a matching change to config/opencode.json here.

First run, gateway mode

If someone gave you an API key, that key is all you need. The gateway URL is built into this repository. You need Linux, macOS, or WSL with curl, sed, install, and OpenCode on PATH -- no ssh, no VPN, no cluster account.

git clone https://github.com/Yejashi/tellico-agentic-setup.git
cd tellico-agentic-setup
./install.sh

With no mode flag it asks which way you want in; choose 1) API key, paste the key when prompted, and that is the install. Then:

opencode-tellico 0

./install.sh --gateway skips the question. To avoid the key prompt too, pass --api-key-file PATH or pipe the key in:

printf '%s' "$KEY" | ./install.sh --gateway

Rerunning ./install.sh later keeps both the mode and the key, so an update after git pull asks nothing. Pass --api-key-file to replace the key, or --gateway-url URL to point at a different gateway.

./doctor.sh knows which mode a device is in once installed. Before the first install there is nothing recorded yet, so say which mode to check:

./doctor.sh --gateway

The rest of this section is tunnel mode, which gateway mode does not need.

First run, tunnel mode

Follow these in order; each step depends on the one before it. Run ./doctor.sh at any point to see which step you are on.

  1. Get the repository. It is private, so the device needs GitHub access first:

    git clone git@github.com:Yejashi/tellico-agentic-setup.git
    cd tellico-agentic-setup

    An HTTPS checkout works too when Git credential authentication is set up: git clone https://github.com/Yejashi/tellico-agentic-setup.git

  2. Install the tools. The device needs Linux, macOS, or WSL with a POSIX shell, plus ssh, curl, sed, install, and OpenCode on PATH.

  3. Reach the network. tellico.icl.utk.edu is only reachable from the site network, so connect to the VPN when off site.

  4. Add an SSH alias to ~/.ssh/config, if the device has none. User is your own Tellico account:

    Host tellico
      HostName tellico.icl.utk.edu
      User YOUR_CLUSTER_ACCOUNT
      IdentityFile ~/.ssh/id_ed25519

    The account that owns the allocation is one person's account and not a shared login, so connect as it only if it is yours. ./doctor.sh reports the account the alias resolves to, which is the one the next step authorizes against.

  5. Authorize this device. Every device gets its own key; never copy a private key between machines. If this one has no key yet:

    ssh-keygen -t ed25519

    Then add its public key to ~/.ssh/authorized_keys of the account from step 4, either with ssh-copy-id -i ~/.ssh/id_ed25519.pub tellico or by appending it from a machine that already has access. ./doctor.sh prints this device's key and both commands, ready to paste, and names the account they apply to.

  6. Install:

    ./install.sh

    The installer is idempotent and can be rerun after pulling updates.

  7. Make sure an allocation is running. The two model servers exist only while a Slurm allocation is active:

    tellico-qwen-tunnel cluster-status

    Only the service owner's account can submit the job. If there is no allocation, ask them to start one, then run tellico-qwen-tunnel restart.

Installer options

With no mode flag, an interactive run asks which mode to install, defaulting to the mode already recorded on the device.

--gateway                Gateway mode against the built-in URL
--ssh, --tunnel          Tunnel mode over your own SSH connection
--gateway-url URL        Gateway mode against a different base URL, for
                         example https://host.example.ts.net/v1
--api-key-file PATH      File holding the gateway API key. Without it the key
                         is piped in, typed, or kept from a previous install.
--ssh-host HOST          SSH hostname or config alias (default: tellico)
--remote-key-path PATH   Model API key path on the cluster, for accounts that
                         read it from somewhere other than the default
--fix-path               Add ~/.local/bin to PATH in the login shell's
                         profile, if it is not there already
--port0 PORT             Local port for tellico-compute0 (default: 18080)
--port1 PORT             Local port for tellico-compute1 (default: 18081)
--no-start               Install the files only, skipping every connectivity
                         check. Use this to set a device up before it has been
                         authorized on Tellico.
--no-t3code, --t3code    Leave t3code alone (and take out the Tellico provider
                         an earlier install added), or put it back. On by
                         default; a rerun keeps the last choice. See t3code.

Platform notes

These notes are about holding the SSH tunnel open, so they apply to tunnel mode only. Gateway mode has no tunnel and behaves the same everywhere.

Linux. The installer enables a systemd user unit, so the tunnel starts at login and restarts on failure.

macOS. There is no systemd, so the tunnel runs as a detached SSH master instead. It survives closing the terminal, and opencode-tellico restarts it whenever it is not running, but it does not auto-start at login or restart itself after a mid-session drop. Run tellico-qwen-tunnel restart if a session loses the servers. If the SSH key has a passphrase, add it to the agent once:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Windows (WSL). Use a WSL distribution with its own OpenSSH client:

sudo apt install openssh-client

Windows' ssh.exe is visible inside WSL through PATH interop, but it reads a different ~/.ssh and cannot hold the tunnel's control socket; the installer stops with instructions if it detects that. Whether the tunnel uses systemd or a detached SSH master depends on whether your distribution has systemd=true in /etc/wsl.conf; both work.

Troubleshooting

./doctor.sh reads the mode this device was installed in and checks accordingly, printing the one next step for the first thing that fails. It is safe to run before installing. After installing, the same checks are available as tellico-qwen-tunnel doctor.

In tunnel mode it checks tools, SSH config, network, device authorization, the API key, the allocation, and tunnel health, in that order. In gateway mode none of that applies, so it checks tools, the config, and one authenticated probe of the gateway, which distinguishes an unreachable gateway from a rejected key from a gateway with no model server behind it.

Use

Four commands start a session: opencode-tellico for the OpenCode harness with its lead and two workers, hermes-tellico for Hermes, which also spreads subagents across both nodes, and pi-tellico and crush-tellico for the single-agent pi and Crush harnesses. All four share the tunnel, the key and the two servers.

Start a dual-node OpenCode session with the lead on node 0:

opencode-tellico 0

Or put the lead on node 1:

opencode-tellico 1

The number chooses the server the session's agent runs on. By default that agent does the work itself; to start on the lead that can also delegate to both node-pinned subagents, add --orchestrate; see Orchestrate or solo.

With no number, the lead node is derived from the device, so that several users do not all lead on node 0 and compete for one server's prompt cache. It is stable for a given device, which keeps that cache warm across sessions; TELLICO_LEAD_NODE=0 or =1 overrides it.

Why it matters: the servers run a 91% prompt-cache hit rate, and a miss on a large agentic context costs minutes of reprocessing. Spreading the leads is free and protects it. See docs/flash-next-evaluation.md for why the window cannot simply be made bigger instead.

Orchestrate or solo

Sessions open on build-tellico by default: one engineer that reads, edits, runs checks and performs authorized Git operations itself, with delegation disabled. (--solo is still accepted and means the same.)

--orchestrate opens on orchestrate-tellico: the same direct workflow, plus delegating substantial independent work when that helps complete the task, rather than requiring a two-worker round for every increment. It prefers the opposite node for the first worker while its own node is busy. tab switches between the two inside a session.

opencode-tellico 0 --orchestrate
TELLICO_SOLO=0 opencode-tellico

Both primary agents default to 200 tool steps per turn. For a long unattended session, --steps extends both primaries, including one selected later with tab; worker limits remain 200:

opencode-tellico 0 --steps 100000
opencode-tellico 0 --orchestrate --steps 100000

Frontier lead

A frontier model plans, decides, briefs and verifies; the two Qwen workers do the investigation, reading, editing and test runs. The local model executes a well-specified task reliably but weighs broad, ambiguous work poorly; the frontier model is the reverse and is the metered part. The lead cannot edit files itself, because a frontier lead that is merely allowed to delegate measurably does not. See docs/frontier-lead.md for the published experience this is built on and what was measured here.

It is frontier-tellico, an OpenCode agent, so everything this setup configures applies: the workers are real subagents you can watch in the sidebar, in t3code (the Tellico Frontier provider, below) and on the phone relay, with Magic Context, the guard plugins and serve. It defaults to OpenAI's GPT-6.1 Terra at high reasoning effort; Sol is one switch away. OpenAI lets a ChatGPT plan be used from other apps, and OpenCode has that sign-in built in, so no API key is needed.

opencode auth login                    # once: OpenAI, then "ChatGPT" sign-in
opencode-tellico --frontier            # Terra, high effort
opencode-tellico --frontier-model openai/gpt-6.1-sol
export TELLICO_FRONTIER_MODEL=openai/gpt-6.1-sol    # make Sol the default

tab (or the VS Code picker) reaches it in any session. TELLICO_FRONTIER=1 and TELLICO_FRONTIER_MODEL are the environment forms, and the model applies when you tab to it as well. TELLICO_FRONTIER_EFFORT (default high) sets its reasoning effort as a model-level default, so t3code, which sends no variant of its own, gets it too; a level picked with ctrl+t or in t3code's picker wins, and OpenCode remembers that pick for the model. --steps applies to it; --think is the workers' setting. The workers think at xhigh (TELLICO_WORKER_THINK).

Its window is held to the workers' 128k, which is required rather than merely cheaper: compaction, titles and the Magic Context historian all run on the local model. The cap covers the model you name plus Terra and Sol (TELLICO_FRONTIER_CAP changes the list), for providers this device has a key or opencode auth login entry for -- capping one without would list its models in every picker. The lead cannot edit files: its agent denies edit, and plugins/frontier-guard repeats that at execution time, because t3code's full-access mode writes session rules that would otherwise override it. secret-guard blocks reads of OpenCode's credential store (~/.local/share/opencode/auth.json), where the sign-in is kept, because the workers allow every tool.

Claude models are not used for this. Anthropic permits subscription credentials only in Claude Code itself, which is why claude-tellico and codex-tellico (Claude Code or Codex leading the workers through an MCP bridge) existed; they were removed on 2026-11-14 in favour of this, which shows the workers' progress and runs through serve and t3code.

pi and Crush have no subagents, so there is no frontier mode for them. Hermes does; see docs/frontier-lead.md for where it stands.

What it costs, and what it saved

tellico-savings              # last 30 days: actual spend, frontier-only estimate, saved
tellico-savings 7d --json
tellico-savings dashboard    # writes ~/.local/state/tellico-qwen/savings.html and opens it

It reads what the tools already record -- Claude Code transcripts, Codex rollouts and OpenCode's own database -- so nothing extra is logged while you work. A session counts as a frontier lead when it ran on frontier-tellico (or, before 2026-11-14, loaded the claude-tellico/codex-tellico worker bridge); ordinary Claude Code and Codex use is left out. Actual is the lead's own tokens at its model's list price (models.dev, cached for a day). Frontier only adds the workers' tokens at that same model's prices, keeping their cached and uncached split, and saved is the difference. That is an estimate, not a measured comparison: the frontier model would not have taken exactly the workers' steps, tokenizers differ, and the lead's coordination overhead is counted as spent either way. On a Claude or ChatGPT plan, read the dollars as plan usage at list prices.

Every device in one place. After each refresh tellico-savings also pushes this device's rows to the relay on the gateway host, authenticated with the gateway key, and Grafana there shows them together in the Frontier savings dashboard (/grafana/, folder Tellico): actual vs. frontier-only per day, by device, by lead and model, and tokens, filterable by user and device. Only numbers travel -- kind, lead tool, model, day, token counts, list-price cost, worker-minutes and a device name (TELLICO_DEVICE, default the hostname); session ids are hashed, and no prompt, path or project name is sent. Only rows that changed since the last push are sent, and a device that is offline catches up next time. A tunnel-mode device pushes only if it also has a gateway key (api-key.gateway). --no-push or TELLICO_SAVINGS_PUSH=0 keeps a device local.

Results go into a ledger (~/.local/state/tellico-qwen/savings-ledger.json) that keeps a session after its source is pruned. opencode-tellico refreshes it, and pushes, in the background on launch at most every ten minutes (TELLICO_SAVINGS_AUTO=0 turns that off), and every tellico-savings run does too. The report also lists worker runs with no frontier lead (not counted as saved) and what your all-local build-tellico/orchestrate-tellico sessions would have cost on GPT-6.1 Terra (--reference picks another model). uninstall.sh leaves the ledger.

Both modes use medium reasoning by default and retain the credential guard and OpenCode's external-directory permission behavior. Repository instructions still apply: a project that restricts the lead to read-only Git or requires a separate commit worker retains that workflow until its own rules are revised.

Dispatch is foreground by default: the lead sends its workers off together and waits until every one has reported before it continues. That gives up some overlap, and in exchange the lead never has to track reports that are still owed or take notifications mid-turn -- bookkeeping the model handled badly in practice -- and its slot is free for other users while the workers run. TELLICO_BACKGROUND_SUBAGENTS=1 turns background dispatch on for interactive sessions, where the lead keeps working and is notified as each worker finishes; opencode run is always foreground. The dispatch plugin tells the agent which mode is active.

For consecutive slices in the same subsystem, the prompt allows resuming the worker session if the tool provides that facility (task_id on OpenCode 1). Use a fresh session for unrelated work or stale/crowded context. This can avoid repeated orientation but does not guarantee prompt-cache reuse.

Measuring agent overhead

The existing gateway monitor shows request load, queueing, generation/prefill rates and aggregate cache behavior. An optional client-side trace adds tool and dispatch durations for both supported plugin APIs:

TELLICO_AGENT_TRACE="$PWD/tellico-agent-trace.jsonl" opencode-tellico 0

The path must be absolute and its parent must exist. The plugin creates a private append-only JSONL file (existing files retain their permissions), logging timestamps, process/session/call IDs, tool names, worker/node, dispatch mode and duration. It records no prompts, command text, file paths, outputs or keys. Trace write failures never fail a tool. Rotate/remove the file between measured runs; it has no automatic size limit. The file is gitignored by basename.

tool_return.duration_ms measures a tool call; dispatch_return.duration_ms measures foreground task latency or background acknowledgement latency. completion_observable: false means the return cannot establish worker completion. Even foreground return is not proof that acceptance passed. The trace does not measure model generation, reasoning tokens or request prefill: use server timings and session exports alongside it. Do not add overlapping agent/tool durations to estimate total wall time.

Compare similar bounded tasks from the same starting code, recording elapsed time, retries and verified acceptance conditions. Prompt-guidance changes and mock regression tests are not a demonstrated speedup.

Watching the workers

The sidebar lists the session's subagents -- which worker, what it was asked, and whether it is still running -- and clicking one opens its session so you can follow its progress. OpenCode 2 hides the sidebar until you press ctrl+x b (or run "Show sidebar" from ctrl+p). On OpenCode 1 the panel is the vendored subagent-watch; on OpenCode 2 it is plugins/tui-v2/subagents.

Controlling how much it thinks

The model is a reasoning model, and how much it reasons is a model variant. Four levels exist, which are what the chat template accepts:

Level Meaning
off no reasoning at all; fastest, for mechanical edits
low brief reasoning
medium the default for the leads
xhigh the most the template allows; the default for the two workers

medium is the default because it is the level this setup is tuned around, and it is now stated in config/opencode.json rather than inherited: the cluster happens to start llama-server with --reasoning-effort medium too, but the client no longer depends on that, and the template's own default is xhigh. Change agent.*.variant there to move the default.

OpenCode 2: fixed as of 2026-10-13. 2.0.26 only forwards settings written as extraBody, so the config now carries both spellings, and the session's title, summary and compaction run on a no-thinking twin of the model. Measured end to end: off produces no reasoning at all, and a level chosen inside the session overrides the one given at launch.

There are three ways to change it, from widest to narrowest scope.

For a whole session, at launch:

opencode-tellico 0 --think low
TELLICO_THINK=off opencode-tellico 1      # same thing, as an environment variable

Inside a running session, with OpenCode's own variant controls:

ctrl+t      cycle the level
ctrl+x t    list the levels and pick one

For one request:

/think-off    fix the import order in this file
/think-xhigh  why does this lock order deadlock under two writers?

The workers default to xhigh (TELLICO_WORKER_THINK changes that): they get narrow, well-specified tasks, where thinking longer is cheap insurance, while a lead's reasoning sits on the critical path of every turn. --think applies to the lead and both workers (verified on 2.2.5 for a dispatched worker). ctrl+t changes the session's model variant, which is the lead's: a worker dispatched afterwards still runs at the level the session was launched with, so reach for --think when you want the whole fleet moved. ctrl+x t is this setup's binding (config/tui.json) and is OpenCode 1 only; ctrl+t is built in and works on both.

Title, summary and compaction never think at all, whatever you ask for: they are the session's own overhead rather than work you asked for, and a summary that spends its output budget on reasoning comes back truncated, which ends the session (see Compaction).

Non-interactive example:

opencode-tellico 0 run 'Review this repository and fix the highest-impact issue'
opencode-tellico 0 --think off run 'Rename this symbol across the repository'

When the disk fills up

On a full disk OpenCode's write tool creates the file, fails to fill it, and reports only "Unable to write" -- which an agent can mistake for a broken tool. plugins/disk-guard checks free space before every write and, when there is not enough, refuses with the reason instead, so no empty file is left behind and the agent knows to free space or stop.

When it repeats itself

Two things in every request keep the model out of thinking loops. It no longer re-reads its own earlier turns' thinking, which it used to get back in full -- including any loop it had fallen into, which it then copied. And no single think can run past 8,192 tokens: the server ends it with a note to act instead.

On top of that, now and then the model still falls into a loop -- the same paragraph of deliberation over and over -- and a looping step never ends by itself: it runs to the 32,768-token output cap, holding one of the cluster's four slots for minutes. plugins/loop-guard watches each reply as it streams. When the newest text is almost entirely text the reply already wrote, it stops the step and tells the session to stop deliberating and take the next action. A looping worker is stopped and its lead is told why, so it can narrow the task or do it itself. It never touches ordinary output, quoted code included, and TELLICO_LOOP_GUARD=0 turns it off.

Compaction

A session that fills its 128k window is compacted: OpenCode summarises the old turns and continues from the summary. That is routine, but on this setup it had two ways to fail, and a failed compaction is terminal -- OpenCode stops the turn, and every turn after it retries the same compaction and stops again, so the session is finished mid-task.

Both are closed now:

  • The summary ran out of output budget. Compaction inherited the session's thinking level -- the compaction messages in this device's history are recorded with variant: xhigh -- so the model reasoned at length and the summary itself could be cut off. Measured across 193 compactions on this device, summaries run 600-7,400 output tokens with thinking on, median 3,215, and the thinking is all of the variance. OpenCode 1.18.30's own compaction allows 32,000, so there was margin here; its newer compaction path hard-caps the summary at 4,096, which 17% of those 193 exceeded. Compaction, summary and title now never think (plugins/compaction-guard/, plus agent.compaction.options in config/opencode.json), which removes the variance instead of relying on the margin.

  • The compaction request was itself too big for the window. Compaction renders the whole conversation so far into one prompt, reasoning blocks included, and reasoning is the bulk of it -- 74% of a 115,562-token payload in the one session on this device where compaction failed outright. compaction.reserved is meant to make compaction fire early enough to leave room, but OpenCode only honours it when the model also declares limit.input, which is why each model here now declares it. Compaction fires at 131,072 - 40,000 = 91,072 tokens, leaving the whole 32,000-token output allowance plus slack for one more turn.

If it ever does fail, the session cannot be rescued: start a new one. Nothing is lost from disk, only the conversation.

Magic Context

opencode-tellico and pi-tellico both replace their harness's compaction with the Magic Context plugin by default. Instead of one blocking summary when the window fills, a background "historian" summarises old history into tiers that stand in for it, stale tool output is dropped by rule, and decisions and conventions it notices are kept as project memory that every later session inherits.

opencode-tellico 0 --no-magic-context   # OpenCode's own compaction instead
TELLICO_MAGIC_CONTEXT=0 opencode-tellico  # the same, for every session

Under pi it is the npm:@cortexkit/pi-magic-context entry in pi/settings.json's packages, which pi installs into its agent directory on the first start after install; delete that entry and rerun ./install.sh --no-start to turn it off there, and pi's own compaction takes over again. Both are pinned to the same version, because they share one database and processes of different versions on it block each other's migrations. Under pi the historian is a child pi --print through the tellico-1 provider, with tellico-0 as the fallback, since pi has no worker providers. Crush has no Magic Context port, so crush-tellico keeps its own auto-summarize.

It is on by default, but it has measured costs worth knowing:

  • The historian is a session on the cluster. It runs as a hidden child session through OpenCode, so it holds one of the four slots while it reads the old history. config/magic-context.jsonc sends it through the tellico-1-worker provider with tellico-0-worker as the fallback, so the gateway queues it behind every lead, and with thinking off, so it cannot spend its output budget on reasoning the way OpenCode's own summary once did. plugins/compaction-guard/ does not cover it: that guard keys on OpenCode's compaction, summary and title agents by name.
  • Every request pays for it. The plugin's guidance block is about 1,400 tokens in its full form and the five ctx_* tool schemas about 2,500 more, on every lead and worker request. The installed config picks the light preset; the tools stay.
  • Workers lose compaction. Turning OpenCode's compaction off is global to the session, and the plugin's own feature matrix gives subagents no historian and no compartments, only rule-based drops and an overflow refusal. A long worker run has less protection than it does today.
  • Its cache deferral was designed for paid prompt caching. Drops wait until the plugin assumes the provider's cached prefix has expired. On llama.cpp the prefix cache is a slot's KV state, which another user's request evicts at any time, so the wait may buy nothing.
  • The package comes from npm. OpenCode fetches @cortexkit/opencode-magic-context itself on first use, which needs the network; the version is pinned in bin/opencode-tellico because OpenCode 2 may remove an unpinned package mid-session when a newer one appears. The installed config turns embeddings off, which avoids a 90 MB model download and a native ONNX runtime; search falls back to keywords. Its state is a SQLite database under ~/.local/share/cortexkit/, outside this setup and outside secret-guard's reach.

Under OpenCode the plugin is named only in the session's runtime config, never in config/opencode.json, so --no-magic-context can leave it out and start a session that does not depend on npm. OpenCode appends a runtime plugin list to the installed file's list rather than merging them, and a plugin named in both loads twice -- measured on 2.0.26 with marker-writing probe plugins -- which is why the launcher names the one package and not the three local plugins as well. Its settings are config/magic-context.jsonc, installed to ~/.config/tellico-qwen/ and linked from ~/.config/cortexkit/magic-context.jsonc, the one user-level path the plugin reads; a file of your own there is left alone and wins. The plugin needs OpenCode 2.0.22 or newer, or OpenCode 1.

The pi harness

pi-tellico runs pi against the same two servers. Install pi first -- curl -fsSL https://pi.dev/install.sh | sh, or npm install -g --ignore-scripts @earendil-works/pi-coding-agent; it needs Node 22.19 or newer -- then:

pi-tellico                       # node derived from this device, as above
pi-tellico 0 --think off
pi-tellico 0 'Review this repository'
pi-tellico 0 --print 'What does install.sh install?'
pi-tellico --continue            # pi's own flags are passed through

Pi is a deliberately small harness: four core tools, no subagents, no plan mode. So there is no orchestrate-or-worker choice here -- one agent, one server, which is the same shape as the default opencode-tellico session. Use it when you want a harness that does less, and opencode-tellico --orchestrate when you want two servers working at once.

What it gets from this setup:

Piece Why
pi/models.json Both nodes as openai-completions providers, 131,072 context and 32,768 output, and the chat_template_kwargs channel that actually reaches the Qwen template. The API key is named as a command, so the file holds no secret.
pi/settings.json Medium thinking, grep/find/ls added to pi's four default tools, and compaction reserve raised so a summary gets the model's whole output allowance. Prompt-cache warming is off: four slots serve every user, so a warming request is a slot taken from someone's real work.
pi/APPEND_SYSTEM.md Appended to pi's own system prompt: the context economy, .agent/PLANS.md as the thing that survives compaction, and the verification and credential rules.
pi/extensions/secret-guard/ Blocks reads of the API key, client.env and SSH private keys. Pi has no permission system -- it "does not ask for approval before every tool call" -- so this is the only thing between the model and the credential.

Thinking works the same way, with pi's own controls: the config defaults to medium, --think sets the session, /thinking changes it inside one, and ctrl+t cycles. Pi has seven levels but the Qwen3.8 template accepts only low, medium and xhigh, so the rest are marked unsupported and never reach it -- asking for high clamps to xhigh rather than being rejected by the template.

Its configuration is kept apart from your own ~/.pi/agent in both directions: a Tellico session is not changed by your personal pi settings, and installing or removing this setup never touches them. The cost is that your own pi extensions, skills and prompts are not loaded; add them to ~/.config/tellico-qwen/pi/, or name them with pi's --extension, --skill and --prompt-template flags, which are passed through. Sessions are kept outside the configuration directory, in ~/.local/share/tellico-qwen/pi-sessions, so uninstalling never deletes a transcript.

The Crush harness

crush-tellico runs Crush against the same two servers. Install Crush first (Homebrew, npm, Arch, Nix and more in its README), then:

crush-tellico                    # node derived from this device, as above
crush-tellico 0 --think off
crush-tellico 0 run 'Review this repository'
crush-tellico 0 --continue       # Crush's own flags are passed through

Crush is single-agent here too: its subagent support is an open pull request, so this is its counterpart of the default build-tellico session. What it adds over OpenCode and pi is a permission prompt in front of every tool call, PreToolUse hooks, and LSP integration, in a single Go binary with no Node dependency.

It is configured differently from the other two, and not by choice. Crush merges ./.crushrc, ./crushrc and $XDG_CONFIG_HOME/crush/crushrc and has no config flag or environment variable; relocating that search with XDG_CONFIG_HOME would also redirect every command the model runs through the bash tool. So this setup integrates instead of isolating: install.sh writes ~/.config/tellico-qwen/crush/tellico.crushrc and appends one marked line to your own ~/.config/crush/crushrc that sources it. ./uninstall.sh takes that line back out, and ./doctor.sh has a crushrc line that tells you if it is missing. Nothing in the fragment touches a setting that is not about Tellico -- no global permission changes, no provider defaults -- because it runs inside a file you own.

Because crushrc is Bash, the launcher passes the node and the thinking level through TELLICO_LEAD_NODE and TELLICO_THINK, which the fragment reads. Crush has no flag that picks a model for an interactive session, so this is the whole mechanism; crush run does accept -m, if you want to override it for one command.

Four providers are registered, which is two more than the other harnesses need:

Provider Why
tellico-0, tellico-1 One per node, carrying the session's thinking level in extra_body as chat_template_kwargs.
tellico-0-quiet, tellico-1-quiet The same servers with thinking off. These hold Crush's small model slot, which it uses for titles and for auto-summarize. A summary that spends its output budget on reasoning comes back truncated -- the failure this setup already paid for once under OpenCode -- so here it simply cannot think.

Two settings are worth knowing about. The models are deliberately not marked --can-reason: Crush's own --reasoning-effort offers high, which the Qwen3.8 template rejects outright, so the level travels in extra_body instead and Crush never adds one of its own. And request-timeout is raised to 3600 from its default of 60 -- Crush aborts on that many seconds of inactivity, and a cold 110k-token prompt on this cluster still waits about 90 seconds before the first token, so a deep session started from scratch would otherwise die. A session the server has already seen comes back in about a second (see below).

The Hermes harness

hermes-tellico runs Hermes against the same two servers. Install Hermes first, then:

hermes-tellico                   # node derived from this device, as above
hermes-tellico 0 --think off
hermes-tellico 0 'Review this repository'
hermes-tellico 0 -z 'What does install.sh install?'
hermes-tellico --continue        # Hermes' own flags are passed through

Hermes has subagents, so unlike pi and Crush this is the orchestrate shape: the session runs on the node you chose and delegate_task sends its children to the other one, marked as workers so the gateway admits a person waiting at a prompt ahead of background work. That is the same arrangement as opencode-tellico --orchestrate, reached through Hermes' own delegation settings rather than through named agents.

Children are capped at two, down from Hermes' default of ten. delegation.provider is a single value, so every child lands on the peer node -- and that node has two slots no matter what a key's per-user allowance says. Queueing past them is not a neutral wait: the gateway rejects a request it cannot admit within 120 seconds, and a worker never takes a slot while a lead is queued, so subagents are the first to starve. Two children plus the lead's own request also lands exactly on the gateway's global MAX_INFLIGHT of 3.

max_concurrent_children is a hard cap on the number of tasks in one delegate_task call, not a queue in front of them. A larger batch is refused with "Too many tasks: N provided, but max_concurrent_children is 2", which names the three ways out, and the model simply splits the work -- measured with five tasks, it re-sent them as 2 + 2 + 1 and all five completed. So nothing is lost, at the cost of a turn. One sharp edge comes with it: when a single turn emits several separate delegate_task calls rather than one batch, the excess calls past the cap are dropped with only a log line (_cap_delegate_task_calls). Raise the cap if you would rather have the parallelism and take the queue; it is one line in hermes/config.yaml.

Two other delegation limits are Hermes' own and this setup leaves them alone. delegation.oneshot_max_children caps a one-shot session -- anything run with -z -- at 2 children in total, whatever the config above says, which is worth knowing before concluding that a cap is not working. And max_spawn_depth is 1, so children cannot spawn children of their own.

It is configured through Hermes' managed scope, which is the one piece of this worth understanding. Hermes keeps everything in a single ~/.hermes/config.yaml -- skills, authentication, the messaging gateway, the kanban board, platform toolsets -- and rewrites that file whenever a setting changes inside a session. Appending to it the way this setup appends to crushrc would therefore be both invasive and unstable. So instead install.sh renders an overlay into ~/.config/tellico-qwen/hermes/nodeN/ and hermes-tellico is the only thing that points HERMES_MANAGED_DIR at it. Hermes deep-merges that file over your own per leaf, leaving sibling keys alone.

Three consequences are worth spelling out:

  • A plain hermes is completely unaffected. Your own model, provider and settings are what it uses, exactly as before this was installed. Nothing in install.sh or uninstall.sh writes inside ~/.hermes at all -- there is a test that asserts it.
  • Your own Hermes keeps working inside a Tellico session. Skills, auth, sessions and the kanban board are the ones you already have, because this runs in your real Hermes home. Only the model settings are replaced.
  • The overlay cannot be changed from inside a session. Managed values are immutable, so /model will not move a hermes-tellico session off the cluster. Start a plain hermes for that.
  • Reinstalling does reach a live session. Hermes folds the managed file's signature into its config cache, so it re-reads the overlay whenever the file changes. ./install.sh --no-start is safe to run while an OpenCode, pi or Crush session is live -- those read their configuration once at startup -- but it would move a running hermes-tellico session onto the new settings mid-run. Stop that one first.

The overlay is rendered once per lead node, into node0/ and node1/, because the two keys that choose a node -- model.provider for the session and delegation.provider for its subagents -- are configuration rather than command-line flags. All four providers are declared in both copies, so switching is still possible; only which one is the default differs.

Provider Why
tellico-0, tellico-1 One per node. transport: chat_completions is declared rather than guessed from the hostname, and the key is a key_cmd, so the file holds no secret and a key the tunnel refetched is picked up without reinstalling.
tellico-0-worker, tellico-1-worker The same two endpoints with X-Tellico-Role: worker, which is what the gateway reads for lead-first admission. delegation.provider points at the peer node's worker entry.

Thinking is the one place where Hermes needs guarding rather than configuring. Its ladder has eight levels -- none, minimal, low, medium, high, xhigh, max, ultra -- and its custom-provider profile forwards whichever it is given as a top-level reasoning_effort without clamping, because a custom endpoint's vocabulary is undiscoverable. The Qwen3.8 chat template accepts exactly four of them and raises on the rest:

Level none minimal low medium high xhigh max ultra
Qwen3.8 template ok 500 ok ok 500 ok 500 500

A rejected level is an HTTP 500 from the chat template, which Hermes retries three times and then reports as an empty answer with no explanation -- measured, not guessed. So the level is pinned in two places: the overlay defaults agent.reasoning_effort to medium, and hermes-tellico --think refuses the four bad levels before any request is made. Use off (spelled none by Hermes, and accepted either way here), low, medium or xhigh.

What a Hermes session costs before you type anything, measured on 2026-10-18 with --usage-file and a one-word prompt:

Configuration Prompt tokens
-t file (the core system prompt, floored) 3,713
-t file,terminal 5,021
10 coding toolsets 10,505
All 17 default CLI toolsets 13,837
The default, in this repository 22,105

That is ~3.7k of system prompt, ~10.1k of tool schemas for 17 toolsets, and ~8.3k of project context -- Hermes loads AGENTS.md and README.md, and this repository's AGENTS.md alone is 50 KB. The 43 KB skills snapshot costs nothing upfront: --skills '' measures identically, because skills are disclosed through the skills tool rather than preloaded.

Against the 131,072-token window that leaves about 76,000 tokens of conversation before the first compaction, because the compaction point is not the 0.50 that compression.threshold defaults to. Models under 512k are floored at 0.75, raise-only (_SMALL_CTX_THRESHOLD_PERCENT in agent/context_compressor.py), since at half a 128k window the incompressible floor eats the headroom and compaction re-fires every turn or two. So it fires near 98k, not 64k. Pass -t with a narrower list for a session you expect to run long; ten coding toolsets save about 3.3k, and vision, image_gen and tts cannot do anything on this text-only model in any case.

Compaction is Hermes' own, not Magic Context, and it works in three layers rather than one. Old tool results are first replaced in place with [Old tool output cleared to save context space], which costs no model call at all; a post-turn micro-compaction pass trims incrementally; and only then does a full summary call happen. The summary targets max(2000, min(content x 0.20, min(context x 0.05, 10000))) tokens, so 2,000 to 6,553 on this window.

Hermes bounds the summary's input -- 160,000 characters of serialized turns, head and tail kept -- and deliberately puts no max_tokens cap on the call ("NEVER add a max_tokens wire cap on the summary call", agent/context_compressor.py). That is the inverse of the arrangement that cost this repository a session under OpenCode, where the summary's output budget was what reasoning ate, so the truncation plugins/compaction-guard/ exists to prevent cannot happen the same way here.

The overlay still pins summaries thinking-off, for a different reason. The compressor passes no reasoning config of its own, and an empty reasoning_effort sends nothing at all, which leaves the server default -- and the Qwen3.8 template's default is thinking on. Free prose is this model's worst shape, acceptance ~0.26 and near 80 tok/s, so an unguarded compaction reasons its way through a summary at the slowest rate the cluster has, at the least convenient moment. auxiliary.compression therefore names a provider, a model and reasoning_effort: none: all three are needed, because Hermes only certifies the non-reasoning path when the configured route matches the one the call resolves to. session_search gets the same treatment; title_generation already runs thinking-off unasked.

Because auxiliary.compression.provider is the session's own node, a summary is a second request on that node and takes one of its two slots. It happens between turns, so it does not compete with the session's own generation.

./doctor.sh checks both overlays, because Hermes is fail-open about a managed file it cannot parse: it logs one warning and carries on with your own provider, which looks like a working session answering from the wrong place. It also warns if your own ~/.hermes/config.yaml has a Tellico provider whose base URL is not the /v1 API path -- a hand-configured /tellico/ points at the relay's web page, which answers HTML and so fails with a bare HTTP 404: Not found.

Tunnel and cluster checks:

tellico-qwen-tunnel status
tellico-qwen-tunnel doctor
tellico-qwen-tunnel restart
tellico-qwen-tunnel cluster-status
tellico-qwen-tunnel stop

The commands install to ~/.local/bin. If that is not on PATH, the installer says so and prints the exact line and file for your login shell. To let it make the change:

./install.sh --fix-path

That appends the line to ~/.zshrc, ~/.bashrc (~/.bash_profile on macOS), ~/.config/fish/config.fish, or ~/.profile, depending on $SHELL, and does nothing if the profile already sets it. Open a new terminal afterwards.

VS Code

tellico-vscode is a chat panel for VS Code over this same setup: the same agents, workers, plugins and Magic Context, with streaming output, a workers tray, native diffs of what changed, and session history shared with the terminal. Download the .vsix from its releases and install it with:

code --install-extension tellico-vscode-<version>.vsix

It needs OpenCode 2 and this client installed first, in either mode -- a gateway user needs only ./install.sh --gateway and a key. The extension runs no agent logic of its own. It starts opencode-tellico serve for the open folder, which is OpenCode's headless API server with exactly the runtime config the TUI gets, and drives that. You can run it yourself:

OPENCODE_PASSWORD=choose-one opencode-tellico serve --port 4123

An install made before serve existed makes the extension say so; rerun ./install.sh --no-start to update the launcher.

The extension can use either connection, whichever way you installed: a plug icon in its chat box switches between the SSH tunnel and the API gateway, and asks once for a gateway key. From a terminal the same thing is --connection:

opencode-tellico --connection gateway      # this launch through the gateway
opencode-tellico --connection tunnel       # this launch over SSH

Each connection keeps its own key: the one you installed with stays in ~/.config/tellico-qwen/api-key, the other goes in api-key.gateway or api-key.tunnel (mode 600), so switching never overwrites anything. A tunnel connection fetches its key over SSH as usual. pi and Crush use only the installed connection.

t3code

t3code drives OpenCode too, and install.sh sets it up whenever t3code has been opened on the device (it has a ~/.t3/userdata directory, or $T3CODE_HOME/userdata). It adds a Tellico provider instance to t3code's settings.json, beside its own OpenCode one, which starts opencode-tellico serve with TELLICO_T3CODE=1. Pick a tellico-0 or tellico-1 model under it. t3code reloads the file itself, so a running t3code shows the provider without a restart.

t3code has no agent picker for OpenCode 2: every turn it switches the session to the agent named build, or plan in its plan mode, and ignores default_agent. So TELLICO_T3CODE=1 redefines OpenCode's own build as build-tellico on the lead node -- the same prompt, model, sampling and step limit, with no delegation. Its explicit allow permissions are left out so that t3code's approval modes still apply. Plan mode is OpenCode's stock plan on the same model, and orchestration is not reachable from t3code.

A second instance, Tellico Frontier (TELLICO_T3CODE=frontier), makes build the frontier lead instead: pick a frontier model in t3code's model picker -- openai/gpt-6.1-terra or -sol after signing in to OpenAI with opencode auth login (a ChatGPT plan works) -- and it plans and verifies while the Qwen workers do the edits; t3code shows each worker as a child thread. Its reasoning effort is TELLICO_FRONTIER_EFFORT (high) unless you pick a level in t3code. The picker opens on TELLICO_FRONTIER_MODEL (default Terra) when this device has a credential for it, otherwise on the local model; picking a Qwen model there gives a Qwen lead that cannot edit. t3code's full-access and auto-accept modes write session rules that would let the lead edit anyway, so plugins/frontier-guard blocks the lead's write tools at execution time; the workers are unaffected. Plan mode is still OpenCode's plan.

Settings given to the instance in t3code -- a name, colour, extra environment variables such as TELLICO_THINK=low, or turning it off -- survive a rerun. ./install.sh --no-t3code takes the instance out, and uninstall.sh does too while it still points at this setup's launcher. A device installed before t3code was opened gets it on the next ./install.sh --no-start.

Images

The Tellico models see images. Since 2026-11-03 both servers load the Qwen3.8-27B vision projector (mmproj-Qwen3.8-27B-Q8_0.gguf), and every model in config/opencode.json and pi/models.json takes image input, so when an agent reads a screenshot the picture goes into the request. A 1920x1080 screenshot is about 2,000 tokens of context and answers in about 3 s through the gateway; speculative decoding keeps working. A conversation with an image in it takes no context checkpoint right after the image, so branching from it re-prefills a little more.

tellico-look shot.png "question" asks the same model about one image through the gateway, for scripts and tools that are not an agent. (It used to ask a separate CPU model, Qwen3-VL-8B, on the gateway host; that was retired on 2026-11-14 and its name now routes to the cluster model. See "Vision" in gateway/README.md.)

Phone

Sign in at https://<gateway host>/tellico/ with your API key to see every session you have running — laptop, VS Code, terminal, any machine — streaming live, and send one a prompt or stop it. Turn it on per computer:

./install.sh --no-start --remote-control

Each OpenCode process then publishes its sessions to the relay through plugins/remote-control, and takes prompts and stops back. It only ever makes outbound requests, so no port on your machine is opened. Off by default; see relay/README.md for how it works and what a phone can and cannot do.

An always-on console on the gateway host

Separately, remote/ puts the same setup behind a browser, so you can read and steer sessions from a phone. It is one supervised opencode-tellico serve on the always-on host, published by the same reverse proxy that already fronts the gateway, and the client is OpenCode's own web app -- a PWA you add to the home screen.

remote/install-remote.sh      # on the gateway host
tellico-remote caddy          # the proxy snippet, hash filled in
tellico-remote pair           # a one-time link; open it on the phone

It runs on the always-on host rather than your laptop because sessions belong to the process running them: another server reading the same storage sees the history, and even the answer text as it is flushed, but gets no events for it and reports it inactive. So a phone can only watch sessions started by the server it is attached to, and a laptop that sleeps takes the endpoint with it. Its sessions therefore live in that host's workspace, which suits asking, planning and reviewing rather than editing files that exist only on your laptop. To drive the same sessions from both, point the laptop's client at it with opencode --server https://your.host.example -- tools then run there, not locally.

Remote sessions go through the gateway on loopback rather than straight down the tunnel, so they queue, cap and log like every other user's. The endpoint can run commands on its host, so it sits behind two locks on one secret: the proxy checks HTTP basic auth at the edge and OpenCode checks it again behind, and a single Authorization header satisfies both. See remote/README.md.

What gets installed

Client side, on each user's own device:

~/.config/tellico-qwen/client.env
~/.config/tellico-qwen/lib/checks.sh
~/.config/tellico-qwen/opencode.json
~/.config/tellico-qwen/prompts/orchestrate.md
~/.config/tellico-qwen/prompts/worker.md
~/.config/tellico-qwen/prompts/build.md
~/.config/tellico-qwen/plugins/
~/.config/tellico-qwen/pi/
~/.config/tellico-qwen/crush/
~/.config/tellico-qwen/hermes/node0/config.yaml
~/.config/tellico-qwen/hermes/node1/config.yaml
~/.config/tellico-qwen/magic-context.jsonc
~/.config/tellico-qwen/api-key
~/.local/bin/opencode-tellico
~/.local/bin/pi-tellico
~/.local/bin/crush-tellico
~/.local/bin/hermes-tellico
~/.local/bin/tellico-qwen-tunnel
~/.local/bin/tellico-look

Plus one marked line in ~/.config/crush/crushrc and a symlink at ~/.config/cortexkit/magic-context.jsonc, the only things this setup writes outside its own directory; ./uninstall.sh removes both again, the symlink only if it is still this setup's.

pi/ is the whole configuration for the pi harness -- models.json, settings.json, APPEND_SYSTEM.md and extensions/ -- and PI_CODING_AGENT_DIR points pi at it. Pi sessions go to ~/.local/share/tellico-qwen/pi-sessions instead, so uninstalling does not delete them.

hermes/node0/ and hermes/node1/ are the two managed-scope overlays, one per lead node, and hermes-tellico points HERMES_MANAGED_DIR at whichever one it was asked for. Unlike pi/ this is not a whole harness configuration: a Hermes session runs in your own ~/.hermes, and the overlay replaces only the model settings on top of it. Nothing here is written inside ~/.hermes, so Hermes sessions, skills and auth are untouched by installing or removing this setup.

On systems with a working systemd user manager, the installer also enables:

~/.config/systemd/user/tellico-qwen-tunnel.service

On macOS and other non-systemd systems, the tunnel helper uses a controlled background SSH master instead.

On a host that also runs the API gateway, gateway/install-gateway.sh adds:

~/.config/tellico-gateway/gateway.env
~/.config/tellico-gateway/keys
~/.config/systemd/user/tellico-gateway.service
~/.local/lib/tellico-gateway/tellico_gateway.py
~/.local/lib/tellico-gateway/tellico_monitor.py
~/.local/bin/tellico-gateway

And remote/install-remote.sh adds the phone-facing server:

~/.config/tellico-remote/remote.env
~/.config/tellico-remote/password
~/.config/tellico-remote/password.bcrypt
~/.config/systemd/user/tellico-remote.service
~/.local/bin/tellico-remote

remote.env and both password files are mode 600, and the password reaches the service by reference, never inside the unit file.

tellico-gateway monitor on that host shows who is using the models, how much of the cluster is busy, and whether anything is queueing -- including sessions that come down an SSH tunnel instead of through the gateway. See gateway/README.md.

OpenCode loads opencode.json through its supported OPENCODE_CONFIG merge layer, so the device's normal providers and settings remain available. Runtime overrides pin title, summary, compaction, lead, and worker calls to Tellico for the launched session.

OpenCode v2 serves plain opencode invocations from a shared background service that was started without OPENCODE_CONFIG, which makes it ignore that variable and report Model unavailable: tellico-0/qwen3.8-27b-gsq-iq3s. So opencode-tellico passes --standalone, giving the session its own server that does read the config. The flag is used only when the installed OpenCode advertises it, so older versions without a background service are unaffected. Force the choice with TELLICO_OPENCODE_STANDALONE=1 or =0.

Allocation lifecycle

The two model servers exist only while the Tellico Slurm allocation is active. Check it with:

tellico-qwen-tunnel cluster-status

That check needs no privileged access: it reads the queue with squeue and probes each server's authenticated /v1/models, both of which any cluster account may do.

In gateway mode there is no cluster account to run squeue with, so tellico-qwen-tunnel status reports what the gateway says instead: models no_allocation means the gateway is healthy but has nothing behind it.

When an allocation has expired, the owner of the service account submits another one:

ssh tellico qwen38-submit
ssh tellico 'qwen38-status --wait'

Everyone else then picks the new servers up with:

tellico-qwen-tunnel restart

Gateway-mode devices need no such step: the gateway notices the new servers within ten seconds by itself.

Using a custom SSH alias requires replacing tellico in the submit commands or setting TELLICO_SSH_HOST for that shell.

Shared access

The model API key lives in the lab's shared space rather than in the service owner's home directory, which is not traversable by other accounts:

/data/gclab/qwen38/secrets/api-key    mode 0640, group gclab

Any gclab member can therefore install this repository under their own cluster account with no extra flags. Someone outside that group needs a copy they can read, passed with --remote-key-path.

These names are defaults, not assumptions. Override them per shell:

TELLICO_REMOTE_KEY_PATH   Path to the API key on the cluster
TELLICO_SERVICE_USER      Account that owns the allocation (default: bbogale)
TELLICO_JOB_NAME          Slurm job name (default: qwen38-api)
TELLICO_COMPUTE_NODES     Space-separated server hostnames
TELLICO_MODEL_PORT        Port the servers listen on (default: 8000)

Update

git pull --ff-only
./install.sh

Uninstall

./uninstall.sh

The uninstaller removes only files installed by this repository, including the gateway if it is present. It does not remove OpenCode, SSH configuration, any cluster files, a PATH line added by --fix-path (other tools in ~/.local/bin may depend on it), or the gateway's keys file, which holds other people's credentials.

Security model

  • Both model forwards bind only to 127.0.0.1.
  • Gateway mode never holds the shared cluster key, never opens an SSH connection, and needs no cluster account; it holds one per-user API key, which the operator can revoke on its own.
  • A gateway-mode device sends its key to whatever --gateway-url names, so that URL should be https://. The installer warns when it is not.
  • The model API key is fetched over SSH and stored with mode 0600.
  • No API key is written into OpenCode JSON or the systemd unit.
  • The repository contains no private credentials and can safely be cloned.
  • Access still requires both Tellico SSH authorization and the cluster API key.
  • The API key is shared by every user of the OpenCode client, so it identifies the service rather than the caller. llama-server --api-key-file accepts one key per line, so moving to per-user keys is the way to get revocation and attribution. Users who come in through gateway/ already get this: each holds only their own key, and the shared cluster key never leaves the gateway host.

Development checks

Run sh tests/run.sh with Node 22+ and Python 3 available. It checks POSIX shell syntax, JSON, launcher overlays/fallback modes and both dispatch-plugin API adapters with local mocks; no cluster, credentials, tunnel or live gateway is needed. GitHub Actions runs the same checks. These tests do not validate live OpenCode schema loading, background notifications or Tellico performance.

After pulling changes to config, prompts, plugins or launchers, run ./install.sh --no-start, restart the OpenCode session, and run ./doctor.sh to check that installed copies match. This updates client files without restarting a live tunnel.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages