the complete
spectroscope user guide
One headless core, many faces. Every feature of the full build, from the terminal, the web UI, the desktop shell, the Lab, subagents, skills, MCP, voice and vision to the fleet on the bus, explained with real screenshots, and every protocol, event and endpoint documented down to the wire.
About this guideHow to read this book
Every screenshot is a real capture of the running app, taken by the
repository's own capture pipeline (capture_screens.mjs) and
reshot per release. The two editions of this guide carry matching sets:
the dark pages show spectro dark, the light pages
spectro bright.
spectroscope is the reference harness of the workshop “build an agent harness
from scratch” — and the full build in the spectroscope/ folder is its final form:
everything the ten core stages and six bonus stages construct, plus the conveniences
that never fit a single stage. This guide documents that full build. Not the journey —
the machine.
The book has five parts, ordered from “I just want to use it” to “I want to know exactly what crosses which wire”:
- Part I — Meet spectroscope: what it is, how to start it, what it can do.
- Part II — The faces: the terminal, the web UI with all its tabs and panels, the Lab, session management, the design system, the desktop app.
- Part III — The agent's powers: tools, permissions, hooks, subagents, skills, providers, vision, voice, image generation, MCP and scheduling.
- Part IV — The file system: where everything lives on disk and what each file looks like inside.
- Part V — Under the hood: the complete technical reference — the architecture dossier, every event, every socket frame, every REST endpoint, every configuration key.
Every screenshot in this guide is a real capture of the running application
(English chrome, taken 2026-07-20 with the reproducible script in
docs/guide-assets/capture_screens.mjs). Every terminal block is real
command output. Every name, constant, wire string and default was read from the
migration phase-6 source tree (2026-07-20) — not from memory. Where behaviour is a
deliberate limitation, the guide says so.
Conventions
monospacemarks anything that exists literally in code, config or on the wire: tool names, event types, JSON keys, commands.- Chips classify tools: free needs no permission, gated asks first, gated + prefix rule asks first and remembers approvals scoped to a prefix.
- German UI labels appear in parentheses where the chrome ships bilingual — the screenshots use the English chrome.
- File paths are relative to the
spectroscope/root unless they start with~.
ContentsThe map of this book
Meet spectroscope
What the harness is, the one idea everything hangs on, how to start it in thirty seconds, and a map of everything it can do.
Chapter 1What is spectroscope?
spectroscope is an agent harness: the machinery around a large language model that turns “a model that can talk” into “an agent that can work” — read files, run commands, ask for permission, delegate to subagents, remember sessions, and show you every step it takes while doing so.
It was built from scratch across a sixteen-stage workshop (Java 21, Gradle,
Spring only where it earns its place), and the spectroscope/ full build is the
sum of all of it. Three properties define the design:
One core, five faces
The heart of spectroscope is spectro-core — a headless Java library with no
container, no UI and no I/O of its own. Everything in it is constructible with plain
new. When the agent runs, the core produces exactly one thing: a typed
stream of RunEvents. Everything else in the project is a renderer
of that stream:
- the CLI (
spectro-cli) renders it as ANSI text with tool cards and a spinner, - the web backend (
spectro-server) pushes it over a WebSocket, - the browser UI (
spectro-web) folds it into chat, graph, trace, Lab and panels, - the desktop shell (
spectro-desktop) wraps that same web UI in a supervised window, - the session store writes it line by line into a JSONL file.
That last renderer is the trick: the same event stream is simultaneously the UI protocol, the storage format and the data source for every visualisation. Replaying an old session is not a separate code path — it is the same array of events, read from a file instead of a socket.
Additive forever
The JSONL wire format is shared byte-for-byte with the sibling TypeScript edition of the workshop. That makes the format a contract: new event types and new optional fields may be added, but nothing is ever renamed, removed or reinterpreted, and every consumer silently skips what it does not know. Sessions recorded in 2026 replay forever; sessions recorded by the other edition replay here (proven by a byte-for-byte round-trip test).
Tool inputs are model output
The course maxim that shapes every security decision in this book: whatever a tool receives as input was written by the model, and the model writes what it wants. So every file path passes a sandbox, every mutation passes a permission gate, network egress asks first, hooks can veto calls before the gate, and even the “always allow” button remembers its approval scoped to a prefix instead of a blanket. You will meet this maxim in almost every chapter.
The stack, in one breath
Iterable<RunEvent>, cancellation is a cooperative CancelSignal.LlmProvider port; behind it Anthropic (official SDK, SSE), Ollama (NDJSON) and any OpenAI-compatible server (SSE) — switchable mid-session.spectro-server; picocli in the CLI; React 19 + React Flow in the browser../gradlew build needs no API key and no network model.Chapter 2Quick start
One executable starts everything. The ./spectro-app launcher
resolves a JDK, loads your .env, and dispatches to the face you ask for —
you never have to touch the Gradle task zoo.
./spectro-app is the developer wrapper in a clone of the
repository; spectro is the shipped CLI. The command you install
from a release is called spectro and this guide calls it that everywhere
else. The wrapper in the repository root carries a different name so that it reads as
one of a set with its two siblings — ./spectro-serve for the server's
lifecycle and ./spectro-env for the local docker stacks. Same product,
two entry points, and the names say which one you are holding.
The launcher
◆ spectroscope
repl interactive agent REPL — /voice push-to-talk, --speak reads aloud
./gradlew -q --console=plain :spectro-cli:run --args="..."
run headless run: ./spectro-app run -p "task" [--json] [--image path] [--speak]
./gradlew -q --console=plain :spectro-cli:run --args="run -p ..."
node fleet node: ./spectro-app node -p "task" --hub host:port --context <fleet>
./gradlew -q --console=plain :spectro-cli:run --args="node -p ..."
cron scheduler: list / status / run <id> (~/.spectro/jobs.json)
./gradlew -q --console=plain :spectro-cli:run --args="cron list"
sessions list the stored JSONL sessions (~/.spectro/sessions)
./gradlew -q --console=plain :spectro-cli:run --args="sessions"
resume continue a session by id: ./spectro-app resume <id>
./gradlew -q --console=plain :spectro-cli:run --args="--resume <id>"
doctor check Java, config layers, provider reachability
./gradlew -q --console=plain :spectro-cli:run --args="doctor"
level the tutorial with its ticks and receipts (no server needed)
./gradlew -q --console=plain :spectro-cli:run --args="level"
tour guided feature tour — menu, tips, settings
./gradlew -q --console=plain tour
web web face: a command group — start, dev, stop, status, logs (./spectro-app web)
./spectro-app web · ./spectro-app web start --port 8090
desktop desktop face — Electron shell + managed Spring Boot process
./gradlew :spectro-server:bootJar && (cd spectro-desktop && npm start)
mcp-notes build the example MCP server (stdio) into a runnable dist
./gradlew -q --console=plain :spectro-mcp-notes:installDist
Extra arguments pass through: ./spectro-app repl --verbose · ./spectro-app run -p 'Say OK.' --json
Vision needs a vision model (attach via --image or the web composer); voice: bash scripts/setup-stt.sh (in) and scripts/setup-tts.sh (out).
MCP: configure external servers in .spectro/settings.json ('mcpServers'); ./spectro-app mcp-notes builds the example; in the REPL, /mcp lists them.Before dispatching, the launcher fixes the two classic host problems for you:
- JDK auto-resolution. The modules compile with
--release 21, but many machines default to an olderjava. IfJAVA_HOMEis not already a JDK 21+, the launcher probes/usr/libexec/java_home, the Homebrewopenjdk@21keg (preferred: a stable versioned keg survives brew upgrades), newer kegs up to 25, and the Cellar globs — and exports the first hit. Verified up to JDK 26; a miss prints a note and continues. .envloading for every face. The gitignoredspectroscope/.envholds API keys and provider switches. The launcher parses it (comments skipped, one quote layer stripped, empty values ignored soKEY=can never blank a shell export) and exports the pairs — deliberately including thedesktopface, which spawns the server through Electron and would otherwise bypass Gradle's own injection.
The thirty-second start
cd spectroscope # 1 — pick a model. Local (no key at all): # .env: SPECTRO_PROVIDER=ollama · SPECTRO_MODEL=qwen3 · SPECTRO_BASE_URL=http://localhost:11434 # cloud: ANTHROPIC_API_KEY=sk-ant-… (model defaults to claude-opus-4-8) # 2 — check the environment ./spectro-app doctor # 3 — go ./spectro-app web start # browser UI on http://localhost:8080, in the background ./spectro-app web # what the group can do, and whether it is running ./spectro-app repl # terminal REPL ./spectro-app run -p "Say OK." # headless one-shot ./spectro-app desktop # the Electron app
What doctor tells you
./spectro-app doctor is the honest pre-flight: it probes what your configuration
actually selects — it loads the config hierarchy with your flags, pings the provider you
chose, connects to every configured MCP server, and write-tests the sessions directory.
This is real output from the machine this guide was built on:
◆ spectro doctor
✓ Java 25.0.2
✓ config: provider=ollama model=qwen3.5:27b permissionMode=ask autoApprove=2 rule(s)
layers: user config absent · project settings present
✓ ollama at http://localhost:11434 (version 0.24.0)
images: gemini — GEMINI_API_KEY not set; generate_image will return a readable error
✓ skills: 4 installed (brainstorming, test-driven-development, verification, writing-plans)
✓ hooks: 0 configured
✓ mcp: notes reachable at /Users/you/spectroscope/spectro/spectro-mcp-notes/build/install/spectro-mcp-notes/bin/spectro-mcp-notes (2 tools)
vision: local provider — attach images only with a vision model (e.g. ollama pull qwen3-vl); a text-only model fails fast
✓ voice input: whisper-cli + ggml-small.bin ready (/voice)
voice output: piper missing · voice missing — run bash scripts/setup-tts.sh to enable --speak
✓ sessions dir writable: /Users/you/.spectro/sessions (7 session(s))
✓ jobs.json: 0 job(s)
Everything looks good.Green checks fail the exit code when broken; the indented dim lines are information, not failures — a missing image-generation key, for example, is deliberately not unhealthy (the tool will return a readable error instead). Chapter 4 documents all twelve checks.
The web face is a command group
Typing ./spectro-app web does not start anything. It shows what the
group can do and whether a server is already up, because a face that fires a
process the moment you name it is a face you cannot ask a question of.
◆ spectro-app web
○ stopped
port 8199 health --
start background: the packaged jar, health-waited, browser opened
--port N · --env-file F · --no-open
dev foreground: gradle bootRun from source, ctrl-c stops it
--port N · this is what a bare ./spectro-app web used to do
stop stop it, and free the port if something still holds it
restart stop, then start again on the same port
--port N
status the block above, on demand
logs the tail of logs/spectro-server.log
-f follows
open open the url in the browser
doctor what the SERVER needs to boot: java, the jar, a backend
./spectro-app doctor checks the agent instead
A bare './spectro-app web' no longer starts anything: 'start' backgrounds it, 'dev' holds the terminal.
Every verb but 'dev' runs ./spectro-serve, which owns the pid file, the port file and the log.Two of those verbs start a server and they are not the same thing.
start runs the packaged jar in the background, waits for health and
opens the browser, which is what you want when you are using spectroscope.
dev runs Gradle's bootRun in the foreground from
source, which is what you want when you are changing it: the terminal stays
attached and ctrl-c stops it. Everything else, stop,
restart, status, logs, open
and doctor, reaches ./spectro-serve. That script owns
the pid file, the port file and the log, and it stays the only implementation of
the lifecycle so the two can never disagree about what is running.
Before this, a bare ./spectro-app web started the server in the
foreground. That behaviour is now ./spectro-app web dev. If you have
the old form in a script, it fails with one line naming the fix rather than
printing help and exiting cleanly, because a script that believes it started a
server and did not is worse than a script that stops.
Note that ./spectro-app doctor and ./spectro-app web doctor
answer different questions. The first checks the agent: Java, the config layers,
whether your provider replies, whether the sessions directory is writable. The
second checks only what the server needs in order to boot at all.
The provider matrix
| provider | transport | needs | default model |
|---|---|---|---|
anthropic | official SDK, SSE streaming | ANTHROPIC_API_KEY | claude-opus-4-8 |
ollama | Spring RestClient, NDJSON | local Ollama on :11434 | qwen3 |
openai | Spring RestClient, SSE | OpenAI itself, or any compatible server you point baseUrl at; key optional | local-model |
lmstudio | Spring RestClient, SSE | LM Studio's server on :1234, or another compatible one | local-model |
llamacpp | Spring RestClient, SSE | your own llama-server on :8080 | local-model |
More ids exist than these five. Chapter 12 compares every backend and chapter 18 lists them exhaustively; this table is the quickstart set, and it is deliberately not a second copy of that list.
Selection rides the settings hierarchy (chapter 30 has every key): defaults
< environment (SPECTRO_PROVIDER / SPECTRO_MODEL /
SPECTRO_BASE_URL, usually from ./.env — the BASE layer)
< ~/.spectro/settings.json (user) < launch-dir
.spectro/settings.json (deprecated) < workspace
.spectro/settings.json + settings.local.json < CLI flags —
any settings file outranks the env. “Local Ollama for this checkout” is three
uncommented lines in .env as long as no settings file overrides them — and
in the web UI you can switch provider and model mid-session from the header.
Chapter 3The capability map
Everything the full build can do, on two pages. Each row names the chapter where the details live.
| Capability | In one sentence | Ch. |
|---|---|---|
| Terminal REPL | Interactive agent with tool cards, spinner, slash commands, voice input and spoken answers. | 4 |
| Headless runs | spectro run -p "…" — one prompt, exit codes for CI, NDJSON event output with --json. | 4 |
| Cron scheduling | Jobs in ~/.spectro/jobs.json run unattended with desktop notifications and audit sessions. | 19 |
| Web chat | Streaming chat with markdown answers, collapsible reasoning, tool cards with gate chips, image attachments, push-to-talk. | 5 |
| Graph view | Every run as a BPMN-style flow overview or a dagre DAG — live and for any archived session. | 6 |
| Trace view | The Wireshark of the harness: every wire frame both directions, filters, Δt, LLM-direction column, three detail modes. | 6 |
| Right panel | Agents roster, the agent's live plan, the exact system context per agent, and a sandboxed file browser. | 6 |
| The Lab | A step-through debugger for agent runs: dam up the event stream, advance it click by click across three synchronized views. | 7 |
| Scenarios | Seven deterministic, compiled demo runs — no LLM, no key — for teaching every mechanism. | 7 |
| Sessions | Every run is a JSONL file: list, replay, resume live (with the context re-upload made visible), delete with a two-step guard. | 8 |
| Import | Replay foreign files: raw spectroscope JSONL verbatim, Claude Code transcripts through an adapter. | 8 |
| Design system | Three brand designs — spectro dark, spectro bright, spectro white — switchable in place, particle/scroll effects, one-step persistence. | 9 |
| Desktop app | Electron shell that spawns and supervises the server JVM: health checks, tray, native cron notifications. | 10 |
| DE/EN chrome | The whole UI chrome switches between German and English live; content keeps its language. | 11 |
| Tool belt | 22 built-in tools — files, shell, grep/glob, web fetch + search, page browse, image + document view, plan, skills, spawns — plus every connected MCP tool. | 12 |
| Permission system | ask/auto/readonly modes, a blocking broker, prefix-scoped allowlist, “always allow” with optional persistence. | 13 |
| Hooks | Config-driven shell guards around every tool call: pre_tool_use can block before the gate, post_tool_use observes. | 13 |
| Subagents | Explore/worker children with their own context, visible A2A task/status/result protocol, four development role tools. | 14 |
| Skills | SKILL.md packages with progressive disclosure — catalog in the prompt, body on demand. | 15 |
| Provider switch | Change LLM backend and model mid-session from the header; history intact; refused cleanly without a key. | 16 |
| Resilience | Transient-error retry with backoff (spectroscope owns it, SDK retry off) and Anthropic prompt caching with an honest compaction trigger. | 16 |
| Vision | Attach images in the composer or --image; blobs stay next to the session file, prompts stay small. | 17 |
| Voice in/out | Local whisper.cpp push-to-talk (editable transcript) and piper speech output that reads answers while they stream. | 17 |
| Image generation | generate_image behind its own provider port (Gemini / OpenAI), content-addressed store, gallery panel. | 17 |
| MCP client | External tool servers over stdio or HTTP/SSE, Claude-Desktop-shaped config, at-most-once calls, example notes server included. | 18 |
| Context management | Automatic compaction at 100k tokens, forced /compact, live context ring with per-part introspection. | 28 |
| Wire tracing | --verbose mirrors the full agent↔provider protocol to stderr; the Trace tab does it in the browser. | 4, 6 |
The faces
Five renderers of one event stream: the terminal, the web UI with its tabs and panels, the step-through Lab, the session archive, the design system, and the desktop shell.
Chapter 4The command line
The CLI is the first face the workshop builds and the most direct one: a picocli application whose root command is an interactive REPL, with subcommands for headless runs, scheduling and diagnostics. Everything it prints is a rendering of the same RunEvent stream that the web UI folds and the session store writes.
Commands and global flags
| invocation | what it does |
|---|---|
spectroscope | the interactive REPL (root command, no subcommand) |
spectro run -p "…" | headless single prompt (see below) |
spectroscope cron [list|status|--once id] | the scheduler — chapter 19 |
spectro sessions | list stored sessions (a positional, not a subcommand) |
spectroscope --resume <id> | continue a stored session in the REPL |
spectro doctor | the environment check |
./spectro-app tour | the interactive menu tour of all faces |
Global flags on the root command apply to every entry point (subcommands reach them
via picocli's @ParentCommand — so flags > env holds on
run and doctor too):
| flag | meaning |
|---|---|
--provider / --model / --base-url | override the configured backend for this invocation |
--compaction-threshold <n> | compaction trigger in input tokens (default 100 000) |
--resume <id> | rebuild the conversation from a session file and continue it |
--verbose | mirror the agent↔provider wire protocol to stderr in cyan |
--speak | read answers aloud while they stream (voice output, chapter 17) |
The REPL
Startup prints a banner naming the provider and model (for Ollama it live-probes the
server version), the session id and file path, and the essential key bindings. The
prompt is a coral ❯. Each turn drives three consumers in order — the JSONL
store, the speech renderer, the ANSI renderer — the CLI-side proof of “one stream, many
renderers”.
◆ spectroscope full build ollama · gpt-oss:20b · ollama 0.24.0 · images: gemini · skills: 4 · allowlist active session 20260716-091500-3fa4b2c1 · ~/.spectro/sessions/20260716-091500-3fa4b2c1.jsonl /help for commands · /mcp servers · /voice push-to-talk · /speak on|off reads answers aloud · Ctrl+C aborts a run
Rendering rules worth knowing (all in EventRenderer):
- Thinking is dimmed. A reasoning model's
thinking_deltastream prints dim under a· thinkinglabel, visually apart from the answer. - Tool cards. Each call prints
⚒ name(coral hammer) with a 100-char input preview; the result line shows✓/✗ <ms> · output preview. - Permissions inline. A gated call prints a sand
run <tool>? [y/N]prompt; onlyyapproves. Allowlisted calls print✓ auto-approved by allowlistinstead — the decision is still recorded in the stream. - Subagents in brackets. Child events render as
[worker-1] …lines with line-buffered text so parallel children stay readable; A2A task/status/result messages appear as one dim line each. - The plan pretty-print. An
update_plancall renders◇ plan (N steps)with[x]done, coral[~]running,[ ]open. - Degradation. Colors and the braille spinner appear only on a real TTY
(and never with
NO_COLORorTERM=dumb); piped output is plain text.
Slash commands
Handled entirely in the CLI — none of these ever reaches the model:
| command | behaviour |
|---|---|
/help | lists all twelve commands with one-liners |
/cost | cumulative session tokens: Session usage: X in / Y out |
/model | current provider · model (+ base URL for local providers) |
/sessions | the stored-session listing, same as spectro sessions |
/skills | installed skills with descriptions (chapter 15) |
/mcp | connected MCP servers, reachability, and every mcp__… tool (chapter 18) |
/think on|off | toggles reasoning visibility; rebuilds the agent and reloads the session history so nothing is lost |
/voice | push-to-talk: record → transcribe → editable confirmation (chapter 17) |
/speak on|off | voice output at runtime; off also stops the current sentence |
/compact | force a context compaction now (Agent.compactNow()); prints the result or “Nothing to compact” |
/clear | fresh agent, fresh session file: New session: <id> |
/exit | leave (empty line or Ctrl-D work too) |
Headless runs: spectro run
The automation face. One prompt, one process, honest exit codes:
# plain: prints only the final answer text
./spectro-app run -p "Count the Java files under spectro-core."
# NDJSON: every RunEvent on stdout, one per line — byte-identical to the JSONL format
./spectro-app run -p "Say OK." --json | jq -c '{type}'
# guard rails
./spectro-app run -p "…" --max-turns 5 --permissions readonly
| flag | meaning |
|---|---|
-p, --prompt | required task text |
--json | NDJSON events on stdout; ALL diagnostics go to stderr, so | jq pipelines stay clean |
--max-turns <n> | cancel when a turn exceeds the limit (stop reason max_turns) |
--permissions readonly|auto | the headless policy. Default readonly — “capability before convenience”: unattended runs deny mutations unless you opt in |
--image <path> | attach an image (repeatable; jpg/jpeg/png/webp/gif) — vision, chapter 17 |
--mcp / --no-mcp | mount the configured MCP servers for this run, overriding the headlessMcp setting (chapter 30) — or decline them. A permission, not a convenience: with --permissions auto this approves every tool every configured server offers, unwatched. Absent, the setting decides; the run prints what it mounted on stderr |
--verbose / --speak | as on the REPL |
Exit code contract: 0 only when the run ended with a regular
end_turn. Anything else (aborted, max turns, error) exits 1 and names the
stop reason and the session id on stderr — the session file is always there for
post-mortem inspection.
The twelve doctor checks
| # | check | what it verifies |
|---|---|---|
| 1 | Java runtime | runtime major version ≥ 21 |
| 2 | Config hierarchy | the layered config loads; shows provider/model/permissionMode/allowlist count and which layers exist. A broken config stops doctor immediately — nothing below would make sense |
| 3 | Provider reachability | anthropic: key present · ollama: live version probe · llamacpp: GET /props, which answers only once a model is loaded · every other OpenAI-compatible id: GET /v1/models (2 s/3 s timeouts) |
| 4 | Image provider | key for gemini/openai; a missing key is informational, not unhealthy |
| 5 | Skills | count + names of installed skills |
| 6 | Hooks | count + event:matcher pairs |
| 7 | MCP servers | connects each configured server: reachable + tool count, or a red UNREACHABLE line (never a crash) |
| 8 | Vision | informational hint matching your provider (vision model needed locally) |
| 9 | Voice input | whisper-cli on PATH + the ggml model file present |
| 10 | Voice output | piper binary + voice model present |
| 11 | Sessions dir | creates the dir, writes and deletes a probe file, counts sessions |
| 12 | jobs.json | the cron file parses (loudly invalid otherwise) |
Wire tracing with --verbose
A decorator around the provider mirrors the whole protocol to stderr
in cyan: the full outgoing request before each call (message count, tool names, clipped
system prompt, every content block) and each incoming provider event as it streams
(⇠ text_delta "…", ⇠ tool_call name {…},
⇠ usage 123 in / 45 out, ⇠ stop end_turn). Because it writes to
stderr, spectro run --json | jq keeps a clean stdout. In the REPL the traced
instance is shared with subagents — you see the children's requests too.
The tour
./spectro-app tour is a menu-driven guide around all faces: launch the REPL,
list or resume sessions, do a headless run, drive cron, start the web jar, run doctor —
plus a settings screen that captures a provider choice and API key for the session
(hidden input, optional save to ./.env with mode 600) and stage-appropriate
“try this” tips. The same file ships in every workshop stage and detects at runtime
which features exist; in the full build it shows everything.
Chapter 5The web face: Chat
One Spring Boot jar serves everything: the built React UI, a handful of
REST endpoints, and a single WebSocket that carries RunEvents in one direction and a
small set of client frames in the other. Open http://localhost:8080 after
./spectro-app web and you are looking at the chat.
The anatomy of the screen
- Sidebar (left, resizable, collapsible): “New chat”, the Scenarios button (chapter 7), the session archive with an Import button (chapter 8). The first row is always the live session of this browser tab.
- Header: session title (the first prompt), image-gallery toggle (appears once images exist), right-panel toggle, the Thinking switch, the DE/EN toggle, the design drawer, the provider picker chip, the context ring (appears once tokens flow) and the Stop button while a run is active.
- Tab bar: chat · spectrum · trace · graph · text · lab — six lowercase lenses over the same state (the labels are wire vocabulary; trace carries a live frame count). Entering a fleet swaps the whole row for the fleet's own bar — chapter 15b.
- Footer: run and session token totals in tabular numerals, a run status dot, and the connection dot.
Behind the scenes, every incoming socket frame is batched once per animation frame
(a text_delta flood costs one React render per frame, not per token) and
folded by one pure reducer into an immutable UiState. Live stream and
archived replay produce the same state shape from the same code — which is why every
view in this chapter works identically for both.
The conversation
- Markdown answers. Assistant turns render through a dependency-free
markdown parser (headings, streaming-safe fenced code with language chip and copy
button, nested lists, GFM tables, quotes). Nothing ever becomes raw HTML, and link
protocols are vetted —
javascript:renders as plain text. Model output is untrusted end to end. - Thinking, apart from the answer. While a reasoning model thinks, a
pulsing “thinking …” dot is the live indicator; the finished reasoning collapses into
a disclosure above the answer showing its size. The header switch turns the stream off
for the next run (
set_thinking). - Tool cards. One collapsed card per call: status dot + name + input
preview + gate chip + duration. The gate chip is the didactic point —
Gate: allowed / denied / waiting appears only for permission-gated calls, so
the three control paths (free tool, gated tool, hook-blocked call) read differently at
a glance. Expanding gives the call three faces:
structureddraws it as the thing it is (a shell call as its command and its output, a read as the file and its content),jsonas collapsible input and output trees, andrawas the untouched pair — with a copy button for the output beside them. The face is a reading mode rather than card state: switching it on one card switches every card, and the browser remembers the choice. Where an imported call handed back a picture, the picture sits under the body — the section after this one. - Subagent threads. Consecutive turns of one child nest under a pastel-edged thread block (header: agent id, role label, task). Interleaving splits threads honestly — the Trace keeps the flat truth. More in chapter 14.
- Info and error lines. Spawns and compactions appear as quiet info lines (localized live); a failed run gets an error card with a “Send again” link.
Pictures out of an imported transcript
An imported Claude Code transcript usually carries pictures — a screenshot pasted into a
prompt, a browser tool handing back what it saw — and the importer used to read the base64
block, count its bytes, write [image/png · 31.0 KB] and drop the bytes on the
floor. The bytes are in the file and are already in memory by the time anything reads them,
so they now ride on the frame itself (attachment_image, a type that exists for
imports and never crosses the wire) and are drawn where they belong. That old sentence
survives as the picture's label. Chapter 8 covers the import itself.
- In the bubble. A picture that arrived with a message sits above its words as a bounded thumbnail (at most 160 px wide, 120 px tall). Where the record said nothing else — no sentence, just the screenshot — the picture is the message and gets a bubble of its own instead of gluing itself to whatever was typed next, which is a different message. A picture pasted into a prompt that was queued behind a running turn arrives the same way.
- On the tool card. What a tool handed back is drawn under the card's body, bounded at 320 px tall and loaded lazily. This is the common case by a wide margin: of the 8,788 image blocks in the corpus this was measured against, roughly 7,300 sat in a tool result rather than in a person's message.
The same picture appears twice more — as its own section in the trace row that carried it
(chapter 6), and on the Lab's agent card (chapter 7). In the bubble and on the card it also
opens: the cursor becomes a magnifier over a picture that does, and an accent hairline draws
itself on hover. Clicking one opens the lightbox, which is mounted at app level
rather than inside the chat, because the walk has to cross surfaces. ← and
→ step through every picture in the session in stream order — the ones a message
brought and the ones a tool returned, interleaved as they happened — wrapping at both ends,
with a counter saying which of how many; Esc closes. The bar above the picture names where it
came from (“from a message”, or “from” and the tool's name) and carries the file's own note
beside it.
A picture also has three faces, in the trace's own idiom:
picture, base64 and file. base64 is the
string itself, headed with the media type and its character count, copyable in one press.
file is the .jsonl line the picture lives on, with three lines
either side for orientation — the neighbours clipped, the line itself cut into segments of
readable JSON with every base64 blob standing in as a measured absence you can click to land
on the string. Every blob, not only this picture's: one record often carries several, and the
head says how many. Nothing is fetched for any of this, because the import holds the file's
lines byte for byte and knows which line produced which frame. The face survives a step to the
next picture, deliberately — comparing where three screenshots sit in one file should not cost
three presses.
Two buttons finish the bar. save writes bytes that are already in the page,
so nothing leaves the machine and the picture really is a file afterwards.
folder opens the transcript's folder, and its tooltip says why it is
not the picture's: the picture is not a file on disk, it is base64 inside the
.jsonl. That button appears only for a session opened from the store — a picked
file and a paste have no address behind them, though both still get the file face,
because the lines came in with them.
Nothing reads what is in a picture: no OCR, no caption, no search by what a screenshot shows, and the viewer neither zooms nor pans. Generated images (chapter 17) keep their own gallery and stay out of this walk — handed over and painted here are different provenances, and the app refuses to blur them. And the frame is import-only by construction, so all three JSONL formats an export can write drop every picture; only the HTML export carries the bytes out.
The composer
- Enter sends, Shift+Enter breaks the line — the only shortcut. The textarea autosizes up to 150 px; the Send button reads “Running …” while a run is active (one run at a time per connection).
- Image attachments. The image button opens a file picker (jpeg/png/webp/gif, multiple); dragging files anywhere onto the chat works too. Every image is downscaled in the browser before sending (longest edge 1568 px, JPEG re-encode when useful; GIFs pass through to keep animation), previewed as removable chips, and never leaves the page until you hit Send. Attachments cross the socket as base64, land in the session's blob store, and the events carry only references.
- Push-to-talk. The mic button records (coral pulsing dot +
m:sstimer), a second press stops; the audio is transcribed server-side by local whisper.cpp and the text lands in the input field — an STT error is reviewable text, never an agent instruction. If STT is not installed the button disables itself with a setup hint. Chapter 17 has the setup.
The permission gate
When the agent wants to run a gated tool and no allowlist rule covers it, the run genuinely pauses — the agent's virtual thread parks on a future server-side — and a gate bar rises at the foot of the chat. A bar rather than a modal, on purpose: the pending request is first-class UI, the conversation it is about stays readable behind it, and the run waits either way.
- The row itself is a glance — the tool and its input as one compact line, the whole of it in the title. The chevron opens the read: the input in full as pretty JSON, unclipped, in a box that scrolls with the buttons above it, so no payload can push them anywhere. You approve what you see, never a summary.
- Neither button is preselected and the bar takes no focus of its own — a decision has to be made deliberately, and the rest of the app stays usable while the run waits.
- The agent that asked wears its own colour beside the word
gate, so a subagent's request is never read as the main agent's. - Further requests queue behind the one on screen and say so:
+2 waiting. - Opened, the bar also keeps the record — the last six decisions of this session, newest first, each as allowed or denied with its tool and its agent.
The two checkboxes appear one behind the other:
always allow (session) remembers the approval for this browser
tab only. save to project, which appears once the first is ticked and only
for a session that has a workspace, appends the rule to that workspace's
.spectro/settings.json — the same autoApprove list the CLI reads,
so a web decision is honored by the terminal on its next run. Remembered rules are
prefix-scoped for risky tools (a git status approval becomes
run_command:git*, never “all commands”) — chapter 13 has the exact rules.
A denied call returns ERROR: the user denied the execution. to the model,
which reads it and adapts.
Live indicators
- The context ring in the header fills with the last reported input tokens against the compaction threshold (green < 70%, amber ≤ 90%, red above); clicking it opens the introspection popover — chapter 28.
- The usage footer counts run and session tokens live in tabular numerals; the status dot switches “run active → ready” and names any non-regular stop reason.
- The connection banner appears under the header when the socket drops, with a reconnect countdown (exponential backoff 1→15 s) and a “Reconnect now” action. Sessions survive: the JSONL file is the state, the socket is just transport.
- The Stop button sends
abort, which fires the run's CancelSignal — the run ends withstopReason: "aborted", tools and children included.
Chapter 6The web face: tabs & panels
Chat is one lens. The Graph tab shows the same run as a process diagram, the Trace tab as wire frames, and the right-docked panel adds the agents roster, the live plan, the exact system context and a sandboxed file browser.
The Graph tab
Two views behind a Flow | Graph toggle (sticky per browser). For archived sessions a replay bar adds Full, Time-lapse (advances along the real timestamp gaps, accelerated), Pause and a scrubber.
Flow is the minimap for humans: rounded activities
(user / think / tool / say / compact / end) stack on the main spine; each
subagent wave becomes a split block with one column per child and a join bar. Think and
say activities accumulate their streamed text, so real runs get sentence labels instead
of “token one”. Tool activities label themselves honestly — read <file>,
$ <command>, fetch <url>, server · tool for MCP.
Gate outcomes render as small dots (pending / allowed / denied). In a replay, clicking
an activity seeks the timeline to its first event.
Graph is the classic DAG (React Flow + dagre): one node per prompt,
turn, tool call, subagent and answer, with t+ relative times, duration and
token badges, error badges, and a pulsing outline for running nodes. Sequentially
spawned subagent subtrees get their own side-by-side lanes. Clicking a node opens a
detail panel with its timing line and its raw evidence — exactly the events
that formed the node, each as a collapsible JSON tree. The right mouse button pans;
the left button only clicks — on both React Flow canvases in the app.
The Trace tab
“What Wireshark is to packets, this is to the harness protocol.” Every frame lands here — RunEvents inbound, client messages outbound — before the reducer even looks at it, so unknown event types are visible too.
- Columns: sequence · wall-clock time with milliseconds · Δt since the previous visible row · LLM direction (↑ going to the model with the next request, ↓ produced by the model, · internal) · proto · host · agent · type · summary. Host and model — the model serving each run — are the two optional ones, switched from the toolbar's columns control; a column that is off is left out of the row entirely.
- The proto column names the wire each payload actually rides, and it
answers for the frame rather than for the work behind it. Most rows therefore
read
WebSocket: atext_deltais our own frame about the model's output, not the model's output arriving, and it never left this machine. Only a recorded exchange says what went out to the provider —HTTPS/SSEfor the cloud streams (Claude and OpenAI-compatible),HTTPS/NDJSONfor Ollama, plainHTTPSfor the one-shot calls (transcription, an image) andprocesswhen the provider runs as a local child process. Tool rows name their execution transport instead:JSON-RPCformcp__*(a result resolves its tool through the callId),HTTPforweb_fetchand image generation,localfor the file tools. - The host column names the network counterpart. A frame that only rode
the socket names this server's own origin; an import, which cannot know the machine the
other app ran on, leaves it at a dash. LLM rows show where the
request really goes (
api.anthropic.com,localhost:11434…) — live from the socket-onlyprovider_infoframe the server sends on connect and after every provider switch, so a switch is a visible trace row, never a silent client-side swap. MCP rows name their server,web_fetchthe fetched URL's host, image events their backend endpoint; replays only know the provider, so anthropic maps to its fixed endpoint and local backends show an honest—. - Filters: free-text (matches type, agent and payload), an LLM-direction segment, and fourteen toggleable category chips — run, turn, text, thinking, tool, workflow, mcp, permission, usage, image, llm, context, client, other. Tool traffic has three of them rather than one, so the MCP calls and the workflow steps can be taken out without taking the local tools with them.
- Summaries with token highlighting: the model's own words in sand,
punctuation faint, numbers tabular, the literal
ERRORred — lossless, the full payload is one click away (collapsible JSON tree + copy). - The synthetic
system_contextrow: the system prompt rides with every request but is not a wire event — the view prepends one synthetic ↑ row (fromGET /api/context) naming prompt size, tools, skills and MCP servers. Display-only: never in the reducer, never in the JSONL. - Reading flow: the tab opens at the first frame — the story reads from the beginning. While a frame is open, Space steps to the next visible one (it opens, takes focus and centres itself; Enter still toggles a focused row), so the trace taps through like the Lab stepper. A slim jump rail on the right edge scrolls to the first or the newest frame.
- Auto-follow engages once you scroll down to the live end; while you are reading further up, arrivals accumulate into a “{n} new ↓” pill instead of yanking the view.
- Four faces per row: structured opens a frame as the thing it
is (a call as its tool card, an answer as its text), insight as an expanded
JSON tree, wire as plain text — exactly the lines that crossed the wire, one
row each — and source as the line of the imported file the frame was read from.
The view control above the list is a master switch: flipping it brings every row back to
that face, and a single row can still be switched afterwards. A face with nothing behind
it is left out rather than shown empty, so a recorded LLM exchange offers no
source; the two retired names,
rawandcompact, both land on wire. Thesession_resumemarker's detail carries the entire re-uploaded history as its JSONL lines — chapter 8. - Pictures. A picture an imported transcript carried is a frame like any
other (
attachment_image), and it answers to theimagechip beside the generated ones — that chip is the one gesture a reader makes to find the pictures, and until it knew this type it hid every one of them. The collapsed row prints the file's own note,[image/png · 31.0 KB], and never the bytes: a row's summary is also the text a find in the page matches, and one 140-picture file put 19.87 MB of base64 there before this had a case of its own. Open the row and the picture is drawn in a section of its own, labelled with the payload field it came out of, that note underneath. An imported picture carries its bytes on the frame, so clicking it opens the lightbox and the arrow-key walk (chapter 5); a generated one is a store URL rather than bytes in the page, so it renders and does not open.
The Text tab
Another lens on the same stream — where the trace shows frames, this shows the text. Two views behind a Text | JSONL toggle (sticky per browser), one copy button for the whole thing:
- Text is the whole session as one readable feed, with the protocol
made visible as literal markers: every reasoning run sits between
<think>and</think>— exactly the tags Ollama really streams inline; Anthropic's thinking blocks render to the same boundaries — followed by the answer text in plain type. Tool use appears as honest indicators with nothing hidden:[tool_call read_file {"path":"…"}], then[tool_result read_file · 9ms]with the full output underneath. Run boundaries ([run_start ollama],[run_end end_turn]), the permission gate, spawns, A2A messages, compaction and errors all keep their place in reading order; subagent lines carry their agent id as a prefix. Prompts render in sand, thinking dimmed italic, markers faint — but it is all ONE selectable text, and the copy button hands it out as plain text. - JSONL is the session exactly as the file on disk stores it: one
compact JSON line per wire event, newline per event — the NDJSON view of the truth.
Socket-only UI frames (
provider_info,workspace_info) are deliberately absent: they never enter the file, and this view IS the file.
An extended toggle sits above the feed, and it is the honesty control: the reading leaves things out, extended puts them back — the assembled request the model was handed, its parts each headed by a character count and a token estimate, plus the usage lines, the turn boundaries and the published plan. The setting travels into the exported document, so the file says what the screen said.
Explain: have a model read the run
Above the feed sits an explain button. It does one thing: it sends a bounded digest of exactly what the Text view shows — the same prompts, reasoning runs, tool calls and gates, nothing hidden and nothing added — to your configured provider, and streams back a plain-prose reading of why the session went the way it did. What the agent was trying to do, why each step followed from the last, where it hesitated or was gated, where it recovered. Because the digest is built from the same feed you can read yourself, imported sessions explain just as well as live ones.
The panel is labelled for what it is — “model-generated reading of the recorded run — not model internals”. That boundary is not just a caption: it is written into the instructions the explain call runs under. The model is told it is reading a recording, that it cannot see hidden state, and that it must ground every claim in the transcript and say so where the transcript is ambiguous. If you want the deterministic view — which gate asked and why, folded straight from the stream with no model in the loop — that is the gates panel in the Trace tab, and it stays exactly as it was.
One honest caveat: the explain call spends your provider key, like any other model call. It carries the same local-origin fences as the key-writing endpoint (it is a loopback-only, same-origin request), and it needs a provider with a key set — otherwise the button reads “set a key in Settings” instead of running.
The right panel
The header's panel button docks a resizable panel into the Chat tab with four tabs.
Agents
One card per agent — main first, then every child in spawn order — with its state dot
and lifecycle badge (submitted / working / done / failed), its task, the latest
report_status line, and its token cost (1.2k in · 0.3k out).
Selecting a card jumps to the System context tab for that agent.
Plan
The read-only, latest-wins snapshot of the agent's plan, fed by the permission-free
update_plan tool: each step with a status dot and badge — done,
coral running …, open. The wire statuses stay canonical English
(pending / in_progress / completed, enforced at the write boundary); only
the labels localize. The tab badge counts steps; the CLI pretty-prints the same event.
System context
The transparency panel: exactly what the LLM receives before any user message. For the
main agent: the full system prompt (working directory, project
SPECTRO.md context, the skills catalog), provider/model/thinking, every tool
in registration order (gate chip = needs your approval), skills, MCP servers, and the
six subagent role profiles. For a selected subagent: its role prompt,
its task, its reduced tool set — and the note that children never get the spawn tools
(nesting ends at depth 1). The data comes from the stateless GET /api/context,
assembled from the same constants the live tools use — what the panel shows is what the
model gets. One row is a list rather than a state: the MCP servers are the ones
your settings file declares, named whether or not they answered, and their tools are not
in the tool list above them. Reachability is what /mcp and
spectro doctor report (chapter 18). A fullscreen Raw modal shows the
whole context untruncated with a character count and copy button.
Files
A read-only view of the agent's workspace — the same sandbox the file
tools see. Every session gets its own folder (<tmpdir>/spectroscope-ws/<session-id>
unless a workspace is configured — chapter 30), the server announces it over
the socket (workspace_info, visible in the trace), and the tab follows it
live: what the agent writes appears here, and the repo you started spectroscope in stays clean.
The panel opens itself on the Files tab when the announcement arrives —
the agent's desk appears where its files land. And you can pick the desk per
session: the “Choose folder …” button opens the native folder dialog on
the spectroscope machine (POST /api/pick-workspace, macOS; a browser cannot hand out
absolute paths, but spectroscope runs locally), and the picked path travels back as the additive
set_workspace socket message. Only before the first run — once the agent ran,
the sandbox and every subagent are anchored, the button locks and a new chat is the way to
a different folder. The pin is shared server state, so the tree you browse and the sandbox
the tools see are always the same folder, and a resume in the same server process finds
the picked folder again. And the browser remembers your pick: every new
chat lands on the same desk with zero clicks (a dashed “↺ folder” chip shows the memory;
clicking it forgets — the next chat gets its per-session temp folder again. Resumes are
never touched).
Hidden entries and build folders (.git, build,
node_modules, target, dist, out) are
skipped, depth and entry count are capped honestly (“list truncated” note). Clicking a
file previews it by type: HTML runs in a CSP-sandboxed iframe (scripts
execute in an opaque origin — no cookies, no API, no parent access), markdown renders
through the shared component, images inline, everything else as text. Binary answers
415, oversized 413 — and the defense in depth is real: the .env with your
API key answers 404 even by direct URL, because every path segment is re-checked
server-side against the hidden/ignored rule.
The image gallery
When the agent generates images (chapter 17), a gallery column opens next to the chat:
cards newest-first with the prompt as caption, provider and model badges, full-size link,
and a provider dropdown (gemini / openai) that switches the image
backend for the next generation. Only generated images live here: a picture an imported
transcript carried belongs to the message or the tool card it came with (chapter 5), and the
two provenances are kept apart on purpose. Bytes never cross the socket — cards load from
GET /api/images/<sha256>.<ext>, where the content address doubles as
the traversal guard (64 hex chars or 400, before any filesystem access).
Each card also carries “→ Workspace”: it copies the image from the global store into this session's workspace — you pick the file name (keep the content-addressed original or type your own; a name without an extension inherits the original's), and the file appears in the Files tab, ready for the agent's tools. The endpoint keeps the house guards: only store names touch the disk, the target name is a single sanitized file name, and an existing file is never overwritten (409).
the spectrum and the new lenses
The design transplant (migration phase 4) brought the brand into the product and six new surfaces with it. All of them read the same event stream you already know; none of them adds a second source of truth.
the spectrum tab
The spectrum is the fleet on one screen. Every agent draws one horizontal lane; every event is a discrete mark on it, colored by type: token teal, reasoning violet, tool amber, gate red, subagent ocean, lifecycle faint. The lane rail carries the agent id, its task, its lifecycle state and its token totals. A pending permission turns the lane's gate mark violet and pulses until the decision lands.
Clicking a lane opens the trace pinned to that agent: the new agent chip row filters the wire view to one lane while session-level frames (decisions, run ends) stay visible. Dense token streams are thinned for display; the counter names the drop, and the JSONL file keeps everything.
the reasoning lens
The reasoning lens is a trace mode for one question: what did the model say it was thinking, and what did it then do? Switched on, thinking frames come to the foreground in violet, everything else steps back, and tool calls and gate frames stay readable as anchors. Every block of thinking ends in a clickable "then:" line that jumps to the action that followed it.
Reasoning text is the model's self-report, recorded next to what it actually did. It is not a window into the weights, and the interface says so. With reasoning capture off, the lens tells you that too instead of showing an empty highlight.
The lens state persists with your design preferences, applies to live runs and replays alike, and combines with every other filter.
the causal chain and the replay scrubber
Every open trace frame now shows its walk-back: result, decision, request, call, turn, run, spawn, parent run. Each link is a chip; click it and the trace jumps there. The chain is computed from the recorded stream alone, with no model involved.
The replay scrubber caps the visible stream at any frame. Drag it back and you read the session exactly as far as it had happened, filters included.
the gate surface
Pending permissions are first-class now. Instead of a blocking modal, a gate bar docks above the footer on every tab: the violet line means the run waits on you. Approve or deny inline, keep the session rule checkbox at hand, and expand the bar for the full input and the recorded outcomes of earlier gates. The Lab keeps its own stepper-owned dialog; replays never ask.
the explain panel
The explain button in the trace toolbar docks the why layer next to the stream: a run summary (duration, turns, tool calls by name, gate tallies, tokens, stop reason) and one entry per gate that answers "why did the gate ask" from the permission mode at ask time. Stored sessions carry no mode frames; the panel says "not recorded" instead of guessing. Everything in the panel folds deterministically from the stream.
spectro doctor, the web face
The reference-lamp button in the header opens the calibration page:
one measured line per subsystem, from this browser's viewpoint. Server
api, live socket, llm backend, session store, default workspace, log
level, permission mode. The page re-measures on every open; the CLI's
spectro doctor checks the machine side.
Chapter 7The Lab
The Lab is spectroscope's step-through debugger for agent runs, and its best classroom. Events do not flow through automatically: they queue behind a client-side dam, and you advance them one boundary at a time while chat, map and JSONL strip move in perfect lockstep. It works on live runs and on any archived replay.
The dam
Live events keep queueing even while the Lab is closed. Each step applies the next event(s) through the same pure reducer the chat uses, plus two pure folds underneath: the scene model the Flow map paints, and a strict Petri marking. The invariant is literal: the stepped UI state always equals “fold of everything applied so far”. Because the server genuinely waits on permission futures, a token sitting at the gate is the real system state, not a simulation.
The transport
Two things frame the map. Above it floats the “now” band: a plain-language line
of what the run is doing at the current step (“the model is thinking”, “writing hello.txt to disk”,
“the permission gate is deciding”, “orchestrating 3 workers”, “done · control is with you”), and
next to it the size of the dam (“n waiting”, or “waiting for server …” when
a live run has drained the queue). Before the first step it reads “ready to run”. Because the band
names the folded scene, a scrubbed replay always tells you where it is.
Below the map sits the scrub bar. Four buttons drive it: reset (⟲),
step back (‹), play / pause (▸ / ❚❚) and step forward (›). A
slider walks the run's coarse-step boundaries, so a drag always lands on a whole
step, never mid-block; a “step 4 / 27” counter reads out where you are. Scrubbing pauses
auto-play. Grain and speed are one rung down, behind a small more disclosure that a
live run rarely needs:
| control | behaviour |
|---|---|
| › Step forward | applies the next block (or one JSONL line in Single grain); the “now” band, chat, map and strip all move together |
| ‹ Step back | undoes the last step group, re-folding chat, map and strip backwards from scratch |
| ▸ Play / ❚❚ Pause | opens the dam into a paced auto-play: a timer steps at the chosen speed instead of teleporting to the end |
| ⟲ Reset | re-queues everything applied and returns to the start |
| Blocks | Single (more) | the grain: one click = one meaningful block (a whole thinking run, a whole answer run, or one event), or exactly one JSONL line |
| Speed (more) | the auto-play slider, roughly 0.5×–16×/s (default 0.8/s) |
Every button has a keyboard twin, guarded so it never eats a keystroke while you type in a field:
space or → steps forward, ← steps back, f toggles
flow (auto-play), and r resets. Press ? at any time for the full sheet
(see the keymap).
The panes left and right of the map, the fully functional chat and the JSONL strip, are
drag-resizable and collapsed by default, so the map gets the room. The strip shows applied lines,
the just-fired line highlighted, the dam divider (“▮▮ dam · n waiting”), then the dimmed
queue; every row expands to the full event, and an “All” toggle drops the render window for very long
sessions.
compact | expanded
A small compact | expanded pill sits by the map (the same pill rides the fleet machine room, and the two share one preference, so the choice follows you between them). It changes how much of each node opens by default, not what the map means.
Compact is the working default: an agent card shows its identity, its state and its live token totals, and the surrounding stations (the context window, the system prompt, the user, the LLM) sit as tidy tiles. Expanded opens every card at once — the context window and the system prompt move to sit BESIDE the agent, the prompt beside the user, and each node card carries its task and history inline. The layout genuinely spreads to make the room, so nothing overlaps; it is the view for reading one run closely, where compact is the view for watching the shape of it. The same pill also carries the Text | JSONL toggle for the strip, so both grains are one click away.
The Flow map (React Flow)
The centre is a literal systems map. On the left, the AGENT SYSTEM · YOUR MAC zone: the user, the Agent hub (live activity line, the Loop row, the Permission gate row that colors with the gate state, and the tool belt) and the OPERATING SYSTEM band, disk (animated platter, the touched file as a pill), shell (the running command), MCP client and network stack. On the right, OUTSIDE: the LLM (neural-net animation while thinking, model name, reasoning and answer stream slices per agent), the network and the MCP server.
One coral packet rides the rails to wherever the current event happens. The map is honest about locality: with a local provider (Ollama) the LLM node sits inside your machine and nothing crosses the boundary; switching to a cloud provider re-places it beyond a dashed network boundary, live, because the layout follows the selected provider.
Every node carries disclosures with the real data. The Agent hub now shows a tool belt:
the chips read_file, write_file, list_dir,
run_command, use_skill, call_mcp and generate_image,
with the active one lit. Its system context disclosure holds the main agent's system prompt,
the live context budget (estimated tokens against the compaction threshold, with a bar per segment)
and the current tool call as a JSON tree, or, when the active tool is generate_image, a
small generated-image thumbnail with its prompt caption. The LLM node reveals per-agent reasoning and
answer streams; the disk shows its file, the shell its command, the MCP client its call. Subagents pop
in as their own loop cards with lifecycle badges and disappear when the root run ends.
Two picture panels can sit on that same shelf, and the label says which is which.
generated image is the last one the agent painted, captioned with the prompt that asked for
it. pictures handed over is what an imported transcript handed the agent — a pasted
screenshot, a browser tool's answer — captioned with the file's own note, in arrival order, and
capped at six per card, because a session that pasted forty screenshots would otherwise draw forty
on one node; the rest are still in the chat and the trace. The panel reads the stream's
root agent rather than the literal main, and that distinction is the
whole feature on an imported subagent transcript: such a file roots at the child's own id, and two
thirds of the pictures in the measured corpus sit in exactly such a sidecar. In
expanded both panels leave the context disclosure and span the whole card under the
two columns. Neither thumbnail opens the lightbox — that walk belongs to the chat bubble, the tool
card and the trace row (chapters 5 and 6).
The keymap
Press ? anywhere (or the header's keyboard button) to open the keymap:
a game-style shortcut sheet that pairs each keycap with a small icon of what it does. It documents the
Lab's real keys, space/→ step forward, ← step back,
f flow, and r reset, alongside ? to reopen the sheet and
esc to close it. Close it with Esc, the ×, or a backdrop click. The footer
states the one rule worth knowing: shortcuts pause while you type in a field.
Under the hood: the Petri marking
The stepper also folds every event into a strict Petri net, for the formally minded: places are the stations a request passes through (User, LLM, Tool, Gate), transitions are the RunEvents, and one token moves. A Log place accumulates a token per fired transition, so its marking literally counts the JSONL lines, which is the invariant the strip's highlight rides on. Guards keep the net consistent: a move only fires when the source place holds the token, otherwise it degrades to a pulse (parallel tool calls, odd foreign replays). The marking has no view of its own; it is the Lab's bookkeeping, not its face.
Scenarios: deterministic demo runs
The sidebar's Scenarios button opens seven built-in demo runs. Each is authored once in a bilingual DSL and compiled to a deterministic RunEvent stream: real wire format, fixed timestamps, no model involved. Picking one lands in the Lab with the stepper at event zero; it rides the same replay path as any stored session, so every tab renders it.
| scenario | what it demonstrates |
|---|---|
build_plan · 1 subagent | dev-tool delegation: a planner child with status updates, verification by main, a gated test run, a denied MCP call and graceful continuation |
Review fan-out · 3 subagents | parallel children (bugs / performance / security) running round-robin, then a prioritized consolidation |
Permission gate · blocked & allowed | the red path: rm -rf data/tmp denied, the agent backs off to read-only listing, then an allowed git status |
Disk & shell · read / write | the disk/shell stations: read → write → list → a gated npm test |
Coding · 4 phases, parallel workers | a realistic dev run: explore → plan (subagent) → implement (two parallel workers writing files) → verify (gated test run) |
Research · consolidate + critical review | parallel source sweep with MCP searches, a consolidation draft, an adversarial critic child that finds a contradiction, the corrected answer |
Context window · fill & compact | no subagents: three big reads fill the context ring (ok → warn → high), then a compaction drops the meter and the run continues |
Live model behaviour is non-deterministic: a workshop demo that depends on the model calling exactly the right tool at the right moment will betray you. Scenarios make every mechanism of the harness (gates, subagents, MCP, compaction) steppable and repeatable, offline, in either language. (Most screenshots in Part II use the “coding” scenario for exactly this reason.)
the tutorial
spectroscope has seven tabs, three lenses, a fleet canvas, a machine room and an observability page. Every one of them earned its place, and together they are a wall. A fresh install therefore starts small and opens up as you use it. That is the tutorial.
The twist is that there is nothing to sit through. The product already watches agents through an event stream; the tutorial watches you through the same instrument. You do not tick a checkbox to say you have read a trace. You read one, and the engine sees it in the events the interface was rendering anyway.
the first question
A home that has never been used opens with one screen and one decision: grow into it, or open everything now. There is no silent default in either direction, the question is asked exactly once, and the answer lives in the settings afterwards. A home that already has sessions or a settings file is never asked at all and never sees a lock. If you have been using spectroscope, nothing about this changes for you.
It is also the only screen at that moment. A first run has a second thing to say — that spectroscope needs a backend and which ones are free — and two welcome dialogs on top of each other is the wall this whole wave exists to remove. So the backend sheet waits until the tutorial question has been answered, and then takes the screen alone.
the seven states
Four of the names are the product's own words. Three are borrowed from telescopes, because they describe the same progression.
| state | you can | it opens |
|---|---|---|
| dark frame | set up a provider, watch scenarios | chat, settings, scenarios |
| first light | run an agent and read its answer | composer, sessions, files, gallery |
| the trace | read what actually happened | trace, text, replay, import |
| the gate | decide what the agent may do | permission mode, workspace |
| the prism | split one run into many lines | spectrum, lab, lenses, graph |
| the fleet | watch agents across processes | fleets, canvas, machine room |
| deep field | point the instrument outward | explain, observability, scaffolder, MCP |
A dark frame is the calibration image a sensor takes with the shutter closed: the picture you make before you make a picture. First light is a telescope's first real image, the night it stops being an assembly of parts. The deep field is what appears when you point the thing at a patch of sky that looks empty and stare long enough.
The header pill carries a small spectrum strip that gains one colored line per state reached. At the end of the tutorial you hold the full spectrum, which is the same five lines as the mark on the cover of this guide.
No new colour was invented for any of this. The five lines are the app's existing event tokens, in the app's existing order, so the strip is correct in all three skins without a palette of its own.
progression with receipts
Click the pill and the panel shows every state, every criterion, and
what each one counts. A met criterion carries its evidence: the session
it happened in and the event inside it. The link opens that session on
the trace, at that frame. The address is a plain hash
(#/session/<id>@<n>), so it can be copied into a
bug report like any other link.
A criterion you tick by hand says so, permanently. The tutorial distinguishes what it saw from what it was told, because a progress display that cannot tell the difference is decoration.
A state you have not reached still shows its tab. Clicking it gives you a card naming what the surface is, what opens it, and a button that opens everything immediately and permanently. Two surfaces are never locked at any level: the settings, because that is where keys live, and the permission gate, because a run waiting on a decision must always be answerable.
The card is the whole mechanism in one screen. The tab stays where it was, because a feature nobody can see is a feature nobody adopts. What it shows you is the one criterion still outstanding, not a percentage, and the button beside it ends the whole arrangement then and there.
reaching a step
A climb is announced once, briefly, and names what it opened. The new line on the strip ignites as it lands. It blocks nothing and needs no dismissing, and after seven seconds it is gone again, because a tutorial that congratulates you at length is a tutorial standing in your way.
turning it off
Settings has three modes. Tutorial opens surfaces as you go. Checklist locks nothing but still tracks and celebrates, which is what every existing home gets. Off removes the pill and the panel and stops recording entirely. The same block restarts the tutorial from the dark frame; restarting does not ask the first question again.
The CLI reports the same state without a server:
spectro level prints the tutorial with its ticks and receipts,
and spectro doctor carries one line of it. All three faces
read the same levels.json from the core, so they cannot
drift apart.
Every criterion is reachable without a paid API key. A local model through any of the free backends counts as light, the bundled scenarios count as fan-outs and fleets, and the walkthrough from dark frame to deep field has been done that way.
Chapter 8Sessions: replay, resume, delete, import
Every run writes a JSONL session file as it happens — there is no “save” step, the file is the state. This chapter is about what you can do with those files from the UI; Part IV shows them on disk, and chapter 22 documents every line.
The archive
The sidebar lists every stored session newest-first: first prompt, relative age, token total. Opening one is a replay — the same events, folded by the same reducer, rendered read-only. Chat, Graph, Trace and Lab all work on it; the composer is replaced by the archive bar.
Resume — with the re-upload made visible
Resume session turns an archive back into the live session: the UI
seeds itself from the stored events, the socket reconnects with ?resume=<id>,
the server reconstructs the provider history from the file (main agent only, orphaned
tool calls dropped — chapter 22), and new events append to the same file. The
header eyebrow flips to Resumed.
The didactic detail: resuming is not free. Your next prompt re-uploads the whole
conversation to the model. The trace makes that visible with a UI-only
session_resume marker between history and new traffic,
summarizing how many events and roughly how many tokens now ride along. The estimate
mirrors what the server actually reconstructs: main agent only (a subagent's inner steps
never ride back up), thinking excluded (it never re-enters the provider history) — and
the marker says explicitly that the system prompt and tool schemas come on top, so the
next usage event's real jump lands where you expect it. The marker's detail
view carries the entire re-uploaded history as its JSONL lines. Scenario and import
replays offer no resume — there is no file to append to.
Delete — the two-step danger button
Deleting a session is the first deliberately destructive surface in spectroscope, so it is
defended in depth: the button must be clicked twice within four seconds (first click
arms, idle disarms); the REST endpoint (DELETE /api/sessions/{id}) guards the
id shape before touching anything; and the store itself refuses any id that does not
resolve to a direct child of the sessions directory — traversal ids delete nothing.
Success removes the JSONL and the session's blob folder, closes the replay and
refreshes the sidebar. The session currently being resumed by the live tab never offers
the button.
Import
The Import dialog replays foreign files through the same pipeline, and it is a room
rather than a picker: full screen, with only the row list scrolling. Three ways in. The
transcripts the server finds under ~/.claude/projects are listed and load on one
click — that folder is Finder-invisible, so a file picker cannot reach it — and beside the list
sit an ordinary file picker and a paste box for everything else. Detection reads records rather
than the first line, because a real transcript opens with metadata records before its first
message, and the first record naming a format decides which of three it is.
- Raw spectroscope JSONL — recognized by its event types, eighteen of them, and replayed verbatim. Files from the TypeScript edition work identically (the format is the contract).
- Claude Code transcripts — recognized by their
messagerecords and translated by an adapter: tool uses becometool_calls,Task/Agentspawns becomeagent_spawn+ A2A messages, thinking becomesthinking_delta, sidechains fold under their owning task. You can watch a real Claude Code session step through the Lab's map. - VS Code agent-mode exports — dotted type names over a
dataobject. That export records which tools ran and whether they succeeded, never what they returned, so the dialog states it in one line rather than leaving a screen of empty tool bodies to read as a broken import.
A file matching none of the three is refused by name: the message lists up to five of the type names it actually found and says where each format lives, because “unrecognized format” sends you back to the file with no idea what to look at.
Two axes narrow the list, and they behave differently on purpose. The model chips
are the families the facts have turned up so far, and picking a second one widens the
result; the three with chips — workflow, subagents,
images — all have to hold, so picking a second one cuts deeper. A search box matches
file, project, model and opening prompt. One statistics line then adds up whatever is left
standing: the transcripts, the span they cover, each model family with the number of sessions it
spoke in, and the workflow, subagent and picture totals. Beside those stands “n not read
yet”, because a total that silently omits forty rows is not a total.
Facts arrive per visible row. The listing is one directory walk and answers in milliseconds; reading transcript bodies does not, so rows render from the listing and fill in behind it. They are batched as you scroll, twenty-four to a request, and the cap comes from a number the server publishes in its own answer so the client cannot drift past it. A row that has not answered is neither a match nor a miss — it is pending, and the dialog says how many are still out instead of quietly shrinking the list to whatever happened to be cached.
A row carries the file name, the opening prompt, the project, the age and the size, then chips
for what is inside it: the model families (the tooltip keeps the verbatim ids), workflow
×n, subagents ×n, workflow-agents ×n, images ×n and
the language. None of that says what the session was about — an opening prompt is often
“read the card first” or four thousand characters of pasted log — so one button asks the
configured model and writes one line per row. It is a press and never a side effect of opening
the dialog, because it is the only thing on the row that costs a model call. One call per
session, sixty per press at most, and the answer is capped at 120 tokens. The instruction is to
write twelve words at most, and to answer unclear from the opening prompt rather than
invent a subject. The lines live on disk keyed by each file's size and modification time, so a
transcript that has grown is offered again and its old line is marked stale; the tooltip names
the model that wrote it.
Two ceilings govern what you get to see. The listing is the three hundred newest transcripts, and it says so when that cap fired — a bigger store is never quietly reported as a store of three hundred. The content endpoint serves at most 128 MiB, so a transcript above that is disabled before the click, with the reason on the row rather than a status code after it. A refusal you still get means the file grew between the render and the press, and the server names both numbers. Nothing about an import is written to disk, and the bar over the replay says so along with the file's own counts: how many lines arrived, how many frames the view is built from, and how many lines nothing was read from.
What an imported transcript says
A Claude Code record carries the conversation and everything the client wrote down around it, and the second half used to reach nobody. It does now, and every reading below obeys the same two rules: a line that does not carry a field produces nothing — not an empty row, not a dash — and values travel verbatim, because each of these is a vocabulary somebody else extends without asking.
- The tool's own return value. A record with a tool result can carry a
toolUseResultbeside it, structured, while the block holds the flattened text the model was shown. AReadsheds the line-number gutter the block welds onto the body, and states the page that came back instead of counting the flattened text.Bashkeepsstdoutandstderrapart, which the block runs together with no marker. An edit'sstructuredPatchsays where the change landed, where the block says only that the file was updated, and a task update carries the state it moved out of. Two shapes are refused rather than half-read — a string, which is always an error the block already states, and an array, which is MCP output the block duplicates byte for byte. - Compaction. A boundary becomes the
compactionframe the wire has carried all along, counting the turns that went and how long the summary ran. The machine-written summary is out of the chat bubble, where it had been rendering as your own words — and it is now readable nowhere in the app, because every face hangs off a row and a line that produces no frame has no row. The bytes stay in the transcript on disk. Giving them a carrier is an open call rather than an oversight. - Failures that read as failures. An API error record, and the outage text
the client writes into the assistant channel, both become
errorframes. A retry ladder used to be a silent gap in the clock, and an overload message used to read as the model's own answer. - What a subagent cost. A modern transcript does not contain its own
children. Everything about one therefore comes off the single record the parent wrote when
the call came back: the child's token bill, which maps onto the wire's
usageevent under the child's own id, its wall time, its tool count, and whether it was launched in the background and never reported back inside this file. The model it ran on travels with it, verbatim down to the context-window suffix. It is read and never inherited: a child runs a different model from its parent often enough, and sometimes a whole tier down. - Who was driving. Chips on the trace row name the skill, the MCP server and its tool, the effort the turn was told to spend, who wrote a user turn when it was not a person, an answer that stopped on a limit rather than on its own ending, and a model swapped under the run. Names travel with their plugin prefix; spectroscope looks nothing up.
- Where the run stood. The working directory, the git branch and the client version are announced once and again at every move, each with what it left behind. Nearly two thirds of the session transcripts this was measured over stand in more than one directory, and every relative path in every tool result after a move means something else than it did before.
None of this touches the wire. These are readings of somebody else's file, and the readings with no home on the wire ride frames that are import-only — no writer in the Java or Python core constructs them, and they can never land in a session file.
The agents beside a session
A session transcript holds only its own start. Every word its agents said lives in sibling files, under a folder derived from the transcript's own name — direct spawns in it, and the agents of a workflow run one level below, under the run id. Opening a session asks the server which of them are there, and that costs one directory walk and no transcript read. The work panel lists what came back: how many sit beside this file, each row an agent id and a size, each openable as a session in its own right. Where the task's own claim and the files disagree, both are shown and neither is corrected.
On the chat and on the trace, buttons open the folders a recorded session left behind — the transcript's folder, the workflow folder, the scratchpad. These are the ones the app reads constantly and could never reach, because the store hides under a dot-folder and the scratchpad under a temp path nobody would guess. The server decides which of the three exist, since only it can stat them, and only the kind travels back: an endpoint that opened a path off the wire would be a way to run your file manager on any file on the machine.
A standalone agent transcript imports as a session too, and the shape decides rather than the name. A file whose every record is a sidechain, with nothing in it that could own them, is not a session with orphans in it — it is one agent's transcript, and it names that agent on every line. The bar says which agent, what kind of agent it was and which session it ran under, for as much of that as the file states. A copied or renamed file is read for what it holds.
The agents beside a session are a listing, not a join. Opening one replaces the view, and the parent's trace, spectrum and graph gain no child rows, because a real join would mean reading several thousand files to open one session. The folder is derived from the transcript's location, so a session split across two project directories answers “none”, indistinguishable from one that spawned nothing. And a pasted or file-picked transcript has no address in the store at all: no facts row, no gist, and no agents beside it.
The address of what you are looking at
There is no copy-link button anywhere in spectroscope, and that is the design: the browser's
address bar is the link, and the app keeps it current as you move. A stored session reads
#/session/<id>, with @<n> for the frame you are standing on
and a trailing segment for the tab — chat, spectrum, graph,
trace, text or lab, the six literals being the six words in
the tab row. A transcript opened from the store is an address as well,
#/import/<path>, the path percent-encoded so its own slashes cannot be misread
as the tab. A paste, a picked file and a scenario are views and not addresses: there is nothing on
disk for them to point at, so the bar falls back to the live default rather than invent an id. Two
tabs add how they are being read — the trace its open row and the categories left on, the
spectrum its zoom window as two fractions of the span. No other tab reports anything, so a lab or
a graph link says only which session and which tab.
Following an address plans the smallest change that gets you there. One naming the session
already on screen is a seek or a tab flip, never a refetch. Anything the app cannot believe — a
malformed frame index, an unknown tab, a filter category this build does not know — is dropped in
favour of what it does understand, because landing wrong is worse than landing home. Reading down
a trace or dragging a zoom replaces the history entry instead of pushing a new one, so
fifty readings of one place do not bury the place you came from. The tab row draws its own back
and forward chevrons, dark when there is nothing behind them, and ⌘←/⌘→ do the same. The desktop
shell is a bare window with no browser chrome, which is why a deep link is something you read in a
browser at localhost and not in the packaged app.
New chat semantics
One socket connection is one server-side session — one agent, one JSONL file. “New
chat” therefore closes and reopens the socket: fresh agent, fresh file, fresh state.
Two browser tabs are two fully independent sessions (separate agents, files, remembered
permission rules); only persisted rules in .spectro/settings.json are shared,
which is why that writer is serialized.
Chapter 9The design system
The whole UI reads its colours, fonts, radii and motion through CSS custom properties. That single indirection makes two things cheap: swapping the entire look (a design) and painting effects that always match.
The token base
The master vocabulary is brand-strict: espresso #17120D as the ground
(never pure black), the logo's amber line #CE9440 exclusively for
interaction and live elements (links, buttons, focus, running indicators, the
packet), violet #8B7CF0 as the editorial accent (eyebrows,
kickers — never clickable), desaturated status colours, and pastel agent accents that
are decoration only. Focus is always visible and always the accent; reduced motion
switches every animation off globally; numbers render tabular so nothing jitters
mid-stream.
Three designs
A design is a token override set — nothing structural changes. Selecting one flips a
data-design attribute on <html> and the UI re-expresses
itself instantly, no reload; a FOUC guard applies the saved design before first paint.
There used to be a shelf of extra skins; the picker was pruned to the three brand
designs, and any retired id left in older browser state folds back to the default.
| design | data-design | character | ground / accent | particles |
|---|---|---|---|---|
| spectro dark | spectroscope | espresso · amber line (the default) | #17120D / #CE9440 | brand dust |
| spectro bright | paper | LIGHT — paper · the logo blue | #F6F4EE / #2E7EA6 | ink dust |
| spectro white | still | LIGHT — minimal white, gray, one blue | #fbfbfd / #0071e3 | — (deliberately still) |
The two light designs flip color-scheme and re-derive
the status colours, agent pastels and the whole shadow set for a light ground — proof
that the token vocabulary carries a full polarity flip, not just a palette swap.
The effects layer
Two independently togglable effects, both reading the active design's colours live: the particle field (a canvas behind the UI with the brand dust drift — spectro white deliberately renders none) and the scroll reveal (panels fade/rise on entry). Both defer to reduced motion; particles pause in hidden tabs. A design choice from the settings page applies and persists in one step; the effect switches work the same way.
Chapter 10The desktop face
A JVM cannot live inside Electron's main process, so the desktop face is honest about it: a thin shell that supervises the real server as a child process and points a window at it. Packaged, it goes one step further and carries everything it supervises — the server jar and a whole Java runtime — so a fresh machine needs no system Java. Double-click, the server starts with it, the cockpit opens. The lesson of this face is process management, not IPC.
The supervision sequence
./spectro-app desktop prepares the ground (stops a stale instance — Electron's
single-instance lock would otherwise hand a relaunch to the old build; builds the
boot jar; clears the macOS quarantine flag from fresh Electron binaries on first run; puts
the resolved JDK on the PATH). The Electron main process then finds a free port, spawns
java -jar spectro-server.jar --server.port=P, polls /api/health
every 500 ms for up to 30 s, and opens a 1200×800 window on
http://127.0.0.1:P — the same React UI, talking over the same WebSocket. The
renderer gets no Node API at all (contextIsolation on,
nodeIntegration off): it is an ordinary web page.
A self-contained run kit
That dev flow expects a JDK on the machine. The packaged app does not.
scripts/build-desktop-runkit.sh bundles two things into the .dmg
as Electron extraResources: the server jar and a
jlink'd Java runtime (a trimmed JRE built straight from your JDK). Run
packaged, the shell resolves both from inside its own bundle — the jar from the app
resources, java from the bundled jre/bin/java — so a fresh
machine needs no system Java at all. Double-click the app, the server boots on the bundled
runtime, the cockpit window opens: the same React UI over the same WebSocket. (In dev the
shell falls back to the java on your PATH and the jar from the Gradle build.)
The app wears the brand — productName spectroscope, a brand app icon that the
build script regenerates from icon.svg when rsvg-convert is
present.
- First launch is a right-click. The default build is
ad-hoc signed (free, no Apple account). That clears the “is damaged” gate,
but on first download macOS still calls it an unidentified developer: right-click the
app → Open once (or
xattr -cr), and it launches normally ever after. - Zero-warning distribution is a paid path. A plain double-click with
no prompt needs an Apple Developer ID plus notarization. The bundled JRE is the one
twist — its dylibs and JIT need signing and the right entitlements. The full recipe,
end to end, is in
docs/DESKTOP-SIGNING.md. - Built for the host. The bundled runtime and the Electron binary are OS- and CPU-specific, so the script builds for the machine you run it on; a Windows, Linux, or Intel kit means running the build there.
Tray, notifications, quitting
- Closing the window does not quit. The tray (an ebony diamond) keeps shell and server alive so cron jobs keep running. Tray menu: New chat, Cron status, Quit.
- Native cron notifications. Every 30 s the shell polls
GET /api/jobs/stateand fires a desktop notification for each job whose status changed — click to open the window. - Quit is graceful, then firm. ⌘Q / tray-Quit sends the JVM a SIGTERM (Spring Boot closes sockets and finishes the current JSONL line); if the process still lives after five seconds, SIGKILL follows.
- Startup failures clean up. If health never turns green, the shell
kills the JVM before showing the error — no headless server left behind. In dev a
missing
javashows “Java 21 is required”; the packaged run kit carries its own runtime, so it cannot miss.
Chapter 11Two languages
The UI chrome ships fully bilingual — English (the default) and German — switched live by the header toggle. This guide's screenshots use the English chrome.
- Chrome only, content never. Sidebar, header, dialogs, panels, tool cards, Lab toolbar, the in-map SVG texts, trace chrome, settings page, footer — all switch live (~460 dictionary entries). Chat content, tool payloads and the JSONL wire never run through translation: a session keeps its own language.
- Wire values stay English. Plan statuses (
pending / in_progress / completed), A2A states and stop reasons are canonical English on the wire — only labels localize. Cross-edition compatibility depends on it. - Scenarios compile per language. Picking a scenario compiles it in the current chrome language; the compiled session then keeps that language, like any other session.
- Late-bound info lines. Reducer-folded info lines (spawns, compactions) carry an additive key + variables, so lines already in the transcript re-render when you flip the language.
The agent's powers
Everything the model may do, and everything that keeps it honest: the tool belt, the permission system, hooks, subagents, skills, providers, the senses — vision, voice, images — MCP and the scheduler.
Chapter 12The tool belt
Twenty-two built-in tools, plus one dynamic adapter per connected MCP tool.
Every tool obeys one iron contract: it never throws — failures come back
as a string starting ERROR:, which the model reads and self-corrects from.
Every model-supplied path passes the sandbox. Every mutation passes the gate.
The shared rules
The path sandbox
Every file tool resolves paths relative to the working directory and refuses anything
that normalizes outside it (ERROR: path is outside the working directory).
Tree-walking tools additionally verify each hit's real path (symlink-escape
guard) and never enter .git, build, node_modules,
target.
The caps
Files: 50 kB read/edit ceiling. Output: 10 000 chars shared clamp (surrogate-safe truncation). Glob: 200 results. Shell: 10 s timeout, pipe drained on a background thread — a timeout means “genuinely hung”, never “output too large”. Web: 512 kB streamed body cap before the text clip.
Files & search
read_filefreeReads a UTF-8 text file (≤ 50 kB) relative to the working directory. Try: “Read build.gradle.kts and tell me the dependencies.”
list_dirfreeLists a directory, sorted, directories marked with a trailing slash; an empty
directory answers (empty).
globfreeFinds files by glob pattern (**/*.java) under a directory — pure Java,
pruned walk, symlink-guarded, results sorted and capped at 200. Also granted to explore
subagents. Try: “Find every *.test.ts file under spectro-web.”
grepfreeSearches file contents by Java regex, returning path:line:text hits —
no shell involved, so it works even where run_command is denied. Skips
oversized and binary files; invalid regex answers ERROR: invalid regex: ….
Try: “Search for TODO across the Java sources.”
Mutation & shell
write_filegated · remembers pathWrites a text file, creating parent directories. Success:
Wrote: <path> (<n> bytes). An “always allow” remembers the full
path, never a blanket approval.
edit_filegated · remembers pathSurgical exact-string replacement. old_string must be unique
(ERROR: old_string is not unique (n matches) otherwise) unless
replace_all; zero matches and empty old_string error readably.
A denied edit writes nothing. Try: “In README.md, change the title X to Y.”
run_commandgated · remembers first tokenRuns a shell command in the working directory via /bin/sh -c: stderr
merged, output drained concurrently and capped, 10 s timeout, killed on abort.
Non-zero exit returns ERROR: exit code <n> plus the output —
exit-code feedback is what lets the model fix its own mistakes. An “always allow”
remembers only the first token (run_command:git*).
web_fetchgated · remembers URLFetches an http/https page and returns readable text (scripts/styles dropped, tags
stripped, entities decoded, whitespace collapsed, 10 k-char clip). Network egress on
model-chosen input is a side effect — hence the gate. Streaming fetch with finite 15 s
timeouts and a 512 kB body cap; non-2xx answers
ERROR: web_fetch got HTTP <status>. Main agent only.
Metadata & delegation
update_planfree · main onlyPublishes the agent's step list to the Plan panel (chapter 6) as the additive
plan event. Latest wins — each call replaces the whole plan. The status enum
(pending / in_progress / completed) is enforced: a model improvising
"done" gets an error instead of polluting the wire. Deliberately not given to
subagents — a worker's plan would clobber the display.
use_skillfreeReturns the full instructions of a named skill (chapter 15). Registered only when skills are installed; children get it too, so dev-tool workers can load their role skill.
generate_imagegatedGenerates an image via the configured image provider (chapter 17). Gated because it
is a paid cloud call. The image lands content-addressed under ~/.spectro/images/;
the model sees only a short confirmation — the user sees the gallery.
spawn_agent / spawn_agentsfree · parent onlyStarts one subagent (or up to four in parallel) and waits for the result — chapter 14. Free of a gate because the child's tools ask for permission individually; delegation is not an escape hatch.
build_plan · write_spec · develop · testfree · parent onlyThe four development role tools — thin wrappers over a worker spawn, each pointing its child at a skill (chapter 14).
report_statusfree · children onlyThe child's side of the A2A protocol: one short progress sentence per milestone, surfacing live in the parent's UI. Telling the parent what you do is never dangerous.
mcp__<server>__<tool>gated · dynamicEvery tool a connected MCP server advertises, wrapped as a first-class spectroscope tool (chapter 18). Always gated: its inputs are model output and its effects are external.
Who gets which tools
| agent | registry |
|---|---|
main | all of the above (21 named + MCP; use_skill when skills exist) |
worker-N | list_dir, read_file, write_file, run_command, edit_file, glob, grep, view_image, view_file, use_skill*, report_status |
explore-N | list_dir, read_file, glob, grep, report_status — read-only by construction: everything else simply is not in its registry |
* when skills are installed. Children never receive: spawn/dev tools (nesting depth 1 is structural), generate_image, web_fetch, web_search, browse_page, update_plan, MCP tools.
Chapter 13Permissions, allowlist & hooks
You stay in control through three layered mechanisms: the permission gate (a human decides), the allowlist (you decide once, scoped), and hooks (your scripts decide, before the gate even asks). All three leave an audit trail in the session file.
The guard pipeline
Every single tool call passes this pipeline, in this order — the order is load-bearing:
Permission modes
| mode | meaning | where |
|---|---|---|
ask | gated tools pause the run until a human decides (terminal y/N, web dialog) — with the allowlist answering first | default; the interactive faces |
readonly | every gated tool is denied automatically — the agent can look, not touch | headless default (spectro run, cron): “capability before convenience” |
auto | every gated tool is approved automatically | opt-in for unattended runs: --permissions auto / per-job "permissions": "auto" |
Even automatic policies emit permission_request/permission_decision
events — the JSONL always shows who allowed what.
The allowlist
Rules live under autoApprove in any settings layer and are consulted
before any human is asked. Since 0.9 every entry also names a tier —
what the tool it approves is allowed to DO:
{ "autoApprove": [
"grep#read", // approves every grep call
"run_command#eval-execute:git status*", // a prefix rule scopes the input as before
"write_file#write:docs/notes.md*",
"mcp__playwright__*#read" // a whole server's READERS, nothing above
] }
| tier | what it covers |
|---|---|
read | looks, never touches — a file read, a page read, a screenshot |
write | acts on the page, the app or the disk — input, navigation, a file write |
eval-execute | runs code — in the page, in a Node context, or on this machine |
Two rules carry the whole idea. An entry that names no tier approves
read and nothing above it. And a family wildcard has to name its
tier: mcp__playwright__* on its own approves nothing at all, because
a wildcard is exactly what somebody writes to stop a prompt storm — and before this,
one such line would have approved a Node-context eval beside the screenshots.
Where the tier comes from. Not from the server. Both MCP transports
pin protocol revision 2024-11-05, which carries no tool annotations at all
(readOnlyHint and its siblings arrived in 2025-03-26), so no
server here can rate its own tools even if it wanted to. The tier is a shipped, versioned
data file inside the jar — resources/permission/tool-tiers.json, one section
per blessed MCP server plus one for the built-in belt. Anything it does not name
is eval-execute and prompts, including every tool of a server the
map has never heard of. The honest cost: a server that grows a tool produces prompts on
the day it updates, and the fix is an entry in the next release rather than a config edit
on your machine.
What happened to the entries you already had. They were migrated once,
in place, each rewritten to carry the tier its tool held on the day of the migration — so
no entry started or stopped approving anything. The record of that pass, entry by entry,
is in ~/.spectro/gate-audit/allowlist-migration.jsonl. Only entries written
after the migration fall under the read-by-default rule. A pre-existing wildcard is left
untouched: it matched nothing under the old name-only gate, and stamping it with a tier
would have turned dead config into a live family grant.
The list has a screen. Settings → Auto-approvals lists every entry with
the tier it carries, per settings layer, says out loud which ones approve running code, and
adds or removes entries in your own settings. It renders what the gate decided and never
re-decides it: GET /api/settings/allowlist reads the entries out through the
gate's own parser and the shipped map.
Every gate decision is written down. One line per decision in
~/.spectro/gate-audit/<session-id>.gate.jsonl — the tool, the tier it
resolved to, which section of the map decided that, the map version, allow or deny, who
decided, and the raw entry that approved it. A sidecar, not a new event: the RunEvent wire
is byte-frozen. The call's input never enters the file — a command argument or a
URL routinely carries a credential.
Matching reads each tool's guarded field — the one input that defines its blast radius. The same table drives what an “always allow” click remembers, so a remembered rule can never silently go dark on the matching side:
| tool | guarded field | an “always allow” click stores |
|---|---|---|
run_command | command | the first token: run_command#eval-execute:git* |
write_file / edit_file | path | the full path: edit_file#write:src/Main.java* |
web_fetch | url | the full URL: web_fetch#read:https://example.com* |
| everything else | — | the bare tool name plus its tier |
Session-remembered rules live per connection (two tabs never share them). Ticking
Persist appends the rule to the session's workspace
project file (<workspace>/.spectro/settings.json) through a single
serialized writer that preserves every other key and dedupes — falling back to the
deprecated launch-dir .spectro/settings.json only for a session with no real
workspace yet. Because the CLI resolves the same workspace, it honours your web decision
there on its next run.
The net fence
web_fetch and browse_page take their address from model output,
and the model reads whatever page it was last shown. So the private world is refused before
a request leaves:
| refused | why |
|---|---|
file://, and every scheme but http/https | these tools reach the network, not the local disk |
| 10/8, 172.16/12, 192.168/16 | the private networks (RFC 1918) |
| 100.64/10 | the range a tailnet hands out |
| 169.254/16, fe80::/10, fc00::/7 | link-local (where a cloud metadata service lives) and unique-local |
| multicast, 255.255.255.255 | "every host on this segment" is not somewhere a page may send an agent |
| loopback | unless allowLocalhost is set — see below |
A host name is judged by where it points, not by how it looks:
localtest.me looks public and resolves to 127.0.0.1, so every name is resolved
and every answer is checked. A name that does not resolve at all is left to fail on its own
— the fence does not invent an answer.
How far the fence reaches, per tool
web_fetch is fenced at every hop. It follows redirects
itself — up to five — and each address in the chain goes to the fence before a request is
made. It has to work that way: the Java HTTP client follows same-protocol redirects on its
own, and a first build that asked the fence once let a public page answer 302
and hand back a LAN page and a tailnet page with the fence never consulted for either.
browse_page is fenced at the address you give it, and no
further. Once Chrome is running it follows its own redirects and runs the page's
JavaScript, which can navigate anywhere, and none of that comes back past the fence. Two
things would close it and both cost something real: pinning Chrome to a single host breaks
every page whose script lives on a CDN — which is the reason this tool exists — and running
Chrome through a fencing proxy closes it completely but is a component of its own. Until
one of them ships, treat browse_page as an entry check and prefer
web_fetch when the page does not need JavaScript.
allowLocalhost: true (Settings → Net fence) is the deliberate opt-in for the
local verify loop: this product on 8746, ollama on 11434. It opens loopback and nothing else
— not the LAN, not the tailnet, never a file URL, and not through a redirect either. It is
process-global: a workspace scope may not set it, because the workspace is the
folder the agent itself writes into, and a switch inside the sandbox is not a switch.
A refusal names the address and the rule and carries nothing else:
ERROR: browse_page refused 192.168.1.10:8746: it is a private network address,
RFC 1918 (rule: rfc1918). No path, no query string, no userinfo — a URL the model
assembled may carry a token in any of the three, and a refusal goes back to the model and
into the transcript.
web_search is deliberately not fenced: it dials the instance
you configured, not an address the model chose, and a SearXNG on localhost is the
normal case.
Hooks
Hooks are config-driven shell commands around every tool call — your policy, in your scripting language, versioned with the project. They come only from config, never from tool input: a pre-hook is arbitrary shell running before the gate, so it must never be model-controlled.
{ "hooks": [
{ "event": "pre_tool_use", // or post_tool_use — anything else fails LOUDLY
"matcher": "run_command", // tool-name glob: * (default), exact, or prefix*
"command": "./guard.sh", // any shell string
"timeoutSeconds": 5 } // optional, default 10
] }
SPECTRO_TOOL_NAME, SPECTRO_TOOL_INPUT (compact JSON) and — post only — SPECTRO_TOOL_RESULT; cwd is the agent's working directory.{"decision":"block","reason":"…"}. A block short-circuits: no dialog, no execution — the model receives ERROR: blocked by pre_tool_use hook: <reason>. First matching block wins.mcpServers: a settings layer that defines hooks replaces the entire block below it.In .spectro/settings.json:
{ "hooks": [ {
"event": "pre_tool_use", "matcher": "run_command",
"command": "case \"$SPECTRO_TOOL_INPUT\" in *'rm '*) echo '{\"decision\":\"block\",\"reason\":\"rm is off limits here\"}' ;; esac"
} ] }
Ask the agent to delete a file with rm and it instantly receives
ERROR: blocked by pre_tool_use hook: rm is off limits here — no permission
dialog, no execution. spectro doctor shows
hooks: 1 configured (pre_tool_use:run_command). Verified live.
The audit trail
Every decision is an event: permission_request (with the full input) and
permission_decision (allowed or not) land in the JSONL even for allowlist
auto-approvals; hook blocks are visible as their error result. The Trace tab's
permission category shows the whole history; the Lab's gate row turns green or red as
you step onto it.
Chapter 14Subagents & the A2A protocol
A subagent is not a special construct — it is another Agent
from the same core, with its own fresh context, its own reduced tool registry, and the
same permission gate and hooks as its parent. What makes spectroscope's delegation special is
that the protocol between the agents is visible: a task/status/result lifecycle
you can watch in every view.
Two roles
| role | purpose | tools |
|---|---|---|
explore | read-only research: map a directory, find usages, summarize | list_dir, read_file, glob, grep + report_status — read-only by construction |
worker | real subtasks that may change things | the full standard set + use_skill + report_status |
Children see only their task text — no conversation history — which is the
point: a fresh, focused context. Their final text is all the parent receives, wrapped
with the token cost so delegation visibly costs something:
[worker-1] result (tokens: 2.1k in / 0.4k out): …
ERROR: [worker-1] timeout after 120 s — cut the subtask smaller.); nesting depth 1 — children simply do not have the spawn tools.The A2A-lite lifecycle
Two additive events carry the protocol — the wire format needed no break. Every spawn
emits agent_spawn (child id, parent id, task) followed by an
agent_message with role task, state submitted.
While working, the child's permission-free report_status tool publishes
role-status messages (“working”) that surface live in the Agents panel, the
chat thread header and the Lab cards. At the end — on every path, including
timeout and cancellation — a role-result message closes the child with state
completed or failed.
Where you see it: chat nests child turns into pastel threads (chapter 5); the Graph
tab draws split/join blocks per wave (chapter 6); the Lab gives every child its own loop
card with its own packet (chapter 7); the Agents panel keeps the roster with token costs
(chapter 6); the CLI renders [worker-1]-prefixed lines (chapter 4).
The development tools
Four role tools wrap a worker spawn with a persona and a skill — no new agent types; specialization is prompt + skill:
| tool | persona | asks for skill | delegates |
|---|---|---|---|
build_plan | senior PLANNER | writing-plans | a feature request → a step-by-step implementation plan (a written document) |
write_spec | requirements ANALYST | brainstorming | a rough idea → a design/spec with decisions and trade-offs |
develop | IMPLEMENTER | test-driven-development | a development task, carried out in small verified steps |
test | TESTER — “Do NOT fix anything.” | verification | runs and inspects, reports evidence, changes nothing |
The child's prompt is the persona preamble + a standing instruction to report progress
via report_status + your task text; the A2A task message shows your
task with the tool name as its label — which is exactly how the UI badges the thread.
Try: “build_plan: draft a plan for adding CSV export to the sessions list.”
Read that third column as a request, not an inventory. The preamble says “if a
use_skill tool is available, load the skill 'writing-plans'
first”, and only verification is still seeded into a fresh home (chapter 15).
The other three live in the catalogue's superpowers pack, where an installed
copy answers to superpowers:writing-plans, so the bare name comes back as
ERROR: unknown skill 'writing-plans'. Available: … with the qualified names in
the list that follows. The child can take the hop itself — nothing rewrites the request
for it.
Chapter 15Skills
Skills extend the agent with data, not code: markdown instruction packages the agent loads when a task matches. The loop is untouched, no new event type exists — a skill use is an ordinary tool call you can see in every view.
The package format
~/.spectro/skills/<skill>/SKILL.md (user scope — a bare name) ~/.spectro/skills/<pack>/<skill>/SKILL.md (user scope, inside a pack — <pack>:<skill>) <project>/.spectro/skills/…/SKILL.md (project scope — wins by name) --- name: humanizer description: | Remove signs of AI-generated writing from text. Use when editing or reviewing text to make it sound more natural and human-written. Based on Wikipedia's comprehensive "Signs of AI writing" guide. Detects and fixes patterns including: inflated symbolism, promotional language, superficial -ing analyses, vague attributions, em dash overuse, rule of three, AI vocabulary words, passive voice, negative parallelisms, and filler phrases. --- # The skill body — full markdown instructions…
The frontmatter above is the catalogue's humanizer, cut after its
description. A directory under a root is a skill when it holds a SKILL.md. A
directory that holds no SKILL.md of its own but does hold skills is a
pack, and its skills are advertised as <pack>:<skill> —
the name the system prompt lists and the name use_skill answers to, not a
label printed beside one. The rule adds exactly one level and stops there: a skill three
directories deep is invisible, and everything installed before packs existed sits at the
top level and keeps its bare name. That is the whole purpose of the colon — somebody else's
brainstorming lands beside yours instead of on top of it.
Both roots are scanned in order, user first, project second, so a project skill of the
same name wins — the settings hierarchy again. A .disabled file inside a skill
folder hides that skill from the loader; the same marker on a pack folder skips the pack and
everything under it. A broken SKILL.md is skipped with one warning line: a bad
skill must never take the harness down.
Frontmatter is hand-parsed — two flat key: value lines do not earn a YAML
dependency — but flat is not the whole story. A value of |, |-,
> or >- opens a block scalar, and the indented lines under it
are folded into one, because both readers of that string put it on a single line: the
prompt's bullet and the settings row. Reading the marker alone was the bug 0.7.0 had to fix
before the catalogue could ship — the skill above reached the model as
- humanizer: |, unfindable, and its continuation line ending in a colon was
then read as a further key. One matching pair of surrounding quotes is stripped, since YAML
treats them as delimiters. Missing keys fall back: the name to the folder, the description
to the first non-blank body line, capped at 120 characters.
Progressive disclosure
Only the catalog — one name: description bullet per skill —
rides in the system prompt under a ## Skills header, with the standing
instruction to call use_skill before matching work. The body loads on
demand. That keeps the context small however many skills you install, and makes skill
usage visible: you can watch the agent fetch its instructions in the chat and trace.
It is also the reason the shelf described below is carried but not installed — every skill
under a root is a bullet in every run's prompt, and fifty-seven foreign bullets would be a
standing tax rather than a gift.
What a fresh home gets
One skill. verification — evidence before claims, report what you observed
in this run and nothing else — is copied out of the artifact into
~/.spectro/skills the first time a server starts, and that is the entire seed
set. Until 0.7.0 there were four; brainstorming, test-driven-development
and writing-plans left it because the catalogue carries versions between three
and five times longer, and only a name and a description ever reach the prompt anyway.
verification stayed because nothing on the shelf replaces it.
Seeding is deliberately timid, and both promises are pinned by tests. An existing folder
is never written over, so your edit survives every upgrade. And a .seeded
ledger in the skills root records every name ever offered, so a skill you deleted stays
deleted instead of reappearing on the next boot. Dropping three skills from the seed set
therefore removed nothing from any machine: an installation that already had them still
has them.
The bundled catalogue
Four collections of foreign skills ride inside every build as classpath resources under
skills-catalogue/, vendored verbatim at a named commit:
| pack | skills | vendored from | licence |
|---|---|---|---|
humanizer | 1 | github.com/blader/humanizer | MIT |
matt-pocock | 35 | github.com/mattpocock/skills | MIT |
superpowers | 14 | github.com/obra/superpowers | MIT |
ui-ux-pro-max | 7 | github.com/nextlevelbuilder/ui-ux-pro-max-skill | MIT |
Fifty-seven skills across those four packs — and both totals carry the commands that produced them rather than a memory of a release note:
$ find .spectro/skills-catalogue -mindepth 1 -maxdepth 1 -type d | wc -l
4
$ find .spectro/skills-catalogue -name SKILL.md | wc -l
57
The shelf is enumerated exactly the way that second command works: a skill is the
directory that holds a SKILL.md, found by walking the resources, so neither of the two
nesting depths the catalogue uses — <pack>/skills/<skill> and
<pack>/skills/<category>/<skill> — ever enters the code as a number. Each
row the settings page receives carries an opaque id (<pack>/<name>), the pack,
the description parsed out of the frontmatter exactly as the model would read it, the
licence, the upstream repo and commit, a file count and a byte count, plus whether that
skill is already on disk.
Every pack keeps its upstream LICENSE beside it and a
PROVENANCE.json naming the repo, the exact commit and the date it was taken —
MIT permits the copy, what it requires is that the notice travels with it. One pack goes
further because it had to: ui-ux-pro-max ships 54 font files, and a typeface's
licence is not covered by a repository's MIT statement, so they carry 27 SIL OFL notices of
their own, checked rather than assumed.
Installing one
The skills section of the settings page lists the shelf under Catalogue, below the skills you already have. Each row that is not installed yet ends in an install button, and its tooltip names the pack and the licence before you press it: “From superpowers, licensed MIT — LICENSE and PROVENANCE.json travel with it”. A row already on disk says installed instead. Nothing is fetched. The whole section works with the network unplugged.
The press posts the row's id to POST /api/skills/install as
{"skill": "<pack>/<name>"}, and that id is matched against the enumerated
index by string equality — never resolved as a path. The destination is not something the
caller can name: it is the pack and leaf folder the index already carries, which is why a
one-line request cannot become a write anywhere else. The copy is assembled in a staging
directory that is a sibling of the skills root, never inside it, because a
half-built folder already holds a SKILL.md and the loader would read it as a
pack mid-write. Files are written byte for byte out of the jar with no permission carried
across — thirteen catalogue files have an execute bit upstream and none of them arrives
runnable — and the finished folder is promoted with one atomic move. Beside the skill land
the pack's LICENSE, its PROVENANCE.json, and a
spectro-install.json recording pack, skill, repo, commit, licence and the date
you installed it.
| answer | when |
|---|---|
| 200 with the facts | installed — name, pack, files, bytes, licence, commit |
| 409 | already installed (“delete it first to install it again” — copying over it would lose your edits and its off switch), or the project already carries it and would win anyway |
| 413 | over the ceilings: 400 files or 24 MiB, with the measured numbers in the body |
| 400 / 404 | no id in the body; an id that is not on the shelf; any request from a non-local origin |
| 500 | the copy failed — including a pack whose LICENSE could not be read, which is refused rather than installed unlicensed |
One install runs at a time, and a refusal is said rather than swallowed: the panel prints the server's own sentence and deliberately does not reload the list, because redrawing rows after a failed copy is how a failure comes to look like a success.
What is installed is managed in the same list. The switch on each row writes or removes
the .disabled marker. The delete button is armed by the first click and deletes
on a second within four seconds, the same two-step the session archive uses, and it appears
only for user-scope skills — a project skill answers a project skill belongs to the
repo — remove it there. Deleting the last skill out of a pack takes the empty pack
folder with it. Every route comes in a
one-segment and a two-segment form (/api/skills/{name}/disabled and
/api/skills/{pack}/{name}/disabled, the same for DELETE), because
the pack is a real directory and travels as a real path segment; the colon in
superpowers:brainstorming is a display name, not a path.
Nothing is installed from a live git remote and there is no update check — each pack is pinned to the commit it was vendored at, and a skill that quietly moves under a run nobody re-read is worse than one that does not move. Re-installing is refused rather than treated as a no-op. The catalogue rides in the artifact whether you ever install from it or not. And spectroscope publishes nothing of its own to it: this is a client, not a registry.
/skills lists what is installed, under the names the model uses;
spectro doctor names the same set on one line, which on a fresh home reads
skills: 1 installed (verification); the System-context panel shows the catalog
exactly as the model sees it. All three read the library rather than the directory, so a
skill you switched off is missing from all three. Changes reach the next session
rather than the running one — the library is loaded once per connection, and the settings
panel says so above the list.
the fleet: many agents, one spectrum
Subagents fan out under one main agent. The fleet is the next level:
several full agents, each with its own model, tools and workspace, run in
parallel and merge into one spectrum. They can share a single JVM, or run
as separate processes on separate machines that report to a hub. Either
way you enter a fleet from the sidebar, and what opens is not a session
with more lanes in it — a fleet has a bar, a rail and a vocabulary of its
own. That is the spectro-orchestrator module (migration
phase 5), and its facade follows the frozen five-lines style exactly:
var panel = Spectro.panel().model(Anthropic.opus());
panel.agent("bugs").task("Find bugs in the diff");
panel.agent("perf").task("Check the hot queries");
for (RunEvent event : panel.run()) {
System.out.println(event); // every lane, one spectrum
}
Every lane is a complete core agent on its own virtual thread. The
panel announces each lane on the stream (agent_spawn, then a
task message), the lane reports in with a working status, its whole event
stream rides along verbatim, and a result message records completed or
failed before the panel's own run_end closes the spectrum.
One broken lane never kills the fleet; it fails loudly and alone.
what rides the bus
Under the surface, every event of every lane is wrapped in a bus
envelope: sender, contextId, taskId, a per-sender sequence, a causal
parentId chain, a topic for session isolation, and the RunEvent itself,
verbatim. Addressing is self-addressing: consumers correlate by taskId
within a contextId instead of from/to routing. The transport behind it is
a seam. In-memory serves a fleet inside one JVM; a process transport
(ProcessBus over a small TCP hub) carries a fleet spread
across processes and machines. The envelopes and their consumers are
identical either way, so nothing downstream can tell the two apart.
The bus publishes through the same tracing seam the JSONL sink uses: one registry in the core where the session file and the bus publisher hang side by side, durability first (JSONL before bus). A node's local session file is always the anchor; the bus is what makes it visible to the rest of the fleet.
spectro node: the process fleet
A fleet does not have to live in one program. The launcher runs one prompt as a fleet node whose whole event stream rides the bus to a hub:
spectro node -p "Review the input handling" \
--hub 127.0.0.1:7700 --context pr-42 --role security
The node runs headless, exactly like spectro run, and
publishes as it goes. --context names the fleet to join (the
bus topic derives from it) and is required; --hub host:port
(or $SPECTRO_HUB) names the hub. Options fill in the rest:
--id (default node-<pid>),
--role (default worker),
--permissions (readonly by default,
auto, or ask), --max-turns, and
--linger. The local session JSONL stays the durability
anchor, so a dead or stalled hub never blocks the run: events past the
buffers are dropped from the bus view only, counted and warned
loudly, while at-least-once still holds for everything the outbox
accepted. Each process start stamps a fresh wall-clock epoch, so a
restarted node reads as a new stream on the bus, delivered rather than
misread as a redelivery.
By default a node connects, runs once, and exits.
--linger keeps it connected and controllable after the run,
until a stop arrives over the hub (or, in production, SIGTERM).
--permissions ask parks each gated tool for an operator to
answer over the hub instead of a fixed yes or no (see the control plane
below).
A node can also be given a run loop, and that is the difference
between a node you can only stop and a node you can talk to. Three flags
do it: --watch <dir> runs the prompt again on changes
in that directory, non-recursively, with a burst of changes debounced
into one run; --listen <port> runs it on a POST to
/trigger at that port, where the bind is always
127.0.0.1 and the bearer token is printed at start; and
--every <n>ms|s|m|h runs it on a timer, at least a
second apart. None of them fires at start — the prompt of a triggered
node is written against an event, and running it with no event executes
half a sentence, so the node boots straight to waiting and says so. Plain
--linger gives no loop at all: that node parks on its stop
latch and has nowhere to put anything you send it.
code fleets on the hub
A Spectro.panel() fleet runs in one JVM by default and
returns one merged EventStream. Set $SPECTRO_HUB (or build
the panel with a hub address) and each lane also mirrors its frames to
the hub as a node, so a code fleet becomes live-visible in a running
cockpit and each lane arrives on the bus rail as its own card. The
invariant holds where it matters: the returned EventStream stays
byte-identical on the local in-memory spine, the hub is a purely additive
mirror, and a dead hub never delays the run's close.
entering a fleet
Fleets are first-class, like sessions. The sidebar's segment row
carries three of them — sessions, fleets and
state graph — and switching to fleets lists every fleet the
hub has seen: a row with its sigil, its context id, and its agent count
as online of total. A fleet with a parked gate floats to the top and
wears a pulsing gate open chip.
Click a row to enter it. A browser opened after the fact is not blind to
history: entering pulls each node's recent frames from the hub's replay
ring, so a gate parked before you arrived is still there waiting. The
ring is bounded; the node's own JSONL stays the full record.
With nothing entered, the segment lands on a lobby rather than on the
last session left standing, and the lobby's machine-dependent offers say
they cannot work before they are pressed rather than after. With
no hub running, the spawn button is disabled and the place where the
ready-made spectro node … command would stand carries the
sentence that names the variable it needs instead. With a hub up, the
lobby adds the part the server cannot answer: spawning also wants
SPECTRO_ALLOW_SPAWN, and the API does not report whether it
is set.
No hub at hand? The lobby's first offer is five recorded fleet
scenarios — deterministic, no hub, no key, no provider — that load as a
replay fleet and enter it. That is the fastest way to see the whole
surface before wiring a real one. A fleet also has an address,
#/fleet/{contextId}, and pasting one moves the sidebar with
it; the address carries the fleet and nothing else, no tab and no
reading, so it lands on the bus.
the fleet bar
While a fleet is entered, the app's tab row is replaced by the fleet's
own bar: bus, spectrum, trace, a
separator, one tab per agent, and a + that docks another
node. Spectrum and trace survive the swap because both read across the
whole fleet rather than one agent of it. Chat, graph, text and lab have
no fleet face and are not on the bar. Each agent tab carries a state dot
— working, failed, completed or idle, pulsing while a working agent is
still connected — and an agent with a decision parked marks its own
tab.
Picking an agent tab pins the trace filter with it, so the trace for one agent is one click away and already scoped. There is no way off the bar itself: leaving a fleet is a sidebar move, to a session, to the live row, or to another fleet. The back and forward chevrons are absent here, because the fleet bar stands where they live, though ⌘← and ⌘→ still walk your history.
Two views 0.6.1 offered have no way in any more. The fleet canvas — one spectral node per agent, spawn and task rails between them — was reached through the graph tab of an entered fleet, and the machine room, the whole fleet drawn as one agent-system diagram, through the lab tab. Neither tab is on the fleet bar. The machine room's fold survives where it matters: a bus card is built from exactly the same scene, with the same loop, gate and wiring chips on it. The canvas is out of the app entirely. The surface decision is not closed.
the rail and its cards
The bus is where an entered fleet opens. Every node docks on one rail as a full agent card, and the rail draws with nothing on it too. That is deliberate: the one surface that explains what this application is used to answer an empty fleet with a line of placeholder text, so it only appeared once it no longer needed to explain anything. Now the sentence sits on the rail — “No nodes on the bus yet.”, beside the button that docks one.
A card carries the id, the role and the provider; a restart marker
when the node came back with a new epoch, since a restart is a new stream
and never a silent continuation; an offline label when the
node has left; and the task line. Under that sit four chips for what the
agent is wired to — disk, shell, mcp and
llm, each naming the file, command, server or provider in
use while it is in use, the llm chip wearing a cloud mark
when that provider is remote — then a gate row while a decision is
parked, the agent's own spectral band, tokens in and out, and a stop
control.
Spawn ancestry orders the row and draws no tree. A spawned node docks
immediately right of its spawner and carries the parent id above it, but
the nodes stay peers on one line, because that is what they are on a bus.
Traffic rides the rail underneath them: every agent_message
is a mark between the two agents' drops, coloured by its role — task,
result or status — with its direction written under it, and the most
recent one animates from sender to receiver. The rail shows the last
twenty-four; the trace keeps every one.
The card head and the card's band are doors. The head opens that agent's own tab; the band opens the trace, already filtered to that agent.
one agent's tab
An agent tab shows that agent's activity as a feed: one row per event, with a timestamp relative to that agent's own first frame, the event type and a one-line preview, dot-coloured across eleven kinds of event. The feed follows the live edge as frames arrive. An agent that has done nothing yet says so instead of showing an empty pane.
talking to a node
Until 0.8.0 both composers were disabled inputs that said why on
hover, because the bus control op carried stop and
gate and nothing else. It carries words now.
One component draws the box on both faces — on every bus card, and at
the foot of an agent feed — so the two can never disagree about who may
be talked to. It is a text input with the placeholder “message this agent
…” and a → button; Enter sends, Shift+Enter does not. The
box is live only for a node that is connected and announced a
trigger, and the border is the tell: dashed is dead, solid is live. Those
two dead states are different problems with different fixes, so the
tooltip names which one you have — “the node is not connected”, or “this
node cannot take messages — it has no trigger, so it has no run loop.
Start it with a trigger to talk to it”.
A send posts to /api/fleet/{node}/message behind the same
local-origin fence as stop and the gate. The server dispatches a
ctl frame carrying your words to the node's live connection,
and the node's bus reader drops them into the fire slot its trigger loop
is already waiting on. The node simply runs again — the operator's words
are another thing that fires it, not a second machinery beside the
triggers — and the words stay out of stderr, because they are content and
that is not where content belongs. Messages that arrive faster than the
node fires merge in order rather than being refused, since the endpoint
already answered and nobody is left holding a retry.
The answer is one of four short sentences under the box, and none of them claims more than it knows. “sent — best-effort, no ack” is the 202: the words reached the node's connection, which is the most anyone can say, because the control plane has no acknowledgement. “the node cannot take that” is the 409 for a node with no run loop. “the node is gone” is the 404. “not sent” covers everything else, a request that never landed included, and a failed send keeps your draft rather than eating it. Stop is idempotent and its own note tells you to re-issue until the node leaves the roster; a message is not, and says so — sending it twice says it twice.
The bus wire version moved from 3 to 4 for this one verb, which is
worth the cost in writing. The gate form could stay on 3, because a
pre-gate node dispatches on the call id and simply never enters that
branch. A message has no such luck: every shipped version-3 handler reads
if ("stop".equals(action)), so a v3 node handed
action:"message" would have dropped the words with no log
line at either end — a 202 at the endpoint, silence at the node. The bump
makes the line unreadable to v3 rather than quietly misread by it.
the control plane
Watching is one thing; driving is another. The honest state is that
most driving still happens from the CLI (spectro node) or
from code (Spectro.panel()). What the cockpit adds is a
small set of first-class controls over the hub.
Spawn. Four affordances open the same form: the
+ at the end of the fleet bar, the “dock a node” button at
the end of the rail, the sidebar's + node once at least one
fleet is listed, and the lobby's own spawn button while none is — that
last one is how a first node gets started before there is a fleet to dock
into. The web spawn is deliberately read-only: the endpoint is reachable
from a browser, so it must never launch a node that writes or runs code,
and the server forces a UI-spawned node to --permissions
readonly rather than trusting the request. It is also opt-in
(SPECTRO_ALLOW_SPAWN, off by default) and needs a running
hub and a reachable node launcher, and because every refusal is a
uniform 404 the form names all three instead of pinning one. For a node with full rights it hands
you a copy-paste spectro node … command, built from the same
fields plus the live hub port, with an ask-or-auto toggle. You run that
in your own terminal. A replayed scenario fleet offers neither the bar's
+ nor the rail's dock button — there is no hub behind it to
dock into.
Stop. A connected agent card carries a
⊗ button that sends a stop to the node over the hub.
Best-effort, and it says so: the click confirms the verb was sent, not
that the node has stopped (the control plane has no ack). The node leaves
the roster when it actually ends; re-click if a stop was lost, and
SIGTERM is the ultimate backstop.
Answer a gate. A node started with
--permissions ask parks each gated tool instead of denying
it. The parked gate surfaces on the node's card, on its lane in the
fleet's spectrum, and in the same gate bar you use for your own session,
docked above the footer — on every tab of the entered fleet, the agent
tabs included, because that bar sits outside the view chain by
construction. Approve or deny inline and the answer travels to the node
over the hub. There is no “remember” here: a remote node keeps its own
allowlist, not yours. Like stop it is best-effort; if the node left
first, its own close denies the orphaned gate, so the gate clears either
way.
Every fleet write, and the roster read itself, is gated to a local origin: a loopback caller and a localhost Host header (the Host check is the DNS-rebinding defense). A remote or rebound page gets a 404 with no hint the endpoint exists. Spawn adds its opt-in flag and forces the node read-only on top. The bus carries prompts, reasoning and tool I/O, so its face is treated as sensitively as the ability to spawn.
A fleet is still not an object anyone creates. No endpoint makes one, and the only path to one is spawning a node into a hub that is already running behind two variables that both default to off. The message verb makes an existing fleet talkable; it does not make a fleet easier to have.
Chapter 15cThe state graph: drawn before the run
Every other view in this book is reconstruction — a run happened, events
arrived, and the picture was folded out of them afterwards. A state graph is the one
drawing that already exists before the first token, because its topology is fixed at
compile(), and the run only lights it up.
Why this one is not a reconstruction
The spectrum, the trace, the flow view, the machine room: each of them reads the same
event stream and folds a picture out of it, so each of them can only ever know what already
happened. A StateGraph is different in kind. You declare its nodes and edges in
your own code, compile() validates and freezes the topology, and the drawing is
knowable at that moment — before a node has run, before a token has moved. The sink receives
the topology before the compile call even returns. What the event stream adds afterwards is
light, not shape: nodes as they enter and leave, edges as they are taken, values as they are
written. The header of a loaded view states the whole idea in one line, on every frame:
“The topology is fixed at compile(), before the first token — so the graph is drawn first and
the event stream only lights it up. Observe without touching.”
The runtime lives in spectro-core, in the published library, not in the app.
The app never runs your graph and never asks a server for one; it reads two files your run
wrote and draws them. The footer of every loaded run says offline · no network
calls, which is the point rather than a disclaimer. One warning before you start: the
session tab called graph is a different thing entirely, and the last section of this
chapter separates the two.
Reaching it, and the address it does not have
The sidebar carries a segmented control with three buttons: sessions, fleets, and the
state graph. The third one is the whole surface. Clicking it hands the entire content area to
the view — no session tabs above it, because a graph is not a session. It needs no hub, no
provider and no key, and unlike the fleets segment beside it, the tutorial in chapter 6c
never closes it: this view reads two files off your disk and starts no process, so there is
nothing here for a level to protect you from. Under the segment sits a note that
says where a graph comes from: “A state graph sits beside the session as a pair of files, not
on the server — load it in the view on the right.” Below that, an eyebrow reading
Scenarios and five rows, one per bundled run.
Now the honest part. The state graph has no address. It is deliberately absent from the
router, so there is no #/stategraph to bookmark, no deep link into a record, and
no way to send a colleague the run you are looking at — the deep links that exist for trace rows
and spectrum windows stop at this door. A reload empties the pane, and it has to:
the browser cannot re-open a file from your disk without you picking it again. Exactly one
piece of view state survives, the horizontal-or-vertical orientation, kept in local storage
under spectroscope:sg-orientation and defaulting to horizontal. Cursor and picked
node reset, because the run they pointed into is gone.
Loading is a gesture, not a state. Reload the page or open a second tab, and the pane is empty again with its five scenarios on offer. A switch to sessions and back costs nothing — the run, the cursor and the pick are held above the pane, so the segment hands them back. If you want to keep what you are looking at, export it — two files, one click each, described at the end of this chapter.
Five bundled runs, under two sets of labels
The empty pane offers a load file … button and, under the line “or load a
scenario — real runs of the engine”, five chips. The sidebar rail offers the same five runs
one column to the left. They do not read the same, and that trips people: the rail prints
human titles, the shelf prints bare file stems in mono. Name the surface when you tell someone
what to click, because “click crag” and “click CRAG · corrective
loop” are instructions for two different places on one screen.
| rail title | shelf chip | what ran | values |
|---|---|---|---|
| Reference run · the measured template | crag-payload |
the widest of the five — ten nodes, fourteen edges, eleven supersteps | summary |
| Simple RAG · linear pipeline | simple-rag |
start, retrieve, rerank, generate, end — one line, no branch | sample |
| CRAG · corrective loop | crag |
a router, a grade that branches, and a rewrite that returns to the router | sample |
| ReAct tools · two turns, one thread | react-tools |
two runs filed under one thread id, with act returning to plan | summary |
| Failing run · an honest node_error | failing-run |
four nodes, three edges, and a generate that ends in an exception | summary |
All five were written by the engine itself in real runs, never typed by hand: the demo files are the output of a test that runs the graphs and keeps what they wrote. The two-turn scenario also shows a limitation the view states rather than hides — when one file pair carries several runs, the footer describes the first while the transport walks them all.
The pair of files, and why the shape alone is enough
A run writes two files beside itself. <stem>.graph.jsonl carries the
structure and the lifecycle — the topology record, graph start and end, node start, node end,
node error, and every edge taken. <stem>.state.jsonl carries what the nodes
wrote. The split is enforced by the writers and not by whoever calls them: the structure file
accepts those seven record types and nothing else, the values file accepts only value records,
and anything misfiled is dropped and counted rather than smuggled through. The counter surfaces
in the footer, next to the count of lines that were not JSON.
The naming is mechanical: the values path is the structure path with its family suffix
re-pointed, so run.graph.jsonl always pairs with run.state.jsonl and
never with run.graph.state.jsonl. The file picker takes both at once, and it is
worth selecting both, because picking only the values half of a pair with nothing drawn yet
earns the reply “That was a .state.jsonl on its own. Values need a shape — load the matching
.graph.jsonl with it.” The other way round is fine, and this is the sentence the empty pane
closes with: the shape alone is enough, the values are optional. A graph with no values file
draws completely, counts its supersteps, colours its edges and marks its loops; only the panel
goes quiet, saying “No .state.jsonl loaded — the shape is complete, only the values are
missing.” while the footer wears a no .state.jsonl chip.
Four tiers, and why the default is off
What a run may write into its values file is decided once, at compile time, and carried unchanged to every node. There are four tiers, and the default is the one that records nothing.
| tier | what lands in the values file |
|---|---|
off | nothing — and nothing is even built. This is the default. |
summary | only the ten channels on the allow list:
question, query_used, route, answer,
citations, confidence, abstained,
grade_ratio, rewrites, trace. Its promise is that no
document corpus reached the file. |
sample | every channel that is not denied, cut to the per-channel
caps — docs keeps its first three entries and clips the strings under it at
512 B, an unnamed channel falls back to 1024 B, and a whole record is capped
at 8192 B. |
full | every channel that is not denied, uncapped. It will not
construct unless you say so: full(false) throws. |
The reason for that default is worth a sentence, because it is a design decision and not an
oversight. A library upgrade that quietly started writing a user's document corpus to disk
would be a data incident, not a feature — so there is no ambient switch, no environment
variable and no settings toggle that turns recording on behind your back. Recording is switched
on by choosing a compile() overload, which means it is one visible line in your own
source, in a file you own, where you will find it again:
var app = graph.compile(sink, StatePolicy.summary());
A null policy or StatePolicy.off() records nothing; the lifecycle-only overload
compile(sink) still writes the structure file, so the drawing exists either way.
The tier is frozen for the length of the run and announced once, in a
state_policy record — the one line that later explains why a channel a node wrote
is missing from the file. Wherever recording happens at all, one channel is denied by default:
principal. And no policy field promises redaction: it says
patterns, never true, because patterns catch credential shapes and
cannot catch confidential prose.
None of the five bundled scenarios ships at the off tier — two record at
sample and three at summary, because a demo of nothing teaches
nothing. The tier you will meet on your own first run is the one the app cannot show you.
The transport, and the speed that is not one
Under the header sits the transport: five glyph buttons — |<,
<, play, >, >| — a speed select, a band of
ticks, a counter and a state chip. Only two of the buttons have a name to quote: the rewind
announces itself as “first record” and the play button as “play (space)”, flipping to a pause
glyph and “pause (space)” while it runs. The other three carry no title and no label, so this
book describes them by glyph, as you will have to as well.
The speed select offers 0.5×, 1×, 2×, 4×
and instant. The last of those is not a speed. It jumps straight to the last record
and pauses — a promise of no waiting, kept literally, because playing a four-hundred record run
at a nominal top speed is still half a minute of watching. The four real speeds divide the
recorded gap between two records by the speed and clamp the result into a 45–760 ms
window. That window is what keeps a run with a two-minute think from stalling the replay, and a
burst of ten records in one millisecond from flickering past. A pair carrying no usable
timestamps steps at 260 ms divided by the speed.
The band is one clickable tick per record, each tick titled with its record type and node,
so you can aim at a node error instead of hunting for it. Every manual seek — any button, any
tick — pauses playback first, because a transport that keeps running under your click is a
transport that argues with you. Pressing play on a finished replay starts it again from record
zero rather than doing nothing. The counter reads record N/M and the chip beside it
reads complete when the cursor stands on the whole run, mid-run
otherwise. That counter is the one string in this view that never translates; a German session
shows record 13/27 next to mitten im Lauf.
The keyboard is shorter than the toolbar: space toggles play, → and
← step one record, Home returns to the first record, and
End jumps to the whole run. Keys stay out of the way where they would do damage —
they are ignored while a modifier is held, while you are typing in a field, and entirely while
your focus sits inside the export dialog; space alone is ignored while focus sits inside the
right-hand panel.
Reading the drawing
The field is organised by rank. Every occupied rank gets a thin ruler drawn in the gutter
before its column, running far past the content in both directions, so the stages read as
separated columns instead of a cloud of boxes. Each ruler carries a lowercase label:
rank 0, rank 1, and so on. Each node card repeats its own rank
in its meta line as · r2, with an r and not an s, because
s means superstep everywhere else in this view and the two numbers are not the same
thing. The meta line also carries the duration in milliseconds once the node has one, and the
lifecycle word until then: never entered, running, done,
error.
Two badges read ×N. They run on different clocks, which is worth knowing before
you catch them disagreeing on screen. On a node card, ×2 means the run entered that
node twice — counted over the whole file, never scoped to where the cursor stands, so a card can
already say ×2 while the transport is still walking the first visit. On an edge,
×2 means that edge has been walked twice up to the cursor, and it climbs as
you play. An edge that closes a cycle is marked ↺ instead — but only while it has
been walked at most once. From the second walk on, the count replaces the marker, because a loop
you have taken three times is better described by the three than by the fact that it loops.
Colour on an edge is three-state and follows the same clock as the edge badge: a quiet arrowhead for an edge not yet taken, the success tint for one that has been walked, and the accent for the edge the cursor is standing on. The untaken edge is the detail to notice. It does not vanish when the run goes the other way; it dims and stays exactly where the compiler put it. The panel explains why, in the view's own words: “Every edge stays on the canvas, the untaken one included. It does not disappear, it steps back — that is what separates this view from a trace.” A trace can only show the path taken. A graph shows the branch that was available and declined, which is usually the more interesting half of the question.
The panel: what one node wrote at this visit
With nothing picked, the right-hand panel describes the cursor: a heading reading
current record over three rows — record, node, superstep, with an em dash where the
file does not carry one — followed by branches, known at compile() and the paragraph
about the edge that steps back. Click a node card and the panel switches to
node detail: the node id in mono, then lifecycle, rank, superstep, duration, bytes
(wrote 592 B) and entry count (2×).
The superstep row is the one that repays attention. It is the superstep of the visit the
cursor is standing in — not the node's last word. Before the cursor has reached the node the row
falls back to its first visit, and it is empty only for a node the run never entered. Everything
below it follows the same rule: the channels written chips, the heading
state · s1, and the values under it are what that node wrote at that
visit, not a merge of everything it ever wrote. A documents strip renders list channels as
numbered rows with an item count, and a clipped badge marks a value a cap cut short.
Where the recorder sampled a list, a badge says how many of how many entries were kept — taken
from the recorder's own count of the original, never from the number of rows on screen.
The most useful thing the panel does is refuse to be silent about absence. A channel a node
wrote but the policy did not record is printed in its own place, followed by
not recorded and the actual reason: recording is off, the channel was denied, or it
is not on the allow list. An empty box would teach you that the node wrote nothing, which is
false and expensive to believe.
A node that failed
A failure shows up in four places at once, and none of them is a shrug. The card takes an
error-tinted border, and its meta line prints error where a duration would be,
since a node that threw has no end to subtract from. In the panel, the lifecycle chip reads
error and an error box carries the exception class in bold on its own line, with the
message under it — for the bundled failing run, IllegalStateException over
“GroundednessError: no retrieved chunk supports an answer”. Both halves come straight from the
record; an earlier reader looked for a nested object, found none, and drew a failed node with no
reason at all.
The box is scoped to the visit the cursor is in: entering the node clears it, the error record sets it, and nothing looks forward. Scrub back to before the failure and the node looks healthy again, because at that record it was.
Export: two files, no options
The export button in the header opens a dialog titled export, with the source
file name as its subtitle, a close button, and Escape as the other way out. There is nothing to
configure — two groups, one download button each.
The SVG is the drawing at the current record, and it is self-contained: colours resolved to
literal values from the export theme, no external references, no fonts to fetch, the source name
as its title. It speaks the same visual language as the screen, rank rulers and lowercase rank
labels included, and it carries the same ×N and ↺ grammar, so a
picture pasted into a ticket does not need a legend. The Markdown is the run as text: a heading
naming the source, a bullet list of source, run and thread identity, node and edge counts,
supersteps and the recording policy, then a six-column table of every node — node, rank,
lifecycle, superstep, duration, channels written — in the order the drawing reads. Both file
names come from the source stem, so crag-payload.graph.jsonl leaves as
crag-payload.svg and crag-payload.md.
Why the layout is not dagre
The session graph tab is laid out by dagre; this drawing is laid out by code written for it.
That was not a preference. Dagre breaks cycles the way most layered layout
engines do — by reversing an offending edge, laying out the resulting acyclic graph, and
declaring the job done. Pointed at the corrective loop, it drew rewrite → router as
router → rewrite: the exact opposite of what ran, rendered cleanly and confidently.
A graph view that shows the reverse of the truth is worse than no graph view, because it is
believed.
What replaced it is a rank layout with gutter routing. A cycle-closing edge is marked as one and routed as a returning lane through the gutters, re-entering its target through the same face every other edge uses, so the arrowhead still points into the box and the direction on screen is the direction that ran. The lane costs vertical space, which is the honest price of drawing a loop as a loop. The session Graph tab keeps dagre and is welcome to it: that graph has no cycles to break.
Two things called graph
The app ships both words, and both of them stay, because they name two different jobs. Chapter 6 documents the session tab: a DAG of one run, one node per prompt, turn, tool call, subagent and answer, reconstructed from the events after they arrived. This chapter documents a segment beside sessions and fleets that draws a topology fixed before the run started. Renaming either one would only have made the other harder to find.
| the session tab named graph | the state graph segment | |
|---|---|---|
| where it lives | a tab inside one session | the third button of the sidebar's segmented control |
| what a node is | a prompt, a turn, a tool call, a subagent, an answer | a node you declared in your own code |
| where it comes from | folded out of the event stream after the fact | the topology record written at compile() |
| when it exists | after events arrive | before the first token |
| layout | React Flow with dagre | this project's own rank layout with gutter routing |
| cycles | none to draw | marked, routed as returning lanes, never reversed |
Chapter 16Providers & models
Everything the loop needs from a model sits behind one narrow port:
Iterable<ProviderEvent> stream(request). Swap the implementation and
the loop never notices, which is why spectroscope can switch between a cloud API and a local
model mid-session.
The backends
Eight provider ids resolve to three implementations. Six of them
(openai, lmstudio, llamacpp, openrouter,
gemini, spectro-local) share one wire protocol and differ in which
host they talk to, whether a key is attached, and which reasoning fields that host
understands.
| anthropic | ollama | openai-compatible | |
|---|---|---|---|
| provider id | anthropic | ollama | openai · lmstudio · llamacpp · openrouter · gemini · spectro-local |
| implementation | AnthropicProvider | OllamaProvider | OpenAiCompatProvider |
| transport | official SDK, SSE | NDJSON over POST /api/chat | SSE over POST /v1/chat/completions |
| endpoint | api.anthropic.com (fixed) | localhost:11434 (preset) | openai → api.openai.com; lmstudio → localhost:1234; llamacpp → localhost:8080; openrouter → openrouter.ai/api; gemini → the Google gateway; spectro-local → a llama-server spectroscope starts itself. The three local ids read their own …BaseUrl key first (chapter 18); for the rest a set baseUrl overrides the preset |
| key | ANTHROPIC_API_KEY | none | OPENAI_API_KEY (openai) · OPENROUTER_API_KEY (openrouter) · GEMINI_API_KEY (gemini) · none (lmstudio, llamacpp, spectro-local) |
| default model | claude-opus-4-8 | qwen3 | openai, lmstudio and llamacpp fall back to the placeholder local-model, which a local server ignores; spectro-local takes the bundled catalogue's default; gemini and openrouter have none, so name one |
| thinking | extended thinking (2048-token budget) | native message.thinking + inline <think> tag splitting | reasoning_content deltas split out (LM Studio, vLLM). Turning it OFF differs per id: openrouter takes reasoning.enabled, spectro-local takes chat_template_kwargs.enable_thinking, llamacpp takes both that and reasoning_effort: none, LM Studio takes nothing at all |
| quirks | usage + parsed tool inputs exist only after the stream ends | mints its own call ids; vision fail-fast for non-vision models | assembles fragmented tool-call deltas per index |
One wire, one endpoint per id, no guessing. spectroscope used to redirect a
keyless openai to a local LM Studio endpoint on your behalf. That guess is
gone: the provider id names the endpoint, the key only authenticates. openai
now always means the cloud. For a local OpenAI-compatible server, pick the id that matches
it: lmstudio for LM Studio, llamacpp for your own
llama-server, and for anything else (vLLM and friends) lmstudio
with an explicit baseUrl. Same wire either way — but not the same questions:
a llamacpp endpoint is asked GET /props for the window the loaded
model really has, where lmstudio is asked for a model listing a llama-server
does not serve. Pointing lmstudio at one works on the wire and loses that.
First run — choosing a backend
Out of the box the provider is anthropic with claude-opus-4-8,
which needs a key. On its first boot every face (spectro run, the web chat,
spectro node, spectro doctor, the server) checks whether a
~/.spectro/settings.json exists yet; when none does, it writes one from your
current environment base (whatever you exported, or put in ./.env), so day one
your settings file mirrors exactly what you already had and nothing changes behind your
back. With nothing set at all you land on the Anthropic default, and a run without a key
stops with a readable “no key” message instead of a mystery failure.
No Anthropic key on hand? Every local backend is keyless — and the first one needs no install either:
- built-in (
spectro-local) — pick a model in the app, it downloads once, done. The first-run sheet leads here, and chapter 16b is all about it. - ollama — install it, pull a model, and it serves
localhost:11434. Setprovider=ollama; no key. - lmstudio — run LM Studio in server mode with a model loaded, and it
serves
localhost:1234. Setprovider=lmstudio; no key. - llamacpp — start your own
llama-serverand it serveslocalhost:8080. Setprovider=llamacpp; no key. It carries the one model it was started with, and it gets asked what window that model is really loaded at.
The cloud backends that speak the OpenAI wire do need a key: openai
(OPENAI_API_KEY), openrouter (OPENROUTER_API_KEY) and
gemini (GEMINI_API_KEY).
Set the provider in ~/.spectro/settings.json, through
SPECTRO_PROVIDER, with a --provider flag, or from the header
picker. The CLI tour is the guided path: spectro tour → Settings
[s] picks a provider, takes the key hidden, and can save it to
./.env for you.
Putting a key in the .env
The simplest home for a secret is a ./.env file in the repo root: it is
gitignored, and spectroscope reads it as the environment base at startup. One
KEY=value per line, no quotes:
# ./.env — gitignored; the environment base layer ANTHROPIC_API_KEY=sk-ant-... # anthropic OPENAI_API_KEY=sk-... # openai (the cloud); also the image backend's key OPENROUTER_API_KEY=sk-or-... # openrouter GEMINI_API_KEY=... # gemini (the LLM and the image backend share it) # the local ids need no key: ollama, lmstudio, llamacpp, spectro-local
The .env is only the base layer: a value in
~/.spectro/settings.json (or a flag) wins over it, and
spectro doctor names any SPECTRO_* variable that is set and
shadowed, so a value that isn't taking effect never fools you. Keys are secrets: they never
enter a session file, the event stream, or the first-run settings seed. In the tour,
[w] “save key to .env” writes the file for you (gitignored, mode 600). The
image-generation keys (GEMINI_API_KEY, OPENAI_API_KEY) live in the
same file — chapter 17.
Switching mid-session
The header chip opens the picker, and it offers every provider id in the table at the
top of this chapter — the cloud backends, the OpenAI-compatible endpoints and the built-in
model alike. Nothing has to be chosen in settings to be reachable from there;
SPECTRO_PROVIDER, ~/.spectro/settings.json and the
--provider flag set the same thing before a session starts, which is the one
difference. The model dropdown is honest: for Ollama it lists
what is actually installed (live from /api/tags), for Anthropic what your key
can use (its Models API), for an OpenAI-compatible endpoint whatever /v1/models
returns, each with a curated fallback and “Custom model …” always available; an empty list
degrades to free text. Apply sends set_provider, and because the agent is built
once per connection (it carries your history), the switch swaps a delegate inside
the provider port instead of rebuilding anything. It takes effect on the next prompt,
history intact. Selecting Anthropic without a key is refused with a readable error, and the
map in the Lab re-places the LLM node inside or beyond the network boundary to match.
Retry — a loop that doesn't tip over
Transient failures (HTTP 408/409/425/429, any 5xx, dropped connections) are retried
with exponential backoff and jitter — but only before the first event of
a response. A stream that breaks mid-answer is never retried (that would duplicate
already-emitted text into the transcript); it surfaces as an honest error. Terminal
errors — 404 model-not-pulled, 401 bad key — never retry. spectroscope owns retry exclusively:
the Anthropic SDK's own retry is disabled so nothing double-retries.
Knobs: maxRetries (default 2, SPECTRO_MAX_RETRIES; 0 disables).
Prompt caching (Anthropic)
With promptCaching on (default), spectroscope places two cache_control
breakpoints per request: one covering the stable prefix (system prompt + tool list), one
on the last stable message — the one just before the current turn. Costs and
latency drop on every consecutive turn. The wire stays honest: the usage
event reports the provider's raw token counts, while the cache-read/-creation counts are
folded only into the compaction trigger — cached tokens still occupy the context
window, so the threshold must see them (chapter 28).
The wrapper chain
Agent loop
└─ TracingProvider (CLI --verbose only: the wire on stderr)
└─ SwitchableProvider (web face only: the mid-session swap point)
└─ RetryingProvider (all faces, unless maxRetries=0)
└─ AnthropicProvider | OllamaProvider | OpenAiCompatProvider
All wrapping happens at one chokepoint (providerFromConfig()), so the
CLI, the server, headless runs and the scheduler behave identically. The one
OpenAiCompatProvider serves all three OpenAI-compatible ids — only its endpoint and key
differ.
Chapter 16bThe built-in models
Every backend in the previous chapter is something you install
or pay for. spectro-local is the one that is already here: a small
catalogue of open models, driven by a llama-server this process
starts and supervises. It exists so a fresh install can produce a real run —
including real tool calls — before anyone has decided whether to pay for a
key.
Choosing a model
The picker entry reads built-in and needs no key at all. Applying it opens a chooser rather than switching blindly, because which model runs is a real decision now. Five models ship in the catalogue, smallest first:
| model | download | tools | thinking | good for |
|---|---|---|---|---|
| Qwen3 1.7B | 1.8 GB | yes | yes | quick answers on light machines |
| Qwen3 4B (default) | 2.5 GB | yes | yes | everyday chat and small agent tasks |
| Qwen2.5 Coder 7B | 4.7 GB | yes | no | coding and tool-driven agent work |
| Qwen3 8B | 5.0 GB | yes | yes | the deepest answers, on 16 GB machines |
| VibeThinker 3B | 1.9 GB | no | yes | watching a model think, without tools |
Under each row the dialog answers the question a spec sheet never does: can this machine hold it? The server reads total memory and the free space on the models volume, and the row says fits this machine, fits, but it will be tight, or names what is missing. A model that cannot fit cannot be chosen — the dialog disables the button rather than letting a five-gigabyte download start toward a full disk. (The REST endpoint underneath does not enforce it; the check is advice to the operator, not a lock on the API.) On a machine whose numbers the JVM cannot read, the check says so and steps aside instead of inventing a refusal.
One more thing the dialog checks: whether a llama-server exists
for this install at all. The desktop run kit bundles one, so there is nothing
to install; the bare server jar and the CLI do not, and there the chooser says
plainly that brew install llama.cpp is still needed — better than
a spawn that fails after five gigabytes have already been fetched.
Where a model comes from
Each catalogue entry is looked for in two places, in order: the app bundle
(a desktop build that ships a model sets spectro.bundle.models)
and then ~/.spectro/models/. Missing means the chooser offers the
download; nothing fails.
Every download is pinned: one URL, one sha256, one expected size. The hash is checked before the file is moved into place, so a download that was cut short or came back as something else is discarded instead of becoming the model your next boot loads. Four of the five come from Qwen's own repository; VibeThinker's GGUF is a community requantization of WeiboAI's model rather than WeiboAI's own file, which is why its row carries a licence note. Each row links both the licence and the source it downloads from. On the server side two different models can download at the same time without disturbing each other; the dialog itself runs one at a time.
The runtime underneath
Nothing runs until the built-in provider is actually used. On first use the
server picks a free loopback port, execs llama-server against the
chosen GGUF with that model's own context budget, and then polls
/v1/models until it answers. The budget is a generous sixty
seconds, because a cold multi-gigabyte load genuinely takes that long on a
laptop.
llama-server -m ~/.spectro/models/Qwen3-1.7B-Q8_0.gguf \
--host 127.0.0.1 --port <free> -c 8192 --jinja
One runtime serves every session in the process, and the handle hangs off a JVM shutdown hook, so quitting spectroscope takes the child with it. Switching to a different catalogue model shuts the old subprocess down and starts a fresh one — a llama-server keeps serving the weights it was started with, so a live runtime is only ever reused for the same model. From there the ordinary OpenAI-compatible provider does the talking — the same class that serves LM Studio and OpenRouter, aimed at localhost with no key.
The desktop app carries its own signed llama-server inside the
bundle (llama.cpp's official macOS build, eleven Mach-O files that reference
each other only through @loader_path, so nothing resolves out of a
Homebrew prefix that a user's machine may not have). Running the bare server
jar instead, you bring your own — brew install llama.cpp puts one
on the PATH.
Honest capability flags, per model
This is the part worth knowing before you judge an answer. Whether a model
speaks the OpenAI tool_calls protocol is a fact about the model,
not about the provider, and the catalogue records it per entry. The four Qwen
models speak it natively — the agent's file tools, shell and subagents all
work. VibeThinker does not: that came out of testing it rather than reading
its card — asked to use a tool, it writes the call out as prose, and it can
keep going in the think channel for a while afterwards.
So the loop is told the truth per model. A tool-capable entry is offered
the full registry; VibeThinker is marked reasoning yes,
nativeTools no, and tools are simply not advertised to it. The
alternative — offering tools that come back as text — would have produced a
stream of confusing half-failures and blamed them on the user's prompt. A
stale selection resolves to the catalogue default the same way the runtime
resolves it, so what runs and what is advertised never disagree.
A local model in these sizes stays below the cloud models: simpler answers, a smaller context. But the default Qwen3 4B genuinely drives the file tools while showing its thinking, and the first-use sheet tells you the limits of whichever model you picked. When the work gets real, the picker is one click away — and the chapter before this one is about everything else in it.
Everything downstream is unchanged by the choice. The events, the permission gate, the trace, the spectrum and the JSONL on disk are the same ones a cloud provider produces. That is exactly why the tutorial in chapter 6c runs from the dark frame to deep field without a paid key.
Chapter 17Seeing, hearing, speaking, painting
Four bonus-stage senses, all built on the same additive principles: binary data never enters the event stream, external engines sit behind injectable seams, and everything degrades readably when a piece is missing.
Vision — image input
- Web: attach images in the composer (picker or drag-and-drop) — they are downscaled client-side, stored content-addressed in the session's blob folder, and restored before the prompt text in the provider history (the position matters to the model).
- CLI:
spectro run -p "…" --image photo.png(repeatable). - Local models fail fast: Ollama models without the vision capability are refused with an actionable message (“use qwen3-vl or llava”) before any tokens are wasted; cloud models handle images natively.
- Events carry only references (
blobPath,sha256) — session files stay small andjq-able; a deleted blob degrades to a placeholder, never a crash.
Voice input — two routes to the same text
A recording becomes text on one of two routes, chosen the way the chat providers are.
The hosted route posts the clip to OpenAI's transcription endpoint
(gpt-transcribe at api.openai.com/v1/audio/transcriptions, keyed
by OPENAI_API_KEY — the same key image generation already uses) and needs
nothing installed, which is what makes dictation work in a downloaded app. The
local route runs whisper.cpp as a child process and sends nothing anywhere,
which is why it stays. Neither is ranked above the other: sttProvider takes
auto (the default — hosted when a key is there, local otherwise),
local, or openai, and an explicit choice wins
even when it cannot run right now, because naming what is missing is more use than a
silent reroute — and in the local direction a reroute would mean somebody's voice leaving
a machine whose owner asked for the offline path.
The local route's one-time setup is bash scripts/setup-stt.sh: whisper.cpp
through Homebrew on macOS (brew install whisper-cpp; on Linux the script names
the build and stops), then the ggml-small model, ~460 MB, against a pinned
SHA-256 — a mismatch deletes the file and aborts. ffmpeg left this path: the
browser converts its own recording now, so the app needs exactly one binary and whisper.cpp
is MIT. The CLI's /voice still records with ffmpeg, a job the browser cannot do
for it, which is why the script reports ffmpeg as absent and never requires it. Recording
for whisper is 16 kHz mono WAV, its one input format. sttLanguage rides both
routes — auto by default, so German dictation is not forced through English,
or de/en to pin it.
Settings → Speech to text shows the whole thing without a terminal: the route, the dictation language, the model file with a download button when it is absent, every binary with the path that proves it was found, and one true install line when it is not. There is a button only for the half an app can honestly do — the model is a checksummed download, the binaries belong to a package manager this app must not drive.
- REPL
/voice: record until Enter, transcribe, then an editable confirmation — Enter sends, typing replaces, blank discards. An STT error never reaches the agent unreviewed. The turn is audited by avoice_inputevent that never enters the provider history. - Web mic button: a toggle, not a held key — one press arms and
records (the tooltip flips to “Stop recording”, a level meter moves with your voice,
a “Recording 0:04” line counts up), a second press commits. The clip goes to
POST /api/transcribeand the transcript lands in the composer draft, never at the agent. Each route reports its own obstacle with a 503: “Speech-to-text is not installed — run bash scripts/setup-stt.sh.” locally, a missingOPENAI_API_KEYhosted. Telling somebody to run a shell script when what is missing is a key is the failure this split was built to end. - Before any of it: the first reach for the microphone opens a sheet titled “Speak instead of typing” instead of recording, so on the hosted route nobody's audio leaves the machine before a sentence has said that it will. A caret beside the glyph opens “Choose a microphone”, which lists “System default” and the named inputs — and before permission has been granted once, a sentence explaining why the browser is withholding the names rather than five blank rows.
- Seven named failures, seven sentences: denied, no device, device busy, STT missing, request failed, convert failed, unknown. Only “no microphone found” and “speech to text is not set up on this server” take the button away — a denial is an event, so somebody who then allows it in the site settings can press again without reloading.
Live transcription — text while you are still speaking
On the hosted route the text does not have to wait for the end of the recording. Tick
“Write along live” (“Live mitschreiben”) in Settings → Speech to text and the next
press opens a websocket to your own server at /ws/stt, which bridges to
OpenAI's realtime transcription session (gpt-live-transcribe) and holds the
key: the browser never talks to the provider directly. Audio goes up in pieces as it is
spoken, at 24 000 Hz, because the realtime session refuses 16 000 and
whisper reads nothing else — the rate is therefore a function of the route, decided before
the audio graph is built rather than discovered by being refused after somebody has already
spoken. Anything spoken before the session is configured is held and flushed in order, so
the cost is a few hundred milliseconds once instead of the first word of every take.
What arrives is two layers of text in one box. The textarea keeps the draft, crisp, with
its caret and its selection; a ghost layer under it repeats that draft in transparent ink
and continues it with the provisional words in faint ink, so the faded part begins exactly
where the typed text ends. Those words are unselectable, are not in the draft and cannot be
sent — only a final frame is committed. A guess is never one Enter away from
being sent as though somebody had typed it, which is the entire design.
The switch is off until you turn it on, and that is a decision rather than caution: a live session is metered by the minute where a clip is billed per clip. It is also never hidden and never quietly honoured. It carries a sentence in every one of its four states, greyed included, because a grey switch with no reason is a vanished button with extra steps: the local route “only writes once the recording is complete — it reads a finished file”, and a hosted route with no key says so and points at the providers page. Wanting live text is not consent to send the audio of somebody who chose the offline path off their machine, so no branch anywhere upgrades one route to the other to satisfy a setting. The server refuses the socket by those same two names, plus a third when the provider itself cannot be reached, and a refused session records nothing at all — no model was called and no audio left. Both paths stop at 300 seconds, and on the live one that ceiling is also the only thing that ends a metered session somebody walked away from mid-sentence.
One live socket is one exchange on the llm-wire record: the handshake is the request, every frame the provider sends is a response line kept verbatim with the provider's own type, so a reader can tell a partial from the answer without this code flattening one into the other. The asymmetry is named rather than discovered — the spoken bytes are not on the request line, because a websocket has no single request body and the audio flows after it. The record is complete about what the model said and incomplete about what it heard; the batch route, which posts one body, carries the audio itself.
Voice output — spoken answers
Setup: bash scripts/setup-tts.sh (piper + the en_US-lessac-medium
voice). Enable per run with --speak, at runtime with /speak on|off,
or permanently via "tts": { "enabled": true } in ~/.spectro/settings.json
(the legacy config.json name is still read beneath it for one release —
chapter 20).
The renderer buffers streamed text to sentence boundaries and speaks while the rest
still streams (synthesis runs up to three sentences ahead of strictly-ordered
playback). Code blocks are never read aloud — you hear “Code block skipped.” Abort stops
mid-sentence with no ghost audio. Ollama + piper is a fully offline agent that talks.
Image generation
The generate_image tool sits behind its own provider port with two
backends — Gemini (gemini-2.5-flash-image, GEMINI_API_KEY) and
OpenAI (gpt-image-1, OPENAI_API_KEY) — selected by
imageProvider / SPECTRO_IMAGE_PROVIDER or live from the gallery's
dropdown. Results land content-addressed under ~/.spectro/images/ (same bytes
= same file, shared across sessions); the additive image_generated event
feeds the gallery panel; the model itself only sees a short confirmation string. Gated,
because it is a paid cloud call. A missing key returns a readable
ERROR: the model can relay — never a crash.
Chapter 18MCP — external tool servers
spectroscope is an MCP client: plug in any Model-Context-Protocol server and its
tools become first-class spectroscope tools — gated, traced, replayable. No new event type was
needed: MCP calls are ordinary tool_call/tool_result lines, so
graph, trace and archive render them for free.
Configuration
The mcpServers block has the same shape Claude Desktop and Claude Code
use — a config written for those drops straight in:
{ "mcpServers": {
"notes": { // stdio: command (+ args, env)
"command": "/path/to/spectro-mcp-notes/bin/spectro-mcp-notes",
"args": ["/path/to/notes-directory"]
},
"search": { "url": "http://localhost:9000/mcp", "type": "sse" } // or HTTP/SSE
} }
Servers connect eagerly at startup. One that fails to start is logged and
skipped, and startup carries on with the rest. How long that costs you
depends on how it is broken: a command that does not exist fails in milliseconds, while
a command that does start and then goes silent waits out the
20-second handshake budget and then the shutdown that follows it: end of
stream, a signal, and force, capped at 5 more seconds. So
25 seconds is the most a silent server can cost, and that is the number
to hold us to. Measured here, an ordinary silent one is written off in 21.4 s, and one
that ignores every step of the shutdown in 23.5 s. Either way the entry is skipped, the
process tree it started is destroyed, and the line you get names the server and says what
went wrong:
mcp: notes UNREACHABLE at /usr/bin/notes-server — no answer to 'initialize'
within 20000 ms. Each advertised tool registers as
mcp__<server>__<tool>, always permission-gated: its inputs are
model output and its effects are external. /mcp lists servers and tools in
the REPL and spectro doctor probes reachability; the System-context panel in
the web face names the servers you configured, which is a different question. It
is read from the settings file, so a server that never answered is listed there too.
Call semantics: at-most-once
Every call is issued exactly once with a 20-second budget. If the transport
died between calls, it is re-established once, lazily. But a call that fails or times
out on an established transport is not retried — a slow-but-successful
add_note must not be doubled. Instead the transport is poisoned (the next
call reconnects) and the model receives a readable
ERROR: MCP tool '…' on server '…' failed: …. A dead or slow server degrades;
it never crashes the harness.
Every close runs the same steps, and their order is the point. First spectroscope asks the server's process tree who is in it, while there is still someone to ask. Then it closes the server's stdin. That end of stream is the ordinary way an MCP server is told to stop, and a server that flushes a cache or says goodbye upstream does it now. Then it waits a second — and asks the tree again, because a server given that second may spend it starting something, and a list drawn up before the goodbye cannot name what the goodbye caused. Only then does it destroy the server process and force whatever ignored that.
The last two steps are also what unparks a stuck read, which is why they are not
optional on a timeout. A read waiting on a child that has gone silent cannot be
cancelled: it ignores an interrupt, and it holds the stream's own lock, so closing the
stream to abandon the read only waits on the read. When the far end dies its pipe closes
and the read returns. Earlier versions closed the stream first, so a
server that spawned and then said nothing parked spectro doctor, the REPL
and any web session for good, and left the child running after the process that started
it was gone. Against a server that never speaks, the probe now reports it unreachable
inside the budget above, the rest of the output prints, and no orphan survives.
What gets destroyed is the whole tree, not only the process
spectroscope started. That matters more often than it sounds, because most published
servers are configured behind a launcher such as npx or a shell wrapper, and
then the thing that actually speaks JSON-RPC is a grandchild. Kill the launcher on its
own and the operating system hands the server to init, where it keeps running for as long
as the machine is up, one leftover per failed connection. This is also why the first count
comes before the goodbye: a launcher that exits when its stdin ends would otherwise be
gone before anyone asked it, taking the only list of its children with it. The second count
is the mirror of that, and it is why there are two: the grace exists so a server can do its
cleanup, cleanup work starts processes, and a count taken a second earlier cannot name
them. What stays out of reach is what nobody is left to be asked about — anything started
after that last count, and anything at all started by a launcher that had
already exited, because a process the operating system has reaped names no
children.
The example server: spectro-mcp-notes
The repository ships a complete, runnable MCP server as its own module — a stdio
JSON-RPC 2.0 program whose only dependency is Jackson (“no heavy MCP SDK, no Lucene”).
Build it with ./spectro-app mcp-notes. It serves two tools over a directory of
plain text files (default ~/.spectro/notes, six seed notes included):
search_notes— hand-rolled ranked full-text search (term frequency + substring bonus) returning snippets with their source files;add_note— appends a note as a new file, with collision-safe naming (two identical texts land in two files, never an overwrite).
Try (with the server configured): “Search my notes for 'permissions' and summarize what you find.” — the call passes the permission gate like any other tool, and the Lab's map lights the whole MCP chain: client → network stack → boundary → server.
Chapter 19Scheduling
The scheduler turns spectroscope into an unattended worker: jobs in a JSON file, 5-field cron expressions, honest state tracking and desktop notifications.
# ~/.spectro/jobs.json — an array of jobs, validated loudly on load
[ { "id": "morning-report",
"cron": "0 8 * * *",
"prompt": "Summarize yesterday's session files under ~/.spectro/sessions.",
"cwd": "/Users/you/projects/demo",
"permissions": "readonly" } ] // readonly (default) | auto
| command | behaviour |
|---|---|
spectroscope cron | the foreground daemon — arms every job, reschedules after each firing, Ctrl+C stops |
spectroscope cron list | each job with its expression, policy and next run time (explicit zone) |
spectroscope cron status | list + last outcome per job: time, ok/failed/skipped, stop reason, session id, result preview |
spectroscope cron --once <id> | run one job now; exit 0 only on ok |
- Overlap guard: a job still running when its next slot fires records a
skipped (overlap)state instead of stacking. - Headless policy: jobs default to
readonly; every run writes a normal session file (full audit) and its outcome into~/.spectro/jobs-state.json. - Notifications: macOS notification per finished job (terminal bell elsewhere); the desktop shell (chapter 10) adds native click-to-open notifications.
- Container-free: cron-utils computes the times, a single ScheduledExecutorService thread fires them — no Spring scheduling in the core.
The file system
Everything spectroscope writes and reads lives in two places: a per-user
home under ~/.spectro and an optional per-project .spectro/ folder.
This part maps every file — and opens one up.
Chapter 20Where everything lives
spectroscope keeps no database, no hidden caches, no binary state. Everything is
plain files you can read, version, back up and delete — most of it line-oriented JSON
that jq understands directly.
The user home: ~/.spectro
~/.spectro/ ├── settings.json user settings layer (optional) — provider, model, autoApprove, │ mcpServers, hooks, tts … every key in chapter 30. │ (config.json, the legacy name, is still read beneath it │ for one release — `spectro doctor --migrate` renames it) ├── sessions/ the session archive — the heart of it all │ ├── 20260716-091500-3fa4b2c1.jsonl one session = one append-only event file │ └── 20260716-091500-3fa4b2c1/ only when images were attached: │ └── blobs/ │ └── ca96adb5…f90157 attachment bytes, file name = sha256 ├── images/ generated images, shared across sessions │ └── 3f7a99c2…81d4.png content-addressed: <sha256>.<png|jpg|webp> ├── .env API KEYS ONLY, mode 0600 — written here when you save a │ key in the UI (chapter 33). Never a settings file's job. ├── leveling.json the tutorial: marks, receipts, the level-up history │ and the mode (chapter 6c) ├── skills/ user-scope skills (chapter 15) │ ├── .seeded ledger of what the first boot planted, so your edits │ │ survive and your deletions stay deleted │ └── my-skill/SKILL.md ├── models/ local model files — everything that runs without a key │ ├── qwen2.5-coder-7b-instruct-q4_k_m.gguf a built-in chat model — whichever │ │ entries you downloaded in the chooser land here (chapter 16b) │ ├── ggml-small.bin whisper.cpp STT model (~460 MB, SHA-pinned) │ ├── piper/piper TTS binary │ └── en_US-lessac-medium.onnx TTS voice ├── logs/spectroscope.log operator diagnostics, one [agentId] prefix per line; │ the level is a settings key, not a code change ├── notes/ the example MCP server's note store (one .txt per note) ├── jobs.json cron jobs — an array, hand-edited (chapter 19) └── jobs-state.json last outcome per job id — written by the scheduler
Directories appear on demand — a fresh install has none of them until
the first session, the first generated image, the first setup script. A home that has
done nothing but boot and answer one prompt holds five things:
settings.json, leveling.json, sessions/ with a
single .jsonl, the seeded skills/, and
logs/.
The launch-dir layer: <launch dir>/.spectro
<launch dir>/.spectro/ ├── settings.json the launch-dir settings layer — the directory spectroscope was │ STARTED in. Still read (deprecated compat layer, one │ release); its role as THE project file moved to the │ workspace's own .spectro/ pair below. The “Persist” │ checkbox appends here only when the session has no │ real workspace (throwaway temp desk). └── skills/ project-scope skills — win over user scope by name ├── brainstorming/SKILL.md ├── test-driven-development/SKILL.md ├── verification/SKILL.md └── writing-plans/SKILL.md
Two more files matter at the launch-dir root: the gitignored ./.env
(secrets only since the precedence flip — API keys, loaded by Gradle and
the launcher, never committed; deprecated SPECTRO_* lines still work as the env
base, but .env.example now documents just the keys) and an optional
SPECTRO.md, whose content is appended to the system prompt as project context —
the agent reads it before your first message.
Two .env files, one job between them. The launch-dir
./.env is the one you write by hand and the launcher loads;
~/.spectro/.env is the one the UI writes for you when you save a key in
the settings, at mode 0600. Both hold secrets and nothing else, both are read the
same way when a provider is built, and neither is ever a place for
provider or model — those belong in a settings file, where
they can be seen and diffed.
Inside a session file
The session id doubles as the file name: yyyyMMdd-HHmmss-<uuid8> —
sortable by start time, collision-free. One event per line, compact JSON, UTF-8,
append-only: the file is never rewritten, not even by compaction. This is a real session
from this machine (a thinking model answering “What is 2+2? Think first.” — 43 lines,
abbreviated in the middle):
{"type":"run_start","runId":"e93e24cf-f415-4873-a21c-cbd73fec673e","agentId":"main",
"prompt":"What is 2+2? Think first.","provider":"ollama","ts":1783086147207}
{"type":"turn_start","agentId":"main","turn":1,"ts":1783086147208}
{"type":"thinking_delta","agentId":"main","text":"The","ts":1783086147561}
{"type":"thinking_delta","agentId":"main","text":" user","ts":1783086147561}
… 30 more thinking deltas — the model reasons token by token …
{"type":"text_delta","agentId":"main","text":"2","ts":1783086148244}
{"type":"text_delta","agentId":"main","text":" +","ts":1783086148263}
{"type":"text_delta","agentId":"main","text":" 2","ts":1783086148282}
{"type":"text_delta","agentId":"main","text":" =","ts":1783086148301}
{"type":"text_delta","agentId":"main","text":" 4","ts":1783086148339}
{"type":"text_delta","agentId":"main","text":".","ts":1783086148358}
{"type":"usage","agentId":"main","inputTokens":297,"outputTokens":49,"ts":1783086148379}
{"type":"run_end","runId":"e93e24cf-…","stopReason":"end_turn","ts":1783086148379}
Everything you saw in Part II — chat, graph, trace, Lab, plan panel — is a fold over lines like these. A REPL session appends multiple runs into one file; resume appends to the same file again. Chapter 22 documents all 18 line types field by field.
Blobs and generated images
- Attachments (what you upload) live next to their session:
sessions/<id>/blobs/<sha256>. Content-addressed — attaching the same image twice stores one file. Events referenceblobPath+sha256only; a deleted blob degrades to a placeholder on replay, never a crash. Accepted types: jpeg, png, webp, gif. - Generated images (what the agent paints) are shared across sessions
in one global store:
~/.spectro/images/<sha256>.<ext>. The web UI loads them viaGET /api/images/<file>, where only64-hex.png|jpg|webpnames are even considered — the content address is the traversal guard.
The workspace: where the agent actually works
The session files above are spectroscope's records; the workspace is
the agent's desk — the directory the file tools sandbox against, glob/grep search
and run_command runs in. Unless a workspace is configured
(chapter 30), every session gets its own folder under the OS temp dir:
<tmpdir>/spectroscope-ws/ ├── 20260716-204301-db5f3af7/ one folder per session, keyed by the SESSION id │ └── add.py ← what “write me a script” produces lands HERE, └── 20260716-204419-39b41a29/ not in the repo you started spectroscope in
The key is the session id, so a resume finds its files again; a New
chat means a fresh desk. The web server announces the folder over the socket
(workspace_info — a UI-only frame, never in the JSONL) and the Files tab
follows it (chapter 6). The launch dir stays anchored: skills, MCP servers, SPECTRO.md and
its own (deprecated) settings layer still load from the directory spectroscope was started in —
while the workspace contributes its own settings pair on top (below).
Three ways to place the desk, highest wins: pick it per session in the
web UI (the Files tab's “Choose folder …” button opens the native macOS dialog on the
spectroscope machine; only before the first run, pinned in server memory for the session),
configure it (workspace key / SPECTRO_WORKSPACE /
--workspace — chapter 30), or let spectroscope mint the per-session
temp folder above.
A real workspace (picked or configured — not the throwaway temp desk) carries its own settings pair, the top file layers of the hierarchy (chapter 30) and the files the composer gear and the permission dialog's “Persist” checkbox write:
<workspace>/.spectro/ ├── settings.json the workspace PROJECT file — travels with that repo. │ Team conventions become checked-in settings: │ permissionMode, autoApprove, hooks, mcpServers … │ Persisted “always allow” rules land here. ├── settings.local.json machine-local overrides for THIS workspace — absolute │ paths, personal rules. Never committed: └── .gitignore spectroscope writes it on the first local save; it lists settings.local.json so the local file stays out of git
This pair is not just read, it is executed:
mcpServers spawns processes, hooks runs shell commands around
every tool call, and a permissive autoApprove/permissionMode can
auto-allow gates that would otherwise ask (chapter 30). Pinning a cloned or otherwise
foreign folder as your workspace means reviewing its .spectro/ first — the same
way you would review its build scripts before running them.
Lifecycle: create, resume, delete
- Created on the first prompt of a connection (web) or at REPL start; headless and cron runs write the same files.
- Resumed by
spectroscope --resume <id>or the web Resume button — the history is reconstructed from the file, new events append. - Deleted only by the guarded two-step (chapter 8):
SessionStore.deleteSessionremoves the JSONL and the blob folder — and only for ids resolving to a direct child of the sessions directory. Nothing else in spectroscope ever deletes or rewrites a session line. - Crash-safe by construction: one
writeper event, no open handle, no save step. A torn last line after a crash is silently discarded on read; everything that streamed is on disk.
The jq cookbook
Because the format keeps its rules (one object per line, no binary, camelCase), the shell is a first-class session browser:
# watch a session live
tail -f ~/.spectro/sessions/<id>.jsonl | jq -c '{type, agentId, name}'
# tool timings
jq -r 'select(.type=="tool_result") | "\(.durationMs)ms\t\(.callId)"' <id>.jsonl
# token totals
jq -s '[.[] | select(.type=="usage")]
| {in: map(.inputTokens)|add, out: map(.outputTokens)|add}' <id>.jsonl
# the agent tree
jq -r 'select(.type=="agent_spawn") | "\(.parentId) -> \(.agentId): \(.task)"' <id>.jsonl
Under the hood
The complete technical reference: the generated architecture dossier, every event on the wire, the provider port, the WebSocket and REST protocols, the loop internals, context management, MCP internals, every configuration key, and the build itself.
Chapter 21The architecture dossier
Nineteen generated diagrams cover the full build — every class name,
wire string, endpoint and count in them was read from this repository, and each comes
from a rerunnable Python generator (docs/diagrams/build_NN_*.py): change the
system, rerun the script, the diagram is current again. The shared visual language:
ocean outlines the core,
lilac the faces,
salmon the model side and external services,
moss the disk; sand carries the event wire and
stats; coral appears only where a human decides, a run is cancelled or the network
boundary is crossed.
The big picture
Chapter 1 already showed diagram 00 (one core, five faces). The build view below adds
the honest module structure: five JVM modules, two JS toolchains, and the one deliberate
cross-dependency — the server reuses the CLI's Transcriber for /api/transcribe.
The protocol layer
The model side
The loop and the belt
Delegation
The web face
Runtime and integration
The tutorial
The codebase itself
Chapter 22The RunEvent protocol
Eighteen event types, one sealed Java interface, one JSON line each.
This chapter is the complete wire reference — every field, every optionality, every
rule. The format is binding and shared byte-for-byte with the TypeScript edition
(docs/concept/JSONL-FORMAT.md is the contract document).
The wire rules
- snake_case types, camelCase fields.
"type":"tool_call"carriescallId, nevercall_id. - One line = one event, compact JSON, UTF-8,
\n-terminated. No arrays, no comments, no blank lines. - Append-only. A written line is never modified, reordered or deleted — compaction included.
- Additive evolution only. New types and new optional fields may appear; renaming, removing or reinterpreting is forbidden. Old files replay forever.
- Tolerant consumers. Unknown types and fields are skipped, never errors; an unparsable (torn) trailing line is silently discarded.
- Null optionals are absent. Optional fields are omitted from the JSON
entirely (
@JsonInclude(NON_NULL)), matching the TS edition. - Structured tool input.
inputtravels as a JSON object, never as a serialized string — consumers parse trees, never string-match JSON. - Only data. No binary, no object references, no cycles; every line round-trips losslessly.
sessionId = file name (yyyyMMdd-HHmmss-<uuid8>) · runId = UUID per run · agentId = "main" or explore-1/worker-2… · callId links tool_call ↔ permission_request/decision ↔ tool_result.ts = epoch milliseconds, monotone within a file (write order).The catalog
run_start| field | type | |
|---|---|---|
runId | string | UUID per run |
agentId | string | "main" or child id |
parentId | string? | only on child runs — the structural marker of a subagent |
prompt | string | the user/task text |
provider | string? additive | kept live-accurate across mid-session switches |
attachments | Attachment[]? additive | image references: {kind, mediaType, blobPath, sha256} — bytes never enter the event |
turn_startagentId, turn (1-based; the loop caps at 15). On resume a turn
boundary flushes the reconstruction buffer.
text_deltaagentId, text — one streamed chunk of the answer. Consecutive
deltas concatenate into the assistant text on resume.
thinking_deltaagentId, text — one chunk of the model's reasoning. A sibling of
text_delta but a separate stream: rendered apart, and it never re-enters
the provider history (the resume fold ignores it).
tool_callagentId, callId, name, input (JSON object).
MCP calls (mcp__server__tool) and skill uses are ordinary tool calls — no
dedicated types exist.
permission_requestagentId, callId, name, input — emitted when a
gated tool is about to run in ask mode; same callId as the pending call.
permission_decisioncallId, allowed (boolean). One of only two types without an
agentId — consumers route it back via callId. Emitted for
every decision, allowlist auto-approvals included (the audit trail).
tool_resultagentId, callId, output (always a string, clipped
surrogate-safely), isError (true iff the output starts ERROR: ),
durationMs.
agent_spawnagentId (the child), parentId, task. The tree edge —
the whole agent hierarchy lives in these two id fields.
compactionagentId, removedTurns, summaryChars. An audit line only:
compaction replaces the in-memory history — the file is never rewritten.
voice_inputagentId, durationMs, model (e.g. "ggml-small").
Written before the run_start of a spoken turn; pure provenance — a
resumed voice turn is byte-identical to a typed one.
usageagentId, inputTokens, outputTokens — per agent, raw
provider counts. The wire never includes cache arithmetic (that lives only in the
compaction trigger) — the token truth.
run_endrunId, stopReason — no agentId. Stop reasons:
end_turn (regular), unfinished (the run answered while its
plan still had open steps — the loop reads the ledger at the exit),
max_tokens, tool_use (edge case),
aborted (cancel/Ctrl+C), max_turns (the 15-turn brake),
error (preceded by an error event). A run that never wrote a
plan keeps end_turn: the absence of the plan event is the fact,
and nobody can grade a run that never said what it was doing.
erroragentId?, message — the failure channel, followed by
run_end {stopReason:"error"}. Also the web server's own error path (a refused
provider switch, a too-large attachment) — always a first-class event, never a separate
frame type.
image_generatedagentId, callId, prompt, provider,
model, mediaType, blobPath (relative to
~/.spectro, e.g. images/3f7a….png), sha256. The event
carries the reference; the bytes live in the content-addressed store.
context_infoagentId, turn, messages, estimatedTokens,
threshold, thresholdSource (override |
window | fallback — which fact produced the threshold; absent in
pre-0.9 sessions), parts[] ({label, chars, estTokens} — system
prompt / tool schemas / conversation). Emitted once per turn when introspection is on;
sizes are chars/4 estimates — usage stays the truth. Drives the
context ring.
agent_messagefrom, to, role, state, text,
label? — note the addressing is from/to, not agentId. Roles:
task (parent→child, state submitted), status
(child→parent, working, fed by report_status), result
(child→parent, completed/failed). label names the dev
tool (build_plan…) and is absent on plain spawns.
planagentId, steps[] ({text, status}). Latest-wins full
replacement from update_plan; statuses are the canonical English enum
pending / in_progress / completed, enforced at the write boundary.
The cross-edition proof
The invariant is tested, not asserted: CrossEditionReplayTest replays the
TypeScript edition's canonical example session verbatim — every line parses into a typed
event, re-serialization reproduces the other edition's bytes line for line,
and the resume fold reconstructs a usable history even though that file uses compact ids
(a0) instead of "main". Two interop hardenings came out of that
work and are now permanent behaviour: the main agent is found structurally (first
run_start without parentId), and text arriving after buffered tool
results flushes the reconstruction even when a foreign file omits turn_start.
Resume: lines become a conversation
Only four line types rebuild the provider history — everything else
(usage, permissions, thinking_delta, compaction,
voice_input…) is display and audit material. Child events never leak into the
history: their results are already inside the parent's tool_result. Orphaned
tool calls (a crash mid-round) are dropped because the provider API would reject the
history; attachment images are restored before their prompt text, at their
original position.
Chapter 23Streams & cancellation
The Java-21 answer to the TypeScript edition's async generators: a
blocking Iterable over a bounded queue, produced by a virtual thread, plus a
cooperative cancel signal. No reactive framework anywhere.
EventStream
public interface EventStream extends Iterable<RunEvent>, AutoCloseable {
void cancel();
@Override void close(); // idempotent; also cancels
static EventStream start(CancelSignal signal, Consumer<Consumer<RunEvent>> body) { … }
}
- Backpressure for free: a bounded queue (64) — a slow renderer blocks the producer instead of ballooning a buffer.
- The producer runs on a virtual thread named
spectroscope-agent; afinallyalways enqueues the end sentinel, so a consumer's for-each can never hang, even when the body throws. - The sentinel is a private instance recognized by reference identity — never handed out, never serialized.
close()also interrupts a producer parked on a full queue — try-with-resources cleans up even when a consumer bails early.
MergedEventStream
Subagents write into a shared merge: one unbounded queue, many producers (the parent pump plus one forwarder per child), exactly one consumer. Unbounded on purpose — child producers must never block behind a slow renderer. This is why A2A needs no network: the “protocol between agents” is just events interleaving on the one stream that every renderer and the session store already consume.
CancelSignal
public final class CancelSignal {
public synchronized void cancel(); // idempotent, runs listeners
public boolean isCancelled();
public synchronized Runnable onCancel(Runnable l); // returns a deregistration handle
}
The loop checks it at safe points; providers close their HTTP streams on it; running
shell commands are force-killed by a per-call listener that deregisters itself
afterwards (the handle return — a run's listener list must not grow per tool call).
onCancel fires immediately if already cancelled, closing the classic
registration race at spawn time. Children get their own signal, cascaded from the
parent's — one Ctrl+C ends the whole tree with
run_end {stopReason:"aborted"}, not a stack trace.
Chapter 24The provider wire
Between the agent loop and any model sits one sealed vocabulary. The loop consumes five event kinds and produces one request record — everything provider-specific stays inside the adapters.
The port
public interface LlmProvider {
Iterable<ProviderEvent> stream(ProviderRequest request); // blocking, lazy
default String providerName() { return null; } // live label for run_start
}
record ProviderRequest(String system, List<ProviderMessage> messages,
List<ToolSpec> tools, int maxTokens, boolean thinking,
CancelSignal signal)
record ProviderMessage(Role role /* USER | ASSISTANT */, List<ProviderContent> content)
sealed interface ProviderContent
permits TextContent, ToolCallContent, ToolResultContent, ImageContent
sealed interface ProviderEvent permits PTextDelta, PThinkingDelta, PToolCall, PUsage, PStop
record PUsage(int inputTokens, int outputTokens,
int cacheReadTokens, int cacheCreationTokens)
record PStop(StopReason reason /* END_TURN | TOOL_USE | MAX_TOKENS | ABORTED */)
Contract points: all three backends return lazy iterables (tokens reach the
UI while the HTTP stream is still open); PUsage.inputTokens stays the
provider's raw count (it feeds the wire usage event) while the two cache
fields ride separately for the compaction trigger only; a cancel mid-stream surfaces as
PStop(ABORTED), never as an exception.
Backend mappings
Anthropic (SSE via the official SDK)
- SDK client built with
maxRetries(0)— spectroscope owns retry. - Deltas translate one-to-one; usage and fully-parsed tool inputs exist only in
the accumulated final message — the classic SDK trap — so
PToolCalls,PUsageandPStopemit after the stream is exhausted. - Extended thinking with a 2048-token budget (clamped below
maxTokens); thinking requests carry no sampling parameters. - Vision: image blocks are reordered before text within a user message.
- Cancel closes the SDK stream; the resulting error maps to
ABORTEDiff the signal fired.
Ollama (NDJSON)
POST /api/chatread line by line; typed wire records (ChatRequest(model, stream, messages, tools, options, think)); tool results becomerole:"tool"messages; images ride as raw base64 arrays.- No call ids from the server — the adapter mints
ollama-call-<nanos>. - Token counts arrive only in the final
done:truechunk. - Thinking, two ways: the native
message.thinkingfield, plus a streaming state machine that splits inline<think>tags even when a tag is torn across chunk boundaries. - Vision fail-fast: before sending images,
POST /api/showchecks the model's capabilities — a text-only model is refused with an actionable message instead of a confusing 400. - Error classification at the source: retryable HTTP →
TransientProviderException; terminal (404 model-not-pulled, 401) →IllegalStateException— never retried.
OpenAI-compatible (SSE)
data:lines untildata: [DONE]; keep-alive comments skipped;stream_options.include_usageputs usage in the final chunk.- Fragmented tool calls accumulate per index until the turn closes;
arguments arrive as a JSON string and are re-parsed (unparseable →
{"raw": …}). - Bearer auth only when a key is present — LM Studio and friends run keyless.
- Endpoint by provider name, no key-based swap:
openai→https://api.openai.com,lmstudio→http://localhost:1234,llamacpp→http://localhost:8080,openrouter→https://openrouter.ai/api,gemini→ the Google gateway; an explicitbaseUrlalways wins, and the local ids read their own key first (chapter 18). One wire protocol, one host per id.
Chapter 25The WebSocket protocol
One endpoint (/ws), one connection per browser tab, one agent
per connection. Server→client traffic is nothing but serialized RunEvents —
there is no separate frame vocabulary in that direction. Client→server is six small
frames.
Client → server
| frame | shape | semantics |
|---|---|---|
user_message | {type, text, attachments?: [{mediaType, dataBase64}]} | starts a run (refused with an error event while one is active). Attachments are decoded and stored before the run starts; 5 MB cap per attachment; supported: jpeg/png/webp/gif |
permission_response | {type, callId, allowed, remember?, persist?} | answers a parked permission future. remember adds a session rule, persist appends it to .spectro/settings.json |
abort | {type} | fires the current run's CancelSignal |
set_provider | {type, provider, model?} | mid-session backend switch; validated; refused without a key; applies on the next run |
set_image_provider | {type, provider} | gemini | openai; effective on the next generation |
set_thinking | {type, enabled} | reasoning visibility; applies on the next run |
Nuances the names hide: the client frame is
permission_response — permission_decision is the RunEvent
that flows back and into the file. Cancel is spelled abort on the wire.
Resume is not a frame: it is the ?resume=<id> query parameter at
connect time (kept across auto-reconnects); archive replay is pure REST.
Server → client
Every event of a run goes to the socket and the session file — the same
object. Server-side errors (invalid frame, refused switch, oversized attachment) arrive
as first-class error RunEvents. One synchronized writer per connection; a
dead socket swallows sends — “a dead socket is not a run failure; the JSONL already has
it.”
Threading model
- The Tomcat thread only parses frames; every run executes on a virtual thread
(
spectroscope-run). - Permission waits park the agent's virtual thread on a
CompletableFuturekeyed bycallId— cheap, and the browser dialog answers with the same id. On disconnect every pending future completes with deny, so no thread ever hangs. - Two tabs = two connections = two agents, two session files, two independent remembered-rule lists. Only the persisted settings file is shared — its writer is serialized.
- Text frames up to 16 MB (base64 images need the headroom; the Spring default of 8 kB would close the socket with status 1009).
Chapter 26The REST API
The session stream is read-only over REST — the socket carries run
mutations — but a handful of control writes exist: the transcription POST, the guarded
session DELETE, the three settings PUTs, the workspace picker and the image copy-to-workspace
POST (plus the fleet control POSTs). Bound to 127.0.0.1; auth and TLS are a deliberate course
boundary. Every route below except GET /api/health passes a single filter first,
which wants a loopback peer and a localhost Host header: a page rebound
from some other name to 127.0.0.1 reaches the port but cannot forge that header, so it reads a
blank 404. The per-row notes name the extra guards each endpoint adds on top.
| endpoint | returns | notes & guards |
|---|---|---|
GET /api/health | {"status":"ok"} | the desktop shell's startup probe |
GET /api/sessions | [{id, startedAt, firstPrompt, tokens, provider}] | fold over the sessions dir; broken files skipped |
GET /api/sessions/{id}/events | the session's RunEvents as a JSON array | replay + resume seeding; torn last line dropped |
DELETE /api/sessions/{id} | 204 · 400 · 404 | the one destructive endpoint: id-shape guard ([A-Za-z0-9][A-Za-z0-9-]*) → 400; unknown → 404; the store additionally refuses ids that do not resolve to a direct child of the sessions dir. Removes JSONL + blob folder |
GET /api/config | {provider, model} | boot truth; live switches are client-side overlay |
GET /api/context | system prompt, tools, skills, MCP servers, thinking, provider/model, subagent role profiles | stateless — assembled from the same constants the live agent uses; no agent built, MCP not connected |
GET /api/models?provider= | [string] | ollama: live from /api/tags with finite 1.5 s/2.5 s timeouts (a stalled Ollama must not pin a worker); cloud: curated; errors → [] |
GET /api/images/{file} | image bytes | name must match [0-9a-f]{64}\.(png|jpg|webp) — 400 before any filesystem access; the content address is the traversal guard |
GET /api/jobs/state | {jobId: JobState} | cron outcomes; corrupt file → {}; polled by the desktop shell |
GET /api/files | the sandboxed workspace tree | that session’s resolved workspace only (no session, no tree); hidden + ignored dirs skipped; depth ≤ 8; ≤ 2000 entries with an honest truncated flag; local-origin fenced |
GET /api/file?path= | one file for preview | resolveInside compares REAL paths (no traversal, no symlink escape) + per-segment hidden/ignored re-check (the .env answers 404 even by direct URL); every document served with Content-Security-Policy: sandbox allow-scripts; 413 oversized, 415 binary; local-origin fenced |
POST /api/transcribe | {text} | browser audio → ffmpeg 16 kHz mono → whisper-cli; 503 with a setup hint when STT is not installed; the transcript goes to the composer, never to the agent |
GET /api/claude/transcripts | {limitBytes, truncated, transcripts: [{path, project, file, size, modifiedAt, loadable}]} | lists ~/.claude/projects/**/*.jsonl (newest first, ≤ 300 rows, truncated says when that cap fired); limitBytes is the ceiling the content endpoint enforces and loadable is its per-row verdict; local-origin fenced |
GET /api/claude/transcripts/content?path= | raw JSONL | must end .jsonl; canonical real-path must stay inside the real base (no traversal, no symlink escape); 128 MiB cap, streamed; local-origin fenced |
Chapter 26bThe llm-wire
The session file records what the app said about a run; this file records what the backend actually posted to a model and what came back, verbatim. One NDJSON sidecar per session, written at send time — a crash mid-generation still leaves the request on record.
The sidecar file
One file per session at ~/.spectro/llm-wire/<session-id>.llm.jsonl — beside
the sessions directory and deliberately not inside it, because the session list folds every
sessions/*.jsonl in full and a wire file can be orders of magnitude larger than the
session it records. The id is never trusted as a path: the one path rule the recorder and all
three read endpoints share refuses anything that is not a plain basename
([A-Za-z0-9][A-Za-z0-9-]*), so a ?resume= parameter cannot steer the
sidecar into another directory.
Voice happens before any session exists, so both speech routes share a day file named
stt-<yyyy-MM-dd>.llm.jsonl in the same folder — the batch transcription POST and
the live socket append to the same one. The sidecar is per session everywhere else, per day for
speech, and a reader hunting their dictation under a session id will not find it.
Every line is one atomic write. The recorder opens the file with O_APPEND, writes
the whole serialized line in a single channel write and closes again, holding no handle in
between, so two recorders on the same file — the speech day file, a doubly resumed session —
cannot interleave fragments mid-line. A write failure prints one line to stderr and is silent
afterwards: the record is an additive mirror of a run and must never be the thing that kills
it.
The ceiling is 256 MiB per file and it latches. The first payload that would cross it
writes a single llm_wire_truncated line carrying reachedBytes and
ceilingBytes; from then on every body is dropped and its line says
omitted:"ceiling" while the ledger keeps the measured bodyBytes — the
history of the calls survives even when their contents do not. The byte count is seeded from a
file that already exists, so a resumed session gets no fresh allowance, and a file already past
the ceiling latches without a second marker because the line that crossed it wrote one.
The three line types
An exchange is two lines. llm_request is written the moment the call leaves,
llm_response when it closes, and the two are joined by xid — a UUID,
never a position, because parallel subagents share one recorder and their lines interleave.
Serialization is NON_NULL throughout: an absent field is missing from the line, never
present as null.
llm_request| field | type | |
|---|---|---|
xid | string | UUID — the only join between the two halves of one exchange |
agentId | string | the calling agent; the provider itself knows neither agent nor turn |
turn | int? | 1-based; absent where no turn exists (speech) |
kind | string | chat | compaction | image | stt — the call site's knowledge, not the provider's |
provider · model | string | the backend label as the session knows it, and the model id the request names |
transport | string | who owned the socket |
method | string? | POST on every HTTP and SDK route; absent where no verb exists (the speech socket, the local process) |
url | string | the full request URL; a route with no host names itself there instead |
headers | object? | the headers as the provider set them, credential values redacted, key-sorted; absent when none were recorded |
fidelity | string | what the recorded body is worth |
body | string? | the request body, verbatim; absent past the ceiling |
bodyBytes | number | UTF-8 size of that body — measured whether or not the body survives |
omitted | string? | "ceiling", and nothing else today |
ts | number | epoch millis at send time |
llm_response| field | type | |
|---|---|---|
xid · agentId · turn | as on the request; the xid is what pairs them | |
status | int? | the HTTP status — absent when the connection never answered, and absent on a natural Anthropic finish, where the SDK exposes none |
fidelity | string | this half's own label, which may differ from the request's |
lines | string[]? | the received stream lines, verbatim, without their trailing newline |
body | string? | a single-payload response (image generation, an error detail) instead of lines — the two are mutually exclusive |
lineCount | int? | appears only when a streamed body was dropped at the ceiling; while the lines themselves are on the line, counting them is the reader's job |
bodyBytes | number | UTF-8 size of the body, or of the summed lines |
aborted | bool? | true when a cancel tore the stream down; absent otherwise, never false |
error | string? | the failure in one line; absent on a clean close |
durationMs | number | close minus send, computed from the request's own stamp |
omitted · ts | as on the request; ts is the close |
llm_wire_truncatedreachedBytes, ceilingBytes, ts — no xid, because
the marker belongs to the file and not to an exchange. It means “bodies stop here”, and the ledger
rows after it are still complete.
An aborted exchange records the lines that were consumed before the teardown. Bytes the server had sent but nothing had read are dropped on purpose — draining them to make the record look tidier would resurrect the slow-cancel bug that the abort path exists to avoid.
A finished exchange is written a third time, into the session file itself, as the additive
RunEvent llm_exchange — file first, socket second — so a session reopened next month
still knows every model call it made even when the sidecar has been deleted. It carries the
summary and no bodies. The llm_request frame is the counterexample and the one
exception to chapter 25's rule that server-to-client traffic is nothing but serialized RunEvents:
it is a hand-built frame, sent and never stored, because the only facts that exist at send time
are the ones that say nothing about how the call went.
Fidelity and transport
Fidelity is part of the record and never an implication. Each value is a separately measured promise about what the recorded string is worth, and the two halves of one exchange can carry different labels.
| fidelity | the promise |
|---|---|
bytes | the recorded string is what went over the socket — the harness's own HTTP providers post the exact string they hand the tap |
sdk-json | the SDK owned the socket and the recorded request body is byte-equal to what it posts, proven over loopback |
sdk-events | the stream reconstructed from the SDK's typed events: content-equal per event, field order not guaranteed, keep-alives absent |
encoded | the recording's own base64 of real input bytes that never rode a socket as a string — the speech audio |
Anthropic therefore writes sdk-json on the request and sdk-events on
the response, while Ollama and the OpenAI-compatible provider write bytes on both
halves: they own their connection and record the string they post. The detail pane prints one
honest sentence per fidelity, per side, rather than one verdict for the exchange.
transport is the second recorded label and names who owned the socket:
sdk a vendor SDK, http the harness's own client, websocket
the live speech route, process the local whisper-cli — whose URL is
spelled process://whisper-cli and whose trace row prints the executable instead of an
invented HTTP verb. No comment in the source lists all four; the field is the word the call site
chose, and those are the words on disk.
Always on, with one exception
There is no flag, no environment variable and no settings key. The web server opens a recorder wherever it mints a session store, fresh and resumed alike; the command line opens one beside its own store; the headless and cron runner does the same. Recording the backend conversation is what running spectroscope means, not a mode you enter — and the state-graph file that ships in the same release is the opposite case, where value recording stays off until you turn it on.
The exception is the library. AgentOptions.llmWire is nullable, and a provider that
gets no tap records nothing and behaves byte-identically to a build without the feature; the
pre-wire constructor arity still exists for exactly that. An embedder who wants the record hands
the builder an LlmWireRecorder, and a session id is all the recorder needs to place
the file.
What is redacted
Exactly one thing: the value of a credential header. Seven names are matched
case-insensitively — authorization, proxy-authorization,
x-api-key, api-key, x-goog-api-key, cookie,
set-cookie — and each match's value becomes the literal
REDACTED(n chars). Every other header passes through untouched, and the map
is sorted by key so the same request writes the same line twice.
The rule lives in the recorder and not in the adapters: a provider hands over the headers it
really set, real values included, so no adapter can forget to scrub. Two ordinary local setups
therefore record no credential at all — a keyless LM Studio sets nothing but
Content-Type, and the Anthropic path records no headers whatsoever because the SDK
owns the socket, which the detail pane says in those words: “no headers recorded — the SDK owns
the socket here.”
Bodies are recorded as they are: the full system prompt, the entire replayed history, every
tool schema, base64 images and documents, the compaction summary. A secret pasted into a prompt
is a secret in this file. The twelve pattern rules that do catch credentials by shape —
private keys, provider keys, bearer strings, JWTs and the rest — belong to a different file, the
state graph's .state.jsonl, where the clipper applies them to recorded values; they
never run over the llm-wire. Treat ~/.spectro/llm-wire/ as being exactly as
sensitive as the conversations it holds.
Reading a recorded exchange
There is no llm-wire tab. The surface is the session's trace tab, where a finished
exchange arrives as three consecutive rows: llm_request with an ↑ at the instant it
was sent, llm_response with a ↓ at the instant it closed, and
llm_exchange as the closing summary. Only the first and the third are lines anywhere
— the response row is manufactured for display, half a step in front of the summary, and exists in
no file.
Two filters find them. llm is a filter chip of its own rather than a fold into
other, and beside it sits an LLM-direction filter with three settings: to the LLM
(↑ request), from the LLM (↓ response), harness-internal (·). A vertical glyph means the row
actually left this machine; the dot means it never did.
An expanded llm row offers three faces where other rows offer four. Structured breaks the request into its parts — the system prompt with its character count, the messages with their per-message block counts, the tool schemas — and a body in a shape it does not recognize says so and hands over to the tree, because an empty pane would read as “the request was empty”. Insight is a tree over the parsed body. Wire prints the request line as HTTP writes it, then the recorded headers, then the body, then the response one numbered line at a time. Source is not offered for these three types: the endpoint re-serializes parsed nodes, so the line this frame was read from, byte for byte, is not a thing that exists here.
Two caps keep that pane readable and both announce themselves. The response stops at 200 lines with “showing X of Y response lines.” and a button for the rest. Base64 is never printed anywhere: each run of it is cut out and replaced by a mark carrying the length measured on the recorded line. The answer is not reassembled either — the reassembled text is the chat, and a second reassembly in the browser would be a second truth. Bodies never travel on the socket at all; the live frames carry metadata only and the pane fetches one exchange's bodies on the gesture that opens it.
Spectrum draws a second baseline under every agent lane, carrying that agent's exchanges on the same time axis as the app-protocol ticks above it, so a call sits under the moment that caused it. The two tracks stay apart on purpose: the app protocol and the conversation with the model are two stories about one moment. A lane whose agent never called a model is empty rather than absent, and a mark is coloured by what was measured and nothing else — it came back, or it never answered.
The whole file downloads from the archive bar of a stored session, as a plain link labelled
llm wire next to export, saving as
<session-id>.llm.jsonl. It is offered only when the index answered non-empty, so
the link never names an empty file, and it is offered only there — a live session shows no download
button, so open the stored session first or read the route directly.
The three reads
All three sit behind the same front door as the session reads of chapter 26: a loopback peer, a
localhost Host header and an Origin that is loopback or absent. Every
refusal answers 404 and explains nothing — a foreign caller, a malformed id and a session that
never called a model are indistinguishable from outside.
| endpoint | returns | notes & guards |
|---|---|---|
GET /api/sessions/{id}/llm-wire | the sidecar verbatim | the wire-record twin of the session export: application/x-ndjson, X-Content-Type-Options: nosniff, served as an attachment named <id>.llm.jsonl — recorded model traffic is caller-shaped text and must never reach an HTML parsing context |
GET /api/sessions/{id}/llm-wire/index | the bodiless ledger: one object per exchange, in file order | the two halves pair by xid while the file streams past; bodies and stream lines are read, counted and never returned, because one sidecar can dwarf the session it records. A torn tail line is skipped; an exchange still open reports status null |
GET /api/sessions/{id}/llm-wire/exchange/{xid} | {request, response}, both lines parsed | the drill-in behind one ledger row: xid must be UUID-shaped, and the scan stops as soon as both lines are found, so an early exchange in a huge file never costs a full read. response is null while the exchange is open |
DELETE /api/sessions/{id} | 204 · 400 · 404 | the session delete of chapter 26 cascades here unconditionally, not gated on the session file existing — which is what makes the speech day files and an orphaned sidecar deletable at all. 404 only when neither the session nor a sidecar was there |
What it does not record
The session-less endpoints — explain and translate — and the orchestrator's panel lanes are declared, not captured, and the interface says so rather than implying coverage it does not have. A live speech session is complete about what the model said and deliberately incomplete about what it heard: the spoken bytes ride the socket in pieces and are not on the request line, which the record names instead of hiding. What the file holds is the model traffic of a session — not every model call the application makes.
One finding is open, recorded rather than quietly carried: the header redaction prints the
credential's exact length. The number in REDACTED(n chars) narrows the search
for anyone who obtains the file, and it is taken with Java's String.length(), so it
counts UTF-16 code units and not UTF-8 bytes — equal for an ASCII key, unequal in general, and not
a byte count whatever it looks like. The graph layer, reviewed in the same week, already answers
with a coarse band instead — 1-8, 9-16, 17-32,
33-64, 65-128, 129+, measured in UTF-8 bytes — and its own
source names the exact length here as the reason. The wire is to follow; until it does, that number
is real.
Chapter 27The agent loop, hop by hop
The whole harness reduces to one loop in one class: build a request, stream the response, execute the tool calls behind the guards, feed the results back, repeat — at most fifteen times. Everything else in this book hangs off one of these hops.
The turn, precisely
- Run start. Fresh UUID,
run_startwith the live provider label (a switched backend reports itself truthfully). Attachments convert to image content before the prompt text in the first user message. - Per turn (1…15):
turn_start; with introspection on, acontext_infoestimate of what the next call will carry;- the compaction check (chapter 28) — a no-op below the threshold;
- the request: system prompt, full history copy, tool specs, maxTokens (default 32 000), thinking flag, the cancel signal;
- streaming: text deltas emit immediately; thinking deltas emit but never join the history; tool calls collect; usage updates the context measure; the stop reason records;
- abort check — a cancelled run ends here with
aborted; - history append: the assistant message (text + tool-call blocks) goes in before any results — the API's ordering rule;
- no tool calls → the exit reads the run's own
planledger andrun_endcarries the mapped stop reason — orunfinished, when the lastupdate_planstill had open steps and the mapped reason would have beenend_turn. Only that one value is displaced: a brake, a token cap or an abort already says a limit intervened, and the verdict must not eat the reason; - the tool round: every call passes the guard pipeline below, in order; all results of the round return to the model in one user message — denials and errors included, as data the model can read and react to.
- The plan is still open and the leash has budget → the harness writes
its own message naming the steps that are still open and starts another turn in
the same loop, recording
continuation {decision:"continued"}. Two things it will not do: continue a run that has nothing to show since the last continuation (decision:"no_progress"), and continue past its budget (decision:"budget_exhausted", and the run then ends asunfinished_after_continuations). A continuation never resets the turn counter, so the cap below is the whole ceiling. - The turn cap exceeded →
run_end {stopReason:"max_turns"}— the runaway brake.15as shipped, settable per agent. - Any exception →
error+run_end {stopReason:"error"}(orabortedif the signal fired). The stream terminates on every path.
The guard pipeline (per tool call)
Chapter 13 showed it from the user's side; this is the implementation order inside
runGuarded, and it is load-bearing:
pre_tool_usehooks — before anything else, so a policy veto costs no dialog. First matching block wins; timeout fails open.- The gate — only for
needsPermission()tools:permission_request→ the broker blocks the virtual thread (terminal read, or a parked future in the web) →permission_decision— always emitted, allowlist answers included. Denial returns the literalERROR: the user denied the execution. - Execution —
tool.execute(input, context); the context carries the sandbox root, the cancel signal, the caller's agent id, the call id and anemitsink through which artifact tools publish their additive events (plan,image_generated,agent_message). post_tool_usehooks — after the fact, advisory, never rewriting.
The headless variant
spectro run and cron jobs share a runner with a fixed system prompt for
unattended operation (“there is no human at the terminal: do not ask questions … if a
tool is denied, do not retry it”), a constant-policy broker (readonly denies, auto
allows), standard tools (never the spawn tools; the configured MCP servers mount only
when the headlessMcp setting or spectro run --mcp asks for
them — chapter 30 has the switch and its auto-approval warning), an external turn
brake, and a result record (final text, stop reason, session id, exit-ok). Every
headless run writes a normal session file — automation is fully auditable after the
fact.
Chapter 28Context: compaction, caching, introspection
Three mechanisms manage the finite context window: compaction shrinks the history when it grows past the threshold, prompt caching makes the stable prefix cheap, and introspection shows you both at work.
Compaction
- Trigger: the last turn's context size vs.
compactionThreshold— checked at the start of each turn. The threshold is derived when nobody sets one: three quarters of the window the backend says it loaded (LM Studio answersloaded_instances[].config.context_lengthon its own/api/v1/models, ollama answerscontext_lengthon/api/ps, and a llama.cpp server —llamacppor the built-in runtime, which is one — answersn_ctxonGET /props), asked once per run. The last quarter is kept back because the very next call after the trip is the summarizer, which re-sends the history and asks for a completion on top. A backend that states no window — anthropic, any OpenAI-compatible server of unknown make — keeps the old 100 000, and a server that answers "no such endpoint" is not asked again for the rest of the session. An explicit setting always wins — and costs no round trip at all, because there is nothing the backend could say that would change it. Children inherit that setting, so one number governs the whole tree. The measure is honest under caching:inputTokens + cacheRead + cacheCreation, because cached tokens still occupy the window even though the provider bills them separately. - Mechanism: everything except the last four messages is summarized by
the same provider (note-taker system prompt, summary-only instruction, no tools,
and a completion budget sized to the quarter of the window that was kept back —
on a model loaded with 8 192 tokens it asks for 2 048, not 32 000);
the new history is one summary message plus the recent four, with a cut repair so no
tool result loses its call. A failed summary is an
errorevent and an unchanged history — never a broken run. - The file never shrinks. A
compactionaudit line is appended; a later resume reconstructs the full history from the file and compacts again if needed — the file is always the unabridged truth. - Force it:
/compactin the REPL calls the same mechanism regardless of the threshold.
Prompt caching
Chapter 16 covered the two breakpoints (system + tools; the last stable message).
The reference detail: only text and tool-result blocks can carry a breakpoint (others
pass through, best effort); the index is recomputed fresh every call, so a history
reshaped by compaction simply gets a new breakpoint; and the wire usage
event never includes the fold — a cache hit must not change the bytes an old consumer
sees.
Introspection
With introspection on (both interactive faces enable it), every turn emits
context_info: message count, an estimated token total against the threshold,
and labeled parts — system prompt, tool schemas, conversation — each measured in chars
and estimated at chars/4. The web header's context ring renders it live
(green < 70 %, amber ≤ 90 %, red above); its popover shows the parts table
with the honest caveat that the estimate is an estimate — the usage line is
the truth. The “Context window” scenario (chapter 7) demonstrates the whole arc:
fill → warn → compact → continue.
Chapter 29MCP internals
Chapter 18 explained MCP for users; this is the wire level. The
dev.spectroscope.core.mcp package is container-free: hand-rolled JSON-RPC 2.0 records,
a transport seam, a client with strict delivery semantics, and a registry that treats
broken servers as skippable.
JSON-RPC framing
- Protocol version pinned:
2024-11-05; client infospectroscope / 1.0. - stdio transport (primary): one JSON object per line; stdout is
protocol, stderr goes to a per-server log file under the system temp dir. Handshake:
initialize→notifications/initialized→tools/list. Reads are bounded (20 s) by a watcher thread; a timed-out channel is poisoned — the uninterruptible reader is unblocked by closing the streams, and the channel refuses further requests until reconnect. - HTTP/SSE transport (optional): each JSON-RPC frame POSTs to the
configured URL; the reply's
data:lines are concatenated per the SSE spec (a plain JSON body is accepted too); 20 s connect/read timeouts; stateless. - Tool results: the
textof allcontent[]blocks joined by newlines; unexpected shapes return the raw JSON rather than failing.
Client semantics
- At-most-once:
tools/callis issued exactly once per tool call. Dead transport before the call → one lazy reconnect. Failure or timeout during the call → poison + readableERROR:, never a re-issue (a slow-but-successfuladd_notemust not run twice). - Registry policy: eager connect at startup, log-and-skip on failure,
descriptors cached, config order preserved; handles (name, target, reachable, tool
count) feed
spectro doctorand/mcp. Closed on shutdown. - Adapter hygiene: a server-supplied schema that is missing or not an
object is replaced by
{"type":"object"}— untrusted server output must not crash the provider advertisement. - Scope: MCP tools bind to the session at connect time, independent of LLM-provider switches; children never see them.
The reference server
spectro-mcp-notes doubles as executable documentation of the server side:
a single-threaded stdio loop answering initialize, tools/list,
tools/call; JSON-RPC errors -32601/-32603 where they
belong; tool-level failures as MCP-style isError results (still JSON-RPC
successes). Its store is a directory of text files with collision-safe creation; its
ranker a few dozen lines of term frequency + substring bonus — search as a small
program, not a search engine. Eighteen tests include one that spawns the real child JVM
and speaks the protocol end to end.
Chapter 30Configuration reference
Seven layers, 33 keys (three of them whole-block), plus a small handful of true environment-only secrets. Missing files are fine at every layer; malformed JSON fails loudly — a broken config is a programming error, not a fallback case.
The layers
built-in defaults
< environment SPECTRO_* (usually injected from ./.env —
now the BASE, directly above defaults)
< ~/.spectro/settings.json (user; config.json read beneath it,
for one release)
< <launch-dir>/.spectro/settings.json (deprecated compat layer)
< <workspace>/.spectro/settings.json (project — travels with the workspace)
< <workspace>/.spectro/settings.local.json (machine-local, gitignored)
< CLI flags --provider --model --base-url --compaction-threshold --workspace
The flip (owner decision 2026-07-18, "settings-productization"):
env used to sit just below the flags, outranking every settings file — editing
./.env was the loudest way to configure spectroscope, and a stale line nobody
remembered could silently shadow a deliberate choice made in the Settings page. Now env
is the BASE, one step above the built-in defaults: it still seeds the starting
configuration (a good fit for bootstrap, CI, or a one-shot experiment), but ANY settings
file — user, project or local — outranks it. "Die env ist die Basis, die Settings
geben den Ton an." Later layers win field by field — except autoApprove,
mcpServers and hooks, which merge as whole blocks:
a layer that defines the block replaces it entirely (no per-entry merge). Unknown JSON
keys are ignored everywhere, so additive fields never break older builds.
Two resolution moments. Config resolves at two different times, and
not every field is meaningful at both. The process moment (server boot,
spectro doctor, CLI startup, a stateless REST call) has no session yet, so
only defaults < env < user < launch-dir < flags applies — this
is where workspace itself resolves, and the one-per-process
logLevel. The session moment (the agent build, the composer gear, a
per-connection re-apply) joins the session's own resolved workspace pair — project, then
local — directly below launch-dir: two concurrent sessions with different workspaces can
legitimately run different providers or permission modes. A workspace scope may not set
workspace itself (a folder must not repoint the agent elsewhere) or
logLevel (one log file per process) — both fail loudly if you try.
Workspace-supplied config is executed, not just read. A workspace's own
.spectro/settings.json can set mcpServers, hooks,
autoApprove and permissionMode — and every one of those takes
effect the moment the workspace resolves: an mcpServers entry spawns its
process, a hooks entry runs a shell command around every tool call, and a
permissive autoApprove/permissionMode can auto-allow gates that
would otherwise ask. Pinning a folder as your workspace therefore runs whatever its
checked-in settings declare — the course maxim "tool inputs are model output and
therefore untrusted" extends one layer further to folder-supplied config is
untrusted too. Review a cloned or otherwise foreign repository's .spectro/
before pointing spectroscope at it, the same way you would before running its build scripts.
The settings API. GET /api/settings[?session=] returns
the resolved configuration alongside per-field provenance (which layer
won, which lower layers were shadowed) and the raw, non-empty layers — without
?session= it is the process-moment view, with one the session-relative view.
PUT /api/settings/{user|project|local} writes a schema-validated partial
patch to exactly one scope; a null value removes that key. Secret-shaped
keys (*_API_KEY, *_TOKEN) are always rejected — a settings file
is never a place to paste a key.
Two gears, two scopes. The header gear (the Settings page) edits the
GLOBAL, user-scope fields only: session defaults (provider/model/thinking/image backend),
the default workspace (origin badge + a "reset" that falls back to the layer below),
logLevel, and the machine-tool paths (chromeBinary /
imageModel / sttModel). A second, compact gear sits in the
composer row and edits THIS session's workspace: permission mode as a keyboard-navigable
text list (live immediately via set_permission_mode, persisted to the
workspace project file once one is pinned), always-allow rules (add and delete), machine-
local overrides, and raw-JSON editors for mcpServers/hooks. An
unpinned session — still in its throwaway temp folder — shows the composer gear's project
sections disabled: there is no durable folder to write to yet.
When a saved setting lands. Every tool that reads a setting reads it
again on the call, so saving one changes what the session you already have open does next:
the web_search tier (searxngUrl and the two search keys), the
image backend and imageModel, chromeBinary, the
allowLocalhost net fence that web_fetch, browse_page
and the browser and launch families share, and the dictation settings
(sttProvider, sttLanguage, sttModel), which the
transcription route reads per request. A call already in flight finishes on what it
started with; the next one asks again.
The one exception, and it is a control rather than a rule. The image
backend also has a dropdown in the composer row. Picking a backend there speaks to
the running session directly, and that choice outranks a saved
imageProvider until the session ends — so if you used it, the settings page
will not move this session's image backend, and the sentence under that dropdown on the
settings page says so. Nothing else on the page has a second live control like this.
(Where the configured backend has no API key and another one does, the backend with a key
is used. That is not a remembered choice: it is re-derived from the settings and the keys
on every call, so giving the configured backend a key ends it.)
Four things are settled when the agent is built and stay settled for that session,
because changing them mid-session would mean killing processes or rewriting a conversation
that already happened: the workspace, the MCP servers,
the shell hooks, and the system prompt with its skills
and SPECTRO.md/AGENTS.md. The allowlist
(autoApprove) is session-scoped too, and that one is a decision rather than a
cost: the belt reads settings live so the operator reaches the agent, and the allowlist is
what protects the operator from the agent — a run that may write files must not be able to
widen its own permissions between two tool calls. The OTLP exporter is
settled the same way: it is built where the session's store is minted.
Provider, model, the provider address and thinking are the third
case, and the one that reads as a contradiction until it is spelt out. Saving them
on this page applies from the next session — the agent is built with the provider it was
built with, and a run does not change backends underneath itself. The picker in the
header is the live path: it swaps the running provider through the same session, which
is a different door from the settings file. So "live via the picker" is true of the picker
and not of the page, and the page now says the page's half rather than the picker's.
Doctor never lets a shadowed variable go unnoticed. Every run,
spectro doctor walks the same provenance and prints one line for every
SPECTRO_* variable that IS set but is no longer the effective source, naming
both the variable and the settings layer that now wins:
env SPECTRO_MODEL is set but shadowed by user settings (effective model comes from user)
On first boot after the flip, if no user settings file exists yet, spectroscope materializes
one FROM the current env base (secrets excluded, only the fields the env actually sets) —
so day one changes nothing functionally, and from then on that file is the truth the
doctor line, the Settings page and every face agree on. spectro doctor --migrate
renames the old ~/.spectro/config.json to its new name, settings.json,
whenever the new name is not yet present.
Deprecation table
Every SPECTRO_* variable below still works exactly as before — as the env
BASE — but is deprecated in favor of the settings field it feeds:
| env variable (deprecated) | settings field |
|---|---|
SPECTRO_PROVIDER | provider |
SPECTRO_MODEL | model |
SPECTRO_BASE_URL | baseUrl |
SPECTRO_WORKSPACE | workspace |
SPECTRO_THINKING | thinking |
SPECTRO_IMAGE_PROVIDER | imageProvider |
SPECTRO_IMAGE_MODEL | imageModel |
SPECTRO_STT_MODEL | sttModel |
SPECTRO_STT_PROVIDER | sttProvider |
SPECTRO_STT_LANGUAGE | sttLanguage |
SPECTRO_CHROME | chromeBinary |
SPECTRO_MAX_RETRIES | maxRetries |
SPECTRO_PROMPT_CACHING | promptCaching |
SPECTRO_LOG_LEVEL | logLevel |
Not deprecated — these never had a settings-field counterpart: compactionThreshold
and permissionMode (aside from its live socket message) are flag/UI-only, no
env form ever existed for them; the API keys in "Environment-only variables" below are
true secrets and never graduate into a settings file.
Also not deprecated, because they arrived together with the fields they feed:
SPECTRO_OLLAMA_BASE_URL → ollamaBaseUrl and
SPECTRO_LMSTUDIO_BASE_URL → lmstudioBaseUrl, and
SPECTRO_LLAMACPP_BASE_URL → llamacppBaseUrl. They are the
environment form of the three per-provider addresses in the table below, and they sit
in the same env layer as everything above — a settings file still outranks them.
Every key
| key | type · default | env base (deprecated) | meaning |
|---|---|---|---|
provider | anthropic | ollama | openai | lmstudio | llamacpp | openrouter | gemini | spectro-local · anthropic | SPECTRO_PROVIDER | the LLM backend; unknown values fail loudly, and the message lists these same eight |
model | string · claude-opus-4-8 | SPECTRO_MODEL | when no layer set a model, the default follows the provider: ollama → qwen3; openai, lmstudio and llamacpp → local-model, a placeholder each of them ignores in favour of what is loaded; spectro-local → whichever model the bundled catalogue leads with; gemini and openrouter have no honest default at all, so one of the layers has to name a model |
baseUrl | URL · http://localhost:11434 | SPECTRO_BASE_URL | the shared FALLBACK address, and no longer the last word: ollama reads ollamaBaseUrl first, lmstudio reads lmstudioBaseUrl first and llamacpp reads llamacppBaseUrl first, whichever layer any of those values came from, so for those three this field only applies when their own is unset. Ignored by anthropic. For the remaining openai-compatible providers the old rule stands: a preset endpoint by name (openai → api.openai.com, openrouter → openrouter.ai/api, gemini → the Google gateway), no key-based swap, and an explicit URL wins. The default doubles as the "unset" sentinel — typing ollama's own http://localhost:11434 here reads as "nothing configured"; the three fields below carry no sentinel and are honored verbatim |
ollamaBaseUrl | URL · unset (preset http://localhost:11434) | SPECTRO_OLLAMA_BASE_URL | where ollama lives — the address the run, the live model list, the reachability probe and the doctor line all dial. Set it to put ollama on another machine (http://gpu-box:11434) without moving any other provider. No sentinel: only unset or blank means "not configured" |
lmstudioBaseUrl | URL · unset (preset http://localhost:1234) | SPECTRO_LMSTUDIO_BASE_URL | the same for LM Studio, kept apart from ollama's — one address per provider was the point: before this, pointing the shared field at LM Studio's port moved ollama there too, wrong port and all |
llamacppBaseUrl | URL · unset (preset http://localhost:8080) | SPECTRO_LLAMACPP_BASE_URL | where your own llama-server lives. The preset is llama.cpp's documented default port. A llama-server is not LM Studio wearing a different hat: it serves exactly the one model it was started with and ignores the model field in a request, and it answers GET /props with the context size that model is actually loaded at — so the harness reads the window instead of guessing it from a table. Pointing lmstudio at a llama-server works on the wire and loses both of those |
compactionThreshold | int · unset (derived; 100000 when nothing is known) | — (flag only) | compaction trigger in input tokens. Unset is not 100 000 any more: the harness asks the backend what window the loaded instance serves and compacts at three quarters of it, so a model loaded with 204 288 tokens is no longer summarized away at 100 000. Set it and your number wins outright |
permissionMode | ask | auto | readonly · ask | — | chapter 13; headless run has its own --permissions; live via the composer gear's set_permission_mode |
workspace | path · unset | SPECTRO_WORKSPACE / --workspace | the agent's working directory (file tools, glob/grep, run_command); unset = a per-session folder <tmpdir>/spectroscope-ws/<session-id> — a resume finds its files again, and the project you started spectroscope in stays clean; process-global — a workspace scope may not set this (a folder must not repoint the agent) |
autoApprove | string[] · [] | — | allowlist rules: tool#tier, tool#tier:prefix* or family*#tier (chapter 13) — whole-block; lives in the workspace project file once one is pinned |
allowLocalhost | bool · false | SPECTRO_ALLOW_LOCALHOST | the net fence's local-verify-loop opt-in: web_fetch/browse_page may reach loopback. Never widens to RFC 1918, the 100.64/10 tailnet or file:// — web_fetch fences every redirect hop, while browse_page is fenced only on the address you hand it (Chrome's own redirects and script navigation are outside it — chapter 13). Process-global: refused in a workspace scope, user scope only |
searxngUrl | URL · unset | SPECTRO_SEARXNG_URL | the SearXNG instance web_search prefers over every other tier (chapter 13). Process-global: refused in a workspace scope, user scope only — the searcher carries no net fence, so this field alone decides an address the tool dials |
imageProvider | gemini | openai · gemini | SPECTRO_IMAGE_PROVIDER | image-generation backend |
thinking | bool · true | SPECTRO_THINKING | reasoning-stream visibility (REPL /think, web header toggle) |
mcpServers | object · {} | — | Claude-Desktop-shaped server map — whole-block (chapter 18). The REPL and a web session mount every entry; a headless run mounts them only with the opt-in below or its --mcp flag |
headlessMcp | bool · false | SPECTRO_HEADLESS_MCP | the headless faces' MCP opt-in (card 220): when true, spectro run, every cron fire and every triggered fleet node mount the configured mcpServers the way the REPL does — read at each headless start, never by an interactive session. A manual spectro run overrides it per invocation with --mcp/--no-mcp. Under --permissions auto the opt-in approves every tool every configured server offers, unwatched. Process-global: refused in a workspace scope, user scope only — the switch that widens an unattended run must not live in the folder the agent writes into |
progressGuardWrites | int · 3 | — | the progress guard's first detector (card 262): how many DISTINCT earlier paths must already carry the exact bytes a write is about to repeat before the run says what it saw and asks the operator — carry on, change course, or end it. 0 turns the detector off. Cut from a measured hour in which a local model wrote the same 283 bytes to 31 paths while the harness said nothing. Content shorter than 64 characters is never compared, so a scaffold's empty __init__.py files cannot trip it. Process-global: refused in a workspace scope, user scope only — a folder the agent writes into must not be able to switch off the detector watching it |
progressGuardFailures | int · 3 | — | the second detector: how many times in a row ONE call with byte-identical input must fail before the same question is asked. The counter resets on any success of that same call, so a flaky test that fails twice and then passes stays silent. 0 turns it off. Process-global: refused in a workspace scope, user scope only — a folder the agent writes into must not be able to switch off the detector watching it |
progressGuardPlanTurns | int · 0 (off) | — | the third net: how many consecutive turns a plan may sit unchanged, with a step still open, before the same question is asked. Off as shipped — it needs a plan that exists and is maintained, and a model whose profile says it was not trained for tool use is handed no tool belt and writes none. A plan whose steps are all completed never stalls. Process-global: refused in a workspace scope, user scope only — a folder the agent writes into must not be able to switch off the detector watching it |
continuationBudget | int · 3 | — | how many times ONE run may be restarted by the harness after it stopped with its own plan still open (card 266). The harness writes the continuation itself and names the steps that are open, so no cooperation from the model is required — which is the point, because the models this was built for cannot call a tool to ask for another turn. It refuses to continue a run that has nothing to show since the last continuation, and it never resets the turn cap, so the real ceiling stays one number. 0 turns the leash off. Attended faces only: the browser session and the interactive REPL wire it, spectro run and cron do not |
maxTurns | int · 15 | — | the runaway-loop brake: how many turns ONE run may take before the harness ends it with stopReason: max_turns. It is a ceiling, not a detector — nothing was noticed, the run simply ran out of room, which is why a run that hits it stops without a closing word from the model. Counted per run, and a continuation (see continuationBudget above) never resets it, so this number is the real ceiling on one prompt. Card 266 made it an option and card 282 gave it this key; before that every browser session ran on the shipped 15 with nowhere to change it |
maxRetries | int · 2 | SPECTRO_MAX_RETRIES | transient retry budget; 0 disables; garbage values fail readably |
promptCaching | bool · true | SPECTRO_PROMPT_CACHING | Anthropic cache_control breakpoints; no-op elsewhere |
logLevel | error|warn|info|debug|trace · info | SPECTRO_LOG_LEVEL | operator-log detail (chapter 31); process-global — one log file per JVM, refused in a workspace scope, user scope only |
imageModel | string · unset | SPECTRO_IMAGE_MODEL | overrides the image backend's own default model; machine-tool path — belongs in user or workspace-local scope |
sttModel | path · unset | SPECTRO_STT_MODEL | absolute path to the whisper.cpp model (default ~/.spectro/models/ggml-small.bin); machine-tool path |
sttProvider | auto | local | openai · auto | SPECTRO_STT_PROVIDER | which way speech to text goes (chapter 17): auto is hosted when a key is there and local otherwise; an explicit choice is never rerouted, even when it cannot run |
sttLanguage | auto | de | en · auto | SPECTRO_STT_LANGUAGE | the language dictation is transcribed in, on both routes; auto lets the model detect it |
chromeBinary | path · unset | SPECTRO_CHROME | override for the system-Chrome binary browse_page discovers automatically; machine-tool path. Process-global: refused in a workspace scope, user scope only — it names an executable browse_page launches, and the workspace is the folder the agent itself writes into |
hooks | array · [] | — | pre/post tool hooks — whole-block (chapter 13) |
otlpEndpoint | URL · unset | SPECTRO_OTLP_ENDPOINT | the OTLP/HTTP traces endpoint; unset = the exporter is off (see Observability below) |
otlpBasicAuth | pk:sk · unset | SPECTRO_OTLP_BASIC_AUTH | optional Basic-auth pair for the OTLP endpoint (Langfuse's public:secret); never echoed by any read |
tts | {enabled, voice} · off, en_US-lessac-medium | — | CLI-side voice output block in the same user settings file (not a SpectroConfig field) |
Every SPECTRO_* name in the "env base" column still works exactly as
before, but only as the BASE layer — see the deprecation table above for the field each
one now defers to, and run spectro doctor to see which of yours are currently
shadowed. The three "machine-tool path" fields are session-scoped like any other field,
but discouraged from a committed project file — the header gear and the composer gear's
local-overrides section are where they belong.
Observability: streaming to OTLP
Every run already writes its session.jsonl; that file is the durable record and
never needs a network. On top of it, a run can ALSO stream its spans to any OTLP/HTTP backend —
Langfuse, Jaeger, Phoenix, anything that speaks the OpenTelemetry protocol over HTTP. The export
is additive: the JSONL is the anchor, and a dead backend warns once and never slows a run.
Two settings turn it on, both off by default — the endpoint and an optional Basic-auth pair,
either through Settings → Observability or the two SPECTRO_OTLP_* env bases
in the table above:
SPECTRO_OTLP_ENDPOINT = http://localhost:3000/api/public/otel/v1/traces (Langfuse) SPECTRO_OTLP_BASIC_AUTH = pk-lf-...:sk-lf-... (optional)
The doctor carries an OTLP line: it posts an empty — but valid — OTLP batch to
the configured endpoint and shows green when the endpoint accepts it, so you learn a real run's
spans will land before you spend a run finding out. The auth value is never echoed. Each session
becomes one trace whose span tree mirrors the run (agent → turns with token usage → tool calls →
gates), carrying both Langfuse's langfuse.observation.type and OpenTelemetry's
gen_ai.* conventions so either kind of consumer reads it. The full attribute mapping,
the endpoint variants (Langfuse / Jaeger / Phoenix) and the backfill path for older sessions live
in docs/OBSERVABILITY.md.
Environment-only variables
Everything that used to live only in the environment has graduated to a settings
field (deprecation table above). What is left is secrets — spectroscope never
writes an API key to a settings file, and the PUT endpoints refuse to accept
one outright (any *_API_KEY/*_TOKEN-shaped key is rejected):
| variable | consumed by |
|---|---|
ANTHROPIC_API_KEY | the Anthropic SDK; checked up front — a missing key refuses provider construction with a readable message |
OPENAI_API_KEY | optional Bearer for the chat provider; required by the OpenAI image backend |
OPENROUTER_API_KEY | Bearer for the openrouter openai-compatible provider |
GEMINI_API_KEY | the Gemini image backend |
TAVILY_API_KEY | the Tavily tier of web_search; absent falls back to the keyless DuckDuckGo tier |
Three .env injection paths, one parse rule. Gradle's run
tasks (CLI and server) and the ./spectro-app launcher all read the project root's
gitignored .env identically: comments and blank lines skipped, first
= splits, one quote layer stripped, empty values excluded —
a KEY= line can never blank a variable you exported in your shell. The
launcher's copy exists because the desktop face bypasses Gradle entirely. All JavaExec
tasks also run from the project root, so the project settings layer is always found.
The single writer. Only SettingsWriter ever writes a
settings file — two entry points, the same serialized read-modify-write discipline:
appendAutoApprove (the permission dialog's “Persist”/“dauerhaft” checkbox)
appends one allowlist rule to the workspace project file; patch (behind
every PUT /api/settings/* call, so behind both gears) applies a
schema-validated partial patch to any of the three files, rejecting secrets and
process-globals in the wrong scope before anything touches disk. Both preserve every
other key already in the file, and a local write also ensures the
workspace's .spectro/.gitignore lists settings.local.json.
Everything else only reads.
SPECTRO.md
A SPECTRO.md in the working directory is appended to the system prompt as a
“Project context” section — provider-neutral, visible in the System-context panel.
The counterpart of a CLAUDE.md, for spectroscope.
Chapter 31Build & test inventory
Five JVM modules, two npm toolchains, one version catalog — and a test suite that never needs an API key.
Modules
| module | role | key facts |
|---|---|---|
spectro-core | the headless library | java-library; Jackson is api, through its BOM (JsonNode appears in public contracts); the Anthropic SDK is implementation — no other class may import it; Spring Framework as a library (RestClient + HTTP interfaces); cron-utils |
spectro-cli | terminal face | application; picocli; Logback keeps the console WARN-quiet so the ANSI face stays clean and the diagnostics go to the log file; registers the tour task; stdin wired for the REPL |
spectro-server | web backend | Spring Boot 3.5.3 (only here); starters web + websocket; depends on core and cli; serves the built UI from static/ |
spectro-mcp-notes | example MCP server | standalone; Jackson only; its test spawns the real child JVM |
spectro-orchestrator | the fleet facade | java-library; Spectro.panel() lanes as core agents on a bus; POM pins spectro-core at the same version; Maven Central library |
spectro-web | browser UI | npm/Vite; React 19, React Flow 12, dagre; npm run build writes straight into the server's resources |
spectro-desktop | desktop shell | npm/Electron 43; TypeScript strict; zero runtime dependencies (health polling via built-in fetch) |
Version pins
| library | version | |
|---|---|---|
com.anthropic:anthropic-java | 2.34.0 | owns SSE; client built with maxRetries(0) |
com.fasterxml.jackson:jackson-bom | 2.18.2 | a platform, not a number: jackson-databind is declared under it with no version of its own and is the one api dependency |
info.picocli:picocli | 4.7.6 | CLI framework |
org.springframework:spring-web | 6.2.8 | Framework as library — no Boot in the core |
com.cronutils:cron-utils | 9.2.1 | computes slots; a ScheduledExecutorService fires them |
org.junit.jupiter | 5.10.2 | |
| Gradle wrapper | 9.6.1 | checked in |
| Spring Boot | 3.5.3 | spectro-server only |
| react / @xyflow/react / dagre | 19.1 / 12.11 / 0.8.5 | spectro-web |
| electron / electron-builder | 43.3 / 26.0 | spectro-desktop |
org.slf4j:slf4j-api / ch.qos.logback:logback-classic | 2.0.18 / 1.5.38 | the two library modules speak the API only; the faces run Logback, lifted off Boot 3.5.3's managed 1.5.18 |
The Jackson row is a platform import, and that is the interesting one.
The published POM used to declare jackson-databind at 2.17.2 while
anthropic-java 2.34.0 brings the 2.18.2 family in behind it, and every build in
this repository passed anyway — Gradle resolves highest-wins and quietly landed the whole
tree on 2.18.2. A resolver that chooses differently does not: Maven's nearest-wins, or the
classpath an editor extension assembles, can pair a 2.18 databind with a 2.17 core, and a
databind newer than its own core throws NoSuchMethodError on
ParserMinimalBase.<init>(StreamReadConstraints) the first time anything is
deserialized. Importing the BOM pins the family together instead, and the import travels
into the generated POM, so the guarantee holds for every consumer and not only inside this
build. For a published library the POM is the contract — and the contract was the part that
was wrong.
The test suite
| module | tests | what they pin |
|---|---|---|
| spectro-core | 270 | event round-trips and wire shapes, the loop against fake providers, sessions/resume/delete, compaction, subagents (parallelism, timeout), scheduler, all three provider mappings against scripted local servers, MCP client/transports, hooks, allowlist, retry |
| spectro-cli | 33 | renderer, overrides plumbing, allowlist, voice seams |
| spectro-server | 33 | REST guards (incl. the delete endpoint), a full WebSocket round-trip against an Ollama mock, workspace sandbox, transcripts controller |
| spectro-mcp-notes | 18 | store, ranker, stdio protocol in-JVM and as a real child process |
| JUnit total | 354 | plus 252 vitest for the web UI (reducer, threads, stepper, scene models, scenario compiler, importer, markdown parser, layout stores) |
| spectro-web (vitest) | 252 |
Every test runs key-free and network-model-free: fake providers, scripted HTTP
servers, injectable process seams, and a redirected user.home so no test can
ever touch your real ~/.spectro. Gate: ./gradlew build +
cd spectro-web && npm test.
Chapter 32Releases & working with the source
Two libraries go to Maven Central, one download per app module goes to the GitHub release, and the whole tree is MIT and reproducible from a script. Rule of thumb: every module ships something. This chapter is the map of what that something is, and how to work with the source that builds it.
What a release ships
The seven modules split cleanly into reusable Java (published to Maven Central so you can depend on it) and runnable faces (attached to the GitHub release, one download each). One module is the exception, and it is called out below.
| module | build | ships as |
|---|---|---|
spectro-core | Gradle · java-library | Maven Central library |
spectro-orchestrator | Gradle · java-library | Maven Central library (POM pins core at the same version) |
spectro-cli | Gradle · application | GitHub asset spectro-<v>.zip (the terminal face, bin/spectro) |
spectro-server | Gradle · Spring Boot | GitHub asset spectro-server-<v>.jar (executable; embeds the built web UI) |
spectro-mcp-notes | Gradle · application | GitHub asset spectro-mcp-notes-<v>.zip (the sample MCP server) |
spectro-desktop | npm · Electron | GitHub asset spectroscope-<v>-<arch>.dmg (the desktop run kit; bundles server + JRE) |
spectro-web | npm · Vite | not a standalone download — built into the server jar (and the desktop kit) |
Why the web has no download. spectro-web is a single-page
app; it needs the server's REST and WebSocket API on the same origin, so a bundle handed
out on its own could not run. The release build compiles the UI first, writing
its output straight into spectro-server's static resources, so the server jar
below already carries the current UI. The desktop kit carries that same server jar, so the
web face is present in three assets and standalone in none.
scripts/build-release-assets.sh produces every GitHub asset in one pass
into build/release-assets/; the two libraries are the exception and go to
Maven Central instead. On a machine without a full JDK/Node for the Electron step,
SKIP_DESKTOP=1 builds everything except the .dmg.
The libraries on Maven Central
The two headless modules are published under the verified namespace
dev.spectroscope. Depend on the orchestrator and you get the core transitively:
dependencies {
implementation("dev.spectroscope:spectro-core:0.1.0")
implementation("dev.spectroscope:spectro-orchestrator:0.1.0") // POM pins spectro-core at 0.1.0
}
v0.1.0 is the first published version. Maven Central is
append-only: a published version cannot be pulled or overwritten, only superseded,
so the next release is always a bump (0.1.0 → 0.1.1 → …), never a re-publish.
That is why the release ritual gates hard before the single irreversible step.
Only these two ship to Maven. spectro-server is a Spring Boot application:
its plain jar is a thin, non-runnable shell, so it ships as an executable
bootJar on the GitHub release, never to Maven. spectro-web and
spectro-desktop are npm/frontend modules and cannot go to Maven Central at all.
The desktop run kit
The .dmg is a self-contained app. The Electron shell
(spectro-desktop) spawns and supervises the server, and the build bundles both
the server jar and a jlink'd JRE into the app, so the target machine
needs no Java installed: double-click, the server starts with it, the cockpit opens. When
the window closes, the shell reaps its child JVM. The script that builds it is
scripts/build-desktop-runkit.sh.
Two honest limits, and the release notes state them:
- Per-platform. Electron and the bundled JRE are OS/arch specific, and the script builds the host target only. Windows, Linux and Intel each need building on and for themselves.
- Ad-hoc signed, not notarized. On first download macOS reports an
"unidentified developer"; open it once with right-click → Open (or clear the
quarantine flag with
xattr -cr) and it launches normally from then on. A local build has no quarantine flag and starts directly.
For a zero-warning, plain double-click app you need a paid Apple Developer
Program membership, a Developer ID Application certificate, and notarization. The
one spectroscope-specific twist is signing the bundled JRE under the hardened runtime; the
full end-to-end procedure is docs/DESKTOP-SIGNING.md. Sign only with the
ad-hoc identity (-) otherwise, never a corporate certificate the machine may
happen to list.
Working with the source
The layout is the module table above, read as folders. settings.gradle.kts
includes the five JVM modules (spectro-core, -cli,
-server, -mcp-notes, -orchestrator); the two npm
modules (spectro-web, spectro-desktop) sit next to the Gradle
build with their own toolchains, on purpose — a Java backend and JS frontends are separate
worlds and the build does not pretend otherwise. The gate is two commands:
./gradlew build # the whole JVM suite + all tests ( cd spectro-web && npm test ) # the web UI suite (vitest)
Neither needs an API key: the tests run against fake providers and scripted local
servers, and the one live contract check self-skips unless ANTHROPIC_API_KEY
is set (chapter 31 has the full inventory). The whole tree is MIT licensed
(LICENSE, © 2026 Christopher Ezell) — use it, fork it, ship it, keep the notice.
The release ritual
Cutting a release is a written runbook, docs/RELEASE-PLAYBOOK.md
(RELEASING.md is the narrower "just Maven Central" companion; the two agree).
The order exists to put every irreversible step behind a green gate:
- Preflight — clean tree, on
main, and a version strictly higher than the last published (Central is append-only). - Bump versions together — the two Maven libs
(
spectro-core,spectro-orchestrator) and the app versions that name the assets (spectro-cli,-server,-mcp-notes, andspectro-desktop/package.json). - Full green gate —
./gradlew testplus the twojavadoctasks, then vitest. Atv0.1.0the baseline was 678 JUnit (one skipped: the live-Opus check) and 383 vitest. - Dry-run the publish —
publishToMavenLocalfor both libs confirms GPG signing and POM generation (every artifact gets a.asc; the orchestrator POM depends onspectro-coreat the same version) without touching the portal. - Commit + tag
v<v>, push both. - Publish the libraries —
./gradlew publishAndReleaseToMavenCentral. This is the one irreversible step; wait for the coordinates to resolve onrepo1.maven.orgbefore trusting them. - Build the assets —
scripts/build-release-assets.sh, thengh release create v<v>with everything inbuild/release-assets/. - Flip the install snippets — the landing page and the dev portal go from "planned" to the real coordinates; push is deploy, so verify the live copy last.
One-time owner prerequisites (a Central Portal token and GPG key in
~/.gradle/gradle.properties, a full JDK for jlink, Node for the
web bundle and Electron, and gh authenticated) are listed at the top of the
playbook. The gitignored build artifacts (jre/, build/) never get
committed.
The brand mark
The mark is a spectrum. Five vertical lines of different weights and colours, read left to right — the same figure the event stream makes when it fans into its lines. It sits on the cover of this guide. Colour lives only on the lines (a coral edge, amber, a broad teal, cyan, a brown edge); the wordmark is always lowercase, one word, set flat with no shadow and no glow. That restraint is deliberate and matches the interface, where colour is reserved for the spectral lines and nothing else carries it.
Chapter 33Saving a key, safely
You can paste an API key into the cockpit and have spectroscope save it for you. That is a web page writing a secret to disk, so it only ships fenced. This is the honest account of the fences: what the feature does, why it takes effect without a restart, and exactly which requests it refuses.
Where the key goes
The key lands in one place: ~/.spectro/.env, written at 0600
(owner read and write only). It is never put in settings.json
— those files are meant to be shared and checked into a project, so a secret there is a
leak waiting to happen. It never enters a session file, a log line, or any GET response:
the API reports a key's presence (true/false), never its
value. There is one write path, the POST below; the value leaves your browser once, as that
request body, and is never sent back.
Why it works without a restart
A running JVM cannot change its own environment: System.getenv is frozen for
the life of the process. So a key merely written to disk would be inert until the next
launch. spectroscope closes that with a read seam. Every provider resolves its key as
environment first, then ~/.spectro/.env
(SpectroConfig.resolveApiKey). Providers are built per chat, so the moment you
save a key the next new chat picks it up: no restart, and it works the same from the jar and
the desktop app, not only the launcher's ./.env. A real environment variable or
a launcher-loaded ./.env still wins, so nothing you set on the command line is
ever overridden.
The two fences on the write
The endpoint is POST /api/onboarding/key. It is bound to 127.0.0.1
like the rest of the server, and it passes only when both of these hold:
| fence | what it checks | what it stops |
|---|---|---|
| local origin | the TCP peer is loopback and the Host header is localhost / 127.0.0.1 | a remote caller, and a DNS-rebinding attack that aims a public name at your loopback |
| same origin | the Origin header is absent (a non-browser client) or points at loopback | CSRF: a page on evil.com firing a fetch at your local server. Its request is still loopback, but its Origin is its own domain, so it is refused |
The second fence matters because the read side of this controller deliberately allows
cross-origin requests, so the Vite dev server on another port can read /api/config.
Reads are harmless; a write is not, so the write checks the Origin itself instead of
trusting the CORS policy. A refused request answers 404, not 403: an
attacker gets no signal that the endpoint is even there.
What an attacker can and cannot do
Plainly, for the sceptics:
- A website you visit cannot write a key here. The Origin fence answers it 404 (verified: an
evil.comOrigin is rejected, a loopback Origin is accepted). - A remote machine cannot reach the endpoint. The server binds loopback, and the Host fence 404s a rebinding attempt.
- Nobody can read a saved key back through the API. Every surface is presence-only; the value sits on disk at
0600and in the process, nowhere else. - The honest residual: a malicious program already running on your machine could POST a key (its Origin is loopback) or just read the file directly. But a hostile local process can do far worse than this anyway, so it is outside the line this feature draws. That line is external websites and remote hosts, and both are closed.
You never have to use it
The dialog is a convenience, not the only door. The same key is read from a real
environment variable, from a ./.env next to the launcher
(./spectro-serve --env-file), or typed hidden on the command line with
./spectro-serve set-key. All four land in the same resolver, so use whichever you
trust. The cockpit field is there for the person who would rather not open a terminal.
Appendix ATroubleshooting
| symptom | cause & fix |
|---|---|
zsh: operation timed out: ./spectro-app | The folder arrived as a download — every file carries the macOS quarantine flag, and Gatekeeper hangs in an online notarization check on exec. Fix: xattr -cr <path>/spectroscope (read-only .git/objects complaints are harmless). |
ollama unreachable in doctor | The Ollama server is not running — open the app or ollama serve. Probe: curl -s http://localhost:11434/api/tags. |
| Gradle: “release version 21 not supported” | Your default JDK is older than 21. The launcher normally solves this; without it, install brew install openjdk@21 or set JAVA_HOME. |
git hangs / Operation timed out on git diff | On a sync filesystem (OneDrive/iCloud) the fsmonitor daemon can wedge. Fix: git config core.fsmonitor false. |
| Model without vision error | You attached an image while a text-only local model is selected. ollama pull qwen3-vl (or llava) and switch, or use a cloud provider. |
| Mic button disabled with a tooltip | STT is not installed — bash scripts/setup-stt.sh; or the browser denied microphone access. |
Voice output disabled: piper, voice or audio player missing | bash scripts/setup-tts.sh (note: its checksums are pin-on-first-run — the script prints the sums it computed and asks you to verify and paste them). |
| Anthropic switch refused in the picker | ANTHROPIC_API_KEY is not set for the server process — put it in spectroscope/.env and restart ./spectro-app web. |
| Desktop window blank after a rebuild | An old instance held Electron's single-instance lock. ./spectro-app desktop stops stale instances first — use it instead of a bare npm start. |
| MCP server UNREACHABLE in doctor | The configured command path does not exist on this machine (the settings file carries machine-specific absolute paths) or the dist was never built: ./spectro-app mcp-notes. |
SPECTRO_MAX_RETRIES must be an integer | Exactly what it says — the env var refuses garbage loudly instead of silently defaulting. |
Appendix BReproducing this guide
This document is generated — never hand-edited — and every asset in it is reproducible from the repository.
docs/guide-assets/capture_screens.mjs — Playwright on the system Chrome against a running ./spectro-app web. Deterministic: EN chrome seeded before load, scenario replays for all feature states, two live local-model runs for the plan tab and the permission dialog (the delete shot only arms the button and lets it disarm). Output: docs/guide-assets/shots/*.png.docs/guide-assets/mermaid-src/*.mmd rendered to SVG by render_mermaid.mjs (mermaid 11, dark theme on the spectroscope tokens) — the guide inlines the SVGs so the PDF needs no JavaScript.docs/diagrams/build_NN_*.py; rerun with for g in build_*.py; do python3 $g; done../spectro-app and ./spectro-app doctor on 2026-07-16, ANSI stripped at build time.python3 docs/guide-assets/build_user_guide.py — inlines the fonts (from the server's static assets), all images as data URIs, and the content parts from docs/guide-assets/parts/ into the self-contained docs/USER-GUIDE.html.--headless --no-pdf-header-footer --print-to-pdf=docs/USER-GUIDE.pdf docs/USER-GUIDE.html. Dark pages incl. margins come from the thead/tfoot page-spacer technique with @page margin 0.Credits and colophon
spectroscope is designed and built by Christopher Ezell, with Claude (Anthropic) doing much of the heavy lifting across code, design and copy. It began as the reference harness of a build-an-agent-harness workshop and grew into its own product; the brand, the orchestrator and this guide came after.
The guide is generated, never hand-written. The type is Inter and JetBrains Mono, both embedded so the pages read the same on any machine. The architecture plates are drawn by Python generators; the sequence and flow figures are Mermaid; the screenshots are real captures of spectro web. Everything assembles into one self-contained HTML file and a PDF, in an espresso-dark and a paper-light edition. The wire format is shared byte-for-byte with the TypeScript edition, so a session recorded by one is replayable by the other.
Fonts: Inter and JetBrains Mono, both SIL Open Font License 1.1. Diagrams: Mermaid (MIT). Rendering and capture: headless Chrome. spectroscope itself is MIT licensed and pre-release. Contact: chris (at) spectroscope.ai
spectroscope — the agent orchestrator you can watch. This guide ships in espresso dark and paper light, like the product. The wire format is shared byte-identically with the TypeScript edition, so sessions are forever.