Tensegrity

Post 5 of the Nexus series: tiny agents, one job each, composable into a DAG.

---

What a plan is made of

Write programs that do one thing and do it well. Write programs to work together. (Doug McIlroy, Bell Labs, 1978)

Plans, not replanning was about the plan library: saved query-shapes that Nexus tries to match against an incoming query before falling through to dynamic planning. Inside, a plan is a DAG: a directed acyclic graph of steps, each feeding the next, with no cycles allowed so execution is guaranteed to terminate. This post is about the nodes of that DAG, which Nexus calls operators.

An operator is a small, single-purpose step. Each one reads the previous step’s output, does one thing with it, and writes the next step’s input. Composed into a DAG, they answer questions that a single retrieve-and-summarize loop can’t reach.

The operator idea comes directly from AgenticScholar, the same paper that gave us the plan library. It sketches a composable library of typed operators with parameter contracts, and shows that analytical queries compile into DAGs over that library. Nexus implements the full §D.2 + §D.4 set with a runner that segments the DAG before dispatching: contiguous operator steps collapse into a single claude -p call so a four-step plan typically runs as one or two subprocesses, not four.

The Nexus operator set

Twelve operators in the library. Each is intentionally small.

• search runs semantic search against one or more corpora with optional dimension filters. Returns ranked chunks.
• traverse walks the catalog graph along typed links from a seed set. Returns adjacent documents with the link-type that reached them.
• extract pulls structured data (claims, citations, tradeoffs, whatever the caller’s schema asks for) from a set of chunks. Returns JSON matching the requested template.
• rank orders a result set against a criterion the caller specifies. Returns scored, ordered items.
• compare checks a set of items for consistency, contradiction, or alignment on a named axis. Returns a comparison matrix.
• summarize condenses a result set in a named mode (short, detailed, evidence-backed). Returns prose with optional chash:<sha256> citations (chash is a content-addressed hash spanning the exact chunk the claim came from).
• generate produces evidence-grounded prose from a context window and an instruction. Returns prose with citations.
• check evaluates whether a claim meets a stated condition. Returns pass/fail with a reason.
• verify compares a claim against indexed evidence. Returns a confidence score.
• filter applies a predicate across a result set. Returns what survives.
• groupby partitions items by a key (field name or natural-language partition). Returns N keyed buckets.
• aggregate reduces each bucket to one record under a named reducer. Returns one row per group.

Together, filter → groupby → aggregate is the analytics quartet that lets a plan ask “for each experimental dataset across these papers, which baseline method won on the reported metric” as a four-step DAG instead of an ad-hoc free-text pipeline.

How a plan runs

The runner doesn’t dispatch one claude -p per step. It segments the DAG first, then dispatches per segment.

Bundled operator chain: contiguous runs of two or more bundleable operators (extract, rank, compare, summarize, generate, filter, check, verify, groupby, aggregate) collapse into a single claude -p invocation via dispatch_bundle. Intermediate step outputs stay inside the model’s reasoning; the host only sees the terminal step’s output. Every spawn skipped is roughly five seconds of fork + auth + model-init overhead the runner doesn’t pay. A research plan like search → extract → summarize becomes two dispatches (one retrieval, one two-operator bundle) instead of three; longer chains amortise more aggressively. Bundle prompts are capped at 200K chars; over that, the segment falls back to per-step.

Isolated claude -p: what’s left. Retrieval (search, traverse) where the host needs the chunks back as input to downstream operators, and any single-operator segment that didn’t have a neighbor to bundle with. Stateless subprocess. No shared conversation memory. Output validated against a JSON schema before it lands in T1 scratch.

The dispatcher itself is async def claude_dispatch, true asyncio, asyncio.create_subprocess_exec, not a thread pool. A plan can fan out N parallel branches and the runtime actually parallelises them at the syscall layer. T1 scratch is the bus: each segment reads its inputs and writes its outputs through a session-scoped tier of Nexus’s three-tier storage.

(There’s also a SQL fast-path: an isolated filter, groupby, or aggregate over inputs with document identity delegates to T2 document_aspects directly and skips the LLM entirely. Useful when the analytics quartet runs against ingest-time aspect data; not the common case.)

The point of segmenting before dispatching: subprocess spawn is the dominant fixed cost on a plan run. Collapsing four operator steps into one claude -p invocation eliminates three spawns. On a typical research chain that’s roughly three-quarters of the per-step overhead gone, and the model holds richer context across the chain because intermediate results never round-trip through the host.

A walk through Search → Extract → Rank

Arcaneum is Chris Wensel’s knowledge-base tooling project, built on Qdrant rather than ChromaDB. It’s one of the direct design inputs into Nexus: PDF extraction patterns, chunking pipelines, and the RDR process itself all trace back to it. Its RDR corpus makes a clean target for this single-plan demo: close to Nexus in spirit, different in implementation, so the operator DAG runs over a corpus that isn’t Nexus’s own.

The question: “Across Arcaneum’s RDRs on collection creation and bulk indexing, what tradeoffs are called out, and how do they stack up against each other?”

A plan for that shape is three operators:

search → extract → rank

Typical, short, and close to what plan_match would assemble from a dimensional shape like {question-type: comparison, intent: analysis, corpus: rdr}.

Step 1, search, runs over the rdr__arcaneum collection with the query embedding and returns the top chunks about collection-creation and bulk-indexing design. The top hits land across RDR-003 (collection creation), RDR-004 (PDF bulk indexing), RDR-017 (collection export/import), and a handful of adjacent design notes. Roughly a dozen chunks. One isolated claude -p (the host needs the chunks back as input to the next segment).

Steps 2 + 3, extract and rank, bundle into one claude -p. Inside that single subprocess, extract pulls the specific tradeoff claims per a schema like {rdr_id, tradeoff, position}, then rank orders them by agreement strength across the item set, weighted by how tightly each claim constrains the others. The intermediate extraction stays inside the model’s reasoning; only the ranked output crosses the host boundary. A few of the ranked tradeoffs from a real run:

• “disable HNSW indexing during bulk ingest (m=0, indexing_threshold=0) for a 30-50% upload speedup”
• “auto-detect bulk mode by collection size, with --bulk-mode as a CLI override”
• “reject two-phase commit across Qdrant + MeiliSearch: 4-6 hours of sequential code vs 60-80 hours for 2PC, a 10-15x effort multiplier”

Two dispatches, three plan steps. Each segment’s output lands in T1 scratch; the next segment reads from scratch. When nx_answer composes the final response, it draws from what’s in scratch and cites back through chash:<sha256> spans to the exact chunks the search returned.

The ten extracted tradeoffs fold into five themes: HNSW index construction, bulk-mode engagement, dual-indexing overhead, simplicity versus distributed correctness, and the cache-disk budget. The synthesis names one axis that runs through all five: “accept recoverable local risk, refuse distributed-transaction complexity, with user-facing simplicity as the top-priority constraint.”

That sentence is not in the source. The compose step constructed it by aligning the ten tradeoffs against each other. A retrieve-and-summarize pipeline from the same chunks cannot produce it, because the structure the sentence names does not exist in the source to be summarized.

The shape is what the library can save and replay. The same three operators answer “across Nexus’s RDRs on retrieval, what tradeoffs…” or “across Delos’s RDRs on membership, what tradeoffs…”. The corpus scope and the extraction schema change per plan; the operator composition stays the same.

Composition is what makes analytical retrieval tractable

Retrieve-and-summarize works for “tell me about X.” It breaks down on “what is the structure of how Arcaneum talks about X, and how does that structure compare to how Nexus talks about X,” because there isn’t a structure available to summarize. The structure has to be constructed.

Operators build that structure step by step. A traverse names the neighborhood. An extract names the claims. A rank names the salience. A compare names the alignment. A groupby partitions by a stated key. An aggregate reduces each partition to one record. Each operator’s output is a structured artifact the next operator can act on. The DAG is a blueprint for assembling that structure; the operator library is the inventory of shapes the blueprint can call for.

Run that same shape across Arcaneum and Nexus and it takes about three minutes. What comes back has three layers:

• Shared axes where both projects address the same concern: throughput, observability, incremental reindex, pagination tuning. Each project’s actual approach sits side by side.
• Divergent decisions per project. Arcaneum auto-detects collection size and re-indexes aggressively. Nexus requires an explicit --force-stale opt-in because re-indexing its largest code collection costs roughly fifteen dollars in embedding calls.
• A philosophy difference that neither corpus states directly: “Arcaneum treats the vector store as a knob-tunable component it owns; Nexus treats it as a metered-quota SaaS it doesn’t control, and a multi-prefix catalog with user-authored entries mixed in. Arcaneum optimizes for ingest throughput; Nexus optimizes for lifecycle correctness and non-loss of hand-curated data.”

That last sentence is nowhere in the source material. The DAG constructed it.

This is consistent with AgenticScholar’s +47% NDCG@3 gain over naive RAG on analytical queries. Not because the underlying retrieval got smarter, but because analytical answers don’t come from retrieval alone. They come from retrieval composed with structure-building operators, arranged in a DAG the system can save and replay.

Going deeper

• AgenticScholar paper: the composable operator library this post builds on.
• Plan-centric retrieval: how operator DAGs sit inside the broader retrieval loop.
• Plan-authoring guide: writing a plan that uses the operator library.
• MCP tools vs agents: why operators are agent-mediated rather than deterministic MCP tool calls.
• RDR-042: AgenticScholar-Inspired Enhancements: the original adoption of the operator model.
• RDR-088: Operator-Set Completion: check / verify / filter (paper §D.2 closure).
• RDR-091: Scope-aware plan matching: how the caller’s scope argument shapes plan selection and retrieval routing.
• RDR-093: GroupBy and Aggregate Operators: paper §D.4 closure, plus the SQL fast-path that makes the analytics quartet (filter → groupby → aggregate) cost microseconds when document identity is present.

What’s next

Up next (forthcoming): The conexus plugin: the substrate, measured. The Claude Code plugin is where everything in this series becomes available to agents working alongside you: MCP tools, skills for the RDR lifecycle, session-start hooks that propagate T1 scratch across a subagent tree, storage-boundary auto-linking. The plumbing that makes Nexus usable in practice.

Follow along: Post 00: Installing Nexus is the install + first-tour walkthrough. After indexing a repo, run nx catalog setup to seed the builtin plans, then nx plan list to see what’s in the library.