Nexus, by example

Network diagram showing interconnected nodes labeled with NLP-related concepts such as Natural Language, Semantic Embedding, Entity Extraction, Document Ranking, Query Understanding, and Knowledge Graph.

Post 1 of the Nexus series: an explainer walking through the Delos corpus.

Revised August 2026: the storage substrate has moved since this post first ran (PostgreSQL and pgvector everywhere; ChromaDB and SQLite retired). Transcripts, numbers, and internals below are re-run against the current system.


What Nexus Is

Nexus is the knowledge system I’ve been building to make my work stick, and to give the AI agents I work with something durable to lean on across sessions. Nexus indexes code, prose, PDFs, and decision records into three tiers of storage, all tied together by a catalog of typed links between these documents. Nothing revolutionary about this architecture and design. What makes it useful is what I decided to put in the catalog and how I reach into it.

This post is the first of a series, and it starts at the bottom of the stack: storage, catalog, retrieval. Everything else rests on that. Each command I run below is also an entry point into one of Nexus’s four architectural layers, and naming those layers as they show up is where this post is headed.

The three tiers are

  • T1 (session scratch, shared across an agent tree while a conversation is alive),
  • T2 (durable project memory: notes, plans, taxonomy, telemetry, and the like),
  • T3 (the permanent-knowledge tier: vector search over everything I’ve actually indexed).

All three live in PostgreSQL now. Every Nexus install carries a small engine service in front of a PostgreSQL 17 with the pgvector extension; a local install bundles its own private PG, and cloud mode points the same client at a managed instance of the same engine. One substrate shape in both modes, which means everything in this post works identically on a laptop with no cloud account. The embedding model is the one place the two diverge, and it is worth being precise about because the collection names below give it away. A default local install embeds with a bundled 768-dimensional model, bge-base-en-v15-768, and needs no account and no API key. This install is opted into Voyage AI instead, which takes both: voyage-code-3 for code, voyage-context-3 for prose and papers. So where the names below read voyage-code-3, a stock local install would read bge-base-en-v15-768.

Every document I index also lands in the catalog alongside storage. The catalog tracks typed links between documents as first-class data, which is what separates Nexus from a flat vector store, but that’s the subject of the next post. Both the storage and catalog sit behind a command line client and an MCP server, so I can hit the same substrate from a shell, from Claude Code, or from a subagent, and get the same answers and functionality.

That’s the data plane. On top of it sits a Claude Code plugin: skills, subagents, hooks, and a structured practice I’ve built around RDRs (decision records) and beads (tracked work items). This is really how I use Nexus day-to-day.

A session-start hook wires the T1 scratch across the in-process subagents of a session so they share context instead of each one starting cold. A storage-boundary hook auto-links new documents into the catalog the moment they’re stored, so I don’t have to remember to perform the linking manually. And RDRs and beads are themselves indexed, which is pretty satisfying when you’re three sessions deep and need to remember why you picked a thing or what things you did could possibly have caused the current nightmare.

The plugin gets its own post at the end of the series. This one is just the foundation.

Nexus is built to run inside Claude. You can drive it entirely from the command line, but most of it only opens up in a Claude session, where the model reaches the MCP server, the skills, and the agent team directly.

If you want to follow along, Post 00: Installing Nexus walks through the CLI install, the plugin install, and a short tour. Source and README at github.com/Hellblazer/nexus.

Everything with an identifier in it below is local to my install: collection names like code__Delos-5af9bfe0 carry a hash of my checkout, the counts are my corpus, and the answers are what my index returned on the day I ran them. None of it is a shared dataset you can query, and your equivalents will differ in every particular. What transfers is the shape of the interaction, not the values. Read the identifiers as placeholders and the commands as real.

The running example throughout is Delos, a public git repo I maintain: an open-source distributed-systems framework for replicated state machine composition, mostly Java. It’s a good demo target because it carries code, prose docs, its own decision records (RDRs, naturally), and a bibliography of foundational papers all in one bundle.

That whole bundle is what we refer to as its corpus, here and throughout the series. Real enough to ask real questions of, small enough to walk through in one post.

Corpus

I’ve got four Delos collections in T3. A collection in Nexus is just a named set of chunks: documents sliced up and stored next to their embeddings (pgvector columns in PostgreSQL, under the hood), ready for similarity matching against a query. The four of them together are what I’m calling the Delos corpus:

code__1-3__voyage-code-3__v1 16088 chunks
docs__1-3__voyage-context-3__v1 4408 chunks
knowledge__distributed-systems__voyage-context-3__v1 2101 chunks
rdr__1-3__voyage-context-3__v1 103 chunks

Those names are doing more work than they used to, so let me unpack one. code__1-3__voyage-code-3__v1 reads as four fields separated by double underscores: the content type (code), the owner (1-3), the embedding model (voyage-code-3), and a schema version (v1).

The owner field is the interesting one. When I point Nexus at a git-backed project, the catalog registers the project as an owner and assigns it an address; Delos is owner 1.3 in my catalog (dots become dashes inside collection names). Collection identity ties to that catalog entry, not to a path on disk and not to any particular commit.

As I advance main, re-indexes write incrementally into the same collection: the current HEAD commit is recorded on every document so staleness checks have something to compare against, working-tree walks respect .gitignore, unchanged files get skipped by content hash, and only new or edited files re-embed.

Baking the embedding model into the name is deliberate too. A collection embedded with voyage-code-3 is only ever searched with voyage-code-3 query embeddings; if I move to a new model someday, that’s a new collection built alongside the old one, not an in-place mutation that silently mixes vector spaces.

knowledge__distributed-systems has a name where the others have an owner address because it’s not backed by a git repo. It was called knowledge__delos when this post first ran; I renamed it once the same papers turned out to be the reference set for more than one project. The chunk count did not move across the rename, which is how I know nothing was dropped in it. I built it from a directory of cited PDFs, independent of any checkout.

Every indexed document also lands in the catalog: a metadata layer alongside the vector store that tracks title, author, source path, collection membership, and the typed links between documents. The next post is where I get into the catalog properly; for now, just treat it as a second index that sees documents, topics, and knowledge graphs instead of simply collections of chunks.

Here’s the breakdown:

  • code__1-3__voyage-code-3__v1 is the Delos repo itself: roughly 1,800 source files, mostly Java.
  • docs__1-3__voyage-context-3__v1 is the framework’s prose documentation: READMEs, design notes, architectural overviews.
  • rdr__1-3__voyage-context-3__v1 is Delos’s own Research/Design Review records: the decision history of the project.
  • knowledge__distributed-systems__voyage-context-3__v1 is fifteen foundational papers cited by Delos:
    • Fireflies
    • DHR’s self-stabilizing Byzantine overlay
    • Rapid
    • HEX-BLOOM
    • Aleph-BFT
    • Bessani’s BFT-to-SMR transformation
    • Vukolić’s From Byzantine Replication to Blockchain
    • Zanzibar
    • PRDTs
    • Aging Bloom Filter
    • Distributed Bloom Filter
    • Lightweight SMR
    • MFAz
    • pBeeGees
    • Permission Systems at Scale

Semantic Search

The first thing I usually reach for is nx search. It does semantic search: find me passages that are about this thing, don’t care whether the exact words line up. The query string gets embedded, pgvector returns the chunks whose embeddings sit nearest in vector space, and back comes a ranked list.

$ nx search "view change safety" \
    --corpus knowledge__distributed-systems -m 6

[0.3831] rapid atc18
  ls can be combined: If a set of processes F as
  above fails, and a set of processes J joins, then
  $(C\backslashF)\cupJ$ is eventually notified of
  the changes.
[0.3955] rapid atc18
  F)\cupJ$ is eventually notified of the changes.
[0.4482] bft to blockchain
  by malicious processes.
[0.4535] bft to blockchain
  s exchanged since this factor significantly
  affects the performance of BFT
[0.4562] bft to blockchain
  trusted client known as the View Manager, which
  uses the aforementioned state machine protocol to
  issue updates to the replica set. To change the
  current replica set (view) of the system, the
  View Manager issues a signed reconfiguration
  request that is submitted just like any other
  client operation...
[0.4585] bft to blockchain
  time bounds for computation and communication
  shall be enforced after T.

Five and a half seconds wall-clock, and most of that is CLI startup and the round trip to the engine. The retrieval itself is well under a second.

Internally, the flow is:

  1. The query embeds through voyage-context-3 (the same model that embedded the papers), so query and chunks live in the same 1024-dimensional space.
  2. The engine runs a pgvector index scan over the collection’s chunks. The --corpus flag routes the query to the named collection before any vector work happens, a pre-filter rather than a post-filter, which matters when a database holds a couple hundred collections (mine currently does).
  3. A 0.65 cosine-distance threshold filters noise; anything beyond it is treated as a miss, and the miss says so out loud, naming the nearest rejected candidate and the threshold that rejected it, rather than returning an empty list you have to interpret.
  4. A taxonomy layer sits above the ranking. Every collection carries discovered topics; this corpus has 52, with names like Rapid cluster membership processes, Fireflies Byzantine membership overlay, and Zanzibar ACL snapshot checks. On the agent-facing search path (the MCP tool the next section uses), results whose topics relate to each other get a small distance boost (−0.1 for the same topic, −0.05 for linked topics), which nudges conceptually related passages up the list; I measured the −0.05 shift live while writing this.

The topic machinery is two passes at index time: HDBSCAN (clustering) and c-TF-IDF (labeling). HDBSCAN groups the chunk embeddings by similarity; c-TF-IDF then picks tokens that distinguish each cluster from its neighbors, and an LLM pass polishes the labels, which is how clusters get names instead of numbers. New chunks added after the initial discovery are assigned to existing clusters by a nearest-centroid lookup; the per-topic centroids live in their own table beside the chunks, so adding content doesn’t trigger a full re-cluster.

Full disclosure: rebuilding this corpus for the rewrite surfaced a real bug right here. The first discovery pass came back with two topics, one holding 94 percent of the chunks; HDBSCAN’s default cluster selection had swallowed the collection whole. The fix (a cap on how much of a collection any single cluster may claim) landed the same day, with a regression test cut from this very corpus’s embeddings. The 52 topics above are the after picture, and the after picture isn’t finished: a sizable share of this corpus’s chunks still land in no topic at all post-fix, which is an open bug in my tracker, not a solved one.

Synthesized Answers

Ranking is one retrieval mode; synthesis is another. Instead of handing me a list of passages to read through myself, synthesis asks Nexus to read them and hand back a structured answer. Still grounded in the indexed text, but composed by an LLM pipeline that Nexus drives on my behalf. nx_answer is the entry point, exposed as an MCP tool so agents can call it the same way I do:

nx_answer(
    question=(
        "What different approaches to membership churn "
        "are described across the papers in this corpus?"
    ),
    scope="knowledge__distributed-systems",
    dimensions={"verb": "research"},
)

Produces:

Approaches to Membership Churn / Reconfiguration Across the Retrieved Papers

Corpus note: the extraction inputs are not Delos papers. No chunk describes virtual consensus, log sealing, or loglet-level reconfiguration. The retrieved chunks come from four other systems, BFT-SMaRt, Rapid, SMARTCHAIN, and an algebraic PRDT-based Paxos formulation, so the synthesis below covers those.

1. BFT-SMaRt, trusted View Manager, reconfiguration through the ordered command stream

A distinguished trusted client (the View Manager) submits a signed reconfiguration request through the same total-order protocol as ordinary client operations. The request is never delivered to the application; it updates the view instead. Joining replicas run state transfer (last checkpoint plus log) before participating. Tradeoffs: simple and uniformly ordered, but reconfiguration depends on a centralized authority with administrative privileges, and a single replica’s log is not self-verifiable (durability requires a Byzantine quorum of logs).

2. Rapid, a dedicated membership service: cut detection + leaderless consensus view change

Membership lives in its own service layer, not in an application log. Processes monitor each other over a K-ring topology; multi-node cut detection aggregates alerts into a single stable membership-change proposal (combining failures F and joiners J into one cut); a view-change protocol then decides the new configuration by fast, leaderless consensus. On partition, Rapid is CP: the majority component reconfigures and the minority is forced to logically depart. Liveness across the hand-off requires a majority of both the old and new configuration to remain correct until the new one is current, and formal guarantees hold only once the system quiesces.

3. SMARTCHAIN, autonomous quorum-certified reconfiguration, hardened with ephemeral per-view keys

Removes BFT-SMaRt’s trusted View Manager. A joiner collects signed acceptances from a quorum of cv.n − cv.f current-view nodes and submits a reconfiguration transaction through the ordering protocol; the outcome is a reconfiguration block recording the new view’s membership and keys. Replicas run a “forgetting protocol”: fresh consensus key pairs each view, certified by permanent identity keys, discarded at view change, so a node compromised after removal cannot resurrect its old key to fork the chain below the reconfiguration block. Tradeoffs: autonomy and post-removal fork-safety cost multiple ordered transactions per reconfiguration and per-view key management.

4. Algebraic PRDT-based Paxos, reconfiguration by composition

In the paper’s proof-relevant data type design style, reconfiguration is not a bolt-on protocol but a composition of existing building blocks: each consensus instance is bound to a specific configuration (which fixes majority thresholds), and the consensus mechanism itself establishes configuration changes. The claimed benefit is uniformity and verifiability; the chunks do not elaborate operational costs (state transfer, hand-off windows) the way the systems papers do.

Cross-cutting contrast

  • Where reconfiguration lives: three of four put it inside the shared log as specially handled, totally ordered commands. Rapid alone factors membership out into a dedicated service below the application.
  • Authority spectrum: trusted central client (BFT-SMaRt) → quorum of the current view (SMARTCHAIN) → the membership itself via monitoring plus leaderless consensus (Rapid) → structurally internalized in the protocol algebra (PRDT-Paxos). The papers’ own tradeoff claims track this spectrum.

(Lightly condensed for length; the full reply ran about 8.5 KB.)

The first thing in that reply is worth dwelling on: the pipeline volunteered a corpus note saying which papers the retrieval actually surfaced, instead of bluffing coverage across all fifteen. There’s an amusing hazard in naming your project Delos: the model knows Facebook’s Delos papers too, and it flags their absence rather than quietly conflating the two. That honesty is the point: a synthesis you can’t scope is a synthesis you can’t trust.

About 100 seconds wall-clock on this run, almost none of it retrieval. Where the time goes:

  1. The question is classified along a few axes: verb (research), scope (knowledge__delos), and dimension signals (cross-document, analytical).
  2. Those dimensions are the key for a plan_match lookup against the plan library: a T2-persisted set of reusable plan templates, each a DAG of operator calls with parameterized inputs.
  3. On a confident match the matched plan runs. On a miss, an inline dynamic planner writes a fresh plan against registered operator schemas.
  4. The plan executes as a handful of steps: search, traverse, extract, rank, generate.
  5. Operators dispatch as claude -p subprocesses; contiguous operator steps in a plan are bundled into a single subprocess call rather than one apiece. That dispatch is still where the minutes live. (A focus for improvement, mind.)
  6. When the generate step emits citations, claims resolve back to chunks via chash: spans, content-addressed references to exact chunk text. A chunk’s identity is the SHA-256 of its text; which chunks compose a document, and in what order, is a manifest the catalog keeps. If the text is re-chunked later, the span either still resolves or fails loudly. No dangling references.

Four Layers

This one nx_answer run exercised all four layers of the system in a single command. The plan-or-planner decision at the plan_match gate lives in the Planning layer. The step DAG the plan then ran through lives in the Execution layer. The chunks those steps retrieved, plus the chash: spans that grounded the answer, live in the Knowledge Representation layer. And the MCP invocation that kicked the whole thing off is a tool the Nexus Application layer exposes.

In general terms:

  • Application: the CLI and MCP surface, including nx search and nx_answer.
  • Planning: the plan_match gate, the plan library, the inline dynamic planner.
  • Execution: plan_run sequencing operators over the step DAG, carrying intermediate results between steps.
  • Knowledge Representation: the three storage tiers, the catalog, the typed links, the topic clusters.

That diagram is Nexus’s own architecture, but the four-layer shape is borrowed from AgenticScholar (Lan et al., 2026), a reference architecture for agentic scholarly data management. It’s a good shape and I saw no reason to invent a different one. Nexus diverges from AgenticScholar where constraints from implementation choices push back: a Xanadu-inspired catalog rather than a property graph; HDBSCAN + c-TF-IDF for taxonomy rather than LLM-driven taxonomy construction; a plan library with empirical promotion rather than pure LLM rerank. Those divergences are what posts 2 through 5 are about.

What the rest of the series covers

The rest of the series takes each layer named above and unpacks it, one post at a time. The progression is bottom-up: from the catalog and typed links that sit under search, to the plan library above retrieval, to the plugin surface agents actually touch.

  • Typed links and the catalog. Tumblers, content-addressed spans, the typed link graph, and an operator walk that grades six of Nexus’s own edges and throws two of them out.
  • Decisions as indexed data. RDRs and beads as first-class graph nodes, walked on Nexus’s own RDR corpus.
  • Plans, not replanning. The plan library, promotion discipline, match signal sources.
  • Operators as building blocks. operator_*, claude_dispatch, plan DAGs, composition.
  • The conexus plugin. Skills, subagents, session-start and storage-boundary hooks, and the session-scoped plumbing that carries context across an agent tree.

By the end of the series, the whole substrate above should be something you can walk through on your own. That’s the plan, anyway.



6 responses to “Nexus, by example”

  1. […] document, and carries typed links between them. It’s what turns the flat vector store from Nexus by Example into a navigable graph. The thesis of this post: the graph itself is just a substrate. What makes […]

  2. […] along with the same setup as Nexus by Example: uv tool install conexus, nx doctor, nx index repo . inside any git […]

  3. […] AgenticScholar: Agentic Data Management with Pipeline Orchestration for Scholarly Corpora (arXiv:2603.13774) describes a four-layer architecture for analytical querying over a scholarly corpus: a taxonomy-anchored knowledge graph, an LLM-driven query planner that compiles natural-language questions into operator DAGs, a composable operator library of about fifteen typed operators, and structured ingestion. Recall the four layer architecture from our first post. […]

  4. […] along with the same setup as Nexus by Example: uv tool install conexus, nx doctor, nx index repo . inside any git checkout. Then nx catalog setup […]

  5. […] Post 1: Nexus, by example: a worked session against the Delos corpus showing the moving parts cooperating on a real research question. […]

  6. […] Post 1: Nexus, by example: a worked session against the Delos corpus showing the moving parts cooperating on a real research question. […]

Leave a Reply