Context Distillation

Context distillation is fabriq's answer to the scale problem: not every agent turn needs to read entity rows — often a well-placed summary of a scope is all the model needs, costing a fraction of the tokens. Distillation builds and maintains a per-tenant Merkle tree of summaries (DigestNode), stored content-addressed in the file-plane CAS, and exposes it through the agent toolkit as altitude-aware recall and three dedicated tools.

The Merkle tree of summaries01

Fabriq maintains a three-level digest tree for each tenant.

The tree is a Merkle tree: each node's ContentHash is a deterministic function of its children's hashes and a recipe version salt. This means the root hash changes if and only if something in the corpus changed (or the summarisation recipe changed). Agents can remember a root hash, hand it back on the next turn, and receive only the diff — re-grounding is proportional to what changed, not to the size of the corpus.

// DigestNode is the entity persisted by the distillation worker
// (domain.DigestNode, table digest_nodes). One row per tree node.
type DigestNode struct {
    ID         string // stable, e.g. "digest:0:asset:uuid-…"
    Level      int    // 0, 1, or 2
    Kind       string // entity kind (L0), "scope"/"cluster" (L1), "tenant" (L2)
    ScopeName  string // declared scope name (L1 scope nodes); else ""
    ScopeID    string // scope instance id (L1 scope nodes); else ""
    SourceKind string // source entity kind (L0 leaves)
    SourceID   string // source entity id (L0 leaves)

    SummaryHash string // CAS address of the summary text in the file plane
    ContentHash string // Merkle hash (see below)
    SemHash     string // 16-hex SimHash over the summary embedding

    ChildIDs  []string // direct children in the tree
    ParentIDs []string // parents (an L0 leaf may belong to several scopes)
    UpdatedAt int64    // unix nanos
}

Opt-in with DistillSpec02

An entity participates in distillation only if it declares a Distill spec on its registry entry. This mirrors the opt-in pattern of Embed: you never distill something you didn't ask for.

type DistillSpec struct {
    // SourceFields names the columns concatenated into the raw L0 source
    // text fed to the Summarizer. Order is preserved.
    SourceFields []string

    // Text is a function alternative to SourceFields: return the verbatim
    // text to summarise. Text takes precedence if both are set.
    Text func(vals map[string]any) string

    // Scopes names the L1 declared scopes this entity belongs to.
    // An entity may belong to more than one scope.
    Scopes []string

    // Budget is the L0 summary token budget the Summarizer should target
    // (0 = config default). The worker passes it as a hint.
    Budget int
}

Register it alongside the entity, in the Distill field of the EntitySpec — the same place Embed, Search, and Live live:

reg.MustRegister(registry.EntitySpec{
    Name: "asset", Kind: registry.KindAggregate, Model: (*Asset)(nil),
    // …vector, search, graph specs…
    Distill: &registry.DistillSpec{
        SourceFields: []string{"name", "description", "tags"},
        Scopes:       []string{"site", "equipment-class"},
        Budget:       256,
    },
})

Three seams03

Distillation introduces two new seam interfaces and reuses one existing one.

Embedder (reused)

The same Embedder the agent toolkit uses for semantic recall is reused here: digests share the entity vector space. The SemHash of every DigestNode is computed from the summary's embedding. Wiring an Embedder is therefore a prerequisite for SemHash-based lookup and for emergent clustering; without one, SemHash is 0 and clustering is skipped.

Summarizer (new)

// Summarizer generates a textual summary from raw source material.
// The host supplies the model; fabriq stays model-agnostic.
type Summarizer interface {
    // Summarize receives the guarded source (Raw for L0, Children for
    // roll-ups), the node kind/scope, and a token budget hint.
    Summarize(ctx context.Context, in SummaryInput) (string, error)
}

// SummaryInput is the host-model input for one summarization. For L0, Raw holds
// the source text; for L1/L2, Children holds the child summaries.
type SummaryInput struct {
    Level    int           // 0 = entity row, 1 = scope/cluster, 2 = tenant root
    Kind     string        // node kind (entity kind / "scope" / "cluster" / "tenant")
    Scope    ScopeRef      // {Name, ID} for scope nodes; zero otherwise
    Children []ChildDigest // child summaries for roll-up nodes ({ID, Kind, Summary})
    Raw      []byte        // raw source text for L0 (already guarded)
    Budget   int           // target token count (hint)
}

Guard (new, optional)

The Guard is an optional, pluggable safety layer. When nil, it behaves as the identity function — raw content flows through unmodified. When set, it is called at two stages — once on the raw source (GuardIngest) and once on the generated summary (GuardEmit) — distinguished by the Stage field of a single Guard method:

// Guard controls what reaches the model and what leaves it.
type Guard interface {
    Guard(ctx context.Context, in GuardInput) (GuardResult, error)
}

type GuardInput struct {
    Stage    GuardStage // GuardIngest (raw source) or GuardEmit (summary)
    TenantID string
    Scope    ScopeRef
    Level    int
    Text     string // the text to inspect / redact
}

type GuardResult struct {
    Text    string // possibly-redacted text to use downstream
    Blocked bool   // true drops the node (it keeps its previous ContentHash)
    Reason  string // audit reason when Blocked
}

The worker is fail-closed by default: if the guard returns an error, the content is treated as Blocked and the node is not updated. Set forgeext.WithDistillFailOpenGuard(true) (DistillConfig.FailOpenGuard) to pass the original text through on guard error instead — useful in development environments where the guard service is unavailable.

The forgeext/shieldguard sub-package provides a ready-made adapter over github.com/xraph/shield for deployments that already use Shield as their policy engine — wire it with forgeext.WithGuard(shieldguard.New(engine)).

Two hashes04

Every DigestNode carries two orthogonal hashes that serve different agent needs.

ContentHash is the freshness oracle: if the entity row's hash hasn't changed since the last run, the worker skips the Summarizer call entirely — no LLM cost, no CAS write. If the recipeVersion component of the hash changes (e.g. you swap the summarisation prompt or model), the entire tree is invalidated on the next run and all nodes are re-summarised.

SemHash is the contextual oracle: the Hamming distance between two SemHash values is a Hamming-graded approximation of semantic similarity. Agents can store a SemHash in long-term memory and later call resolve(hash) to find related digests without re-embedding — the "contextual hash" is cheap to persist and cheap to compare.

Emergent clusters05

Beyond declared Scopes, distillation identifies emergent clusters: groups of L0 nodes that are semantically similar even if they span different declared scopes or entity kinds.

When a new or updated L0 node's embedding is computed, the top-p bits of its SemHash form its bucket key — the prefix IS the cluster ID. The prefix is stable across membership drift: entities that were in the cluster yesterday and left today do not change the cluster's identity, only its membership count.

A bucket is promoted to a live DigestNode (with a proper summary) only when it meets the noise floor (DistillConfig.NoiseFloor, default 2). Below the floor the bucket exists only in the index; above it the worker issues a rollup summarisation on the next debounce tick.

The proj:distill worker06

The distillation worker runs inside the forgeext extension alongside proj:embed. It is a debounced, batched, per-tenant single-flight consumer over the same event stream: each entity write event that touches a distillation-enabled entity triggers the worker, which accumulates a batch over a debounce window (forgeext.WithDistillDebounce, default 1s when unset) before processing.

Two Merkle short-circuits eliminate LLM calls:

1. Unchanged entity — if the entity row's computed input hash matches the stored ContentHash, the L0 node is skipped entirely (no Summarizer call, no CAS write).

2. Unchanged parent — if all dirty L0 children of an L1 or L2 node are processed and their net hashes did not move the parent's input hash, the parent rollup is also skipped.

The tenant root (L2) is re-summarised at most once per debounce window, regardless of how many L0/L1 nodes changed in that window.

The agent surface07

map(scope?) — the tree as a compact outline

Returns the distillation tree (or a subtree if scope is given) as a compact, line-per-node outline. Each line carries the node's ID, level, label, ContentHash, and SemHash. Pass knownHashes to receive a Merkle diff: nodes whose subtree ContentHash matches a hash in knownHashes are emitted as a single "unchanged" line — the agent only receives the diff, not the full tree.

outline, err := tk.Map(ctx, agent.MapRequest{
    Scope:       "site",                          // optional; omit for the full tree
    KnownHashes: map[string]string{"digest:2:tenant": "h1…"}, // nodeID → known ContentHash
})
// outline is []agent.MapLine{ID, Level, Kind, Scope, ContentHash, SemHash, Unchanged, Summary}

Re-grounding cost is proportional to what changed: an agent that holds the previous root ContentHash pays only for new or modified nodes.

digest(nodeId) — a node's summary text

Returns the stored summary text for a given DigestNode ID, fetched from the file-plane CAS by SummaryHash, plus the summaries of its immediate children.

d, err := tk.Digest(ctx, "digest:1:scope:site:site-42") // returns agent.DigestView
// d.Node      — the node's MapLine (id, level, hashes)
// d.Summary   — the scope rollup text
// d.Children  — []agent.DigestChild{ID, Kind, Summary, ContentHash, SemHash}

resolve(hash) — from hash to node

Resolve takes a single hash string and attempts both lookups at once, returning an agent.ResolveResult{Exact, Near}:

• Exact ContentHash — if any node's ContentHash equals the string, it is returned in Exact. Useful for confirming that a remembered hash still points to the same subtree.

• Nearest SemHash — if the string parses as a 16-hex SemHash, every node within the Hamming threshold is returned in Near (sorted ascending by Hamming bits), without re-embedding.

res, err := tk.Resolve(ctx, "abc123…") // a ContentHash or a 16-hex SemHash
// res.Exact  — *agent.MapLine when a ContentHash matched (else nil)
// res.Near   — []agent.ResolveMatch{Node, HammingBits} for SemHash neighbours

Altitude-aware recall

See the altitude parameter on Recall for how the altitude field integrates with RecallRequest to drive descent through the digest tree. In short: auto uses the token budget to choose the right level; scope and tenant pin the agent to high-altitude digests regardless of budget.

Enabling distillation08

Wire the Summarizer (required) alongside the existing Embedder, and optionally a Guard, through the forgeext extension options:

ext := forgeext.New(
    forgeext.WithEmbedder(myEmbedder),
    forgeext.WithSummarizer(mySummarizer),
    forgeext.WithGuard(shieldguard.New(shieldEngine)), // optional
)

Summary text is stored content-addressed in the file-plane CAS. Live DigestNode.SummaryHash values are GC roots: the CAS garbage-collector will not collect a summary blob as long as a DigestNode references it — so safe deletion of a DigestNode also orphans the blob for eventual collection.

Metrics09

The proj:distill worker exposes the following Prometheus counters:

These counters ride the same Prometheus registry as the rest of fabriq's metrics. A high shortcircuit_total relative to summaries_total is healthy — it means the debounce and Merkle gates are doing their job and LLM calls are minimal.