Skip to content

core.dreaming

core.dreaming

Zone A — theme clustering / synthesis (cron, mirror-not-oracle). BUILD-SPEC §9.

The dreaming agent clusters the AUTHORED corpus into themes (deterministic, model-free) and reflects each back to the owner as a lens on their own notes — never external truth. Output is INTERPRETED + regenerable, self-checked before it is kept, and only runs in troughs.

The Phase-7 Dreamer is the LIVE path. The interpreter panel (R0) + adjudicator (R1) are a flag-OFF R&D track (design-notes/dream-phase-rnd-charter.md) — exported here for tests but never wired into scheduler/cron.py; their entry points refuse to run unless the [dream_rnd] enabled flag is set (core.dreaming.rnd).

Synthesizer = Callable[[list[Message]], str] module-attribute

DreamLogEntry dataclass

One consensus group in the ranked dream log: a hypothesis, never a verdict.

statement instance-attribute

methods instance-attribute

evidence instance-attribute

grounding instance-attribute

agreement instance-attribute

depth instance-attribute

confidence instance-attribute

terminates_in_authored instance-attribute

members = field(default_factory=tuple) class-attribute instance-attribute

Cluster dataclass

members instance-attribute

titles property

digests property

size property

NoteVector dataclass

digest instance-attribute

title instance-attribute

vector instance-attribute

Dreamer dataclass

store instance-attribute

synthesize instance-attribute

derived instance-attribute

threshold = 0.62 class-attribute instance-attribute

min_cluster_size = 2 class-attribute instance-attribute

max_clusters = 8 class-attribute instance-attribute

clusterer = cluster_notes class-attribute instance-attribute

judge = None class-attribute instance-attribute

attestor = None class-attribute instance-attribute

edge_store = None class-attribute instance-attribute

snapshots = None class-attribute instance-attribute

clusters()

Deterministic note-level clustering over the AUTHORED mirror (model-free, §9).

The rows come from a MirrorView (Invariant 6, structural): its only constructor is the MIRROR_READABLE projection and it cannot hold a non-authored row, so observed exhaust reaching this clustering is unrepresentable — not merely filtered out.

dream()

Run a full dreaming pass: cluster -> synthesize each theme -> self-check -> store. Returns the themes produced (also persisted as INTERPRETED dreams).

dream_v2(*, config=None)

The strong-Dreamer pass — deterministic structure first, the earned model call last:

1. BUILD 𝔎|_MR (firewall structural: the complex is built from a MirrorView)

2–5. LOCATE / THEME / TENSION / GAPS — the interpreter panel over the complex 6. SUPPORT — noisy-OR multi-path grounding on the derivation DAG (§6.1) 7. ADJUDICATE — c = min{1, γ^d·g·(1+λ(|Agr|−1))}, confidence-ordered 8. SYNTHESIZE — the ONLY model call(s): narrate each selected candidate, grounded in its authored evidence, mirror-not-oracle 9. STORE — interpreted-only, derives→authored leaves, acyclic, attested 10. MEASURE — append the structural snapshot (§5.4; feeds the A2 drift axes)

Gated by the dream-R&D hard boundary ([dream_rnd] enabled, OFF by default): the live dream() above is UNCHANGED and remains the cron path until a deliberate flip. Every structural judgment here is deterministic and model-free; self.synthesize is the only model seam, invoked once per stored dream and nowhere else. Trough-only when wired (a dream job routes to the synthesis tier, which the foreground gate blocks).

Theme dataclass

titles instance-attribute

summary instance-attribute

check instance-attribute

artifact instance-attribute

MirrorGraph dataclass

Note-level similarity graph over the authored mirror. sim is the full cosine matrix; edges are pairs with sim >= sigma. Deterministic given the MirrorView's row order.

notes instance-attribute

sim instance-attribute

sigma instance-attribute

n property

build(view, *, sigma) classmethod

π_MR → centroids → σ-adjacency. Input is a MirrorView, so the graph is provably over authored notes only (a non-authored node is unrepresentable upstream).

digest(i)

title(i)

neighbors(i)

Indices j != i with cos(i, j) >= sigma — deterministic ascending order.

degree(i)

local_clustering(i)

Clustering coefficient of node i: fraction of its neighbour pairs that are themselves adjacent. LOW clustering at a high-degree node = a structural hole / bridge (Burt): the neighbours form separate groups this node holds together. Degree<2 => undefined => 0.0.

digests_for(indices)

The authored content digests for a set of node indices — the evidence refs (G1).

titles_for(indices)

Claim dataclass

A candidate pattern-claim from one interpreter. support is the set of authored note digests the claim rests on — content-addressed evidence (G1), and the LEAVES that ground it (G2). data is method-specific. No confidence here; ranking is the adjudicator's job (R1).

method instance-attribute

statement instance-attribute

support instance-attribute

data = field(default_factory=(dict[str, Any])) class-attribute instance-attribute

DreamRnDDisabledError

Bases: RuntimeError

An R&D dream entry point was called while the feature flag is OFF (the default).

This is the boundary: the interpreter panel / adjudicator never run unless [dream_rnd] enabled = true is set deliberately. Not reachable from the live path.

adjudicate(claims, *, authored_digests, agreement_jaccard, gamma=DEFAULT_GAMMA, lam=DEFAULT_LAMBDA, support_of=None)

Rank claims into a confidence-ordered dream log (no model; grounding decides). Agreement across DISTINCT interpreters multiplies confidence; a claim found by one lens still appears (disagreement is information, not noise) — just lower.

support_of (H8, §6.1): an optional multi-path grounding scorer — evidence refs → g via noisy-OR message passing on the derivation DAG (core.complex.support.grounding_with_support partially applied). None (the default) keeps the flat resolvability grounding_score; the two are numerically identical whenever no evidence ref is an interpreted node (property-tested), so the confidence law (R1's clamp) is untouched — only g's computation generalizes.

run_dream_rnd(view, derived, *, config=None)

R0+R1 end to end behind the flag: run the interpreter panel, adjudicate, and persist the confidence-ordered dream log as INTERPRETED artifacts. Refuses unless the R&D flag is on.

cluster_notes(notes, *, threshold, min_size=2)

Single-linkage connected components: notes whose cosine similarity >= threshold are joined into a theme. Returns clusters of at least min_size members, largest first (ties broken by the first member's title — deterministic ordering).

near_duplicate_pairs(notes, *, threshold)

Distinct note pairs whose centroids are near-identical (cosine >= threshold). These are near-duplicate CANDIDATES the curator flags — never auto-merged, because authored notes are immutable ground truth (§8). Sorted most-similar first.

note_centroids(rows)

Aggregate chunk rows (one per vector) to one centroid per note (keyed by digest). Insertion order follows first appearance, so the result is deterministic.

note_snippets(rows, *, limit=600)

Per-note grounding text: that note's chunk texts concatenated in row order and truncated to limit chars. Keeps a synthesis/contradiction context lean (§13) while still grounded in the owner's actual words.

build_dreamer(config=None, *, tier='synthesis')

Wire a Dreamer against the real configured stores + synthesis model. Pulling the synthesis-tier model is required only to actually run it (the unit path injects a fake).

run_panel(view, *, config=None)

Run every registered interpreter (σ-graph lenses + structural lenses over one shared reasoning complex) and return all candidate claims (R0 — no adjudication). Refuses unless the R&D flag is on (hard boundary).

require_rnd_enabled(config=None)

Fail closed unless the R&D flag is explicitly on. Called by every R0/R1 entry point.

adjudicator

R1 — the evidence-based adjudicator (design-notes/dreaming-v2-interpreter-panel.md; §6/§8).

Ranks the panel's competing claims and logs a confidence-ordered dream log, each entry carrying content-addressed authored evidence. The load-bearing rule: evidence, not persuasion. The currency is resolvable grounding (core.selfcheck.grounding_score), never rhetoric — no model scores argument quality here, so the most eloquent claim cannot win.

Confidence of a group κ (the single clamped definition, core.recursion.claim_confidence):

c(κ) = min{1, γ^{d(κ)} · g(κ) · (1 + λ(|Agr(κ)| − 1))}

where g(κ) is the authored-grounding score (the gate: g=0 ⇒ c=0, so agreement can never manufacture confidence from nothing — agreement is a multiplier, not a vote), |Agr(κ)| is the number of DISTINCT interpreters that corroborate the claim, λ is the bounded corroboration bonus (core.recursion.DEFAULT_LAMBDA), and γ^d is the recursion decay (DEFAULT_GAMMA). d(κ)=1 here: every claim's support is authored leaves (R0 reads a MirrorView), so the chain terminates in authored ground (Invariant 10) and depth is uniform — γ^d is order-preserving and present for honesty (a first-order interpretation is γ·g, explicitly a hypothesis) and for R3 readiness. Confidence is NOT a probability and never combined with utility (a separate axis, R2): c decides what to believe, utility decides what to surface; one scalar is forbidden.

Output is stored INTERPRETED-only (DerivedStore, no provenance parameter — structural §8) with derived_from = the authored evidence digests (acyclic, depth-1 — G2). Reproducible and tamper-evident: a dream log entry is pinned to sha256 evidence refs.

AUTHORED_LEAF_DEPTH = 1 module-attribute

DreamLogEntry dataclass

One consensus group in the ranked dream log: a hypothesis, never a verdict.

statement instance-attribute
methods instance-attribute
evidence instance-attribute
grounding instance-attribute
agreement instance-attribute
depth instance-attribute
confidence instance-attribute
terminates_in_authored instance-attribute
members = field(default_factory=tuple) class-attribute instance-attribute

adjudicate(claims, *, authored_digests, agreement_jaccard, gamma=DEFAULT_GAMMA, lam=DEFAULT_LAMBDA, support_of=None)

Rank claims into a confidence-ordered dream log (no model; grounding decides). Agreement across DISTINCT interpreters multiplies confidence; a claim found by one lens still appears (disagreement is information, not noise) — just lower.

support_of (H8, §6.1): an optional multi-path grounding scorer — evidence refs → g via noisy-OR message passing on the derivation DAG (core.complex.support.grounding_with_support partially applied). None (the default) keeps the flat resolvability grounding_score; the two are numerically identical whenever no evidence ref is an interpreted node (property-tested), so the confidence law (R1's clamp) is untouched — only g's computation generalizes.

run_dream_rnd(view, derived, *, config=None)

R0+R1 end to end behind the flag: run the interpreter panel, adjudicate, and persist the confidence-ordered dream log as INTERPRETED artifacts. Refuses unless the R&D flag is on.

cluster

Deterministic embedding clustering for the dreaming + curator agents (BUILD-SPEC §9).

No model, no scikit-learn — just NumPy cosine similarity and single-linkage connected components over a similarity threshold. This is the §9 principle in miniature: the heavy, deterministic work (grouping notes by theme) is done in cheap code so the scarce inference slot is earned only for the synthesis step (the model reflects a cluster back as a theme).

Clustering is at the NOTE level: each note's chunk vectors are averaged into one centroid, because chunk-level clustering over-fragments a single note across many themes. Given a stable input order the output is fully reproducible — a dream run is deterministic.

NoteVector dataclass

digest instance-attribute
title instance-attribute
vector instance-attribute

Cluster dataclass

members instance-attribute
titles property
digests property
size property

note_snippets(rows, *, limit=600)

Per-note grounding text: that note's chunk texts concatenated in row order and truncated to limit chars. Keeps a synthesis/contradiction context lean (§13) while still grounded in the owner's actual words.

note_centroids(rows)

Aggregate chunk rows (one per vector) to one centroid per note (keyed by digest). Insertion order follows first appearance, so the result is deterministic.

similarity_matrix(notes)

Pairwise cosine similarity over note centroids.

cluster_notes(notes, *, threshold, min_size=2)

Single-linkage connected components: notes whose cosine similarity >= threshold are joined into a theme. Returns clusters of at least min_size members, largest first (ties broken by the first member's title — deterministic ordering).

near_duplicate_pairs(notes, *, threshold)

Distinct note pairs whose centroids are near-identical (cosine >= threshold). These are near-duplicate CANDIDATES the curator flags — never auto-merged, because authored notes are immutable ground truth (§8). Sorted most-similar first.

dreamer

The dreaming agent (BUILD-SPEC §9) — Zone A, cron/trough tier.

Clusters the AUTHORED corpus into themes, then asks the synthesis-tier model to reflect each theme back to the owner as a lens on their own notesmirror, not oracle (Constitution §III.2): grounded in the clustered notes, citing them, never presented as external truth.

Firewall: it reads MIRROR_READABLE (AUTHORED only), so third-party observed exhaust can never seed a dream and dreams never enter the behavioral baselines (§15). Output is stored as INTERPRETED artifacts in the derived store (regenerable), and each synthesis runs the Constitution pre-return check (core.selfcheck) before it is kept — a dream that fabricates a citation is flagged exactly like a librarian answer that does.

Trough-only: a dream job routes to the synthesis tier (scheduler.router), which the supervisor's foreground gate (HEAVY_TIERS, Phase 3) blocks while the owner is present — so dreaming is never concurrent with foreground use (§13). The synthesizer is injected so this module is testable without the (large, possibly unpulled) synthesis model.

Synthesizer = Callable[[list[Message]], str] module-attribute

Clusterer = Callable[..., list[Cluster]] module-attribute

DREAMER_ROLE = "You are the dreaming agent of a sealed personal mind-palace. You are shown a cluster of the owner's OWN notes that group together by theme. Reflect that theme back to the owner as a lens on their own thinking — a mirror, never an oracle.\n\nGround every observation in the notes shown and cite them by title in double brackets, e.g. [[note title]]. Do not introduce facts that are not in these notes, do not invent notes or quotes, and do not present anything as external truth. Surface the recurring patterns, tensions, and open questions you see across these notes, and leave the conclusions to the owner. If the notes do not actually cohere into a theme, say so." module-attribute

Theme dataclass

titles instance-attribute
summary instance-attribute
check instance-attribute
artifact instance-attribute

Dreamer dataclass

store instance-attribute
synthesize instance-attribute
derived instance-attribute
threshold = 0.62 class-attribute instance-attribute
min_cluster_size = 2 class-attribute instance-attribute
max_clusters = 8 class-attribute instance-attribute
clusterer = cluster_notes class-attribute instance-attribute
judge = None class-attribute instance-attribute
attestor = None class-attribute instance-attribute
edge_store = None class-attribute instance-attribute
snapshots = None class-attribute instance-attribute
clusters()

Deterministic note-level clustering over the AUTHORED mirror (model-free, §9).

The rows come from a MirrorView (Invariant 6, structural): its only constructor is the MIRROR_READABLE projection and it cannot hold a non-authored row, so observed exhaust reaching this clustering is unrepresentable — not merely filtered out.

dream()

Run a full dreaming pass: cluster -> synthesize each theme -> self-check -> store. Returns the themes produced (also persisted as INTERPRETED dreams).

dream_v2(*, config=None)

The strong-Dreamer pass — deterministic structure first, the earned model call last:

1. BUILD 𝔎|_MR (firewall structural: the complex is built from a MirrorView)

2–5. LOCATE / THEME / TENSION / GAPS — the interpreter panel over the complex 6. SUPPORT — noisy-OR multi-path grounding on the derivation DAG (§6.1) 7. ADJUDICATE — c = min{1, γ^d·g·(1+λ(|Agr|−1))}, confidence-ordered 8. SYNTHESIZE — the ONLY model call(s): narrate each selected candidate, grounded in its authored evidence, mirror-not-oracle 9. STORE — interpreted-only, derives→authored leaves, acyclic, attested 10. MEASURE — append the structural snapshot (§5.4; feeds the A2 drift axes)

Gated by the dream-R&D hard boundary ([dream_rnd] enabled, OFF by default): the live dream() above is UNCHANGED and remains the cron path until a deliberate flip. Every structural judgment here is deterministic and model-free; self.synthesize is the only model seam, invoked once per stored dream and nowhere else. Trough-only when wired (a dream job routes to the synthesis tier, which the foreground gate blocks).

build_dreamer(config=None, *, tier='synthesis')

Wire a Dreamer against the real configured stores + synthesis model. Pulling the synthesis-tier model is required only to actually run it (the unit path injects a fake).

graph

The mirror graph G_MR — the shared substrate every interpreter reads (R0; §6/§8).

Built ONCE from a MirrorView (authored-only, Invariant 6 — structural firewall) and handed to each method-specialist, so the panel computes note centroids + the σ-thresholded adjacency a single time. Fully deterministic and model-free (NumPy cosine only): the §9 deterministic floor. The graph is $G=(V,E)$ with $E_{\text{sim}}={(u,v):\cos\ge\sigma}$ over note centroids.

This is read-only structure; the interpreters (interpreters.py) derive claims from it and the adjudicator (adjudicator.py) ranks them. Nothing here touches a model or the network.

MirrorGraph dataclass

Note-level similarity graph over the authored mirror. sim is the full cosine matrix; edges are pairs with sim >= sigma. Deterministic given the MirrorView's row order.

notes instance-attribute
sim instance-attribute
sigma instance-attribute
n property
build(view, *, sigma) classmethod

π_MR → centroids → σ-adjacency. Input is a MirrorView, so the graph is provably over authored notes only (a non-authored node is unrepresentable upstream).

digest(i)
title(i)
neighbors(i)

Indices j != i with cos(i, j) >= sigma — deterministic ascending order.

degree(i)
local_clustering(i)

Clustering coefficient of node i: fraction of its neighbour pairs that are themselves adjacent. LOW clustering at a high-degree node = a structural hole / bridge (Burt): the neighbours form separate groups this node holds together. Degree<2 => undefined => 0.0.

digests_for(indices)

The authored content digests for a set of node indices — the evidence refs (G1).

titles_for(indices)

interpreters

R0 — the interpreter panel (design-notes/dreaming-v2-interpreter-panel.md; §6/§8; BUILD §3.2).

Generalizes the Phase-7 single clusterer into a REGISTRY of deterministic interpreters — the "workers", specialists by METHOD (not source). Each is a different lens on the same authored mirror graph and emits candidate pattern-claims plus the authored graph evidence that supports them:

φ_i : G_MR → 2^K,   κ = (statement, support ⊆ authored notes)

All are model-free — the §9 deterministic floor; the model is earned only for narration/judging, which R0 does NOT do. No adjudication here (that is R1): R0 just produces the raw claims. Inputs are a MirrorView, so every claim's support is authored (Invariant 6, structural) — observed exhaust can never seed a claim.

Two generations of lens share the panel (BUILD §3.2 — "each interpreter is a thin adapter over a core/complex/ function"):

  • the original NumPy lenses over the σ-graph: community (connected components), centrality (degree hubs), density (cores + explicit noise);
  • the STRUCTURAL lenses over the reasoning complex (H4–H7): bridge (Forman–Ricci curvature — upgraded from the local-clustering proxy to the real instrument, companion III §3.2), hole (persistent H₁ — conceptual gaps, NEVER contradictions, §4.2), theme (DC-SBM posterior with a model-selected count + a spectral cross-check, §6.2).

Change-point is a registered but DEFERRED seam — it needs a per-note temporal axis the MirrorView does not yet carry, so it returns nothing rather than fake a trend (the honest-seam pattern). Contradiction (tension) stays routed through the signed Laplacian (core/complex/balance.py) + typed contradicts edges; it joins the panel when a contradiction detector exists to assert them.

COMMUNITY = 'community' module-attribute

CENTRALITY = 'centrality' module-attribute

BRIDGE = 'bridge' module-attribute

DENSITY = 'density' module-attribute

THEME = 'theme' module-attribute

HOLE = 'hole' module-attribute

TENSION = 'tension' module-attribute

CHANGE_POINT = 'change_point' module-attribute

Interpreter = Callable[[MirrorGraph, DreamRnDConfig], 'list[Claim]'] module-attribute

StructuralInterpreter = Callable[[StructuralContext, DreamRnDConfig], 'list[Claim]'] module-attribute

INTERPRETERS = {COMMUNITY: community_interpreter, CENTRALITY: centrality_interpreter, DENSITY: density_interpreter, CHANGE_POINT: change_point_interpreter} module-attribute

STRUCTURAL_INTERPRETERS = {BRIDGE: bridge_interpreter, HOLE: hole_interpreter, THEME: theme_interpreter} module-attribute

Claim dataclass

A candidate pattern-claim from one interpreter. support is the set of authored note digests the claim rests on — content-addressed evidence (G1), and the LEAVES that ground it (G2). data is method-specific. No confidence here; ranking is the adjudicator's job (R1).

method instance-attribute
statement instance-attribute
support instance-attribute
data = field(default_factory=(dict[str, Any])) class-attribute instance-attribute

StructuralContext dataclass

One pass's shared structural state: the reasoning complex 𝔎|_MR at the working σ, plus the full cosine-distance matrix (the persistence filtration sweeps thresholds, so it must not be pre-thresholded). Built by run_panel; consumed by the structural interpreters.

complex instance-attribute
distances instance-attribute

community_interpreter(graph, cfg)

Connected components over the σ-graph — thematic groups (the Phase-7 lens, as one of many). Each component of >=2 notes is a theme; support = the component's notes.

centrality_interpreter(graph, cfg)

Degree centrality — which notes are load-bearing hubs. The top-k highest-degree notes (degree >= min_degree); support = the hub plus the notes it links.

bridge_interpreter(ctx, cfg)

H4 — the curvature bridge lens: Forman–Ricci over the σ-backbone, most negative first (companion III §3.2 — the principled replacement for the local-clustering proxy). A very negative edge is a surprising cross-domain link: exactly what a synthesis pass should look at first. Support = the two linked notes.

hole_interpreter(ctx, cfg)

H5 — the persistence lens: long-lived H₁ features of the flag complex are conceptual holes — notes pairwise related in a ring with no center. A hole is a GAP to surface (a utility-axis prompt), never a contradiction (§4.2: dissonance is balance.py's job).

theme_interpreter(ctx, cfg)

H7 — the SBM lens: degree-corrected blocks give theme membership WITH a posterior and a model-selected theme count (§6.2 — "how many concerns, how sure"), cross-checked against the spectral partition (agreement = robust, disagreement = fragile — a signal in itself). The posterior organizes the graph; it never certifies a thought (§6.3).

density_interpreter(graph, cfg)

Density split (the HDBSCAN-style contribution): notes with >= min_degree neighbours are 'core'; notes with no σ-neighbour are explicit NOISE/outliers. Emits a core-region claim and, distinctively, an outliers claim — the signal the connected-components lens hides.

change_point_interpreter(graph, cfg)

DEFERRED seam: temporal change-point detection needs a per-note timestamp, which the MirrorView does not yet carry. Returns nothing rather than fabricate a trend — the same honest-seam discipline as the §4 judge and the contradiction detector. Wire it when an authored temporal axis lands on the mirror rows.

build_structural_context(view, cfg, *, edges=None)

One pass's shared structural state: the σ-backbone complex (optionally with persisted typed/signed edges overlaid — the tension lens's input) + the unthresholded distance matrix (the persistence filtration). Authored-only by the constructor's input type.

tension_claims(kx)

The tension lens (§2.3): every frustrated triangle — an odd number of − edges — is three commitments that cannot all co-hold ("you keep circling this"). Consumes the signed adjacency (contradiction = a persisted contradicts edge overlaid by build_complex); with no asserted contradictions the graph is all-support and this honestly emits nothing. Dissonance lives HERE, never in H₁ (§4.2).

collect_claims(graph, ctx, cfg)

Run both registries (σ-graph lenses + structural lenses) plus the tension lens over already-built state. The un-gated core of run_panel, shared with the loop-v2 dream pass (which builds its own context so it can overlay persisted edges).

run_panel(view, *, config=None)

Run every registered interpreter (σ-graph lenses + structural lenses over one shared reasoning complex) and return all candidate claims (R0 — no adjudication). Refuses unless the R&D flag is on (hard boundary).

rnd

The hard feature-flag boundary for the dream-phase R&D track (design-notes/ dream-phase-rnd-charter.md).

The interpreter panel (R0) and the evidence-based adjudicator (R1) are RESEARCH, behind a flag that is OFF by default. They are deliberately NOT wired into the live dream path — scheduler/cron.py runs the Phase-7 Dreamer, never this. As a second line of defense, every R&D entry point calls require_rnd_enabled(config) and raises if the flag is off, so even a direct import-and-call cannot run the R&D engine in a normal session. To exercise it you must explicitly construct a config with [dream_rnd] enabled = true — a conscious R&D act.

This keeps the riskiest layer (the one place the system reasons over its own outputs, once R3 lands) gated by construction, not by remembering to be careful.

DreamRnDDisabledError

Bases: RuntimeError

An R&D dream entry point was called while the feature flag is OFF (the default).

This is the boundary: the interpreter panel / adjudicator never run unless [dream_rnd] enabled = true is set deliberately. Not reachable from the live path.

rnd_config(config=None)

require_rnd_enabled(config=None)

Fail closed unless the R&D flag is explicitly on. Called by every R0/R1 entry point.