research preview · frozen synthetic checkpoint
Learning the structure of what our models fail to expect.
Traditional models learn the patterns of the world. Zeno learns the boundaries of those patterns — where a reference stops being valid, which unexpected futures remain credible, and what should be observed or reconsidered next.
- learned object
- z^S · 10 axes
- heads
- failure · regime · tail · impact
- output
- SurpriseReport
the falsifiable claim
A learned representation of model-relative surprise contains information about future regime change and model failure that predictive uncertainty alone does not capture.
Stated so it can fail. The surprise index and eight conventional detectors are scored on identical inputs and identical labels; if a baseline matches it, the claim is not supported on that corpus and the harness says so.
held-out evaluation · surprisebench/1.0.0 · thresholds locked on validation
Every detector, identical inputs, identical labels — controls included
The primary column is AUPRC, read against a label prevalence of 0.399. A constant predictor and a uniform random predictor are scored in the same table, and every method is re-scored against shuffled labels; a method that does not clearly beat all three has not been shown to work. Lead time at a fixed 10% step false-positive rate is reported only when a method respects that operating point — an always-firing detector shows —, not a long warning.
| method | AUPRC | AUROC | lead@FAR | step FPR | detect | ECE | flags |
|---|---|---|---|---|---|---|---|
| SI — z^S failure head | 0.990 | 0.993 | 7.25 | 0.100 | 1.000 | 0.030 | |
| Logistic on hand-engineered signals | 0.894 | 0.921 | 4.10 | 0.069 | 0.649 | 0.042 | |
| Conformal width | 0.715 | 0.706 | 4.83 | 0.075 | 0.508 | 0.139 | |
| CUSUM change-point | 0.699 | 0.803 | 2.66 | 0.080 | 0.340 | 0.162 | |
| OOD score | 0.691 | 0.719 | 5.13 | 0.078 | 0.518 | 0.186 | |
| Anomaly score | 0.686 | 0.720 | 2.91 | 0.069 | 0.429 | 0.144 | |
| Disagreement × impact | 0.604 | 0.696 | 3.29 | 0.080 | 0.393 | 0.278 | |
| kNN novelty | 0.589 | 0.720 | 7.40 | 0.139 | 0.707 | 0.258 | |
| Control — uniform random | 0.399 | 0.500 | 2.99 | 0.099 | 0.220 | 0.260 | |
| Ensemble variance | 0.396 | 0.371 | 1.95 | 0.086 | 0.178 | 0.327 | inverted |
| Control — constant predictor | 0.394 | 0.500 | — | 1.000 | 1.000 | 0.101 | over-alarming · always-on · constant |
| Predictive entropy | 0.284 | 0.239 | 5.64 | 0.104 | 0.319 | 0.598 | inverted |
Synthetic corpus, ten labelled mechanisms including recovery and false_anomaly episodes that contain no structural break. Disjoint train/validation/test seeds, thresholds locked on validation and applied unchanged to test, confidence intervals bootstrapped over whole episodes. These are properties of the harness, not evidence about the world.
structure
The hierarchy
The model is the intellectual centre. Everything above it is subordinate to it.
Surprise Intelligence Model
the learned object
S_θ(X_t, R_t, C_t, M_t) → (z_t^S, U_t, T_t, I_t, A_t). A reference encoder with attention over models, a ten-axis surprise latent with a region topology, horizon heads for model failure and regime change, counter-expectation outputs, and an update policy.
Surprise Kernel
the interface to what already exists
Adapters wrap existing predictors, simulators, observations, constraints, memory and consequence models. Nothing is replaced and no model is blended; the kernel supplies the shared reasoning machinery.
Domain products
where it is used and contested
Earth Observation, Financial Services and Critical Infrastructure — three packs over one kernel, coupled explicitly and evaluated in their own units.
kernel
Six primitives, one kernel
Three domain packs share the same reasoning machinery while keeping their own units, references and constraints.
Reference graphs
Multiple independent expectations per question, with an explicit disagreement structure instead of one consensus number.
Typed surprise
Surprise is classified — magnitude, timing, structure, mechanism, coverage — never collapsed into a single anomaly score.
Unknown / blindness
A first-class channel for what could not be observed, separating low risk from low visibility.
Credible tail search
Bounded search over physically and institutionally admissible futures the current references do not carry.
Model validity
Admission control decides whether a model may still speak: ignore, remember, open a regime, request information, rebuild.
Surprise memory
Events are stored with magnitude and novelty kept apart, so recurrence is never mistaken for novelty.
integration
Three contracts, nothing else
Zeno runs as a sidecar: you keep your predictors, send what reality did and what your models expected, and receive one report. These are the exact schemas in model/sim/schemas.py.
ProcessObservation
What reality did, in the sending organisation's own units. One record per entity per timestamp; sensor health is carried explicitly so absence of signal is never read as absence of risk.
- domain*string
- Domain pack the record belongs to (earth | finance | critical | custom).
- entity_id*string
- Stable identifier of the observed entity (parcel, book, substation).
- episode_id*string
- Groups a contiguous run of observations for lead-time evaluation.
- timestamp*string (ISO 8601)
- Observation time, UTC.
- stateRecord<string, number>
- Observed state variables in native units.
- contextRecord<string, number>
- Exogenous conditions that are not the target of prediction.
- modalitiesSensorHealth[]
- Per-modality availability, freshness, coverage, quality, expected availability.
- data_qualityRecord<string, number>
- Per-field quality scores in [0, 1].
- source_provenanceRecord<string, unknown>
- Where the record came from; travels into the report unchanged.
- labelsRecord<string, unknown>
- Optional supervision: break onsets, operator judgements, realised consequence.
{
"domain": "earth",
"entity_id": "parcel-0421",
"episode_id": "2026-fire-season-north",
"timestamp": "2026-08-23T12:00:00Z",
"state": { "fuel_moisture": 0.11, "wind_speed": 42.0 },
"context": { "days_since_rain": 31 },
"modalities": [
{ "modality": "thermal_ir", "available": false, "coverage": 0.0,
"freshness_seconds": 86400, "expected_availability": 0.95 }
],
"data_quality": { "fuel_moisture": 0.7 },
"labels": { "break_onset": null }
}domains
Three domains, coupled explicitly
Each domain names its sub-areas, its reference set and the unit its consequence is measured in. Nothing here is live in production; maturity is stated per sub-area. Coupling edges convert an output in one domain into the receiving domain's own unit — there is no universal risk score and models are never blended.
Earth Observation
unit · physical magnitude × exposure
- wildfire & fuel statebuild
- extreme weatherbuild
- flood & hydrologybuild
- drought & vegetationbuild
- ocean & sea-surface temperatureresearch
- land-use & urban changeresearch
refs: physics model · ensemble · persistence · satellite retrieval
Financial Services
unit · money
- onchain protocol & liquidity riskbuild
- uncollateralized credit (lender-side)build
- payroll & payout railsnear
- treasury & liquiditybuild
- desk / market risk overlaybuild
- insurance & catastrophe pricingresearch
refs: originator model · peer cohort · cash-vs-tape · yesterday's model
Critical Infrastructure
unit · safety & continuity
- identity / UEBAbuild
- OT & industrial controlbuild
- grid & energybuild
- water & municipal systemsresearch
- detection-model monitoringnear
- defence picture assuranceresearch
refs: role cohort · plant grammar · change calendar · sensor health
cross-cutting
near = implemented · build = needs a tenant and labels · research = weak labels or physics
Model assurance / SurpriseBench
The part that exists today: grading other models' stale worlds. Any external predictor can be run as the subject under the frozen protocol — typed miss, lead time at a fixed false-alert rate, calibration and degeneracy guards, with mandatory constant, random and shuffled-label controls. It is domain-independent and it publishes its losses.
open SurpriseBench →coupling edges — unit conversion, not a universal risk score
fire / drought Ψ → catastrophe price and borrower cash-flow
km² → USD
heat + drought → grid reserve margin and trip potential
°C → service-hours
outage → payroll, working capital and insured loss
service-hours → USD
documentation
Model card
One document covering intended use and what is explicitly out of scope, the synthetic training corpus, the fixed-false-alarm-rate evaluation protocol, eight catalogued failure modes, and how the model behaves when it is uncalibrated or blind.
- —Intended use & out of scope
- —Failure modes with mitigations
- —Calibration & unknown behaviour
- —Limitations and path to publication
zeno-divergent-v1 0.1.0 · synthetic corpus · claim not supported at this release