# Zeno Divergent (`zeno-divergent-v1`)

A model for **model-relative surprise**: it does not forecast the world, it
forecasts when the models that describe the world are about to stop being valid.

Named for Zeno's dichotomy — an approach that never quite arrives — and for
divergence in its literal sense: the accumulating gap between what a reference
expectation asserts and what is actually observed. The learned object is that
gap and its structure, not the observation itself.

---

## 1. What the model does

```
S_θ(X_t, R_t, C_t, M_t) → (z_t^S, U_t, T_t, I_t, A_t)
```

| Symbol | Meaning |
|---|---|
| `X_t` | observed state |
| `R_t` | the set of reference expectations (predictive, physical, historical, ensemble, policy…) |
| `C_t` | context |
| `M_t` | sensor / modality health |
| `z_t^S` | ten-axis surprise latent with a region topology |
| `U_t` | unknown / blindness channel |
| `T_t` | typed surprise (which *kind* of departure) |
| `I_t` | consequence estimate |
| `A_t` | admission action — including `REQUEST_INFORMATION` |

The unit of representation is a reference model's expectation and the way that
expectation is failing. That is what makes the architecture domain-transferable
in principle — and what remains untested in practice.

### The claim under test

> A learned representation of model-relative surprise contains information about
> future regime change and model failure that is **not** captured by predictive
> uncertainty alone.

The harness is built to be able to reject this. On the shipped synthetic corpus
it currently does.

## 2. Quick start

```python
from transformers import pipeline

detector = pipeline("surprise-detection",
                    model="mbarbosa1/zeno-divergent-v1",
                    trust_remote_code=True)

events = [{"action": "transfer", "amount": 1000 + 40 * i, "gap_ms": 900} for i in range(28)]
events += [{"action": "transfer", "amount": 9000 + 1500 * i, "gap_ms": 25} for i in range(12)]

report = detector(events, entity_id="account_1042")
print(report["kind"], report["potential_units"], report["blindness"], report["action"])
```

Lower level, with your own reference expectations:

```python
from transformers import AutoModel

model = AutoModel.from_pretrained("mbarbosa1/zeno-divergent-v1", trust_remote_code=True).eval()
report = model.score_sequence(reference_steps)   # [T][K] dicts, or a [T, K, 6] array
```

Each step is a list of references with the fields
`prediction, uncertainty, residual, role, validity, freshness`. If you pass raw
events instead, a documented light adapter in `pipeline.py` maps them to a single
reference stream — adequate for a demo, not for evaluation.

## 3. Reading the output

| Field | How to read it |
|---|---|
| `kind` | which region of `z^S` the state sits in — the typed surprise, not a severity |
| `potential_units` | Ψ, the surprise potential; a **ranking**, not a probability of harm |
| `blindness` | evidence loss. **High blindness with a low index means "not observed", never "not at risk"** |
| `action` | admission control; `REQUEST_INFORMATION` means the model declines to make a validity claim |
| `horizons` | near / medium / long model-failure probabilities, Brier-scored, not calibrated in the absolute sense |

`calibration_table.json` in this repo gives held-out percentiles of the index and
a reliability breakdown. `kind_rules.json` gives the region taxonomy and the read
rule for each region. Below roughly 15 events an entity is cold-start and the
report should be treated as provisional.

## 4. Intended use

**In scope**
- Research on representations of model-relative surprise and model-validity forecasting.
- Advisory decision support where a named human owns the decision and can see the rationale.
- Prioritising observation: which measurement would most reduce ambiguity about validity.
- Benchmarking new surprise or change-point detectors against a shared, locked protocol.
- Reproducing or falsifying the claim in section 1 with your own corpus.

**Out of scope**
- Any automated or safety-critical action — evacuation, trade execution, grid switching, dispatch.
- Use as a probability of a physical event. This predicts *model failure*, not the hazard.
- Aggregating domains into a single risk number; the output contract has no such number.
- Any use of this synthetic checkpoint that implies real-world calibration.
- Scoring people. The architecture is entity-conditioned; entities are meant to be
  *models, sensors, instruments and processes*, not individuals.

### Ethical limits

These are constraints on use, not aspirations.

| Limit | Why | What it means in practice |
|---|---|---|
| No decisions about individuals | The surprise index is a statement about a *model's* validity; applied to a person it becomes an unaccountable suspicion score with no right of reply. | Do not bind entity ids to natural persons, accounts, or households in a deployed system. |
| No unattended enforcement | Every reported number comes from a simulator that also wrote the labels. | Every output must reach a human who can see the rule trace before anything is actuated. |
| Blindness must never be read as safety | A low index inside an unobserved region is absence of evidence. | Any integration must surface the blindness channel next to the index; suppressing it is a misuse. |
| No implied real-world calibration | Percentiles come from a synthetic held-out run. | Do not present percentiles or Ψ to end users as risk without recalibrating on your own data. |
| Provenance must travel | A reference silently retrained underneath the model invalidates its surprise memory. | Carry `version` and `role` for every reference; drop memory when a version changes. |
| Disparate-impact risk is unmeasured | No fairness evaluation exists, because there is no real corpus. | Treat any population-level use as unevaluated until you run your own subgroup analysis. |
| Dual use | A detector for "where a model is about to fail" is also a map of where a system is blind. | Do not publish live blindness maps of infrastructure you do not operate. |

## 5. Training data and provenance

**Origin.** Entirely synthetic. Generated by `reference_implementation/data.py`
(`make_corpus`) inside this repository — no scraping, no third-party dataset, no
licensed corpus, no human subjects, no personal data, no real observational,
financial or infrastructure records anywhere in this release.

**Composition.** Episodes of reference tensors `[T, K, 6]` with per-step context
`[T, 4]`, labelled at the generator by mechanism:

| field | value |
|---|---|
| split sizes | 384 train / 192 validation / 256 test episodes |
| seeds | disjoint per split; corpus seed `0` |
| references per step (K) | multi-reference, role- and version-tagged |
| per-reference features | prediction, residual, calibration, sensor health, coverage, age |
| horizons | 1, 4, 10 steps (default report horizon index 1) |
| mechanisms | `none`, `cross_model_conflict`, `support_drift`, `constraint_tension`, `sensor_blindness`, `class_failure`, `modality_loss`, `recovery`, `delayed_consequence`, `false_anomaly` |
| negative-control mechanisms | `recovery` and `false_anomaly` — divergence with **no** break |
| prevalence | reported in `benchmark.json` with every result; never implied |

**Labelling.** Labels come from the generator, not from a residual threshold
applied after the fact, so a detector cannot recover them by thresholding the
signal it is scored on. `recovery` and `false_anomaly` exist specifically to
separate a detector from a magnitude threshold.

**Splits and leakage control.** Train, validation and test corpora use disjoint
seeds. Thresholds are locked on validation at a fixed step-FPR and applied
unchanged to test; no method, including this one, sees the test split before its
operating point is fixed.

**Known provenance limitation.** Because labels are generator-side, the
representation may separate *generator* mechanisms rather than real ones. Only a
real corpus resolves this, and none is used here.

**Digests of record for this release** are in `release.json` (`checkpoint_sha256`,
`benchmark_sha256`, data hash). If those digests do not match your rebuild, you
are not looking at the same artifact.


## 6. Evaluation protocol — SurpriseBench 1.0.0

All methods see identical inputs and identical labels.

| Metric | Meaning |
|---|---|
| **AUPRC** | primary metric, reported with prevalence; AUROC is never reported alone |
| lead@FAR | mean steps of warning before onset at a fixed step false-positive rate |
| step-FPR | realised alarm rate at the locked threshold |
| FA(clean) | fraction of clean episodes that fire at all |
| detection | fraction of break episodes detected before onset |
| ECE | calibration error of the probability itself |

Mandatory controls ship in every result file: `control_constant`,
`control_random`, and a label-shuffled copy of each method. A method that fails
to beat all three is flagged, and the guard block records why in plain text.
Baselines: ensemble variance, predictive entropy, OOD score, anomaly score,
conformal width, CUSUM change-point, kNN novelty, disagreement × impact, and a
supervised logistic model over the same signals — the honest ceiling.

**Verdict rule:** the claim counts as supported only if the surprise index beats
every control and the best baseline on AUPRC on held-out episodes.

## 7. Current result — supported on synthetic data only

SurpriseBench 1.0.0, held-out test split, seed 0, prevalence 0.399, thresholds
locked on validation at step-FPR 0.10, horizon 4.

Intervals are 95% bootstrap CIs resampled over whole episodes (never individual
steps), so correlated steps inside one episode cannot narrow them artificially.
Two methods whose intervals overlap are not separated by this benchmark.

| Method | AUPRC | AUPRC 95% CI | AUROC | AUROC 95% CI | lead@FAR | FA(clean) | ECE |
|---|---|---|---|---|---|---|---|
| **surprise_index (this model)** | **0.990** | [0.988, 0.992] | 0.993 | [0.992, 0.995] | 7.25 | 0.262 | 0.030 |
| logistic_signals (supervised ceiling) | 0.894 | [0.869, 0.916] | 0.921 | [0.906, 0.934] | 4.10 | 0.338 | 0.042 |
| conformal_width | 0.715 | [0.673, 0.756] | 0.706 | [0.661, 0.753] | 4.83 | 0.000 | 0.139 |
| cusum_changepoint | 0.699 | [0.647, 0.744] | 0.803 | [0.781, 0.826] | 2.66 | 0.662 | 0.162 |
| ood_score | 0.691 | [0.656, 0.723] | 0.719 | [0.694, 0.742] | 5.13 | 0.354 | 0.186 |
| anomaly_score | 0.686 | [0.644, 0.723] | 0.720 | [0.694, 0.750] | 2.91 | 0.538 | 0.144 |
| disagreement_x_impact | 0.604 | [0.558, 0.649] | 0.696 | [0.674, 0.717] | 3.29 | 0.369 | 0.278 |
| knn_novelty | 0.589 | [0.553, 0.622] | 0.720 | [0.696, 0.743] | 7.40 | 0.615 | 0.258 |
| ensemble_variance | 0.396 | [0.353, 0.440] | 0.371 | [0.338, 0.407] | 1.95 | 0.354 | 0.327 |
| predictive_entropy | 0.284 | [0.256, 0.316] | 0.239 | [0.214, 0.268] | 5.64 | 0.462 | 0.598 |
| control_constant | 0.394 | [0.354, 0.442] | 0.500 | [0.500, 0.500] | — | — | — |
| control_random | 0.399 | [0.367, 0.434] | 0.500 | [0.489, 0.511] | — | — | — |
| surprise_index, labels shuffled | 0.399 | — | 0.502 | — | — | — | — |

The surprise-index interval [0.988, 0.992] does not overlap the supervised
ceiling's [0.869, 0.916], so the separation is not a single-seed artefact of the
point estimates — it is still a single simulator and a single seed, which is the
larger caveat below.


Guards pass: the index beats both controls, beats its own label-shuffled copy at
chance, and beats the supervised ceiling on AUPRC. Verdict recorded in
`benchmark.json` as `supported`.

**How to discount this number.** The corpus is generated by the same simulator
that defines the labels, so the ceiling is unnaturally reachable and the margin
over baselines is not transferable evidence. Two uncertainty baselines score
*below* chance (AUROC 0.24 and 0.37), which says the generator makes predictive
uncertainty actively misleading — a property of this corpus, not a property of
the world. Read this release as: the architecture trains, the protocol can
return a negative verdict, and the claim survives its first falsification
attempt on synthetic data. Nothing more.


## 8. Failure modes

| Failure mode | Symptom | Mitigation in the current design |
|---|---|---|
| Blindness read as calm | a modality disappears, residuals fall, surprise falls with them | separate blindness channel; sensor health is an explicit input, never imputed away |
| False anomaly | large transient with no structural change | `false_anomaly` episodes count against FA(clean) |
| Recovery misread as break | divergence builds then resolves | `recovery` episodes are labelled negative |
| Correlated references | "independent" references share lineage, so agreement is fake | roles and versions are carried; correlation is **not** yet detected — open limitation |
| Alarm inflation | lead time bought by alarming constantly | fixed-FPR thresholds locked on validation; lead is `n/a` above the operating point |
| Delayed consequence | divergence appears long before the break lands | `delayed_consequence` mechanism; long-horizon head |
| Label leakage from the generator | the representation separates *generator* mechanisms, not real ones | acknowledged; only a real corpus resolves it |
| Silent reference retraining | a reference changes underneath the model | the `version` field invalidates surprise memory for that reference |

## 9. Limitations — read before citing anything

- The corpus is synthetic; the generator encodes an assumption about how expectations break.
- No real-world validation in Earth observation, finance or infrastructure.
- No cross-domain transfer test; the model-family claim is untested.
- Labels are generator-side mechanism labels, not operator judgement of harm.
- Baselines are proxies, not independently fitted deep ensembles or a real conformal calibration set.
- The margin over baselines is measured inside the generator that produced the labels; treat it as a sanity result, not evidence of skill.
- The result is a single seed at one operating point; no bootstrap CI is reported for the headline table.
- Nothing here should be deployed.

## 10. Path to a validated release

1. Real corpora with operator-labelled structural breaks per domain.
2. Retrain with domain-held-out splits; report per-domain and pooled metrics with bootstrap CIs.
3. Strengthen baselines: trained deep ensembles, real conformal calibration, supervised change-point detection.
4. Transfer test: train on two domains, evaluate cold on the third.
5. Only then publish weights that claim anything about the world.

## 11. How to reproduce — complete checklist

Every number in this card comes from these commands. Run them in order from the
repository root. Expected wall time on a laptop CPU is given per step; no GPU is
required at this scale.

**0. Environment** — Python 3.11+, CPU is enough.

```bash
git clone https://huggingface.co/mbarbosa1/zeno-divergent-v1 && cd zeno-divergent-v1
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -c "import torch, transformers, safetensors; print(torch.__version__)"
```

**1. Verify you have the artifact of record** (~seconds). The digests must match
`release.json`; if they do not, stop — you are reproducing a different model.

```bash
python - <<'PY'
import hashlib, json, pathlib
rel = json.loads(pathlib.Path("release.json").read_text())
for f, key in (("model.safetensors", "checkpoint_sha256"), ("benchmark.json", "benchmark_sha256")):
    got = hashlib.sha256(pathlib.Path(f).read_bytes()).hexdigest()[:12]
    print(f, got, "== " + rel[key], "OK" if got == rel[key] else "MISMATCH")
PY
```

**2. Load the shipped weights and score one sequence** (~1 min, first run downloads nothing else).

```bash
python - <<'PY'
from transformers import AutoModel
m = AutoModel.from_pretrained(".", trust_remote_code=True).eval()
r = m.score_sequence([{"amount": 1000 + 40*i, "gap_ms": 900} for i in range(30)], entity_id="demo")
print(r.to_dict())
PY
```

**3. Re-run SurpriseBench on held-out episodes** (~3-6 min). This regenerates
`benchmark.json` — the file this card, the leaderboard and the Space all render.

```bash
python -m model.bench.run --seed 0 --train 384 --val 192 --test 256
```

Expected: thresholds locked on validation at step-FPR 0.10, controls present
(`control_constant`, `control_random`, label-shuffled), guards passing, and
**verdict `supported`** for `surprise_index` at AUPRC ~0.99 against a supervised
ceiling of ~0.89. If the controls are missing from your output file, the verdict
is meaningless regardless of what it says.

**4. Check the protocol itself** (~30 s).

```bash
python -m pytest model/bench/test_protocol.py model/sawm -q
```

**5. Retrain from scratch** (~20-40 min CPU). Deterministic given the seed.

```bash
python -m model.sim.train --epochs 30 --seed 0
python -m model.bench.run --seed 0            # re-benchmark the new checkpoint
```

**6. Rebuild the published bundles without pushing** (~1 min).

```bash
python -m model.sim.export_hf                 # dry run: builds model/artifacts/hf/{model,space}
```

**7. Publish** (requires a write token; the token is read from `HF_TOKEN`,
`--token-file`, or a hidden prompt — never passed as a flag).

```bash
export HF_TOKEN=...     # or omit and let the exporter prompt
python -m model.sim.export_hf --owner <your-org> --push
```

The exporter resolves a semver from the checkpoint and benchmark digests, tags
both repos, then re-downloads the published safetensors and re-runs a sanity
inference that must match `benchmark.json` before it reports success.

**8. Run the demo Space locally** (~1 min).

```bash
pip install -r model/space/requirements.txt
ZENO_MODEL_ID=. python model/space/app.py     # tabs: stream, SurpriseBench, memory & admission control
```

**What "reproduced" means here.** Matching digests in step 1, and in step 3 the
same verdict, the same passing guards, and the same method ordering on AUPRC.
Exact metric values can drift in the last decimal across BLAS builds; the verdict
and the ordering must not.

## 12. Citation

```bibtex
@software{zeno_divergent_2026,
  title  = {Zeno Divergent: a learned representation of model-relative surprise},
  year   = {2026},
  note   = {Research preview; synthetic data only; central claim supported on the shipped synthetic corpus and unvalidated on real data.},
  url    = {https://huggingface.co/mbarbosa1/zeno-divergent-v1}
}
```

Licence: research preview, not for operational use.