Research record saved
Zeno’s open
learning record.
Training complete · exact replay checked
- Saved
- 9 / 9
- Updates
- 24,570
- Fields
- 13
- Now
- —
One model serves every field; unproven fields retain its reference. Forecast gains do not establish warning accuracy.
The race · held-back error
0.604
0.613
0.601
0.600
0.612
0.591
0.625
0.601
0.589
Spotlight / crypto & markets
Where prices meet surprise.
Crypto
BTC/ETH hourly · price-feed references + Deribit option-trade ranges
Exam 58.5% vs tree model (GBM) · 55.4…61.2 · n 19,202
Prediction markets
venue crowd odds vs resolution
Exam 0.2% vs calibrated crowd · -0.2…0.5 · n 11,333
Cross-venue
same question, two venues
Exam -9.7% vs calibrated crowd · -32.1…32.7 · n 33
Held-back days, not a trading signal. These exam scores compare with fitted rivals, not the previous model.
00 / who is in the room
Independent families, field by field.
Agreement only counts when it comes from different families. Bars = independent families in a typical question; copies of one family count once.
Teal = several independent families · ochre = one producer (single-forecast correction, not a crowd)
00b / the test
Fixed before the first update.
Sealed before training
5efecd314ec23acbdcebfa58927e53ebc853fb4cbaa33007cabcc93f0e2fdac1
Fingerprint of the test the run wrote before its first update. Results are judged only by it.
- 01RivalsStrongest of panel average, calibrated crowd and tree model, picked on selection weeks only.
- 02Corrected winA gain counts only if its lower bound stays above zero after correcting for testing every field at once.
- 03Later weeksThe later half of the test weeks must still show a gain (95% lower bound above zero).
- 04WarningThe 'everyone misses together' warning is useful only if its ranking range stays above chance.
- 05ServingA field moves to v1.06 only with a corrected win AND a later-weeks hold; otherwise it keeps its v1.05 answer.
- 06HumansForecastBench is also compared with median humans and superforecasters (reported, never used to pick the rival).
- 07Late recordsAnything received after its outcome was known is excluded.
00c / results
Each field, judged on its own.
| Field | Edge | Corrected range | Later weeks | Warning | Decision |
|---|---|---|---|---|---|
| crypto | +58.5% | +53.7% to +62.3% | holds +52.7% | not testable | exam gate passed · release pending |
| economy ecb | +24.4% | +3.8% to +52.8% | holds +59.0% | not testable | exam gate passed · release pending |
| economy philfed | +0.5% | -4.5% to +2.4% | does not hold | not testable | retain public model |
| energy | +1.0% | -9.0% to +6.4% | does not hold | not testable | retain public model |
| forecastbench | -1.9% | -7.6% to +3.0% | does not hold | not testable | retain public model |
| health covid | -1.6% | -9.0% to +5.6% | does not hold | 0.55 | retain public model |
| health flu | +99.9% | -25.5% to +100.0% | does not hold | 0.68 useful | retain public model |
| health rsv | -17.2% | -48.5% to +1.1% | does not hold | 0.60 | retain public model |
| markets | +0.2% | -0.4% to +0.7% | does not hold | not testable | retain public model |
| markets cross venue | -9.7% | -44.7% to +72.6% | does not hold | not testable | retain public model |
| sea | +93.8% | +92.0% to +95.6% | does not hold | not testable | retain public model |
| weather ensemble | -1.6% | -4.9% to +2.5% | does not hold | 0.41 | retain public model |
| weather history | +34.3% | +29.4% to +38.8% | holds +28.4% | not testable | exam gate passed · release pending |
01 / data river
Every panel, where it went.
02 / preparation
Reading the corpus, field by field.
● checked · converted · admitted ● split by time
03 / exposure
Proof each field was learned from.
Darker = more of the field's training panels read. Blank = learner not started.
04 / architecture
One model reading every model.
- Forecasters≤256 per panel
- Familiescopies count once
- Past missesknown at decision
- Shared coreS · M · L
- Field adapters13 fields
- AnswerSit outMember warningFamily warningPanel warningAll-families warning
05 / size
Does bigger learn more?
06 / answers
What it is allowed to say.
07 / exam
Unseen weeks.
crypto · research only · release test not passed
58.5%
lower error vs tree model (GBM) · 95% interval 55.4 to 61.2 · n 19202
economy-ecb · research only · release test not passed
24.4%
lower error vs tree model (GBM) · 95% interval 9.4 to 41.8 · n 82
economy-philfed · research only · release test not passed
0.5%
lower error vs panel average · 95% interval -2.2 to 1.9 · n 475
energy · research only · release test not passed
1.0%
lower error vs tree model (GBM) · 95% interval -7.0 to 5.3 · n 2096
forecastbench · research only · release test not passed
-1.9%
lower error vs panel average · 95% interval -5.5 to 1.5 · n 14355
health-covid · research only · release test not passed
-1.6%
lower error vs panel average · 95% interval -7.3 to 3.4 · n 1850
health-flu · research only · release test not passed
99.9%
lower error vs tree model (GBM) · 95% interval -10.4 to 100.0 · n 6415
health-rsv · research only · release test not passed
-17.2%
lower error vs panel average · 95% interval -39.2 to -2.1 · n 624
markets · research only · release test not passed
0.2%
lower error vs calibrated crowd · 95% interval -0.2 to 0.5 · n 11333
markets-cross-venue · research only · release test not passed
-9.7%
lower error vs calibrated crowd · 95% interval -32.1 to 32.7 · n 33
sea · research only · release test not passed
93.8%
lower error vs panel average · 95% interval 92.6 to 95.1 · n 2815
weather-ensemble · research only · release test not passed
-1.6%
lower error vs calibrated crowd · 95% interval -4.5 to 1.2 · n 6543
weather-history · research only · release test not passed
34.3%
lower error vs panel average · 95% interval 30.7 to 37.5 · n 2464
08 / log
Nothing behind the curtain.
00:20:57 UTC · 47 of 47
Execution stopped: ArrowTypeError: ("Expected bytes, got a 'Timestamp' object", 'Conversion failed for column available_at with type object')
Exam weeks · better than each field's best simple rival (95%)
How each field's module inside Zeno v1.05 was chosen: the most honest test
| Field | v0.98 (earlier preview) | v1.0 | v1.01 (published) | n (v1.01) |
|---|---|---|---|---|
| Crypto | +12.3% | +9.9% | +7.4% | 17138 |
| Cross-venue | +26.0% | not measured | +12.1% | 16 |
| Flu | not measured | not measured | +16.4% | 3553 |
| COVID | +6.7% | +3.5% | -2.4% | 1331 |
| Markets | -0.1% | not measured | +0.4% | 10220 |
| ForecastBench | not measured | -0.7% | -1.9% | 4819 |
| RSV | -5.2% | not measured | -1.0% | 521 |
| Philly Fed | not measured | not measured | -2.4% | 380 |
| ECB survey | not measured | -9.0% | -17.3% | 58 |
| How the test was run | ||||
| ForecastBench dates | corpus dates only | public resolution files | public resolution files | |
| Flu outcomes | not trained | not trained | earliest report only | |
| Sitting out | any expected gain | only above +2% | only above +2% | |
| Fields trained | 10 | 10 | 12 | |
| Fresh-reload check | exact | exact | exact | |
Energy, sea and weather: fewer than 5 exam weeks in every run · not measured. A higher score under a weaker test is not a better model.
Zeno30 integration · forward results only
Does Zeno earn a say in the team's answers?
A topic switches on only after Zeno beats the team's frozen first forecast on 40 of the team's resolved questions, with the 95% range still above zero. Backtests don't count.
Loading…
Earlier runs · open record
Previous run · forecast-gain test (stopped)
Live Read · learning vs memorising
Watch Zeno learn — or memorise.
Each test changes one thing and trains matched models side by side. The gap between the dashed and solid lines is memorising.
No readings saved yet. No earlier test is substituted.
One question at a time
9 matched tests, one question each.
Audit scope: reused timing splits can retain late answers; wider ranges also changed spread and loss balance. These are exploratory selection readings, not clean unseen-week proof.
Does learning to raise warnings hurt the forecasts?
0 of 6 models saved · comparison unavailable
Does a smaller model or stronger limits on memorising raise skill on unseen weeks?
0 of 9 models saved · comparison unavailable
Does telling the model each forecaster's past misses help on weeks it hasn't seen?
0 of 6 models saved · comparison unavailable
Does giving COVID-19 and RSV room for extreme outcomes help without hurting other fields?
0 of 6 models saved · comparison unavailable
Does a dedicated part for rare extreme outcomes beat the standard range?
0 of 6 models saved · comparison unavailable
Do distinct past misses help when late training answers are excluded?
0 of 9 models saved · comparison unavailable
Do the past misses themselves carry information, once they can be read fast enough to finish?
0 of 12 models saved · comparison unavailable
Do past misses carry information? Completing the four-way test with verified recovery.
0 of 12 models saved · comparison unavailable
Does a bigger model, fairer training weights or past misses beat a calibrated crowd on unseen weeks?
0 of 21 models saved · comparison unavailable
Field by field, on unseen weeks
Waiting for the first finished model.
Practice test · selection weeks · held-back weeks never opened · n 0 fixed-sample outcomes
evidence +−
The trained model, answering from its saved files
Reloaded from scratch, it reproduced every graded score. Its combined three-seed answer beat the calibrated crowd by 5.5% on unseen weeks. Warnings show no measurable edge yet (24 misses, 4 weeks). Shadow only: not served.
Right of centre = better than the plain panel. Orange = model, teal = calibrated crowd. Fresh requests rebuilt from raw inputs matched the graded answers on 728 of 728 rows. Each field has 1–3 weeks, so field ranges are not measured.
Practice test · Selection weeks; not a live test · n 600 outcomes, 13 week blocks
evidence +−
Where past misses helped, field by field
Uniform history helps some fields and hurts others. Exploratory, not a per-field claim.
Right of centre = better. Each field has too few weeks for a range. RSV gains as much with swapped values, so that gain is not credited to real history.
Practice test · 13 unseen selection weeks; 1–3 weeks per field · n 64 sampled outcomes per field × 3 seeds
evidence +−
What goes in, and which parts are working.
- ○ no data yetForecasters' answers0 training cases
- ○ no data yetModel of modelsreads every forecaster together
- ○ no data yetOutcome range + warningstandard range
- ○ no data yetLinked eventsnot measured
- ○ no data yetMultiple choiceuntested
- ○ no data yetZeno30 answersnot measured
Latest experiment
Learning vs memorising
So far: warning training doesn't hurt forecasts, and model size and stronger limits don't lift unseen skill above about 5–7%. The earlier past-misses test kept late answers in training, so the current test repeats it with them removed and a shuffled control.
Does a bigger model, fairer training weights or past misses beat a calibrated crowd on unseen weeks?
Stopped 31 Oct: Hugging Face canceled the job (account out of credits) after 14 of 21 fits were saved. No final matched comparison against the rivals was produced. Interim readings from two finished seeds lean towards the larger size (about 5.7% vs 4.8% skill against the reference), which is not a result. Saved fits are kept for hash-checked recovery.
Small, no past misses · 0 seeds
Small, past misses · 0 seeds
Larger, no past misses · 0 seeds
Larger, past misses · 0 seeds
Small, every outcome weighted equally · 0 seeds
Small, past misses, outcomes equal · 0 seeds
Small, errors swapped · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 21 models
evidence +−
Earlier experiments · 8
Do past misses carry information? Completing the four-way test with verified recovery.
Finished 31 Oct: all 12 fits, 600 unseen outcomes × 3 seeds, 13 weeks. Switched on for every field, past misses did not reliably help the small model: −1.5% [−2.9%, +1.5%] vs no past misses; real vs swapped misses +1.5% [−0.4%, +2.2%], leaning real but not proven. Effects differ by field (exploratory): COVID-19 and RSV lean up, crypto, sea and ensemble weather lean down; RSV's lean also appears with swapped values, so it is not credited to real history. This tests one uniform history switch, not whether a unified model can work.
Without past misses · 0 seeds
Distinct past misses · 0 seeds
Misses dealt to wrong forecasters · 0 seeds
Errors swapped, dates kept · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 12 models
evidence +−
Do the past misses themselves carry information, once they can be read fast enough to finish?
Eleven of twelve fits completed and saved. Hugging Face reports this job as canceled; the final errors-swapped fit stopped at step 1,750 of 4,000. No final matched comparison was produced. Completed fits are preserved for hash-checked recovery; the unfinished fit is rerun from its original seed.
Without past misses · 0 seeds
Distinct past misses · 0 seeds
Misses dealt to wrong forecasters · 0 seeds
Errors swapped, dates kept · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 12 models
evidence +−
Do distinct past misses help when late training answers are excluded?
A narrow history-input test on the existing small model, not a test of the best architecture. Cannot finish: the past-miss fits measured about 33 minutes per 250 steps (the plain fit took 16 seconds), so the run's 7.5-hour limit stops it before any past-miss fit is saved. Only the no-past-miss fit finished (+4.96% unseen skill, one seed). Its final comparison also had a reader bug. Stopped 31 Oct and replaced by v0.95.1. The whole-list shuffle moves how many past entries a task has and how old they are, not just the error values. Purging is correct within each field; no claim that this would have worked live.
Without past misses · 0 seeds
Distinct past misses · 0 seeds
Shuffled past misses · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 9 models
evidence +−
Does a dedicated part for rare extreme outcomes beat the standard range?
No detectable overall gain: selection skill 5.02% vs 5.05%. Raw paired score difference −0.0003 [−0.0008, +0.0001] on 685 outcomes × 3 seeds. Checkpoint audit (31 Oct): the tails were used (COVID-19 median mass 1.1% on trained-on rows, up to 13%, from a 0.05% start) and scored better on the 8 trained-on outcomes beyond the range (+0.60 raw [+0.23, +1.05]), at a small cost inside it. The unseen sample holds only 1 such outcome, so the benefit on extremes is untested there.
Standard range · 0 seeds
With extreme-outcome part · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 6 models
evidence +−
Does giving COVID-19 and RSV room for extreme outcomes help without hurting other fields?
Worse, but the cause is not isolated. Raw paired score difference −0.042 [−0.069, −0.031] on 685 outcomes × 3 seeds. Scaling COVID-19/RSV also widened the assumed spread of single-number forecasts and shifted the loss balance, so this does not show the range itself was the problem.
Current ranges · 0 seeds
Wider COVID-19/RSV ranges · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 6 models
evidence +−
Does telling the model each forecaster's past misses help on weeks it hasn't seen?
Mixed, and not a clean test. Two defects found 31 Oct: the 8 history slots can hold 8 updates of one past miss, and the reused split can keep training answers released after an unseen forecast was made. Fixed versions are in v0.95.
Without track records · 0 seeds
With last 8 past misses · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 6 models
evidence +−
Does a smaller model or stronger limits on memorising raise skill on unseen weeks?
No. Stronger limits changed nothing; the smaller model memorised less (7% vs 12%) with the same unseen skill. This tested model size, not which inputs matter.
Current setup · 0 seeds
Stronger limits · 0 seeds
Smaller model · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 9 models
evidence +−
Does learning to raise warnings hurt the forecasts?
No. Paired difference −0.003 [−0.017, +0.011]; warning training slightly reduces memorising.
Learns warnings · 0 seeds
Forecasts only · 0 seeds
Skill = shrunk, outcome-weighted error reduction vs the reference forecast
Practice test · training and selection weeks only · held-back weeks never opened · n 0 of 6 models
evidence +−
Cancelled v0.90 / v0.91 campaign
Live Read · Zeno Surprise Intelligence
Watch Zeno learn.
One model. Every field. The current campaign, measured as it happens.
This campaign · unified model
What makes this run different.
No version finished yet
Better than the reference forecast? Plus = better. Early: not yet shown better than the reference on fresh weeks.
Outcome-weighted skill (fields ≥30 outcomes)
—
All fields pooled (−error vs reference)
—
Run stage: unknown
A repaired multi-domain experiment. Not yet proven: better than the reference on fresh weeks, use of every data source (flu, Numinous, rivers, notflat and judgments are held), linked events (not measured) and multiple choice (untested).
Practice test · selection weeks of this run · n 0
evidence +−
One model for every field · size M
Every field goes through the same core. Six capabilities sit on top, and each one has a control version with it switched off.
Question
Task type, field, deadline
Forecasters
Each forecast + its past misses
Dependence groups
Copies count once
Shared core
width 512 · 6 layers
Field adapters
Small per-field tuning
Answer
Range + 3-part warning, else reference
Question
Task type, field, deadline
Forecasters
Each forecast + its past misses
Dependence groups
Copies count once
Shared core
width 512 · 6 layers
Field adapters
Small per-field tuning
Answer
Range + 3-part warning, else reference
Dependence
Knows which forecasters copy each other
Timing & memory
Remembers each forecaster's past misses
Linked events
Uses related questions
Adaptation
Small per-field tuning on one shared core
Distributions
Answers with a full range, not one number
Costed caution
Warns when a confident miss is likely
Zeno replay · design from this run's own config · n 0 planned versions
evidence +−
Campaign map
0 of 0 versions finished. Main score: outcome-weighted skill vs the reference (plus = better; fields under 30 outcomes left out). Pooled loss shown second.
Main · 0
Controls · 0
Single-field · 0
Leave-one-out · 0
Zeno replay · selection weeks of this run · n 0 finished
evidence +−
Does each capability help?
Full model vs the same model with one capability off, on the same held-back weeks. Filled only by the final gate.
Dependence
waiting for the final gate
Timing & memory
waiting for the final gate
Linked events
waiting for the final gate
Adaptation
waiting for the final gate
Distributions
waiting for the final gate
Costed caution
waiting for the final gate
Zeno replay · held-back weeks · n 6 capabilities
evidence +−
What each field trains on and is judged on
Training rows, selection rows and held-back rows per field, as the run split them by time.
teal = training · ochre = selection · orange = held back
Zeno replay · time split recorded at startup · n 0 training rows
evidence +−
Release gate
Rules frozen before training. Best possible result this run: a provisional pass, until fresh weeks are scored.
- Startup check passed on real datapending
- Beats the reference in more than 4 fields (5+ weeks each)pending
- Beats panel median, calibrated crowd and recent-error rivalspending
- v0.89.4 compared on weeks after 29 Oct (pending)pending
Zeno replay · frozen 29 Oct 2026 · n 4 rules
evidence +−
Historical release · v0.89.4 · open summary ForecastBenchmarketsCOVID-19RSV
This earlier release answered in 4 fields where it beat its reference on held-back weeks: ForecastBench, markets, COVID-19 and RSV. It is not the current public model.
Model page →Research archive · earlier runs
one model
13 fields, one model.
crypto
299k
forecasts
- panels
- 84k
- outcomes
- 84k
- state
- in training
● correction tested · ● in training · · 2 held sources, outer ring
every run
Every run. Every miss.
v0.82.1
Stopped during practice
Practice stopped · full training not started
Evidence →v0.85.11
Stopped during practice
Trial failed its planted-pattern check · full run not started
Evidence →v0.86.0
Stopped during practice
Full run stopped · held-back error diverged
Evidence →v0.86.1
Stopped during practice
Full run stopped at 13 of 27 models · save limit
Evidence →v0.86.2
Trained · being judged
Checks passed · full 27-model run training on H200
Evidence →
Each verdict links to its frozen run record
today
What Zeno can actually do.
capability map
5 sources corrected live. Everything else stays honest: your forecast, unchanged.
Correct
Warn
Combine
Next check
served default
v1.06
fallback
v0.57
opt-in beta
v0.80.2
research
v0.82.7
research
v0.86.7
research
v0.86.8
Past record · today · served + research records · n 19 sources × 4 capabilities
evidence +−
data honesty
When did the answers exist?
answer clocks
350 of 1,110 answers existed in time. The rest arrived as modern recaptures — usable for later training cutoffs, never backdated.
ECB panel · original slice
1,110 cases · 81 outcomes
answer existed by the cutoff350
archived vintage, after the cutoff16
modern recapture · clock unverified744
ECB panel · extended slice
1,216 cases · 91 outcomes
answer existed by the cutoff350
archived vintage, after the cutoff16
target not identical · excluded732
modern recapture · clock unverified118
An answer published after its forecast is normal, not leakage. It may train a later run only if it existed by that run's fitting cutoff. First-print status: not established by this audit.
Past record · live audit · checked 2026-10-24 · cutoff 2025-10-31 · n 2,326 cases · 172 outcomes across 2 slices
evidence +−
full record
The full ledger.
Show the evidence
24 fields · 7 areas · 6 already beat every rival
- Warnings · v0.82 warning test
- Brier score of confident-miss warnings vs each forecaster's own track record (tree). historical: development and later confirmation slices of past data. exploratory · release gate not yet passed.
- Corrections · v0.80.2 public beta
- forecast loss vs calibrated reference vs calibrated reference. historical: two sealed test periods. passed fields are served in the opt-in beta.
Atmosphere
- Sea-level pressure1.13M outcomes · 2 forecastersWarnings: no edge yetCorrections: calibrated reference
- Wind1.13M outcomes · 2 forecastersWarnings: no edge yetCorrections: calibrated reference
- Urban heat154k outcomes · 2 forecastersWarnings: no edge yetCorrections: insufficient evidence
- Weather vs analysis49k outcomes · 6 forecastersWarnings: beats every rival (+7.9% vs track record, +10.5% later)Corrections: calibrated reference
- Single-run weather23k outcomes · 7 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
- Weather vs airports22k outcomes · 5 forecastersWarnings: beats every rival (+5.6% vs track record, +6.3% later)Corrections: validated improvement
- Air quality644 outcomes · 1 forecastersWarnings: too few misses to judgeCorrections: insufficient evidence
- Farming rain31 outcomes · 2 forecastersWarnings: too small to judgeCorrections: insufficient evidence
- Ensemble weather0 outcomes · 6 forecastersWarnings: outcomes arrivingCorrections: not trained
Space and sea
- Space weather26k outcomes · 4 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
- Sea12k outcomes · 24 forecastersWarnings: beats every rival (+28% vs track record, +23% later)Corrections: calibrated reference
Markets and crypto
- Crypto84k outcomes · 3 forecastersWarnings: beats every rival (+14.2% vs track record, +13.4% later)Corrections: calibrated reference
- Prediction markets66k outcomes · 1 forecastersWarnings: no edge yetCorrections: calibrated reference
- Market traders4k outcomes · 8248 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
- Cross-venue markets325 outcomes · 617 forecastersWarnings: too few misses to judgeCorrections: insufficient evidence
Health
- COVID-196k outcomes · 16 forecastersWarnings: ahead of most rivals (+0.1% vs track record, +1.1% later)Corrections: calibrated reference
- RSV3k outcomes · 6 forecastersWarnings: ahead of most rivals (+2.3% vs track record, +24% later)Corrections: calibrated reference
- Flu2k outcomes · 66 forecastersWarnings: no edge yet (-15.1% vs track record, +9% later)Corrections: calibrated reference
Economy
- Philadelphia Fed (points)4k outcomes · 197 forecastersWarnings: ahead of most rivals (+37.8% vs track record, +23.8% later)Corrections: calibrated reference
- Philadelphia Fed (ranges)2k outcomes · 195 forecastersWarnings: too small to judgeCorrections: calibrated reference
- ECB1k outcomes · 116 forecastersWarnings: ahead of most rivals (+7.8% vs track record, +16.8% later)Corrections: calibrated reference
Energy
- Energy demand3k outcomes · 1 forecastersWarnings: too few misses to judgeCorrections: validated improvement
General forecasting
- ForecastBench109k outcomes · 806 forecastersWarnings: beats every rival (+45.3% vs track record, +23.1% later)Corrections: not trained
- Numinous154 outcomes · 2 forecastersWarnings: too small to judgeCorrections: not trained
- v0.86.2Trained · being judgedChecks passed · full 27-model run training on H200
v0.86.1 plus the 25 Oct review fixes: error floors set per quantity and unit, a stability check that works for negative scores, dropped-forecaster counts kept with the saved data, and saves that wait out Hugging Face limits. Same 13-field, 7.24M-forecast snapshot; 11 fields trained (ECB and energy admitted, not trained). At 07:10 UTC 25 Oct, 7 of 9 check models had finished 4,000 steps with finite losses (first large model: held-back error 5.01 → best 1.405); all 9 checks then passed the scale check on every trained field, and the full 27-model run (job 6addaf0a) started at 07:26 UTC. An exact measurement found the per-quantity floors changed no training-split scale. No win verdict yet: the matched comparison against simple methods and the served model decides.
- v0.86.1Practice stoppedFull run stopped at 13 of 27 models · save limit
Same 7.24M-forecast snapshot with a train-fitted resolution floor, a bounded loss and a confidence floor. All 9 full-length checks passed. The full run restarted at 04:18 UTC on 25 Oct after a storage stop; 13 of 27 models finished stably (held-back error about 1.3–1.4) before it stopped at about 05:47 UTC on the Hugging Face limit of 320 saves per hour. Its models are kept but not used. ECB and energy are admitted but not trained (no separate development history). Stable loss is not yet evidence of better forecasts: the matched comparison against simple methods and the served model decides.
- v0.86.0Practice stoppedFull run stopped · held-back error diverged
On Offdiagonal v6.13 (7.24M forecasts) the checks passed, and the full run started at 23:47 UTC on 24 Oct. Training error stayed near 1, but error on held-back cases jumped to about 1.6 million in most models after step 600. The cause was sea panels where every forecaster said 0: those got a near-zero scale, so a 0.1 outcome counted as about 15,000 scales off. The run was stopped at 01:27 UTC on 25 Oct. No model from it is used.
- v0.85.11Practice stoppedTrial failed its planted-pattern check · full run not started
Trained 45 practice models (3 sizes × 5 versions × 3 seeds) on about 858,000 verified forecasts in 10 fields. Small and medium models cut their error on unseen cases by about 85%. But 9 of the 15 large models got stuck on one constant score, and the planted-pattern check failed (0.57 against a 0.5 bar) because it trained and scored in different frames. The full run was not started.
- v0.82.1Practice stoppedPractice stopped · full training not started
Checked on Hugging Face on 22 Oct at 02:03 UTC. Nine practice checkpoints were saved before the transfer test requested ensemble weather, which has zero labelled panels in this build. The job stopped at 01:22 UTC; the full 30-round, three-seed run never started. The frozen attempt and saved checkpoints remain unchanged. No final verdict or warning-skill pass is available.
- v0.83Too few outcomesInflation rematch: keep the mean
On 366 eligible HICP inflation cases, the tested ridge and tree corrections underperformed the panel mean: +0.058 and +0.056 MAE on 190 test cases across 12 held-out outcomes. Their 90% outcome-bootstrap intervals are above zero. This tests two summary-feature recipes, not the shared model of models or other ECB targets. Offdiagonal's archive recovery found no qualifying identical-target matches for the 744 excluded cases; first-print status remains unestablished. Valid inflation examples can still contribute to joint training. Serving is unchanged.
- v0.82CandidateFinished · energy, sea and crypto lead
The full v0.82 neural run finished on the H200 on 22 Oct with no failures and was judged against its locked rules; the result is public on Hugging Face. On the confirmation period it corrects forecasts best in energy (17.5% better than track records and the tree learner, 15.6% better than the calibrated reference), sea (25% better than the calibrated reference) and crypto (23%), and helps weather vs airports (10% better than track records). The two warning skills were not saved by this run, and two rivals (the market's own price and a scrambled-relations control) were not measured, so no skill passes on this run alone. v0.82.1 stopped during practice before full training. Serving does not change.
- v0.80.2CandidatePublic on Hugging Face · 21 fields · 2 beat strong references
Rebuilt on Offdiagonal v6.5: 2,840,152 forecast panels with known outcomes in 21 trained fields, including air quality, urban heat, agriculture, world conditions and shared answer sources. The time audit found no forecast used after its decision time. No real-world event is scored in two periods. The approved full run trained all 21 starts on 21 Oct (7 variants × 3 seeds). The corrected judge (averaged forecasts, one test across all fields) passed two fields on both test periods: energy demand (one forecasting family) and weather vs airports (airport timing still assumed). Flu improved on development but failed confirmation. Every other field keeps the calibrated reference or has too few outcomes. It now answers as an opt-in beta (/api/public/v1/beta/check, same answers as the frozen files on 721 real cases) and powers the public Forecast Failure Watch; its weights are public on Hugging Face (ZenoDivergent/zeno-divergent-v0802) as a research beta; the default answer stays v0.57.
- v0.80CandidateFull H200 run done: two research candidates, now being repaired
Trained on 20 Oct on 454,490 cases from 16 sources (5 versions × 3 seeds, 13 minutes on an H200). Flu (+32% against the reference on confirmation) and energy (+4% against a tree model) passed both test periods; every other field kept the calibrated reference, and crypto, RSV, economy and Polymarket traders scored worse. An outside audit then found defects: forecasts dated after the run, energy missing its outcome clock, events grouped by question instead of real-world event, and two empty inputs. So both wins stay research candidates until v0.80.1 reproduces them on repaired data and a fresh period.
- v0.79eExploratory13-source panel reader: scored, research only
Owner approved on 20 Oct. Data rebuilt with corrected answer timing and defect flags; the H200 measured its own speed, then trained all 134 research models (0.5M and 2M) that learn forecaster miss risk and confidently wrong consensus; all were saved and re-checked. Official verdict (v0.79f, 20 Oct): it beats four time-safe references in 7 of 10 testable sources, ties on COVID-19 and Phil. Fed ranges, and loses to a tree on crypto; pair history and the 2M size add nothing. Training is not a result: comparisons against simple references decide anything, and nothing switches the served model.
- v0.79FailedQuick rerun on older data: stopped
A 16-model rerun of the v0.78 recipe on the older data. It was stopped on 20 Oct because it skipped the new sources and the agreed simpler comparisons. Its partial numbers are diagnostic only.
- v0.78tToo few outcomesNeural trust campaign: all 50 models trained, saved and scored
Reading the whole panel together beat each forecaster's track record alone, crypto gained a little, and ensemble weather looked better on few test groups. ForecastBench did worse than the tree, pair history gave no reliable edge, and 2M was no better than 0.5M. Not good enough everywhere to replace the served model.
- v0.77ExploratoryCPU tree experiment across seven sources (provisional)
Not a neural training run: a small tree model on a CPU in about 4 minutes. An outside review found two average rates computed with future answers included, so we re-ran it with every input time-valid and every model calibrated the same way (v0.77r, a diagnostic re-check of weeks already seen). Against each forecaster's own track record it cut error by about 11% on ensemble weather, 21% on single-run weather and 8% on crypto. The earlier ~25% ForecastBench gain disappeared: it now only matches the track record. The economy and Numinous sets are too small to judge.
- v0.76bToo few outcomesFailure Atlas: who misses together
The earlier 35% → 45% readout used ranges drawn too narrow. With ranges corrected on earlier weeks, the test window holds zero group failures, so warnings cannot be judged yet.
- v0.75FailedOne shared model for all sources
Did not beat separate per-source models, so the larger run never started.
- v0.74CandidateEnsemble-wind correction
+7.8% against the team forecast on its sealed wind weeks — but simply removing each system's usual error at each place scored +9.1% on the same cases. The gain is local habit, not learned failure structure; it will not be promoted.
- v1.06ServedServed model
Answers the public Check, API and MCP. Corrects only the sources it passed on, keeps the reference everywhere else.
Served now · one network for every field
One brain. A sealed exam.
1 · Data
7.93M forecasts
Offdiagonal v6.42 @ c76303bc · 107 late records excluded
2 · Rivals
Deribit options · human crowd
independent families counted once
3 · Model
one network · 13 fields
3 sizes × 3 starts, trained jointly
4 · Exam
sealed before training
fingerprint 5efecd314e
- Flutie
- Sea levelnot held
- Crypto+58.5%
- Weather hist.+34.3%
- ECB+24.4%
- Energytie
- Phil Fedtie
- Marketstie
- COVIDtie
- Weather ens.tie
- ForecastBenchtie
- Cross-venuetie
- RSVtie
● gain that held on later weeks · ● gain that did not hold · ● no measurable difference. Training and the sealed exam finished. The package was rebuilt from the saved fits on 8 Nov without retraining and reproduces every exam answer exactly. Now the served model: fields without a proven edge return the reference.
Head to head with the previous model · same cases, same grading
- Crypto +7.9% (95% +4.9% to +10.4%)
- Weather hist. -1.0% (95% -1.7% to -0.4%)
- ECB -0.2% (95% -1.0% to +0.8%)
The exam's stricter grading raises every score, so most of the jump above is grading. Against the previous three-model package, the single network wins on crypto and gives up 1% on weather history. One brain instead of three stapled together, and a sixth of the size. Bigger wasn't better: we checked, so you don't have to.
Checked for the next model
● admitted · Multi-answer crowd odds
11,294 answer odds · 388 questions
venue-reported clocks: research exams only; 2 lottery markets incomplete, excluded
● admitted · Hurricane official forecasts
603 forecasts · 41 storms
proven release time (cycle + 3 h)
● held · Hurricane computer models
9.6M track points
no proven release time yet
● held · Five-model weather
31,755 place-days
archive only: no recorded release time
Data checked for the next model. Not used by the served model. Counts are our own recounts at Offdiagonal's pinned versions.