Research

Every experiment, including the failures.

One open model of models across 24 fields. Two already beat strong references, and early warnings beat every rival in crypto, sea and two weather fields. Every run links to its evidence.

figure 5 · runs and their verdicts
  1. v0.57

    Served

    Served model

    Answers the public Check, API and MCP. Corrects only the sources it passed on, keeps the reference everywhere else.

    Evidence →
  2. v0.74

    Research candidate

    Ensemble-wind correction

    +7.8% against the team forecast on its sealed wind weeks — but simply removing each system's usual error at each place scored +9.1% on the same cases. The gain is local habit, not learned failure structure; it will not be promoted.

    Evidence →
  3. v0.75

    Failed its test

    One shared model for all sources

    Did not beat separate per-source models, so the larger run never started.

    Evidence →
  4. v0.76b

    Not enough evidence

    Failure Atlas: who misses together

    The earlier 35% → 45% readout used ranges drawn too narrow. With ranges corrected on earlier weeks, the test window holds zero group failures, so warnings cannot be judged yet.

    Evidence →
  5. v0.77

    Exploratory

    CPU tree experiment across seven sources (provisional)

    Not a neural training run: a small tree model on a CPU in about 4 minutes. An outside review found two average rates computed with future answers included, so we re-ran it with every input time-valid and every model calibrated the same way (v0.77r, a diagnostic re-check of weeks already seen). Against each forecaster's own track record it cut error by about 11% on ensemble weather, 21% on single-run weather and 8% on crypto. The earlier ~25% ForecastBench gain disappeared: it now only matches the track record. The economy and Numinous sets are too small to judge.

    Evidence →
  6. v0.78t

    Not enough evidence

    Neural trust campaign: all 50 models trained, saved and scored

    Reading the whole panel together beat each forecaster's track record alone, crypto gained a little, and ensemble weather looked better on few test groups. ForecastBench did worse than the tree, pair history gave no reliable edge, and 2M was no better than 0.5M. Not good enough everywhere to replace the served model.

    Evidence →
  7. v0.79

    Failed its test

    Quick rerun on older data: stopped

    A 16-model rerun of the v0.78 recipe on the older data. It was stopped on 20 Oct because it skipped the new sources and the agreed simpler comparisons. Its partial numbers are diagnostic only.

    Evidence →
  8. v0.79e

    Exploratory

    13-source panel reader: scored, research only

    Owner approved on 20 Oct. Data rebuilt with corrected answer timing and defect flags; the H200 measured its own speed, then trained all 134 research models (0.5M and 2M) that learn forecaster miss risk and confidently wrong consensus; all were saved and re-checked. Official verdict (v0.79f, 20 Oct): it beats four time-safe references in 7 of 10 testable sources, ties on COVID-19 and Phil. Fed ranges, and loses to a tree on crypto; pair history and the 2M size add nothing. Training is not a result: comparisons against simple references decide anything, and nothing switches the served model.

    Evidence →
  9. v0.80

    Research candidate

    Full H200 run done: two research candidates, now being repaired

    Trained on 20 Oct on 454,490 cases from 16 sources (5 versions × 3 seeds, 13 minutes on an H200). Flu (+32% against the reference on confirmation) and energy (+4% against a tree model) passed both test periods; every other field kept the calibrated reference, and crypto, RSV, economy and Polymarket traders scored worse. An outside audit then found defects: forecasts dated after the run, energy missing its outcome clock, events grouped by question instead of real-world event, and two empty inputs. So both wins stay research candidates until v0.80.1 reproduces them on repaired data and a fresh period.

    Evidence →
  10. v0.80.2

    Research candidate

    Public on Hugging Face · 21 fields · 2 beat strong references

    Rebuilt on Offdiagonal v6.5: 2,840,152 forecast panels with known outcomes in 21 trained fields, including air quality, urban heat, agriculture, world conditions and shared answer sources. The time audit found no forecast used after its decision time. No real-world event is scored in two periods. The approved full run trained all 21 starts on 21 Oct (7 variants × 3 seeds). The corrected judge (averaged forecasts, one test across all fields) passed two fields on both test periods: energy demand (one forecasting family) and weather vs airports (airport timing still assumed). Flu improved on development but failed confirmation. Every other field keeps the calibrated reference or has too few outcomes. It now answers as an opt-in beta (/api/public/v1/beta/check, same answers as the frozen files on 721 real cases) and powers the public Forecast Failure Watch; its weights are public on Hugging Face (ZenoDivergent/zeno-divergent-v0802) as a research beta; the default answer stays v0.57.

    Evidence →
  11. v0.82

    Research candidate

    Finished · energy, sea and crypto lead

    The full v0.82 neural run finished on the H200 on 22 Oct with no failures and was judged against its locked rules; the result is public on Hugging Face. On the confirmation period it corrects forecasts best in energy (17.5% better than track records and the tree learner, 15.6% better than the calibrated reference), sea (25% better than the calibrated reference) and crypto (23%), and helps weather vs airports (10% better than track records). The two warning skills were not saved by this run, and two rivals (the market's own price and a scrambled-relations control) were not measured, so no skill passes on this run alone. v0.82.1 stopped during practice before full training. Serving does not change.

    Evidence →
  12. v0.82.1

    Stopped during practice

    Practice stopped · full training not started

    Checked on Hugging Face on 22 Oct at 02:03 UTC. Nine practice checkpoints were saved before the transfer test requested ensemble weather, which has zero labelled panels in this build. The job stopped at 01:22 UTC; the full 30-round, three-seed run never started. The frozen attempt and saved checkpoints remain unchanged. No final verdict or warning-skill pass is available.

    Evidence →

src/data/site-status.ts · served identity from model-source · research from frozen result files

where things stand

Served, beta, testing.

One line per stage, from the same record every page reads.

Serving
v0.57
Completed research
v0.78t · neural campaign scored · not good enough to switch
Scored research
v0.79e · beats references in 7 of 10 sources · research only
Previous run
v0.80 · flu and energy candidates · others keep the reference
Public beta
v0.80.2 · finished · energy + weather vs airports passed · opt-in beta · fixed-scoring re-check adds weather vs analysis (retrospective) · others keep the reference
Live test
Forecast Failure Watch · every open weather case registered before its outcome · scored hourly
Market test
v0.81 · weather models vs markets on the same airport-day · market beats models and the 50/50 blend
Latest attempt
v0.82.1 · practice stopped at 01:22 UTC on 22 Oct · 9 checkpoints saved · full training never started · checked 02:03 UTC
Last finished
v0.82 · judged 22 Oct · energy +17.5% vs track records, +15.6% vs calibrated reference; sea +25% and crypto +23% vs calibrated reference · no skill passes alone (two rivals not measured) · published on Hugging Face
Next training
Separate corrected successor · requires approval before launch; complete Offdiagonal v6.11 prices remain a separate data milestone
Market prices
Offdiagonal all-markets collection · 13,492 of 19,030 markets priced (11 have no price history) · 5,527 to go · about a day at the current pace · frozen as v6.11 when complete

results by field

Every field, every label.

Each figure says which test produced it, against what, and how far it is validated.

24 fields · 7 areas · 6 already beat every rival

Warnings · v0.82 warning test
Brier score of confident-miss warnings vs each forecaster's own track record (tree). historical: development and later confirmation slices of past data. exploratory · release gate not yet passed.
Corrections · v0.80.2 public beta
forecast loss vs calibrated reference vs calibrated reference. historical: two sealed test periods. passed fields are served in the opt-in beta.

Atmosphere

  • Sea-level pressure1.13M outcomes · 2 forecastersWarnings: no edge yetCorrections: calibrated reference
  • Wind1.13M outcomes · 2 forecastersWarnings: no edge yetCorrections: calibrated reference
  • Urban heat154k outcomes · 2 forecastersWarnings: no edge yetCorrections: insufficient evidence
  • Weather vs analysis49k outcomes · 6 forecastersWarnings: beats every rival (+7.9% vs track record, +10.5% later)Corrections: calibrated reference
  • Single-run weather23k outcomes · 7 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
  • Weather vs airports22k outcomes · 5 forecastersWarnings: beats every rival (+5.6% vs track record, +6.3% later)Corrections: validated improvement
  • Air quality644 outcomes · 1 forecastersWarnings: too few misses to judgeCorrections: insufficient evidence
  • Farming rain31 outcomes · 2 forecastersWarnings: too small to judgeCorrections: insufficient evidence
  • Ensemble weather0 outcomes · 6 forecastersWarnings: outcomes arrivingCorrections: not trained

Space and sea

  • Space weather26k outcomes · 4 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
  • Sea12k outcomes · 24 forecastersWarnings: beats every rival (+28% vs track record, +23% later)Corrections: calibrated reference

Markets and crypto

  • Crypto84k outcomes · 3 forecastersWarnings: beats every rival (+14.2% vs track record, +13.4% later)Corrections: calibrated reference
  • Prediction markets66k outcomes · 1 forecastersWarnings: no edge yetCorrections: calibrated reference
  • Market traders4k outcomes · 8248 forecastersWarnings: too few misses to judgeCorrections: calibrated reference
  • Cross-venue markets325 outcomes · 617 forecastersWarnings: too few misses to judgeCorrections: insufficient evidence

Health

  • COVID-196k outcomes · 16 forecastersWarnings: ahead of most rivals (+0.1% vs track record, +1.1% later)Corrections: calibrated reference
  • RSV3k outcomes · 6 forecastersWarnings: ahead of most rivals (+2.3% vs track record, +24% later)Corrections: calibrated reference
  • Flu2k outcomes · 66 forecastersWarnings: no edge yet (-15.1% vs track record, +9% later)Corrections: calibrated reference

Economy

  • Philadelphia Fed (points)4k outcomes · 197 forecastersWarnings: ahead of most rivals (+37.8% vs track record, +23.8% later)Corrections: calibrated reference
  • Philadelphia Fed (ranges)2k outcomes · 195 forecastersWarnings: too small to judgeCorrections: calibrated reference
  • ECB1k outcomes · 116 forecastersWarnings: ahead of most rivals (+7.8% vs track record, +16.8% later)Corrections: calibrated reference

Energy

  • Energy demand3k outcomes · 1 forecastersWarnings: too few misses to judgeCorrections: validated improvement

General forecasting

  • ForecastBench109k outcomes · 806 forecastersWarnings: beats every rival (+45.3% vs track record, +23.1% later)Corrections: not trained
  • Numinous154 outcomes · 2 forecastersWarnings: too small to judgeCorrections: not trained

archive

Every experiment, newest first.

Filter by outcome. Failures stay on the record.

  1. v0.82.1Practice stopped
    Practice stopped · full training not started

    Checked on Hugging Face on 22 Oct at 02:03 UTC. Nine practice checkpoints were saved before the transfer test requested ensemble weather, which has zero labelled panels in this build. The job stopped at 01:22 UTC; the full 30-round, three-seed run never started. The frozen attempt and saved checkpoints remain unchanged. No final verdict or warning-skill pass is available.

  2. v0.82Candidate
    Finished · energy, sea and crypto lead

    The full v0.82 neural run finished on the H200 on 22 Oct with no failures and was judged against its locked rules; the result is public on Hugging Face. On the confirmation period it corrects forecasts best in energy (17.5% better than track records and the tree learner, 15.6% better than the calibrated reference), sea (25% better than the calibrated reference) and crypto (23%), and helps weather vs airports (10% better than track records). The two warning skills were not saved by this run, and two rivals (the market's own price and a scrambled-relations control) were not measured, so no skill passes on this run alone. v0.82.1 stopped during practice before full training. Serving does not change.

  3. v0.80.2Candidate
    Public on Hugging Face · 21 fields · 2 beat strong references

    Rebuilt on Offdiagonal v6.5: 2,840,152 forecast panels with known outcomes in 21 trained fields, including air quality, urban heat, agriculture, world conditions and shared answer sources. The time audit found no forecast used after its decision time. No real-world event is scored in two periods. The approved full run trained all 21 starts on 21 Oct (7 variants × 3 seeds). The corrected judge (averaged forecasts, one test across all fields) passed two fields on both test periods: energy demand (one forecasting family) and weather vs airports (airport timing still assumed). Flu improved on development but failed confirmation. Every other field keeps the calibrated reference or has too few outcomes. It now answers as an opt-in beta (/api/public/v1/beta/check, same answers as the frozen files on 721 real cases) and powers the public Forecast Failure Watch; its weights are public on Hugging Face (ZenoDivergent/zeno-divergent-v0802) as a research beta; the default answer stays v0.57.

  4. v0.80Candidate
    Full H200 run done: two research candidates, now being repaired

    Trained on 20 Oct on 454,490 cases from 16 sources (5 versions × 3 seeds, 13 minutes on an H200). Flu (+32% against the reference on confirmation) and energy (+4% against a tree model) passed both test periods; every other field kept the calibrated reference, and crypto, RSV, economy and Polymarket traders scored worse. An outside audit then found defects: forecasts dated after the run, energy missing its outcome clock, events grouped by question instead of real-world event, and two empty inputs. So both wins stay research candidates until v0.80.1 reproduces them on repaired data and a fresh period.

  5. v0.79eExploratory
    13-source panel reader: scored, research only

    Owner approved on 20 Oct. Data rebuilt with corrected answer timing and defect flags; the H200 measured its own speed, then trained all 134 research models (0.5M and 2M) that learn forecaster miss risk and confidently wrong consensus; all were saved and re-checked. Official verdict (v0.79f, 20 Oct): it beats four time-safe references in 7 of 10 testable sources, ties on COVID-19 and Phil. Fed ranges, and loses to a tree on crypto; pair history and the 2M size add nothing. Training is not a result: comparisons against simple references decide anything, and nothing switches the served model.

  6. v0.79Failed
    Quick rerun on older data: stopped

    A 16-model rerun of the v0.78 recipe on the older data. It was stopped on 20 Oct because it skipped the new sources and the agreed simpler comparisons. Its partial numbers are diagnostic only.

  7. v0.78tToo few outcomes
    Neural trust campaign: all 50 models trained, saved and scored

    Reading the whole panel together beat each forecaster's track record alone, crypto gained a little, and ensemble weather looked better on few test groups. ForecastBench did worse than the tree, pair history gave no reliable edge, and 2M was no better than 0.5M. Not good enough everywhere to replace the served model.

  8. v0.77Exploratory
    CPU tree experiment across seven sources (provisional)

    Not a neural training run: a small tree model on a CPU in about 4 minutes. An outside review found two average rates computed with future answers included, so we re-ran it with every input time-valid and every model calibrated the same way (v0.77r, a diagnostic re-check of weeks already seen). Against each forecaster's own track record it cut error by about 11% on ensemble weather, 21% on single-run weather and 8% on crypto. The earlier ~25% ForecastBench gain disappeared: it now only matches the track record. The economy and Numinous sets are too small to judge.

  9. v0.76bToo few outcomes
    Failure Atlas: who misses together

    The earlier 35% → 45% readout used ranges drawn too narrow. With ranges corrected on earlier weeks, the test window holds zero group failures, so warnings cannot be judged yet.

  10. v0.75Failed
    One shared model for all sources

    Did not beat separate per-source models, so the larger run never started.

  11. v0.74Candidate
    Ensemble-wind correction

    +7.8% against the team forecast on its sealed wind weeks — but simply removing each system's usual error at each place scored +9.1% on the same cases. The gain is local habit, not learned failure structure; it will not be promoted.

  12. v0.57Served
    Served model

    Answers the public Check, API and MCP. Corrects only the sources it passed on, keeps the reference everywhere else.