Deep Wave Research
Wavelet-Domain ENSO ForecastingReport 9 of 9 · Reference re-basing and handoff

Software companion

Causal Observational References for Tropical-Pacific SST Forecasting

Language
Python 3.10+ · CUDA
Scale
Shared repository · 87 modules · ~42,000 lines
Source
Not publicly released

Overview

An earlier finding about the observational reference was missing its control: the reference had changed at the same time as the model was reused, so degradation could not be attributed to either. The software answer is a configuration family. Five data-build profiles inherit from one base and override exactly one thing — the anomaly reduction — producing five parallel sets of anomalies, window indices, coefficient stores, normalizers and preprocessing reports, each kept entirely separate on disk. Each pipeline is then retrained from scratch rather than adapted, which is what makes retraining and reference-updating separable explanations.

The five arms are a fixed legacy climatology, a fixed 1991–2020 climatology, a trailing block, a relative sea-surface-temperature frame, and a linear detrend. Because one arm deliberately uses a climatology period outside the training window, the configuration records that as a predeclared exception with a stated reason rather than letting it pass as an ordinary setting — the repository refuses that silently otherwise.

Around the comparison sits the record work: extending the observed record to the present under a frozen transport preprocessing path, joining newly acquired months without disturbing the earlier ones, restoring background fields, and producing a live ensemble. The report ends by freezing the deployment contract as a protocol document with its own override amendment.

Implementation

Arms
Five anomaly reductions — fixed legacy climatology, fixed 1991–2020 climatology, trailing block, relative SST frame, linear detrend
Design
Each arm gets its own anomalies, window index, coefficient store, normalizer and preprocessing report, and is retrained from scratch
Configuration
One base profile with five single-override children, so the varied factor is visible in the diff
Record extension
Frozen transport preprocessing, acquisition-only joins for new months, and background restoration for the live ensemble
Deployment
A frozen protocol document for the extended period, plus a recorded override amendment

Components

anomaly_reference.py
The anomaly reference abstraction the five arms instantiate
data/anomalies.py
Climatology construction and every reduction variant, including the declared out-of-training-window exception path
real_data.py
Observed dataset construction across the extended record
index_series.py
Regional index extraction under each reference
final_protocol.py
Protocol freezing and gate checks for the extended period
scripts/phase_h_frozen_transport_preprocessing.py
Preprocessing held fixed while the record grows
scripts/phase_h_join_acquired_era5.py, phase_i_prime_join_acquired_era5.py
Joining newly acquired months without disturbing earlier ones
scripts/phase_h_select_winner.py, phase_h_r1_calibration.py
Variant selection and its recalibration
scripts/phase_i_prime_background_restore.py, phase_i_prime_live_ensemble.py
Background restoration and the live ensemble path
scripts/phase_j_build_anomaly_variant.py
Building one arm end to end from its configuration
scripts/phase_j_audit_stores.py
Cross-arm store audit before any comparison runs
scripts/phase_j_scorecard.py, phase_j_prediction_gallery.py
The cross-arm scorecard and its case gallery

Controlled comparison

  1. extend the record Acquire and join new months under frozen transport preprocessing.
  2. build arms Materialize five parallel data builds differing only in the anomaly reduction.
  3. audit stores Verify geometry, schema and normalization scope across all five before training.
  4. retrain Train each arm from scratch under the identical protocol.
  5. score Produce the cross-arm scorecard and decomposition.
  6. freeze Write the deployment contract for the extended period and record any override.

Reproducibility and validation

  • Retraining from scratch in every arm is the control the earlier finding lacked; it is what allows retraining, rather than climatology updating, to be identified as the dominant explanation.
  • A climatology period outside the training window is only permitted through an explicit predeclared exception carrying a written reason, so it cannot be introduced as an ordinary configuration change.
  • A cross-arm store audit runs before any comparison, so an arm that differs in more than the anomaly reduction is caught before it becomes a result.
  • The deployment contract is a frozen document with an explicit override amendment rather than a moving default, so what was deployed at a given time is recoverable.

Availability

Not publicly released. The five parallel coefficient stores across the extended record are large, and the extension depends on continuing Copernicus access. The configuration family is the transferable part: it is a small set of YAML overrides describing exactly what was held fixed.

Companion research

Five strictly separate ERA5 pipelines change only the anomaly reduction: fixed, updated-fixed, causal trailing, relative sea-surface temperature, or detrended. Each is freshly retrained under one frozen recipe.

Read the research report