Deep Wave Research
Regime-Adaptive Probabilistic ForecastingReport 3 of 3 · Comparative study

Software companion

Regime-Adaptive Probabilistic Foundation Models for Extrapolating Nonlinear Stochastic Systems

Language
Python · CUDA
Scale
29 modules · ~3,400 lines
Source
Not publicly released

Overview

The package implements four components end to end — multi-resolution temporal encoding, exogenous feature fusion, a probabilistic quantile head, and regime-adaptive online adaptation — and is arranged so that the backbone is one swappable slot. A compact from-scratch encoder, a frozen foundation-model lane and a LoRA-fine-tuned lane all present the same interface to the same fusion, head, loss and evaluation code, which is what makes the quality-versus-cost comparison a like-for-like one.

Evaluation is strict walk-forward with purge and embargo, and each fold’s test block is scored under three pre-registered arms: plain inference, sequential conformal recalibration, and conformal plus volatility-triggered adaptation restricted to normalization parameters with a proximal penalty against the pre-adaptation model. The arms are fixed in advance, so adaptation cannot be introduced after the fact as an explanation for a good result.

The study layer closes the loop to the write-up. A study YAML names a base experiment and a list of dotted-key variant overrides; running it writes a planning and execution matrix, a per-variant result and configuration, and the LaTeX table slots the manuscript reads. --dry emits the matrix and the static contract tables before any training happens, so the document can be rendered contract-first and its claims cannot outrun its runs.

Implementation

Backbone lanes
A roughly two-million-parameter from-scratch multi-resolution encoder, plus frozen and LoRA-fine-tuned Chronos-T5 and Moirai-MoE lanes behind one interface
Fine-tuning
LoRA injected into attention query and value projections; a fine-tuned lane starts numerically identical to its frozen twin, isolating the value of adapting the representation
Head
Monotone-by-construction quantiles plus volatility, drawdown and direction outputs, with scale statistics reinjected at the trunk
Arms
Three pre-registered arms per fold: static, conformal, and conformal plus triggered adaptation
Study matrix
Eleven registered variants — two benchmark lanes, four ablations, and five backbone lanes
Offline path
A synthetic regime-switching stochastic-volatility generator, so the pipeline runs with no data files

Components

models/backbones/scratch.py
The self-contained multi-resolution encoder
models/backbones/pretrained.py
Lazy foundation-model adapters and the LoRA injection used by the fine-tuned lanes
models/backbones/cached.py
Fold-invariant embedding caching for the frozen lanes
models/fusion.py
Exogenous fusion by context token or feature-wise modulation
models/heads.py
The multi-output readout with monotone quantiles
adaptation/online.py
Triggered online adaptation with a proximal stability penalty against the pre-adaptation model
adaptation/conformal.py
Rolling pairwise conformalized-quantile recalibration that preserves monotonicity
data/splits.py
Walk-forward folds with purge and embargo
data/features.py, scales.py, targets.py
Causal and engineered exogenous features, multi-resolution views, and leak-free label-window targets
sequential.py
The sequential test-block loop running the three arms
metrics.py
Statistical, volatility and drawdown skill, and cost-aware trading diagnostics
study.py
The variant matrix, its execution, and the manuscript table slots it emits

Study execution

  1. smoke Run the synthetic generator end to end to verify the pipeline without data.
  2. dry Emit the study matrix and static contract tables before any training.
  3. fold Build walk-forward folds with purge and embargo over the real store.
  4. train Fit each variant under mixed precision, one lane at a time.
  5. arms Score each fold’s test block under static, conformal and conformal-plus-adaptation.
  6. aggregate Write per-variant results and the manuscript’s generated tables.

Reproducibility and validation

  • Every lane shares fusion, head, loss, split and evaluation code, so a backbone comparison is not confounded by a difference in the surrounding harness.
  • A LoRA lane is verified to start identical to its frozen counterpart, which is what makes it a measurement of adaptation rather than of two different models.
  • The three evaluation arms are pre-registered, so triggered adaptation cannot be added afterwards to rescue a result.
  • Quantile monotonicity and leakage invariants are asserted in an end-to-end test that runs on CPU in seconds.
  • The package documents its own scope honestly: the earlier head-lab finding that the frozen representation encodes volatility rather than cost-surviving direction is the stated reason the distributional objectives carry the headline and the trading metrics are read as diagnostics.

Availability

Not publicly released. The foundation-model backbones are third-party checkpoints installed separately, and the study reads a multi-year canonical perpetual-futures store that is too large to distribute. The synthetic generator is in the package so the pipeline can be run and tested without either.

Companion research

A completed comparative study of regime-adaptive probabilistic forecasting on the BTCUSDT perpetual surface: a from-scratch multi-resolution encoder against frozen and LoRA-fine-tuned Chronos-T5 and Moirai-MoE lanes, evaluated under strict purged walk-forward splits with rolling conformal recalibration.

Read the research report