The Frozen-Backbone Head Lab: Where the Predictable Structure in BTCUSDT Actually Lives

Language: Python · CUDA
Scale: 23 modules · ~3,300 lines
Source: Not publicly released

Overview

The harness exists because iterating forecasting heads inside a compiled trainer was too slow to learn anything from. It reads the trainer’s cached frozen backbone output — per window, the token tensor the backbone produced — and exports it to a Python-native store: a memory-mapped array for zero-copy minibatches, a columnar index carrying targets, funding and provenance hashes, and a manifest recording dimensions and source. Head and loss iteration then becomes a seconds-per-epoch loop with no backbone forward pass in it.

The scientific care is concentrated in two places. The first is leakage: dataset.purged_embargoed_splits purges each earlier split’s tail within an embargo of the next split’s start, measured in time rather than in rows — the store’s anchors are not uniformly spaced, so a row-counted embargo would silently under-purge — and it enforces that the embargo is at least the maximum label horizon. The second is collapse: the head this work started from converged by fitting a better-calibrated spread while its median stayed nearly constant, so median_dispersion is a first-class metric and the conditional-distribution suite uses a location-scale residual head that starts exactly at the unconditional marginal, making any skill earned rather than assumed.

The three studies in this series live in one workspace repository but in separate packages; this is the first of them.

Implementation

Core libraries: torch with a CUDA build, Polars, pyarrow, NumPy, PyYAML
Store: A memory-mapped float32 embedding array plus a columnar index of targets, funding and hashes, and a dimensions-and-provenance manifest
Sample: 158,528 windows of 15-minute perpetual-futures bars, 2019-09 to 2026-04
Horizons: 15 minutes, 1 hour and 4 hours in the base study, extended to 3 days in the long-horizon probe
Cache validity: Keyed by normalization, input-adapter and backbone-artifact hashes — change any and the embeddings are rebuilt rather than reused
Evaluation: Non-overlapping backtest, one position at a time, full-horizon hold, at 2, 5 and 8 basis points round trip

Components

cache_io.py: Reading the compiled trainer’s sharded embedding cache, validated against the live cache
export.py: Cache to memory-mapped store plus columnar index and manifest
targets.py: Future log-return targets and as-of funding, recomputed from canonical bars rather than taken from the cache
dataset.py: The store wrapper, the torch dataset, and the purged and embargoed time splits
representation.py: The frozen-cache versus live-backbone seam, so unfreezing is a configuration change rather than a rewrite
heads.py: The reference horizon-query monotone head and an MLP quantile head, both monotone by construction
residual_heads.py, residual_losses.py: The location-scale residual head that starts at the unconditional marginal, and its objectives
losses.py: Pinball, calibration, direction and cost-aware terms
metrics.py: Pinball, coverage, directional accuracy, the median-dispersion collapse detector, and edge-over-cost profit
baselines.py, controls.py: The unconditional and regime marginals, and the permutation controls that locate where skill lives
regimes.py, skill.py: Regime labelling and skill scores against the marginal baselines
cd_config.py, cd_prepare.py, cd_runner.py: The conditional-distribution suite: configuration, preparation and the run harness with device guards

Workflow

export Convert the compiled trainer’s embedding cache into the memory-mapped store and index.
split Build purged, embargoed, time-measured train, validation and test blocks.
baseline Fit the unconditional and regime marginals a head has to beat.
train Iterate heads and losses on GPU, logging collapse diagnostics each epoch.
control Run permutation controls to test whether skill comes from the representation or from the regime.
evaluate Score out of sample and run the non-overlapping cost-aware backtest.

Reproducibility and validation

Embargoes are measured in time, not rows, because only about seven in ten consecutive store anchors are actually one bar apart — a row-counted embargo would leak.
The non-embargoed splitter is retained but explicitly labelled as a leaky reference, so the difference the embargo makes stays measurable.
A volatility probe is used as a pipeline validity check: the harness must find the signal that is known to exist before its directional null is trusted.
Median dispersion is logged every epoch, so a head that improves pinball loss by collapsing to a constant median is caught rather than celebrated.
Selection happens on validation and the test split is read sparingly, with the confidence-gating threshold chosen on validation and reported on test.

Availability

Not publicly released. The store runs to tens of gigabytes of cached embeddings and is derived from a frozen third-party backbone, so it is neither distributable nor meaningful without the compiled trainer that produced it. The export path and the store schema are documented in the repository precisely so the cache can be rebuilt rather than shipped.

Companion research

This investigation asks where the predictable structure in BTCUSDT actually lives. Prompted by a production quantile head that settled on a near-constant median over frozen Moirai-MoE embeddings, it uses a fast GPU harness over a 158,528-window cached embedding store under purged and embargoed splits.

Read the research report