Papers
What the named ensembles were built to measure, and what they found.
The audit
Covariance-Method Claims Are Ensemble-Relative: Named Measures and an Audit Program
Peter Cotton · August 2026 · preprint · PDF · LaTeX · seeded scripts
A claim about a covariance or correlation method is a claim about a
distribution over matrices, and the distribution is rarely stated. Across
fifteen named, seeded ensembles at identical dimension and sample size the
Ledoit–Wolf shrinkage intensity ranges from $0.07$ to $0.86$, and
eleven audited claims from eleven fields — shrinkage, hierarchical
risk parity, eigenvalue clipping, kriging screening, parallel analysis,
covariance localization, diagonal loading, the graphical lasso, random
skewers, North's rule, the Mantel test — flip verdict as the ensemble
changes, each for a stated structural reason. The results assemble into a
matrix of claims against measures; a joint leaderboard of scikit-learn and
precise estimators has five different winners.
The pattern survives scale. Seven claims re-run from $n=30$ to $n=3000$ keep their verdict in six cases; the seventh, hierarchical risk parity, flips under its own home-turf ensemble as dimension grows. Ensemble-invariant theorems are run as null controls: the Hartlap factor comes back at $0.98$–$1.01$ under every ensemble, which is what a working pipeline must report.
Working notes
Kept beside the paper · updated as audits land
Prominent covariance-method claims from astronomy, climate, ecology, genomics, neuroscience, chemometrics and econometrics, each classified before it is audited: claims that impose structure on the truth are ensemble-relative and can flip; theorems about Wishart sampling noise hold for every true covariance and serve as controls. Finding ensemble-dependence in a control means the pipeline has a bug.
Peter Cotton · working paper · PDF
An example of the kind of claim the ensembles exist to test: how estimated correlations inflate, and under which generating measures the effect is large enough to matter.
Reproducing
Every number in the audit comes from a seeded script beside the paper
(*_sweep.py), and every ensemble is one line of this package:
pip install randomcov
from randomcov import random_correlation_matrix
C = random_correlation_matrix(n=300, corr_method="hierarchical", corr_kwargs={"rng": 7})The paper compiles on every push (paper.yml), which checks the build for unresolved references, undefined citations and duplicate labels rather than trusting one laptop.