Auditing Your Own Edge: Results From a Cross-Asset Backtesting Program

Abstract. Over twenty-one research rounds, a backtesting program tested momentum, seasonality, breakout, and regime-gating ideas across US stocks and crypto under a fixed audit discipline: chronological splits, untouched holdouts, multiple-testing correction, and realistic costs. One configuration survived. This paper reports its audited numbers, the deflation applied to them for live planning, and - just as importantly - what died on the way, because the kill list is where most of the learning lives.

Method

The program's rules were set before its results existed. Data splits are chronological (train, validation, and a holdout that is opened once). Every family of ideas pays a multiple-testing penalty via a Benjamini-Hochberg bar that tightens as the cumulative count of tested configurations grows - by the end, roughly four hundred configurations meant any new idea faced a far stricter significance bar than the first did. Costs are charged on every turn (6 basis points round-trip on stocks, 20 on crypto), and bootstrap resampling produces tail estimates rather than single-path optimism.

The survivor, with its numbers

The surviving configuration is a 70/30 book: 70% in a six-month stock momentum sleeve (roughly 160 liquid US names, top 3 by momentum with staggered holding tranches, plus a crisis sleeve that shorts the weakest names when book volatility exceeds 30% annualized), and 30% in a crypto long/short momentum sleeve. On the 2018 to 2026 overlap (2,058 trading days), the audited backtest shows an average year of +37.7%, Sharpe 1.39, maximum drawdown of -23.8%, and eight winning years in eight, including 2022.

Those numbers come with mandatory deflation. The stock universe was built from present-day reference lists, which injects survivorship bias that the program estimates - but has not yet measured pointwise - at a substantial fraction of headline return. The written planning assumption is therefore +20 to 30% per year with a Sharpe near 1.0, and a bootstrap 5% tail path that loses 45% peak to trough. The drawdown ladder and de-risking rules were written down before going live specifically so they cannot be renegotiated mid-drawdown.

The kill list

More ideas died than lived, each with a recorded cause of death:

• Intraday opening-range breakout, index version: the only statistically robust finding was that it reliably loses 10 to 18 basis points a day. The liquid-ETF variant died at 2.2 cents per share of slippage.
• Turn-of-month and day-of-week seasonality: passed validation spectacularly, collapsed on the holdout - a textbook overfit caught by the one dataset nobody had touched.
• Quality gates, dual momentum, sector rotation, correlation and volatility regime filters: each subtracted validation performance from the plain momentum it wrapped, or duplicated it outright.
• Ninety crypto strategy configurations across five families: zero passed the multiple-testing bar.

# The shape of the discipline: the holdout is read once, after everything is frozen.
# (round-numbered scripts; each round's config pre-registered before running)

Limitations

Backtests are upper bounds on live results, not predictions. The survivorship deflation is estimated, not yet measured point-in-time. Crypto costs are volatile and the sleeve's historical drawdown (-35%) is severe. Eight winning years in eight is partly a property of the era tested. The program treats all of this as standing risk, with monthly drift detection between live and modeled returns - and the honest summary of eight months of work is: one durable-looking edge, dozens of seductive dead ends, and a process that told them apart.

Full implementation available on request.

Key definitions

Momentum - A strategy that buys assets with the strongest recent price performance and sells those with the weakest, based on the empirical tendency of price trends to persist over intermediate horizons.

Survivorship bias - Systematic overstatement of returns that occurs when a historical dataset includes only securities that survived to the present, excluding those that delisted or failed during the period.

Sharpe ratio - The average excess return of a strategy divided by its standard deviation, measuring risk-adjusted performance; higher values indicate better return per unit of volatility taken.

Maximum drawdown - The largest peak-to-trough decline in cumulative returns over a specified period, expressed as a percentage.

Multiple-testing correction - A statistical adjustment that raises the significance threshold for accepting results as real when many hypotheses are tested on the same dataset, reducing the probability of false discoveries due to chance.

Chronological walk-forward split - A backtesting discipline in which a dataset is divided into sequential in-sample and out-of-sample periods to simulate live deployment without lookahead bias.

Holdout set - A portion of historical data reserved and untouched during model development and validation, opened only once at the end to provide an unbiased performance estimate free from data-mining bias.

References

• Benjamini, Y. & Hochberg, Y., "Controlling the false discovery rate: a practical and powerful approach to multiple testing," Journal of the Royal Statistical Society, 57(1), 1995. doi.org/10.1111/j.2517-6161.1995.tb02031.x

• CME Group, "Futures Specifications and Contract Details," CME Education. https://www.cmegroup.com/education/

• De Moor, L., Sercu, P. & Vanpée, R., "Valuation of small cap stocks: lessons from Belgium," Journal of Empirical Finance, 19(5), 2012. doi.org/10.1016/j.jempfin.2012.06.003

• Federal Reserve Economic Data (FRED), "US Economic Data," https://fred.stlouisfed.org/

• Novy-Marx, R. & Velikov, M., "A taxonomy of anomalies and their trading costs," Financial Analysts Journal, 72(4), 2016. doi.org/10.2469/faj.v72.n4.4

• Pring, M. J., Technical Analysis Explained, McGraw-Hill (5th ed., 2014).

Educational research on historical data only - not investment advice, not a signal, and never a performance promise. Past results do not predict future performance. Drafting uses AI assistance; every citation is link-verified before publication and every paper is re-audited weekly against the library's editorial standard. Last reviewed by the PropLedger research pipeline: 2026-08-26. Educational research on historical data; not financial advice.