Quantitative··3 min read
Auditing Your Own Edge: Results From a Cross-Asset Backtesting Program
0 references, link-verifiedEditor of record: Shane CantyStandards review editorial standard · audit log
## Limitations
Backtests are upper bounds on live results, not predictions. The survivorship deflation is estimated, not yet measured point-in-time. Crypto costs are volatile and the sleeve's historical drawdown (-35%) is severe. Eight winning years in eight is partly a property of the era tested. The program treats all of this as standing risk, with monthly drift detection between live and modeled returns, and the honest summary of eight months of work is: one durable-looking edge, dozens of seductive dead ends, and a process that told them apart.
## Key definitions
**Momentum**: A strategy that buys assets with the strongest recent price performance and sells those with the weakest, based on the empirical tendency of price trends to persist over intermediate horizons.
**Survivorship bias**: Systematic overstatement of returns that occurs when a historical dataset includes only securities that survived to the present, excluding those that delisted or failed during the period.
**Sharpe ratio**: The average excess return of a strategy divided by its standard deviation, measuring risk-adjusted performance; higher values indicate better return per unit of volatility taken.
**Maximum drawdown**: The largest peak-to-trough decline in cumulative returns over a specified period, expressed as a percentage.
**Multiple-testing correction**: A statistical adjustment that raises the significance threshold for accepting results as real when many hypotheses are tested on the same dataset, reducing the probability of false discoveries due to chance.
**Chronological walk-forward split**: A backtesting discipline in which a dataset is divided into sequential in-sample and out-of-sample periods to simulate live deployment without lookahead bias.
**Holdout set**: A portion of historical data reserved and untouched during model development and validation, opened only once at the end to provide an unbiased performance estimate free from data-mining bias.
## References
1. Benjamini, Y. & Hochberg, Y., "Controlling the false discovery rate: a practical and powerful approach to multiple testing," *Journal of the Royal Statistical Society*, 57(1), 1995. doi.org/10.1111/j.2517-6161.1995.tb02031.x
2. CME Group, "Futures Specifications and Contract Details," CME Education. Https://www.cmegroup.com/education/ (primary)
3. De Moor, L., Sercu, P. & Vanpée, R., "Valuation of small cap stocks: lessons from Belgium," *Journal of Empirical Finance*, 19(5), 2012. doi.org/10.1016/j.jempfin.2012.06.003
4. Federal Reserve Economic Data (FRED), "US Economic Data," (link no longer resolves) (primary)
5. Novy-Marx, R. & Velikov, M., "A taxonomy of anomalies and their trading costs," *Financial Analysts Journal*, 72(4), 2016. doi.org/10.2469/faj.v72.n4.4
6. Pring, M. J., *Technical Analysis Explained*, McGraw-Hill (5th ed., 2014).
---
*Educational research on historical data only. Not investment advice, not a signal, and never a performance promise. Past results do not predict future performance. Every reference is link-verified before publication and every paper is re-audited weekly against the library's editorial standard.*
*Last reviewed by the PropLedger research pipeline: 2026-09-20. Educational research on historical data, not financial advice.*
Keep reading
Regime Detection: Rolling Volatility, Markov Switching, and the Overfitting Trap
Regime detection aims to identify structural shifts in market behavior, typically modeled as transitions between distinct statistical states (high volatility/low correlation, crisis regimes, trending/ranging). Two common methods are
Multiple-Testing Correction for Strategy Research: Why Many Tested Rules Produce a Winner by Chance
When a researcher tests hundreds or thousands of trading strategies against historical data, statistical chance alone guarantees that some will appear to work purely by accident. Multiple-testing correction
Multiple-Testing Correction in Strategy Research
When researchers test many trading rules on historical data, some will appear profitable by pure chance, not signal. Multiple-testing correction adjusts statistical thresholds to account for the number
Educational research on historical data only. Not investment advice, not a signal, and never a performance promise. Past results do not predict future performance. Every reference is link-verified before publication and every paper is re-audited weekly against the library's editorial standard. Found an error? Email support@prop-ledger.org and the paper is corrected or withdrawn.