How it works

The pipeline

Think of an assembly line with a quality check at every step. Strategies are bred on the past, tested on days they never saw, compared with luck, and finally left to trade days that had not happened. All of it is paper money. Click a box to jump to its section.

1. Data

What it does. Daily prices come from Alpaca's full-market (SIP) feed: about 9.5 years, which includes the 2018, 2020 and 2022 selloffs. Each day's stock pool is the index as it was on that day. S&P 500 membership uses real join and leave dates; Nasdaq-100 membership is rebuilt from monthly snapshots from March 2020. Small-cap members are rebuilt each month from that month's data (top 400 by trading volume, $2 to $50), including delisted stocks recovered from SEC filings (about 89% covered). Trades are decided after the close and filled at the next open, with a small cost on every fill. A stock that stops trading is sold at its last real close.

Why it exists. A test is only as honest as its data. A stock list built from today's index would quietly drop every company that failed.

Defends against: Look-ahead Survivorship

2. Evolution

What it does. Settings cards are scored on 4 training windows (folds). Fitness is the average fold score minus half its spread, measured against textbook momentum, so steady beats lucky. Each generation the top quarter survive and the rest are bred: two cards are mixed, then values are nudged. 4 islands evolve apart and swap their best bot every 5 generations. An island stuck for 3 generations gets a shake-up: hard mutation and random newcomers. At the end the top 5 finalists meet the neighbor check: each is re-run with 6 copies that change one setting, and ranks by the lower of its own score and its neighbors' median.

Why it exists. Evolution is built to find what fit the past. Folds, the momentum target and the neighbor check make it harder to win by a lucky spike on one exact setting.

Defends against: Overfitting

3. Unseen holdout

What it does. The last 126 trading days (about six months) are cut off before evolution starts. The champion trades them once, at the end.

Why it exists. Training results are always flattering. The holdout is the first moment the champion meets prices it had no chance to fit.

Defends against: Overfitting Look-ahead

4. Luck checks

What it does. The champion is ranked against at least 100 fair random portfolios: 5 random eligible stocks, fully invested, rebalanced on the champion's own schedule. It is also compared with textbook momentum, given a deflated Sharpe ratio (marked down for the number of strategies tried), and checked for concentration: the share of the gain from its two biggest stocks.

Why it exists. A high return alone proves nothing. The percentile says how often luck does as well; the deflated Sharpe prices in the many tries; a percentile under 95 is consistent with luck.

Defends against: Luck Multiple testing

5. Walk-forward

What it does. For each of the last few windows of 126 trading days (about half a year), evolution runs using only data before the window, seeded only with the baselines. The champion, a team of the top 5 and the best-in-training baseline then trade the window blind.

Why it exists. One holdout is one roll of the dice. Several windows show whether results swing from window to window.

Defends against: Overfitting Look-ahead

6. Forward test

What it does. Strategies named in the locked rules are frozen on the start date. A workflow records what each would trade after every close, using only that day's data, and commits the paper ledger to git. Each starts with $10,000 of paper money and pays the same costs as in backtests. A strategy stops if it loses 35%.

Why it exists. Every other test here is run after the strategies were chosen, with the past in view. These days had not happened yet.

Defends against: Look-ahead Overfitting Luck

7. Forecasts and scoring

What it does. Every Monday we state the probability that each forward strategy beats SPY, beats momentum or falls 10%. Every evolution run does the same for its own holdout. Each is written to git before its answer is known and scored with the Brier score against a coin-flip twin.

Why it exists. It stops us telling a story after the fact. A forecast is either checked or it is not.

Defends against: Luck

8. Reports

What it does. Evolution, walk-forward and honest-run reports post silently to Discord, with the Saturday scorecard and the CEO digest. This site is rebuilt from the committed result files after each results workflow. A report with no comparison says so and claims no result.

Why it exists. A result nobody can find, or one shown without its luck check, is a result that will be misread.