Reading the reports
What each post and file contains, with an example of each built from our latest data.
The evolution post
Posted silently to Discord after each run. The coloured bar is green, yellow or red by verdict.
This is the real report builder run on the latest S&P 500 scoreboard row (sp500-20261014-201443): champion, benchmark, momentum, percentile and concentration are that run's figures. The fitness line, drawdowns, the random-portfolio spread, deflated Sharpe and the recipe are filler to show the layout, not that run's values.
- Title. Arena and verdict. If the comparison is missing, the title says "comparison missing: no result claimed".
- Description. Trained and unseen dates, and the data note (for example point-in-time membership).
- Best training fitness. How the best bot's training score rose by generation. It is training, so it flatters; it says nothing about the future.
- Champion recipe. The champion's settings in one line. If it has a stop-loss, a filter or a short lookback, you can read it here.
- Unseen months. Return, worst drop and score on days the run never saw, next to the baselines and the index. Higher is not the point; compare it with the next field.
- Luck check. The percentile among fair random portfolios, textbook momentum, the deflated Sharpe, the number of strategies tried and how concentrated the gain was. This field is what decides whether the title means anything.
The weekly scorecard
Posted silently on Saturday mornings. Built now from the forward test and the forecast scorecard; the weekly file for this week does not exist yet.
- Forward test. The day count, each frozen strategy's paper return and its gap to SPY, in points. Days are few, so read it as a log.
- Forecasts. How many forecasts resolved. The Brier scores appear only after 30 have resolved, then base rate against coin flip.
- Shipped this week. Tickets merged in the last 7 days, so you can see what changed in the engine.
The CEO digest
One silent Discord post per digest; a corrected digest replaces the old post. This is the real latest one.
- Title. The date and time of the CEO run.
- Shipped, Results, Learning. What changed, the newest numbers with their luck checks, and what we learned from them.
- Problems and Needs you. What is broken and what only the board can decide.
- Budget and Forecasts. Actions minutes used, and the CEO's own forecasts for the next run, scored like any others.
The scoreboard files
One row per run in scoreboard/<arena>.csv. Walk-forward windows have a similar file with test_start and test_end.
| Column | Meaning |
|---|---|
date | The last day of the unseen period (the data date). |
run_id | Unique name of the run: arena, date, start time. Rows are never replaced. |
train_start | First day of the history the run trained on. |
holdout_start, holdout_end | The unseen period the champion traded once, at the end. |
benchmark, bench_return | The index fund (SPY, QQQ or IWM) and its percent return over the unseen period. |
champion, champion_return | The strategy the run ranked first after the neighbor check, and its percent return on the unseen period. |
champion_score | Headline score: return minus the benchmark's, minus half of any extra drawdown, in percentage points. |
team_return, team_score | The top 5 strategies with money split evenly, and their score. |
best_baseline, best_baseline_return, best_baseline_score | The textbook strategy that scored best on the same unseen days. |
random_median_return | Median return of the fair random portfolios. |
verdict | A one-line label written by rule from the luck check. See below. |
momentum_return | Return of textbook momentum 12-1 on the same days. Blank on older runs. |
champion_percentile | Share of fair random portfolios the champion beat, 0 to 100. Blank on older runs. |
top2_share, top2_symbols | Share of the gain made by the two biggest contributors, and which stocks they were. |
The honest-run report
One genome run by hand on unseen days. This is the real latest report, for base-momentum-12-1 on sp500.
- Return in the test window. The one genome against the index and textbook momentum, on days after its warm-up.
- Max drawdown. The deepest fall from a peak. A big return with a deep fall is a bumpy ride.
- Luck check. The percentile among fair random portfolios and the deflated Sharpe. For a single genome,
n_trialsis how many strategies stand behind it. - Where it came from. Which stocks made the gain, when present.
- Read this first. The plain statement of what the numbers can and cannot show.
Four words you will meet
- Percentile
- Where the champion ranks among at least 100 fair random portfolios on the same days. 58 means it beat 58%. 50 is average luck. Under 95 is consistent with luck.
- Top-2 share
- The share of the gain made by the two biggest contributors. Above 50% the verdict adds "concentrated: mostly one or two stocks", because that is one bet.
- Deflated Sharpe
- A risk-adjusted return marked down for how many strategies were tried to find this one. Above 0.95 would be convincing. Far below it means the result is what a big search would produce by chance.
- Verdict
- One line, chosen by rule. "Beat the index, textbook momentum and N% of random portfolios" needs a positive score, a percentile of at least 95 and a win over momentum. "No signal yet" means the champion sits inside the range of random portfolios. "Unverified" means the comparison is missing, and no result is claimed.