What we've learned

Findings the CEO has checked, what we are testing, and the decisions behind it.

Findings

The CEO adds findings here after checking them. Each one names its evidence and how sure we are. Monthly post-mortems (biggest misses, luckiest hits) go at the bottom. See ops/FORECASTING.md for the method.

2026-10-15

  • No stock-ranking signal predicted next month's returns in the S&P 500, 2018–2026. Monthly rank IC over 100 months: momentum −0.003 (t −0.1), trend +0.004, pullback +0.004, low volatility −0.014, reversal −0.011; all |t| < 0.7. Source: reports/signal-ic-sp500-2026-10-14.json. Caveat: stocks that delisted mid-month drop out of that month's IC. Confidence: moderate. It means any edge an evolved bot shows needs a reason beyond these signals.
  • Evolution's answer depends heavily on the run. On the same unseen window (2026-04-16 to 2026-10-14), the evolution run's champion made +7.5% and the walk-forward champion made +51.1%. Both were trained on data up to Apr 2026. Source: scoreboard/sp500.csv and scoreboard/walkforward-sp500.csv. A single champion's result is mostly the luck of the search plus the market.
  • The +51% was real, and it was one bet. The walk-forward champion was 5-stock, 6-month momentum. MU made 47% of its gain and SNDK 26% (memory chips), and its worst drop was 32%. Textbook momentum held the same names but more diluted (+8.6%). Data checks found no split or bad-bar artifacts; the largest one-day move was 33% (DELL). Source: reports/honest-run-wf-sp500-w3-i0-g28-11-2026-10-14.json. Lesson: high percentiles against random portfolios can come from one theme in one window. Judge champions across windows, on the forward test, and by concentration.
  • Run forecasts started. Base-rate forecasts of each run's own holdout are scored against coin-flip twins (dashboard, Forecasting scorecard). Fewer than 30 have resolved, so there's no verdict yet.
  • Nasdaq-100 (now reportable), unseen Apr 16 to Oct 14, 2026: the champion made +99.8%, against QQQ +14.3% and textbook momentum +32.0%. That's the 100th percentile of fair random portfolios, and MU and MRNA made 38% of the gain. Run nasdaq100-20261014-201445. One 6-month window in a strong theme market. Treat it as a lead to test forward, not as evidence.
  • Run forecasts are inflated by my verification reruns. 33 base-rate run forecasts have resolved, with Brier 0.123 against 0.25 for the coin flip. But several are repeats of the same question (the same S&P window rerun 3 times, and the walk-forward run twice), so they aren't independent. Fixed the same day: the scorecard now counts each (arena, window, question) once. After deduplication, 18 base-rate forecasts have resolved, with Brier 0.197 against 0.25 for the coin flip. That's still under 30, so no verdict yet.
  • Delisted stocks: Alpaca's free plan is incomplete. It lists 1,544 inactive stocks on major exchanges and serves SIP bars for 139 of a 150-stock sample. But SIVB, FRC, BBBY, TWTR and ATVI are missing (XLNX and CERN are present). A small-cap list built only from Alpaca would still lean toward survivors. T17 needs a second source for delisted names (SEC EDGAR filings index). Source: reports/probe-inactive-2026-10-15.json.
  • Small caps are now point-in-time (T17, T17b). The top 400 most-traded stocks priced $2–50 at each month-end, Mar 2020 to Sep 2026, with index members and exchange-traded products excluded. SEC delisting notices (Form 25-NSE) for operating companies from 2020 on: we know the ticker for 90% and Alpaca has prices for 89% (by year, 76% to 95%; 2026 is lowest). 314 of the 1,692 stocks that were ever members came only from tickers recovered from SEC filings, so the old list really did lean toward survivors. Sources: reports/smallcap-coverage-2026-10-15.json, universes/smallcap_snapshots.csv.
  • First point-in-time small-cap run, unseen Apr 17 to Oct 15, 2026: the champion made −17.7%, against IWM +1.0% and textbook momentum −35.1%. That's the 14th percentile of fair random portfolios: no signal. Momentum lost badly in small caps over this window. Run smallcap-20261015-225409.

Experiments

IdChangeExpectedObservedVerdict
E-T7Full-market SIP feed, 9.5-year default history, end date clamped to the last completed trading dayMore training and test data; less noise in holdout results and fewer missing-bar artifactsRan on real SIP data, 9.5 years; walk-forward and evolve runs green. Signal ICs over 2018-2026 are ~0 (see ops/LEARNING.md).works; no edge found
E-T10Training fitness measured against textbook momentum per fold; top 5 finalists ranked by min(own, median of 6 one-setting neighbors)Fewer lucky or fragile champions; more champions beat textbook momentum on unseen days1 S&P run each on the same unseen Apr-Oct 2026 window: before T10 champion +0.5% (31st pct); after T10 +7.5% (58th pct); momentum +8.6%. Still below momentum.too early (n=1)
E-T18Deflated Sharpe: discount the champion's Sharpe for the number of strategies triedHonest probability that the champion's Sharpe is real; most champions should not clear itpendingpending

Signal information (IC)

Information coefficient (IC): the rank correlation between each signal at month-end and the next month's stock returns. Near 0 means no predictive power; |t| of 2 or more is the usual bar.

S&P 500

SignalMean ICt-statHit rateMonthsLast-12 mean IC
momentum-0.0029-0.140.53100+0.0368
trend+0.0043+0.200.52101+0.0251
pullback+0.0044+0.240.51101-0.0241
low_volatility-0.0136-0.530.48101-0.0602
reversal-0.0106-0.650.47101-0.0314

Takeaway (2026-10-14): |t| < 2 for every signal: no signal reliably predicted next month.

Nasdaq-100

Not yet: no signal-ic report for nasdaq100 yet.

Small caps

Not yet: no signal-ic report for smallcap yet.

Decisions timeline

Board and CEO entries from ops/DECISIONS.md, newest first.