How ideas are tested
The same tests for every idea, written down before the first one was run and locked since. The author of an idea cannot change them.
The prices
Gold against the US dollar, in five-minute steps, from January 2020 to December 2025. The prices from 2026 are kept apart and are looked at once, as a last check, for an idea that has passed everything else.
Tuned on one stretch, judged on the next
An idea may choose its settings on two years of prices. It is then judged on the following six months, which it has never seen. The window moves forward half a year and the whole thing is repeated, eight times, covering 2022 to 2025. Only the judged half-years count.
The costs
Every test trade pays the spread, the broker's commission and slippage, and pays financing for each night it is held, three nights' worth on Wednesdays. Together that is roughly 0.013% of the trade's value for getting in and out. The whole test is then run a second time with all costs doubled.
The pass marks
| Test | To pass | What it guards against |
|---|---|---|
| Enough trades | 200 or more in the judged period | A result built on a handful of trades |
| Score after costs | 1.00 or higher | A small edge not worth its risk |
| Score with costs doubled | 0.50 or higher | An edge that only exists when trading is cheap |
| Profitable half-years | 5 of 8 or more | One good spell carrying four bad years |
| Largest half-year | No more than half of all profit | The same thing, from the other side |
| Timing | Better than 99% of 2,000 shuffled copies of itself | Profit that comes from gold rising, not from when the trades were placed |
| Luck | 95% confidence after allowing for every idea ever tested | The one lucky winner among many tries |
| Neighbouring settings | 6 in 10 nearby settings also work | One magic number |
| No peeking | Identical results when future prices are cut off | A rule that secretly uses tomorrow's price |
After the tests
An idea that passes all of this goes to a demo account. It needs thirty closed trades over at least three weeks, at no less than half the pace its test promised, before it is allowed onto a live account. On a live account it trades at small size, with a stop on every trade, a daily loss limit, and a floor that switches it off.
What this cannot do
It cannot promise the future. Markets change, and a strategy that passed every test can stop working. That is why the record stays public, and why a strategy that stops earning goes back a step.