English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

10,000 Backtests in One Night: Why AI Sped Up Research But Not Validation in Quant Investing

Forum topic · QianXun · 2026-10-20

Summary

At a quant industry conference in Shanghai, Zhang Xiaoquan, founder of Chaoquantum Fund, argued that AI has accelerated only one of quant investing's two clocks: the research clock. AI agents can now run tens of thousands of backtests overnight, and the best purely random strategy among 10,000 non-predictive models could post a ten-year Sharpe ratio around 1.2—indistinguishable from legitimate products in the industry's typical 1–2 range. The validation clock, however, cannot be sped up: markets generate only one day of new data per day, and distinguishing a true strategy with Sharpe 1 from noise takes roughly four years of live trading. Firms on the panel—Mingxi Capital, Pansong Asset, and Kuanfu Fund—described multi-agent research frameworks that cut iteration cycles from about 6 months to 1.5 months. Zhang proposed industry fixes including logging backtest attempts, disclosing detection rates, anonymous incident reporting, and public benchmark datasets. He also warned that excess returns have finite sources, and that when every firm rents intelligence from AI simultaneously, the total pool does not grow—data, execution, and live track records remain the real moats.

Imagine an absurd beauty pageant: 10,000 models that cannot predict anything, each submitting a ten-year backtest. Judges who know nothing rank them purely by Sharpe ratio. A winner will emerge—and likely look quite respectable. According to Zhang Xiaoquan, founder of Chaoquantum Fund, such a "champion" can post a ten-year backtest Sharpe of around 1.2.

For context: presentable industry products typically fall in a Sharpe range of 1 to 2. A first place won purely by luck lands squarely in the good-student zone. Zhang made these remarks at the recent Quantitative Industry High-Quality Development Conference in Shanghai, reported by China Securities Journal on October 19. His entire speech starts from this premise: AI has only sped up one of quant investing's clocks.

⏩ The Research Clock: The One That Got Sped Up

AI entered investment research by first consuming the most exhausting work. Zhang's summary: what AI currently replaces is the first stage—reading materials, writing code, running backtests, producing analyses. A single agent can run tens of thousands of backtests in one night, so candidate strategy production is no longer constrained by headcount.

Institutions on the conference panel confirmed the same curve. Mingxi Capital introduced multi-agent collaboration in its core research framework NOVA, with different agents handling data processing and factor research. Pansong Asset embedded AI into its research workflow, cross-checked against human review by researchers. Kuanfu Fund built a three-layer architecture of skills, agents, and a harness, where Auto Research handles parameter tuning—cutting iteration cycles for new models of comparable scale from about 6 months to 1.5 months.

The cost hides in production speed. Candidates have surged in number while each becomes less trustworthy. The more backtests you run, the more "looks great" candidates you can pick from—and the larger the share that slipped in through multiple-comparison noise. In a lottery with ten thousand participants, the winner's ticket reads "luck."

⏸ The Validation Clock: The One That Did Not Move

The other clock runs slowly, and AI cannot wind it. Markets produce only one day of new data per day. To distinguish a true strategy with a Sharpe ratio of 1 from noise in live trading, Zhang estimates it takes roughly 4 years.

That number deserves attention. It means a tenfold efficiency gain on the research side shortens the validation queue by zero seconds. A team that could once validate tens of strategies per year in live trading can now generate tens of thousands of candidates overnight—but a year later, only a few dozen get live-trading answers. The vast majority of extra candidates never live to see their verdict.

Zhang's definition of "discernment" is plain: which questions to pick, which information to look at, which evidence to accept, when to stop. As tooling advantages converge, differentiation increasingly lies in selection standards—and in who keeps refining them. He also has a colder line: every unit of excess return has a source. When all institutions get smarter through AI at the same time, the total pie does not grow. Intelligence can be rented; data, execution, capital, and live track records cannot.

| Clock | What AI Changed | What AI Did Not Change | |---|---|---| | Research clock | 10,000 backtests overnight; mass candidate production | Each candidate's credibility falls with selection count | | Validation clock | Higher process automation | Markets add only one day of data per day | | Live-trading verdict | Faster monitoring and attribution | ~4 years to separate true strategies from noise | | Total returns | Individual efficiency gains | Whole industry gets smarter simultaneously; total unchanged |

🧭 The Ecosystem Prescription

The second half of the speech is an industry prescription, centered on accounting for tests: managers should log backtest counts in internal ledgers and put false-strategy detection rates into performance reviews; allocators should ask "how many attempts are behind this equity curve"; peers should share lessons via anonymous incident reports; universities should plant known true signals and traps in real market data within public problem sets, so both students and agents get tested; institutions should write junior-analyst development into contracts.

The bitterest ingredient comes last. Zhang warns against over-relying on AI—the risk that research capability gains are only AI's, and you never truly learned. The industry backdrop adds a footnote: the number of 10-billion-yuan private funds just hit a record 159, and several leading quant funds lost 3% to 5% in the first week after the holiday, with some products drawing down more than 10% since October. When scale grows, what gets tested is precisely discernment.

Sources and caveats: The speech content comes from China Securities Journal's October 19 report on the 2026 Quantitative Industry High-Quality Development Conference (republished by 10jqka). The "10,000 backtests," "Sharpe 1.2," and "~4 years" figures are as stated in Zhang's speech; no calculation methodology was presented. Panel institutions' statements come from the same report. Private fund drawdown data comes from third-party channels cited by Eastmoney on October 19, not official audit figures.

Tags

#quant-investing#artificial-intelligence#backtesting#sharpe-ratio#multi-agent-systems#asset-management#multiple-comparisons#china-finance

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636218