FactorMiner: An Open-Source AI Agent That Discovers Alpha Factors Like a Top Quant Researcher

In quantitative investing, discovering Alpha factors that consistently generate excess returns is the core, labor-intensive work of researchers. The traditional process relies heavily on personal experience, intuition, and extensive manual backtesting—inefficient and poorly suited to capturing nonlinear, complex patterns in fast-changing markets.

FactorMiner, an open-source project, introduces a tireless, self-improving "super researcher" into quant research. It is not just an automation tool but an agent with "skills" and "experience memory," aiming for autonomous discovery and evolution of financial Alpha factors.

What Problem Does FactorMiner Solve?

The core question: how can a machine think, experiment, summarize, and evolve like a top quant researcher? FactorMiner's answer is an agent framework with a "skill library" and an "experience memory," enabling AI to:

1. Autonomously compose basic data-processing skills to generate candidate factors 2. Evaluate effectiveness through historical backtests 3. Store successes and failures into memory 4. Use memory to guide the next round of exploration

This self-driven loop can, in principle, continuously refine its exploration strategy as markets and data evolve.

Core Architecture: Two Pillars of Self-Evolution

Skill Library: "LEGO Bricks" for Factors

Skills are atomic, reusable data-operation and feature-computation functions:

• Basic data acquisition skills: e.g., fetch_daily_price, calculate_returns, get_financial_statement, wrapping database/API interactions and returning standardized data objects.
• Feature engineering skills (the core of the library):
• rolling_mean(data, window=20): 20-day rolling mean
• ts_rank(series, window=10): time-series rank over the last 10 periods
• decay_linear(series, window=5): linearly weighted decay
• Composition and validation skills: e.g., backtest_factor, analyze_ic

Experience Memory: Distilling Successes and Failures

Without recording which ideas were tried and which failed, the agent is amnesiac. The memory stores:

• Which factor combinations produced significant IC (Information Coefficient)
• Which parameter combinations failed under certain market regimes
• Which preprocessing methods were robust on certain asset classes
• A reflection mechanism: what characterized failed factors and how to avoid similar failures

This dual success + failure memory avoids the classic pitfall of factor mining: recording only successes and repeatedly stepping on the same failure modes.

Key Difference from Traditional AI Quant

Traditional AI quant is a one-way pipeline: data → factor mining (genetic programming, AutoEncoder) → prediction (GBDT, LSTM) → portfolio & risk → execution.

FactorMiner turns this pipeline into a loop. Each attempt—skills used, composition order, parameters, market state—is recorded, and the next exploration builds on this memory. The underlying insight: the process of Alpha discovery is itself learnable. Top researchers' intuition comes largely from having seen many failures; FactorMiner outsources that to the experience memory.

Practical Deployment Steps

1. Define an internal research task library: Convert the research process behind the last three years of successful Alpha factors into standardized natural-language task descriptions, archived with data sources, scripts, and performance records. 2. Build an internal tool platform: Standardize data APIs (Wind, Tushare, internal platforms), text-processing utilities, and factor analysis toolkits (Alphalens, Qlib adapters) behind a unified, secure interface. 3. Design the agent engine: a practical architecture is "LLM + Code Interpreter + validation loop":

• The LLM (GPT-4, Claude, etc.) acts as commander—parsing tasks, planning steps, generating code outlines and key functions
• The Code Interpreter executes generated code in a controlled sandbox
• The validation loop checks results: if IC is insignificant, the LLM analyzes possible causes (feature definition? data? neutralization?) and iterates
4. Establish evaluation and iteration: run new ideas through the agent for baseline factors and initial performance, then let researchers deepen and optimize. Each task record (success or failure) becomes fine-tuning data for smaller, proprietary models—a positive human-AI co-evolution loop.

Boundaries the Public Materials Don't Emphasize

• Backtest rigor determines everything: look-ahead bias, survivorship bias, or underestimated transaction costs can turn recorded "successes" into false positives. The biggest risk is treating flawed backtest results as ground truth.
• Memory can go stale under regime changes: an Alpha from 2007 may fail by 2020 as market structure, liquidity, and regulation shift. Memories need "validity period" labels with automatic decay of old strategy experience.
• LLM reliability: LLMs can hallucinate plausible-but-buggy feature functions, contaminating the memory so subsequent factors inherit the same defects. Sandbox validation, unit tests, and factor interpretability checks are necessary defenses.

What This Means for Quant Teams

A senior researcher might produce 5–10 valid factors per year; a 10-person team perhaps 50–100. A self-evolving system in the FactorMiner style could push that up by an order of magnitude while shifting researchers from "manual mining" to "designing the mining process." For individual investors, it offers a reproducible Alpha-discovery platform that no longer depends on personal expertise barriers.

The deeper shift: Alpha factor research is becoming an engineering pipeline rather than an expert craft—a transition accelerating through 2025–2026. The business logic changes accordingly: future Alpha sellers will boast self-evolving agents, not star researchers.

Next Steps on the Self-Evolution Roadmap

• Multi-agent collaboration: separate feature-engineering, backtesting, and factor-combination agents cooperating via a message bus.
• Cross-market transfer: can factors validated in A-shares transfer to HK, US, or crypto markets? What memory retrieval and retraining mechanisms does transfer require?
• Active learning: the agent proposes its own research directions for under-explored asset classes rather than waiting for assignments.

Taken to its extreme, this line of work points to a new human-machine division of labor: researchers become the architects and supervisors of Alpha agents, rather than the creators of Alpha themselves.