Buying more inference compute makes a model write longer, more complete analysis. That holds for math and code, where answers can be judged on the spot. In trading, the question changes: after transaction costs, do the extra reasoning tokens turn into portfolio money?
Jiayi Chen and Guiling Wang of the New Jersey Institute of Technology turned this question into a controlled experiment. Their paper, *The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?*, was uploaded to arXiv on September 25, 2026 (arXiv:2609.30705). The conclusion appears right after the first sentence of the abstract: across three model families, adding reasoning did not deliver reliable net-return improvements.
How the Experiment Locked Its Variables
Most LLM trading studies swap the model, prompts, tools, memory, and decision pipeline together, measuring an entire system. This paper moves a single knob: reasoning effort.
| Setting | Value | |---|---| | Period | Full year 2024, 241 entry days (Jan 10 – Dec 23) | | Universe | 100 US stocks, split into 4 cohorts of 25 | | Identification | Opaque codes, company names hidden | | Portfolio | Long top 10, short bottom 10, equal weight both sides | | Exposure | Gross 2, net 0 (market neutral) | | Holding | 5 trading days per cohort, overlapping on the calendar | | Costs | 10 bps per side; sensitivity at 5, 20, 50 | | Economic threshold | 0.25 bps/day | | Models | DeepSeek-Flash, GPT-5.6 Luna, Gemini 3.1 Flash-Lite | | Inputs | Numeric, identifiable news, masked news | | Scale | Over 800,000 asset predictions |
Baseline reasoning levels: none for DeepSeek and GPT; minimal for Gemini, whose API cannot guarantee internal thinking is fully disabled.
Decision rules were pre-registered: an effect counts as economically meaningful only if the lower bound of the two-sided 95% CI exceeds 0.25; harmful if the upper bound is below 0; no meaningful benefit if the upper bound is below 0.25; otherwise inconclusive. Importantly, inconclusive does not mean zero effect — most intervals still contain both economically meaningful positive and negative values.
Nine Annual Comparisons, None Cleared the Bar
Nine annual low-vs-baseline comparisons (3 model families × 3 input conditions), estimates in basis points per day:
| Model | Input | Estimate | 95% CI | Verdict | |---|---|---:|---|---| | DeepSeek | Numeric | +6.159 | −7.243 to 19.560 | inconclusive | | DeepSeek | Identifiable news | +0.378 | −9.033 to 9.790 | inconclusive | | DeepSeek | Masked news | +1.449 | −6.768 to 9.666 | inconclusive | | GPT | Numeric | +3.540 | −2.531 to 9.611 | inconclusive | | GPT | Identifiable news | −0.626 | −5.905 to 4.653 | inconclusive | | GPT | Masked news | +0.040 | −4.297 to 4.378 | inconclusive | | Gemini | Numeric | +0.758 | −6.233 to 7.749 | inconclusive | | Gemini | Identifiable news | −0.142 | −8.128 to 7.844 | inconclusive | | Gemini | Masked news | −1.123 | −8.418 to 6.173 | inconclusive |
Six estimates positive, three negative; all nine intervals cross both zero and the 0.25 threshold. Verdicts are unchanged after Holm correction.
Absolute returns are colder still. Of 18 configurations (9 baseline + 9 low-reasoning), 13 had negative net returns. Positive returns appeared only for Gemini identifiable news (+1.417% / +1.275%) and Gemini masked-news minimal baseline (+2.122%). Sharpe ratios were mostly negative; maximum drawdowns generally ran −20% to −34%.
Non-Monotonic, Not Just Diminishing Returns
DeepSeek is the only family with a full curve tested: none, low, high, maximum. On an audit sample of 48 frozen entry days and 68 return days, with three generations per backbone:
| Input | Comparison | Estimate | 95% CI | |---|---|---:|---| | Numeric | Maximum − none | −4.875 | −20.319 to 10.569 | | Identifiable news | Maximum − none | −4.053 | −12.949 to 4.843 | | Masked news | Low − none | −9.016 | −16.248 to −1.784 | | Masked news | Maximum − none | −8.246 | −17.641 to 1.149 |
The −9.016 for masked news (low − none) is the paper's only result judged a harm, with the CI upper bound at −1.784, entirely below zero. The three independent generations gave −7.490, −9.327, and −10.231 — consistent in direction. Moving-block bootstrap (5 sessions per block) gives −17.929 to −3.477; cohort bootstrap gives −15.482 to −2.281. The finding survives both.
The single decisive positive adjacent comparison is identifiable-news Maximum − high: +4.639 (CI 0.376 to 8.903). Yet maximum still failed to beat none overall. One favorable step is not the same as "more is better" — a sentence worth writing down.
Ask the Same Question Three Times, Get Different Answers
Repeated generation is more revealing. On frozen market dates, the same task asked three times:
- GPT's net returns flip sign between positive and negative.
- Gemini swings between gains and losses.
- Across audited GPT or Gemini entry days, no pair of generations selected identical long and short tails (identical-tails rate of 0.000). GPT numeric low-reasoning has Spearman ρ of 0.946 but tail Jaccard of only 0.657; Gemini masked-news low-reasoning has ρ of 0.669 and Jaccard 0.355.
- Jiayi Chen, Guiling Wang (New Jersey Institute of Technology), *The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?*, arXiv:2609.30705, 2026-09-25
- Test-time compute background cited by the paper: Wei et al. 2022; Wang et al. 2023; Snell et al. 2025; Muennighoff et al. 2025
Similar scores, different stocks. At the boundary of ten longs and ten shorts, a little rank jitter swaps a holding, and a swapped holding creates turnover.
Turnover data support the chain. Across nine conditions, turnover ran 24.5% to 31.2%. DeepSeek's added reasoning lowered turnover (numeric: 30.83% → 24.48%); Gemini's slightly raised it (masked news: 28.38% → 29.51%). In cost sensitivity at 5, 20, and 50 bps, all GPT and Gemini comparisons remain inconclusive, and DeepSeek's masked-news harm still holds.
Better Formatting, Same Returns
Reasoning's effect on output-format compliance varies by model — in inconsistent directions:
| Model | Effect of reasoning on format-failure rate | |---|---| | GPT | Numeric: 0.41% → 1.45% (p=.030); identifiable news: 17 → 5 failures (p=.016) | | Gemini | Identifiable news: 54 → 14 (p < 0.000001); masked news: 4.56% → 2.07% (p=.0032) | | DeepSeek | No final invalid tasks after 17 deterministic repairs |
Baseline invalid-task rates: GPT kept 57 invalid tasks (0.99%), Gemini 136 (2.35%), DeepSeek 0.
The paper's own words: reasoning changed output structural reliability, with direction varying by model, and reliability improvement does not itself imply return improvement.
That sentence deserves a frame of its own. An agent can emit perfectly compliant JSON — every field right, every type correct — and still consistently lose money on its stock picks. Mistaking syntactic compliance for judgment is among the costliest errors in enterprise agent deployments.
Why It's Worth Watching
First, it measures an economic quantity, not a proxy. Most test-time scaling research lands on math and code, where verifiable ground truth exists. This paper sets the endpoint at net return after costs; every link in the conversion chain can eat the gains.
Second, the result is directly actionable for quants. Under 241 trading days, 100 stocks, 10-long/10-short, 10 bps per side, none of nine comparisons across three families could prove that "thinking a bit longer is worth the money." This does not extrapolate to "reasoning is useless" — only to "in your own pipeline, reasoning's benefit must be validated separately."
Third, it dismantles the assumption that stable scores mean stable decisions. ρ of 0.946 alongside a Jaccard of 0.657 shows aggregate metrics can hide tail-selection instability — and the tails are what determine holdings.
Limits
The paper is a preprint. The sample covers only 2024, 100 US stocks, and a single portfolio-construction rule. The baseline is zero reasoning, not "current best practice." Costs are fixed basis points with no slippage or market-impact model. An inconclusive verdict is not equivalence — most intervals still contain meaningful positive and negative values. DeepSeek's −9.016 comes from the 48-frozen-day audit sample, not the 241-day main sample. Also, the model choices (Flash tier, Luna, Flash-Lite) are not each vendor's strongest reasoning models, which limits how far conclusions generalize.
---