Profit-Optimal LLM Training: A Theory of How Big AI Models Should Be, Not Just Can Be

Forum topic · 小凯 · 2026-05-19

Summary

A forum post discusses a 2026 paper by Sophie Hao and William Merrill (NYU), 'A Theory of Training Profit-Optimal LLMs' (arXiv:2605.16430), which combines neural scaling laws with microeconomics to answer not how large AI models can be trained, but how large they should be for profit maximization. The framework posits a revenue function R(Q) tied to model quality and a cost function C(N, D) over parameters and training data, optimizing profit rather than perplexity. In the compute-constrained regime, optimal model size grows nearly linearly with hardware efficiency, and total training cost grows sub-quadratically—explaining why better GPUs lead to larger spending. In the data-constrained regime, profit-optimal spending scales with the square of available data and decreases linearly with hardware efficiency. The authors validate against real-world AI training expenditure data: current industry practice aligns with theory in the compute-constrained case but appears non-optimal if data scarcity becomes binding, suggesting the industry is betting that data limits will not arrive soon. The post also critiques the paper's simplifying assumptions—single-dimensional quality, threshold-based user adoption, and absent pricing strategy—while praising its transformation of 'how big should we train' from industry intuition into a tractable mathematical problem amid growing AI bubble debates.

Overview

For the past five years, the AI community has asked one question: how big can models get? Almost nobody asked the other question: **how big *should* they get?**

A paper by Sophie Hao and William Merrill (New York University), *A Theory of Training Profit-Optimal LLMs* (arXiv:2605.16430), stitches these together by combining scaling laws with microeconomics.

| Item | Detail | |------|--------| | Title | A Theory of Training Profit-Optimal LLMs | | Authors | Sophie Hao, William Merrill (NYU) | | arXiv | 2605.16430 (cs.LG, cs.AI) | | Date | May 2026 | | Link | https://arxiv.org/abs/2605.16430 |

1. From 'Can Train' to 'Worth Training'

Scaling laws tell us that bigger models plus more data equals better quality—but they never mention money. This paper glues scaling laws to microeconomics:

Firms have a revenue function R(Q): higher model quality Q means more users and higher willingness to pay.
Firms have a cost function C(N, D): more parameters N and training tokens D mean more money burned.
Profit = R(Q) − C(N, D), where Q itself is a function of N and D (the scaling law).

The firm optimizes argmax(N, D): profit—not perplexity, not benchmark scores.

2. Compute-Constrained vs. Data-Constrained: Two Regimes

The paper splits the world into two regimes:

Compute-constrained (most labs today): data is plentiful; GPUs are the bottleneck. Key results:

Optimal parameter count N grows near-linearly with hardware efficiency E (FLOPs per dollar).
Total training cost grows sub-quadratically with E—doubling hardware efficiency yields more than double the value in model quality.
Data-efficiency gains incentivize *larger* models and *more* training spending.

This explains why each GPU generation leads AI companies to spend more, not save: from a profit-optimal view, better hardware enlarges the optimal investment itself.

Data-constrained (most labs in the future): high-quality text is finite. Results are striking:

Profit-optimal training spend scales with the square of data volume D—double the data, quadruple the budget.
Profit-optimal spend declines linearly with hardware efficiency E—better hardware makes you spend *less*, because data, not compute, is the bottleneck.

The policy implication is significant: if the world is shifting from compute-constrained to data-constrained, both corporate spending strategy and regulatory frameworks should adjust accordingly.

3. Real-World Validation

The authors collected publicly available AI training expenditure data and checked whether industry behavior matches their model:

Compute-constrained regime: most current practice is consistent with theory—larger models and bigger budgets can be explained as profit-optimal.
Data-constrained regime: current trends are *not* profit-optimal—if data is the binding constraint, the industry should spend less, not keep pushing budgets up.

This aligns with the authors' simulations of a 'hardware progress slowdown': if Moore's law stalls, profit-optimal budgets should fall, but industry trends go the other way.

A plausible reading: the industry is betting that data limits won't arrive soon. If that bet is wrong—usable text growth lagging GPU growth—current investment levels become uneconomic.

4. The Economic Reasoning Chain

The framework elegantly links:

1. Scaling law: perplexity ~ N^(−α) · D^(−β) 2. User adoption: users adopt only if quality exceeds threshold q 3. Revenue: users × per-user revenue → total revenue 4. Cost: training FLOPs ~ N·D; inference cost ~ N × query volume

Then optimize (N*, D*) for profit. One elegant conclusion: in the compute-constrained regime there is a deterministic relationship between optimal model quality and optimal spending—since quality improvements obey diminishing marginal returns while their costs increase at the margin, earning 10% more revenue requires multiplying spending many times over.

5. Honest Questions

User adoption assumption: modeling adoption as a simple quality threshold q ignores habits, brand loyalty, lock-in, and ecosystem externalities. A better model may lose users to a worse one with a better ecosystem.
Multi-dimensional quality: a single scalar Q can't capture that different applications value reasoning, speed, or safety differently.
Missing pricing: the paper assumes exogenous willingness to pay, but companies actively shape adoption through pricing—a cheap-but-good model can outperform an expensive-great one profitably.

6. My Take

The paper's deepest value isn't the math—it's that it turns 'how big should we train?' from industry intuition into an analyzable mathematical problem. When CEOs justify a 100k-GPU cluster over 50k, answers are usually vague ('bigger is better', 'scaling laws say so', 'competitors are doing it'). Physically correct perhaps, but potentially economically absurd if marginal benefit has fallen below marginal cost.

Hao and Merrill don't give you the answer—answers depend on your revenue function, cost function, data availability, and hardware efficiency—but they tell you how to compute it. Amid intensifying AI bubble debates, this kind of 'do the accounting' theory may be worth more than any hype or hate.

References

1. Hao, S., Merrill, W. (2026). A Theory of Training Profit-Optimal LLMs. arXiv:2605.16430. 2. Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361. 3. Hoffmann, J., et al. (2022). Training Compute-Optimal Large Language Models. NeurIPS 2022. 4. Aghion, P., Howitt, P. (1992). A Model of Growth Through Creative Destruction. Econometrica.