Beyond 'How Big Can We Train?': A Theory of Profit-Optimal LLM Training

| Item | Content | |------|---------| | Paper | A Theory of Training Profit-Optimal LLMs | | Authors | Sophie Hao, William Merrill (New York University) | | arXiv | 2605.16430 (cs.LG, cs.AI) | | Date | May 2026 | | Core contribution | Combines scaling laws with microeconomics to derive a theory of profit-optimal LLM training — answering not "how big *can* we train?" but "how big *should* we train?" | | Link | https://arxiv.org/abs/2605.16430 |

Over the past five years, everyone has been asking the same question: how big can AI models get?

Almost no one asks the other question: how big should AI models be?

The gap between these two questions opens up when you put the physics of scaling laws and the economics of costs side by side. This paper by Sophie Hao and William Merrill (both NYU) stitches them together.

1. From "Can We Train It" to "Is It Worth Training"

Scaling laws tell us that bigger models + more data = better quality. But scaling laws never mention money. They only say how much perplexity will drop, not how many zeros will vanish from your bank account.

This paper glues scaling laws to microeconomics. The core idea is simple:

• A firm has a revenue function R(Q) — the higher the model quality Q, the more users adopt it and the more each will pay.
• A firm has a cost function C(N, D) — the more parameters N and training tokens D, the more it burns.
• Profit = R(Q) − C(N, D)

Q itself is a function of N and D — that is what scaling laws tell us. The firm maximizes profit over (N, D), not perplexity or benchmark scores.

2. Compute-Constrained vs. Data-Constrained: Two Regimes, Two Sets of Rules

Compute-constrained: data is effectively unlimited; only your GPU budget binds. This is the current situation of most AI labs.

Data-constrained: you can afford GPUs, but there isn't enough high-quality text. This is the situation most labs will eventually face — the internet's text supply is finite.

In the compute-constrained regime, the derived results are very concrete:

• Optimal parameter count N grows near-linearly with hardware efficiency E (FLOPs per dollar).
• Total training cost grows sub-quadratically with E — a 2× hardware-efficiency gain yields more than 2× in value from model quality.
• Data-efficiency improvements incentivize both larger models and more total training spend.

This explains why each GPU generation leads AI companies to spend more, not less. From a profit-optimal standpoint, better hardware enlarges the optimal investment itself.

In the data-constrained regime, the results are more striking:

• Profit-optimal training spend scales with the square of data volume D — double the data, quadruple the budget.
• Profit-optimal training spend decreases linearly with hardware efficiency E — better hardware makes you spend *less*, because data, not compute, is the bottleneck.

This distinction matters for policy: if the world is shifting from compute-constrained to data-constrained, both corporate spending strategies and regulatory frameworks should adjust.

3. Real-World Validation: Does Current Spending Match the Optimum?

The authors collected publicly available data on AI training expenditure and checked whether current industry behavior matches their model's predictions.

• Compute-constrained regime: most current practice aligns with theory — training larger models and spending more can be explained as profit-optimal.
• Data-constrained regime: current trends do not match profit-optimality — if data were truly the bottleneck, the industry should be spending less, not continuously raising training budgets.

This observation is consistent with the authors' simulations of a "hardware progress slowdown" scenario: if Moore's Law slows, profit-optimal budgets should fall, not rise — the opposite of current industry trends.

One plausible reading: the industry is betting that data limits won't arrive anytime soon. If that bet is wrong — if usable text growth lags GPU growth — current investment levels become uneconomical.

4. The Economic Reasoning Chain: From Perplexity to Profit

1. Scaling law: perplexity ~ N^(−α) · D^(−β) 2. User adoption: users adopt only when model quality exceeds a threshold q 3. Revenue: number of users × revenue per user → total revenue 4. Cost model: training FLOPs ~ N · D; inference cost ~ N × query volume

Then optimize: find (N*, D*) maximizing profit.

One of the most elegant conclusions: in the compute-constrained regime, there is a deterministic relationship between optimal model quality and optimal training spend — earning 10% more revenue requires spending many times more on the model, because quality improvements exhibit diminishing marginal returns while their costs exhibit increasing marginal cost.

5. Honest Questions

First, the user-adoption assumption. The paper assumes each user has a quality threshold q. In reality, adoption involves habits, brand loyalty, lock-in, and ecosystem externalities. A higher-quality model might have fewer users due to a weaker ecosystem — the model collapses all of this into a single threshold.

Second, quality is multidimensional. The paper represents quality as a scalar Q, but different applications value reasoning, speed, or safety differently. Firms cannot optimize a single scalar.

Third, the missing price. The paper assumes an exogenous willingness-to-pay. In reality, AI companies actively shape adoption and revenue through pricing. A cheaper, good-enough model may be more profitable than an expensive excellent one — this dimension is not modeled.

6. My Take

The deepest value of this paper is not its mathematics — rigorous though it is — but that it turns "how big should we train?" from industry intuition into an analyzable mathematical question.

In the past, when CEOs were asked why they train on 100,000 GPUs instead of 50,000, the answers were vague — "bigger is better," "scaling laws say so," "competitors are doing it." These answers may be physically correct but economically absurd if marginal revenue already falls below marginal cost.

Hao and Merrill's framework provides a structured way to answer the question. It doesn't tell you the answer — that depends on your revenue function, cost function, data availability, and hardware efficiency — but it tells you how to compute it.

In today's heated debate about an AI bubble, this kind of "do the math" theory may be more valuable than any hype or hate.

References

1. Hao, S., Merrill, W. (2026). A Theory of Training Profit-Optimal LLMs. arXiv:2605.16430. 2. Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361. 3. Hoffmann, J., et al. (2022). Training Compute-Optimal Large Language Models. NeurIPS 2022. 4. Aghion, P., Howitt, P. (1992). A Model of Growth Through Creative Destruction. Econometrica.