Grok 4 Fast Tops Extended NYT Connections Benchmark: xAI's Fast, Cheap Reasoning Model

Forum topic · ✨步子哥 · 2025-09-21

Summary

xAI released Grok 4 Fast on September 19, 2025, a cost-efficient reasoning model with a 2 million token context window, priced at $0.2 per million input tokens and $0.5 per million output tokens — roughly 47x cheaper than Grok 4. Benchmarks show it nearly matching Grok 4 (85.7% vs 87.5% on GPQA Diamond) while using about 40% fewer thinking tokens, and independent tests by Artificial Analysis rate its intelligence at Gemini 2.5 Pro level for roughly 1/25 the cost, with output speeds of 344 tokens per second. On Lech Mazur's extended NYT Connections benchmark (759 puzzles with added distractor words), Grok 4 Fast Reasoning leads at 92.1%, ahead of Grok 4 (91.7%), GPT-5, o3-pro, Gemini 2.5 Pro, DeepSeek and Qwen 3 — above the estimated 71% average human solve rate. It also ranked first on LiveCodeBench (80.0%) and demonstrated multimodal reasoning in community-shared visual puzzle tests. The model is available via grok.com, x.com and mobile apps, with access to Grok 4 Fast requiring a SuperGrok or PremiumPlus subscription.

Grok 4 Fast: Fast, Cheap Reasoning That Tops the NYT Connections Benchmark

Key points

Release: xAI launched Grok 4 Fast on September 19, 2025, positioning it as a frontier reasoning model with dramatically better cost efficiency than Grok 4.
Pricing: $0.20 per million input tokens and $0.50 per million output tokens — about 47x cheaper than Grok 4.
Context window: 2 million tokens, with a unified architecture that switches between reasoning and non-reasoning modes.
Efficiency: Matches Grok 4's benchmark performance while using ~40% fewer thinking tokens (GPQA Diamond: 85.7% vs 87.5%).
Benchmark leadership: Tops the extended NYT Connections benchmark at 92.1% (reasoning mode), ahead of GPT-5, o3-pro, Gemini 2.5 Pro, DeepSeek and Qwen 3, and above the ~71% average human solve rate.
Speed: Pre-release API tests showed 344 tokens/second output and ~3.8s end-to-end latency.
Coding & search: First place on LiveCodeBench (80.0%), integrated web and X search, 1163 Elo in the LMArena search arena.
Availability: Free Grok 3 access (limited) on grok.com, x.com and apps; Grok 4 Fast requires a SuperGrok or PremiumPlus subscription.

Release and capabilities

According to xAI's announcement, Grok 4 Fast was trained via large-scale reinforcement learning to maximize "intelligence density" — more capability per parameter and per token. Additional results cited: AIME 2025 (no tools) 92.0%, HMMT 2025 93.3%, LiveCodeBench 80.0%.

Artificial Analysis' independent evaluation (X post) found Grok 4 Fast delivers Gemini 2.5 Pro / Claude 4.1 Opus-level intelligence (score of 60 in reasoning mode) at roughly 1/25 the price, completing its intelligence index tests with 61M tokens versus Gemini 2.5 Pro's 93M.

Extended NYT Connections benchmark

Lech Mazur's extended NYT Connections benchmark expands the standard 436 puzzles to 759, adding up to four carefully vetted distractor words per puzzle to prevent saturation and data contamination. Models propose four groups of four words from 16+ words, with at most four mistakes allowed; all four groups must be correct. The latest 100 puzzles are reserved to guard against training-data leakage.

Leaderboard highlights:

| Model | Solve rate | |---|---| | Grok 4 Fast (reasoning) | 92.1% | | Grok 4 | 91.7% | | GPT-5, o3-pro, Gemini 2.5 Pro, DeepSeek, Qwen 3 | trailing | | Human average | ~71% (easy days 98%, hard days 39%) | | o1 | 98.9% |

Multimodal reasoning demo

A community-shared demo showed Grok 4 Fast correctly solving a visual puzzle — inferring the fill order of partially occluded glass cups (7, 6, 3). Commenters raised training-data concerns; the poster clarified this is a fully retrained multimodal model, not comparable to earlier Grok versions.

Community reaction

@ArtificialAnlys: "xAI has released Grok 4 Fast — Gemini 2.5 Pro-level intelligence at ~25x lower cost."
@Prashant_1722: "Grok 4 Fast Reasoning ranks first in the new record, extended NYT Connections 759 puzzles."
@PromptrAI_: The commoditization of frontier AI opens real-time, high-volume use cases like customer support and coding assistants.
@burkov joked that "as expected" was written deliberately to show the model wasn't tuned to the benchmark.

Sources

1. xAI. (2025). Grok 4 Fast. https://x.ai/news/grok-4-fast 2. Mazur, L. (2025). NYT Connections Benchmark. https://github.com/lechmazur/nyt-connections/ 3. Prashant_1722. (2025). X Post on Grok Benchmark. https://x.com/Prashant_1722/status/1969352801290436855 4. Artificial Analysis. (2025). Grok 4 Fast Review. https://x.com/ArtificialAnlys/status/1969180023107305846 5. Mark K. (2025). Multimodal Demo. https://x.com/mark_k/status/1969423645463150990