Compute-Aware Evaluation of LLM Adversarial Robustness: Why 90% Attack Success Can Be Reported as 0%

Forum topic · 小凯 · 2026-06-14

Summary

A paper from the University of Toronto, Vector Institute, and Hugging Face ('Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models', arXiv:2606.11409) argues that conventional jailbreak evaluation based on Attack Success Rate (ASR) at fixed query budgets conceals the true cost of attacks. Borrowing the 'work factor' concept from classical security, the authors measure attack cost uniformly in FLOPs and introduce new metrics: C@τ (compute needed to reach τ% risk) and AE (average efficiency, risk per FLOP). Key findings: SFT-stage models are more robust than DPO/RLVR stages; scaling from 0.5B to 7B parameters raises GCG attack cost 20× but cheap template attacks only 2.8×; transfer attacks via open proxy models let attackers target closed models at near-zero extra cost; safety RL can degrade robustness on out-of-distribution attacks like GCG; and per-category harm coverage in safety data creates up to 5× differences in attack cost. The paper concludes that safety evaluation must report attack cost, not just success frequency.

> Paper: Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models > Authors: Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik, Colin Raffel > Institutions: University of Toronto, Vector Institute, Hugging Face > arXiv:2606.11409 > Code: r-three/risk-under-pressure (to be open-sourced) > Date: 2026-06-09

TL;DR

Current LLM safety evaluations measure defense effectiveness with Attack Success Rate (ASR) under fixed query budgets. A model broken on the first try and one broken on the tenth are rated as equally unsafe. This paper from the University of Toronto and Hugging Face introduces the classical security concept of work factor into LLM safety, measuring attack cost uniformly in FLOPs. It shows existing evaluations can severely overstate defense effectiveness (a 90%-successful attack can be advertised as 0% ASR), and proposes two new metrics: C@τ (compute required to reach τ% risk) and AE (risk gained per unit of compute).

The Problem: What Current Evaluations Hide

A Thought Experiment

Model A: attack succeeds on the first try, ASR@10 = 1.0
Model B: attack succeeds only on the 10th try, ASR@10 = 1.0

Existing evaluations report both as "100% broken." From a security standpoint, Model B is clearly harder to break — the attacker pays 10× the compute.

Fatal Flaws of Current Practice

| Current practice | Problem | |---|---| | "Attack X achieves Y% success after Z queries" | Answers "how often," not "at what cost" | | ASR under fixed query budgets | Treats all attacks as equal cost, masking real attack expense | | Leaderboard-style comparison | Same ASR → rated equally unsafe despite vastly different costs |

Key insight: evaluations use *query counts* as the budget unit, but per-query costs differ enormously. Template attacks (JailBroken) cost one forward pass per query; white-box gradient attacks (GCG) cost 128 forward passes plus backward gradients plus candidate evaluation per query.

Core Method: Compute-Aware Evaluation

The framework imports work factor from classical security: the computational effort an attacker must expend. FLOPs serve as the unified currency:

Hardware-independent (unaffected by GPU model, kernel optimizations, batching)
Comparable across components (gradient steps, auxiliary model calls, target/judge queries)
Can derive energy, wall-clock time, GPU-hours, and cost-per-break

FLOPs Cost Accounting per Attack

| Attack | Type | Per-step cost | Character | |---|---|---|---| | JailBroken (JB) | Template | 1 forward + judge | Cheapest | | PAIR | Iterative optimization | 1 forward + judge + attacker LLM | Moderate | | GCG | White-box gradient | 128 candidate evals + backward pass (β=3) + generation + judge | Most expensive |

New Metrics: Risk-Compute Curves

Empirical risk: $\hat{R}(M, \pi, \lambda) = \frac{1}{N}\sum_{i=1}^N \mathbb{1}[\text{trial } i \text{ succeeds within } \lambda \text{ steps}]$
C@τ: compute required to reach risk τ — higher is safer
CAURC: area under the risk-compute curve
AE (Average Efficiency): CAURC / C_max — risk per FLOP, lower is safer

Five Disruptive Findings

1. SFT Is More Robust Than DPO/RLVR — Non-Monotonic Alignment

For Tulu3-8B stages (C@0.5):

| Model | GCG | PAIR | JB | JB AE | |---|---|---|---|---| | Base | 59.3 | 11.2 | 9.2 | 53.3 | | SFT | ∞ | ∞ | 52.4 | 8.9 | | DPO | 521.2 | 79.9 | 40.9 | 10.4 | | RLVR | 503.6 | 72.4 | 25.7 | 18.9 |

SFT is the most robust stage (GCG and PAIR never reach the 50% risk threshold). Later alignment stages *degrade* compute-space robustness — DPO overfits fixed preference data, and RLVR's binary rewards may reduce calibrated refusal. Safety is not monotonic across post-training.

2. Model Scale Barely Helps Against Cheap Attacks

Qwen2.5 scaling (0.5B → 7B): GCG C@0.5 grows 20× (superlinear vs. 15× parameters), but JB only 2.8×. At 7B, JB's AE is still 18× GCG's. Simply making models bigger barely defends against cheap template attacks.

3. Transfer Attacks Slash Attacker Cost

Transferring suffixes from Qwen2.5-0.5B to Qwen3-8B yields identical metrics to direct attack (AE = 4.9×10⁻³, ASR@10 = 0.15) at zero extra optimization cost. This is an Achilles' heel for closed models: attackers can optimize on open proxy models and transfer to GPT-4/Claude without ever querying the target.

4. Safety RL Can Be Worse on Some Attack Types

For Qwen3-4B, SafeRL made GCG robustness *worse* (C@0.5: ∞ → 189.0). Cause: training-distribution mismatch — SafeRL trains on natural-language adversarial prompts, while GCG finds logit-level token sequences outside the learned distribution. Safety training is not a panacea; it must match attack types.

5. Per-Category Attack Costs Differ by 5×

For Qwen3-4B-SafeRL, C@0.5 spans roughly 5× across harm categories. Harassment/bullying, misinformation, and general harm improve under SafeRL; illegal activity and cybercrime get *worse*, attributed to imbalanced safety fine-tuning coverage.

Why This Matters

1. Current evaluations can overstate defenses: ASR@10 framing can turn a 90%-successful attack into a reported 0% success rate. Evaluations ignore attack cost, cost differences, and curve shape. 2. Defense needs a "cost floor" mindset: defenders should raise the attacker's cost floor rather than chase perfect robustness; evaluators must report compute-aware metrics; deployers should audit per harm category. 3. Scale is double-edged: 20× protection against expensive GCG, only 2.8× against cheap JB. 4. Transfer attacks are the open-secret threat to closed models.

Limitations

| Limitation | Note | |---|---| | FLOPs as proxy | Ignores memory bandwidth, latency, parallelism | | Judge reliability | False positives/negatives, category bias in LLM judges | | Attack coverage | No adaptive attacks co-optimized against defenses | | Binary success | All jailbreaks weighted equally, no harm-severity weighting | | Scale limits | Largest closed frontier models not covered | | Modality | Text only; multimodal extension open |

> "Security is not perfect defense — it is making attackers pay a high enough price."

Reference: Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik, Colin Raffel. "Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models." arXiv:2606.11409, 2026.