Compute-Aware Adversarial Robustness Evaluation: Why 90% Attack Success Can Be Reported as 0%

Forum topic · 小凯 · 2026-06-14

Summary

A forum post on zhichai.net discusses the paper "Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in LLMs" (arXiv:2606.11409) by researchers from the University of Toronto, Vector Institute, and Hugging Face. The paper argues that conventional jailbreak evaluation using Attack Success Rate (ASR) under fixed query budgets hides the true cost of attacks. By introducing the classical security concept of "work factor" and measuring attack cost uniformly in FLOPs, the authors propose new metrics: C@τ (compute needed to reach a given risk level) and AE (average efficiency, risk per FLOP). Key findings: (1) SFT checkpoints are more robust than DPO/RLVR-aligned models, suggesting alignment safety is non-monotonic; (2) scaling models from 0.5B to 7B raises cost against expensive GCG attacks ~20× but only ~2.8× against cheap template attacks; (3) transfer attacks let adversaries compromise closed-source models at near-zero extra optimization cost; (4) safety RL can be worse than base models for some attack types and harm categories; (5) per-category compute costs vary up to 5×. The post concludes that safety evaluation must report attack cost, not just success frequency, and that defenders should aim to raise the attacker's cost floor.

Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in LLMs

> Paper: Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models > Authors: Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik, Colin Raffel > Institutions: University of Toronto, Vector Institute, Hugging Face > arXiv:2606.11409 > Code: r-three/risk-under-pressure (to be released) > Date: 2026-06-09

TL;DR

Current LLM safety evaluation uses Attack Success Rate (ASR) under fixed query budgets. A model broken on the first try and one broken on the tenth try are rated equally unsafe. This paper imports the classical security concept of work factor into LLM safety, measuring attack cost uniformly in FLOPs. It shows existing evaluations can severely overstate defenses (a 90%-success attack can be reported as 0% ASR), and proposes two new metrics: C@τ (compute needed to reach τ risk) and AE (average efficiency, risk per unit compute).

The Problem

Fixed-budget ASR treats all attacks as equal cost, hiding true attacker expense.
Query counts are a poor budget unit: template attacks (JailBroken) cost one forward pass per query, while white-box gradient attacks (GCG) cost ~128 forward passes plus backprop and candidate scoring per step.
Leaderboard-style comparisons rate two models with identical ASR as equally safe despite vastly different costs.

Method

| Metric | Meaning | Reading | |--------|---------|---------| | C@τ | Compute required to reach risk level τ | Higher = safer | | CAURC | Area under the risk-compute curve | Overall risk-cost trade-off | | AE | CAURC / C_max, risk per FLOP | Lower = safer |

FLOPs are hardware-agnostic, comparable across components (gradients, auxiliary model calls, judge queries), and can derive energy, wall-clock time, GPU-hours, and cost-per-break.

Five Key Findings

1. SFT is more robust than DPO/RLVR — alignment safety is non-monotonic

For Tulu3-8B: the SFT checkpoint never reaches 50% risk under GCG or PAIR (C@0.5 = ∞), while DPO and RLVR checkpoints regress. DPO overfits fixed preference data; RLVR's binary rewards may weaken calibrated refusals.

2. Scale barely helps against cheap attacks

Qwen2.5, 0.5B → 7B: GCG cost grows ~20× (superlinear in the 15× parameter increase), but JailBroken cost grows only ~2.8×. ASR@10 barely changes. Scaling alone offers little protection against cheap template attacks.

3. Transfer attacks slash attacker cost

Suffixes optimized on an open proxy model (e.g., Qwen2.5-0.5B) transfer to larger targets at near-zero additional optimization cost — a critical threat to closed models like GPT-4 and Claude.

4. Safety RL can be worse than base models

Qwen3-4B: SafeRL makes the model *less* robust to GCG (C@0.5 drops from ∞ to 189) due to training-distribution mismatch; improvements vary by harm category, with illegal-activity and cybercrime categories getting worse.

5. Harm categories differ by up to 5× in attack cost

Safety fine-tuning data is unevenly distributed across categories (e.g., harassment well covered, cybercrime not).

Why It Matters

ASR@10 conflates frequency with cost; the risk-compute curve shape (linear rise vs. fast plateau) is invisible in single-point metrics.
Defenders should aim to raise the attacker's cost floor, not chase perfect robustness, focusing specifically on cheap attacks.
Evaluators must report compute-aware metrics; deployers should audit per harm category, not just aggregate scores.

Limitations

FLOPs as proxy ignores memory bandwidth/latency; judge reliability issues (false positives/negatives); no adaptive attacks against joint defenses; all jailbreaks weighted equally; only text modality; largest closed frontier models not covered.

Citation

Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik, Colin Raffel. "Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models." arXiv:2606.11409, 2026.

> "Safety is not perfect defense — it is making attackers pay enough."