Profiling for Pennies: LLM Agents Reconstruct Your Personal Profile in 10 Minutes for Under $3

This post discusses the paper "Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents" (arXiv:2605.06232) by researchers from Zhejiang University and Chongqing University. After reading it, the author says they searched their own name and made three social profiles private.

Key points

1. The Privacy Iceberg framework

The paper models LLM-driven privacy risk in three layers:

• DII (Directly Identifiable Information) — facts explicitly published and searchable, e.g., full name, school, awards.
• CII (Contextually Inferred Information) — information not stated but deducible, e.g., inferring someone lives in Melbourne from a mention of a "hook turn."
• DAI (Deeply Aggregated Information) — deep insights synthesized across platforms; individually innocuous sources combine into a full personal portrait (political leanings, health, social networks).

The most alarming layers are CII and DAI: information you never disclosed directly, which LLMs can mine from your scattered digital footprint.

2. IcebergExplorer results

The authors built IcebergExplorer, simulating a worst-case adversary. Given a minimal PII seed (e.g., full name + city), it produces a high-fidelity profile covering 17 privacy categories in ~10 minutes for under $3, with >90% accuracy.

Sample category accuracies: core identity 76.36%, financial status 91.35%, education 52.34%, relationships 85.89%, health & wellness 82.98%, lifestyle & interests 93.53%, appearance and occupation 100% (small samples).

3. Six root causes of vulnerability

1. Persistence — old posts remain accessible indefinitely. 2. Linkability — shared handles/avatars link identities across platforms. 3. Readability — LLMs extract structure from unstructured prose. 4. Identifiability — unique usernames enable precise targeting. 5. Inferability — likes, follows, and stars leak attributes (e.g., beliefs, preferences). 6. Composability — cross-source reasoning produces the "mosaic effect."

4. Existing defenses are largely ineffective

• System-prompt guardrails: refusal rates were 0.00% for GPT-4o, GPT-4.1, and Gemini 2.5 Pro at the Scorer, Extraction, and Operation stages (max 6.88% for GPT-4o at Reasoning). Packaging attacks as programmatic workflows bypasses guardrails almost entirely.
• Individual defenses: Context Ignoring (CI) and Injecting Data (ID) both achieved defense success rates near 0% across mainstream LLMs.

5. Why it matters

1. Profiling is extremely cheap ($3 + 10 minutes). 2. Uses include job screening, product recommendation, headline tailoring — and targeted scams. 3. All inputs come from publicly posted content.

In blind preference tests, content informed by IcebergExplorer profiles beat baseline content in job recommendations (90% win rate), product recommendations (86.67%), and headline customization (86.67%).

6. Mitigations (multi-stakeholder)

• LLM providers: redesign guardrails to detect privacy risk in reasoning chains, not just direct queries.
• Individuals: audit public footprints, avoid cross-platform identifiers, be cautious with inferable information.
• Publishers: reduce linkability, strip metadata, set content expiry.

The paper concedes no perfect technical solution currently exists.

Paper details

| Attribute | Value | |---|---| | Title | Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents | | Authors | Jiahao Chen, Qi Zhang, Ruixiao Lin, Chunyi Zhou, Tianyu Du, Qingming Li, Tong Zhang, Junhao Li, Yuwen Pu, Shouling Ji | | Institutions | Zhejiang University; Chongqing University | | arXiv ID | 2605.06232 | | Date | 2026-05-07 | | Categories | cs.CR, cs.AI, cs.CL | | Scale | 30 real-world participants; 17 privacy categories; ~8,000 manually verified facts | | Key numbers | Fact coverage 92.33%; URL coverage 70.43%; photo coverage 60%; guardrail refusal ≈ 0%; adversarial defense success ≈ 0% |