Structured Intent as a Protocol-Like Communication Layer: Cross-Model and Cross-Language Evaluation of Prompt Frameworks

Forum topic · 小凯 · 2026-04-02

Summary

This paper investigates how reliably structured intent representations preserve user goals across different AI models, languages, and prompting frameworks. Building on PPS (Prompt Protocol Specification), a 5W3H-based structured intent framework, the study extends prior Chinese-language findings in three directions: cross-model robustness across Claude, GPT-4o, and Gemini 2.5 Pro; controlled comparison against CO-STAR and RISEN frameworks; and a user study (N=50) on AI-assisted intent expansion in ecologically valid settings. Analyzing 3,240 model outputs (3 languages x 6 conditions x 3 models x 3 domains x 20 tasks), judged independently by DeepSeek-V3, the authors find that structured prompting substantially reduces cross-language score variance, with the strongest structured condition lowering cross-language sigma from 0.470 to approximately 0.020. A notable weak-model compensation effect emerges: Gemini, the lowest-scoring baseline model, shows a much larger D-A gain (+1.006) than the strongest model, Claude (+0.217). At current evaluation resolution, the 5W3H, CO-STAR, and RISEN frameworks achieve similarly high goal alignment, suggesting dimensional decomposition itself is a key active ingredient. Full paper: arXiv 2603.11113.

Paper Overview

Field: AI
Author: Peng Gang
Published: 2026-03-31
arXiv: 2603.11113

Abstract (translated from the Chinese post)

How reliably can structured intent representations preserve user goals across different AI models, languages, and prompting frameworks? Prior work showed that PPS (Prompt Protocol Specification), a 5W3H-based structured intent framework, improves goal alignment in Chinese and generalizes to English and Japanese. This paper extends that line of inquiry in three directions:

1. Cross-model robustness across Claude, GPT-4o, and Gemini 2.5 Pro 2. Controlled comparison with CO-STAR and RISEN 3. User study (N=50) of AI-assisted intent expansion in ecologically valid settings

Key findings

Across 3,240 model outputs (3 languages x 6 conditions x 3 models x 3 domains x 20 tasks), evaluated by an independent judge (DeepSeek-V3), structured prompting substantially reduces cross-language score variance compared to unstructured baselines.
The strongest structured condition reduces cross-language sigma from 0.470 to approximately 0.020.
A weak-model compensation pattern was observed: the lowest baseline model (Gemini) showed a much larger D-A gain (+1.006) than the strongest model (Claude, +0.217).
At current evaluation resolution, the 5W3H (PPS), CO-STAR, and RISEN frameworks achieved similarly high goal alignment scores, suggesting that dimensional decomposition itself is an important active ingredient.

---

*Archived from zhichai.net on 2026-04-02.*