Register Gap: AI Reads Every Nigerian Word but Misses the Meaning

A Nigerian politician tweets: "What a wonderful day for our great nation."

Ask a sentiment analysis model, and it reports: positive emotion, optimistic attitude, all good.

Ask any Nigerian, and they'll tell you: it's sarcasm. He's criticizing the government.

This is the core problem Celestine Achi identifies: AI's dominant failure mode when processing Nigerian public discourse is not translation failure, but contextual failure.

The same sentence can mean the opposite in different contexts.

The Ceiling of NaijaSenti

Existing Nigerian-language sentiment benchmarks—NaijaSenti, AfriSenti—define the task as three-way classification: positive, negative, neutral. This framework is imported directly from English sentiment analysis, assuming word meaning and intent roughly align.

But Nigerian public discourse doesn't work that way.

Achi points out that in Nigerian contexts, there is often a wide gap between a sentence's surface sentiment and its true communicative intent. "Well done" may be sincere praise—or sarcasm. The difference depends on who is speaking, to whom, on what occasion, and in what register.

"Register" is a classic linguistic concept: variation of the same language across social contexts. Nigerian public discourse involves at least four registers: Standard English, Nigerian English, Nigerian Pidgin, and mixed registers. The same person tweeting in Pidgin and issuing a formal statement in Standard English may convey entirely different messages—even with identical literal content.

The Nine-Dimension Meaning Intelligence Framework

Achi proposes the Meaning Intelligence Framework (MIF), which annotates and evaluates utterances along nine dimensions:

1. Register: Standard English? Nigerian English? Pidgin? Mixed? 2. Surface Sentiment: literal positive/negative/neutral 3. True Intent: what the speaker actually aims to achieve 4. Irony: whether literal meaning and intent diverge 5. Coded Subtext: hidden meaning requiring cultural background to decode 6. Risk Tier: how severe the consequences of misreading this utterance would be 7. Annotator Confidence: how certain human annotators are themselves 8. Speaker Emotion: the speaker's actual emotional state 9. Recommended Communications Action: how a brand or government monitoring this should respond

The design logic: sentiment is not a property of a sentence, but of a communicative event. The same words, said by different people, on different occasions, to different audiences, mean entirely different things.

A 30-Item Calibration Dataset

The paper builds a 30-item calibration dataset covering the four registers and tests Gemini 2.5 Flash under two conditions:

• Zero-shot: direct classification
• Schema-informed: the model is given MIF's nine dimension definitions and asked to analyze accordingly

The result gives the paper its title, the Register Gap:

• Zero-shot register classification accuracy: 33.3% (barely above guessing; the four-class random baseline is 25%)
• Schema-informed: 73.3% (+40 percentage points)

A 40-point jump—just from telling the model "you must first determine the social context of this utterance."

The composite Mean Intelligence Score (MIS) rose from 73.2 to 78.6, a 5.4-point gain. The biggest improvements:

• Coded subtext detection: +10 points
• Recommended communications action: +10.3 points

These two dimensions are the most notable. Coded subtext means the model begins to "hear what's between the lines"; recommended communications action means it doesn't just understand—it can propose actionable response strategies.

Translation Isn't the Problem, Context Is

The paper's most important insight is not the 40-point gain, but its reframing of "why AI underperforms on African languages."

The prevailing narrative: AI performs poorly on African languages due to scarce training data and poor translation quality, so the solution is to collect more data, translate more, train more.

Achi says: wait. A model that achieves 90% accuracy on Standard English dropping to 33% on Nigerian English—is that really because it "doesn't know Nigerian English"? No. The model recognizes the vocabulary and grammar of Nigerian English. What it doesn't know is: how far apart surface meaning and real meaning can be in Nigeria.

This isn't a language problem. It's a cultural-context problem.

An analogy: give a foreigner who doesn't understand Chinese humor a Chinese joke. They recognize every character but don't know where the punchline is. They don't fail at Chinese—they fail at Chinese "register": what's said on what occasion, what's meant ironically, what's a coded signal.

Is 30 Items Enough?

The paper's biggest limitation is data size. Thirty calibration items is far too small by any standard. The author acknowledges this and keeps a private test set for contamination-safe evaluation.

But a 40-point gain on 30 items says something deeper than a 2-point gain on 3,000 items: the problem isn't data quantity, it's the framework. You're giving the model not more data, but a better way of looking at the problem.

This aligns with the logic of prompt engineering: the same model with a different prompt performs drastically differently. But MIF is not a mere prompt trick—it's a structured annotation framework whose nine dimensions mutually constrain, forming a reusable evaluation system.

Who Needs This

The paper's direct audience isn't NLP researchers, but three groups:

1. Companies doing social listening in African markets: if you monitor Nigerian social media sentiment with standard sentiment analysis, you may read sarcasm as praise and coded subtext as neutral. MIF offers a more accurate evaluation framework.

2. Researchers doing cross-cultural AI evaluation: MIF's nine dimensions aren't limited to Nigeria. Any language environment where "surface meaning ≠ real meaning"—which is essentially all of them—can borrow this framework.

3. The AI safety and governance community: if a model can't even detect "this sentence is irony," would you trust it with content moderation? In multicultural contexts, AI "understanding" that stops at the literal level makes any decision potentially dangerous.

Epilogue

This paper raises a bigger question: are we evaluating AI in a fundamentally wrong way?

We measure models with accuracy, F1, BLEU—essentially measuring "how well the model matches annotator judgment." But if annotators themselves—humans—need nine dimensions to judge what a sentence means, why do we assume a three-class label suffices?

Meaning is not a property of a sentence; it's a property of a communicative event. That sentence deserves to be taped to every NLP researcher's monitor.

Paper link: https://arxiv.org/abs/2606.20255