EngramEdit: Performing Knowledge Surgery on LLMs via Conditional Memory Without Retraining

When a Large Language Model Gets a Fact Wrong, What Do You Do?

Imagine a encyclopedia editor who knows everything but has one entry wrong: it says "the capital of France is London." Traditional fixes all have problems:

1. Retraining: like sending the editor back to university for four years — too expensive 2. Fine-tuning: like a few days of remedial classes — but it may scramble other knowledge 3. RAG (retrieval-augmented generation): like giving the editor an assistant who keeps reminding them — a symptom-level fix that doesn't cure the root problem

The DeepSeek Engram architecture offers a new angle: store factual knowledge and reasoning separately. Facts live in "conditional memory" (an n-gram lookup table), while reasoning is handled by the Transformer backbone. It's like separating the encyclopedia from the editor's brain — editing the encyclopedia doesn't affect the editor's thinking ability.

But a question arises: how do you edit the encyclopedia?

Three Hard Problems

The EngramEdit authors found that editing knowledge in conditional memory is much harder than expected, with three pitfalls:

Pitfall 1: Shared embedding conflicts

The same n-gram embedding can serve multiple facts. For example, the n-gram "France" appears both in "capital of France" and "population of France" contexts. Editing the embedding for "capital of France" may also change the answer for "population of France."

It's like editing one entry in a shared notebook, only to find other entries referenced that page and all broke too.

Pitfall 2: Expression generalization

A single fact has many phrasings: "The capital of France is Paris," "Paris is the capital of France," "What is the capital city of France? Paris." Different phrasings activate different n-grams; editing the embedding for one phrasing fails when the question is rephrased.

It's like teaching the editor only the phrasing "The capital of France is Paris" — ask "Paris is the capital of which country?" and they're wrong again.

Pitfall 3: Unrelated knowledge contamination

Shorter, more common n-grams are activated more often. Editing these high-frequency embeddings affects many unrelated facts.

It's like editing a notebook's table of contents — every entry that references the table of contents changes too.

EngramEdit's Three-Step Surgery

EngramEdit's core idea: first compute target memory representations, then jointly update the embeddings. Three steps:

Step 1: Generate paraphrases, compute target memories

Generate multiple paraphrases for each fact (e.g., 5 phrasings of "The capital of France is Paris"), then add a learnable perturbation to each paraphrase's memory representation. Only the perturbation is optimized, so the model predicts the correct new fact. Once optimized, you have each paraphrase's "target memory representation."

The key point: don't touch the embeddings directly — first figure out what the memory should look like.

Step 2: Map back to n-gram embeddings

Map each target memory representation back to the n-gram embeddings it uses. Different paraphrases may share embeddings, and different edits may share the same batch of embeddings. EngramEdit jointly solves for all edits' embedding updates so that each embedding is updated exactly once while satisfying every paraphrase and edit that uses it.

Like performing multiple surgeries at once — but a shared organ is only cut once.

Step 3: Penalize large changes to high-frequency embeddings

During joint optimization, embeddings that are reused frequently get a stronger penalty — if an embedding is used by 1,000 facts, changing it costs roughly 1,000x more than one used by 3 facts. This protects unrelated knowledge from contamination.

Like making edits to a table of contents far more expensive than edits to an obscure entry — because the table of contents has too wide a blast radius.

Experimental Results: Near-Perfect Knowledge Surgery

Tested on the CounterFact and ZsRE knowledge-editing benchmarks:

• Edit success rate near 100%: modifying only conditional memory makes the model predict the updated fact
• Generalizes to unseen phrasings: after editing "The capital of France is Paris," the model also answers "Paris is the capital of which country?" correctly
• Multi-hop reasoning works: on the MQuAKE multi-hop benchmark, EngramEdit's accuracy is 3x the strongest baseline (with CoT prompting)
• Unrelated knowledge preserved: even with cumulative fact updates, unrelated knowledge and general capabilities remain essentially unaffected

Why This Matters

True decoupling of knowledge and reasoning

EngramEdit isn't just a knowledge-editing tool — it validates a deeper hypothesis: conditional memory architectures really do separate knowledge from reasoning. You can change the knowledge without touching the reasoning. This is impossible in a standard Transformer: change one weight, and both knowledge and reasoning are affected.

It's like finally finding the anatomical boundary between the brain's "knowledge base" and "thinking engine," allowing surgery on the knowledge base alone.

An editable knowledge interface

Conditional memory is no longer just a "capacity expander" (helping the model remember more) but an "editable knowledge interface" (letting model knowledge be updated independently). This enables:

• Fact correction: model got it wrong? Edit conditional memory directly, no retraining
• Knowledge updates: the world changed (a new president, a company acquired) — batch-update conditional memory
• Personalization: different users see different fact versions by editing their own memory layers

A general framework for shared-resource updates

The "shared embedding conflict" problem EngramEdit solves is quite general: when many users share one resource, how do you update it without interference?

The framework applies beyond conditional memory to:

• MoE expert networks: multiple tokens share experts — how to edit an expert without affecting other tokens
• RAG vector stores: multiple queries share documents — how to update a document without affecting other queries
• Knowledge graphs: multiple entities share relations — how to change a relation without affecting other entities

Honest Assessment

EngramEdit is not a panacea:

1. Depends on conditional memory architectures: only works on DeepSeek Engram-style architectures, not standard Transformers 2. Limited edit scale: the paper tests single or few-fact edits; large-scale knowledge updates (e.g., 10,000 facts at once) are untested 3. n-gram coverage issues: if a fact's phrasings are too rare, the generated paraphrases may not cover all relevant n-grams 4. Joint optimization cost: computation grows with the number of edits

But as a proof of concept, it's already compelling: conditional memory architectures make knowledge editable.

Conclusion

EngramEdit suggests an analogy: a traditional Transformer is like a monolith cast in one piece — you can't change one corner without affecting the whole. Conditional memory architecture is like Lego bricks: you can swap a single block. EngramEdit finds the block that needs replacing and swaps it precisely, leaving the rest untouched.

When model knowledge can be created, deleted, modified, and queried like a database, we're one step closer to "models that never go out of date."

---

Paper: EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory

Code: github.com/ModalityDance/EngramEdit