Co-LMLM: Continuous-Query Limited Memory Language Models Let LLMs Look Up Knowledge Instead of Memorizing It

Forum topic · 小凯 · 2026-07-10

Summary

Co-LMLM (Continuous-Query Limited Memory Language Models), a paper by Yair Feldman, Linxi Zhao, Nathan Godey et al. from Cornell and the University of Washington (arXiv 2607.07707), proposes an architecture that decouples knowledge storage from model reasoning. Instead of compressing facts into weights during pretraining, the model stores knowledge in an external repository of continuous vector keys paired with raw text values, and generates vector queries at inference time to retrieve relevant passages—no structured knowledge base like Wikidata or hand-designed query languages required. An automatic annotation pipeline can build the knowledge base from any raw text source, including FineWeb-Edu, arXiv papers, or web pages. Experiments show a 360M-parameter Co-LMLM achieves lower perplexity than a standard LLM pretrained on 40x more data, and on SimpleQA factual QA it matches GPT-4o-mini and outperforms Claude Sonnet 4.5. The architecture also enables knowledge control: updating, deleting, restricting, and auditing knowledge by editing the key-value store without retraining, with implications for interpretability and AI safety.

Overview

Paper: Co-LMLM: Continuous-Query Limited Memory Language Models
arXiv: https://arxiv.org/abs/2607.07707
Authors: Yair Feldman, Linxi Zhao, Nathan Godey et al. (Cornell, University of Washington)
Categories: cs.CL, cs.AI, cs.LG

Key points

The motivation: knowledge shouldn't live in weights

Conventional LLM pretraining compresses internet text into model weights. This causes well-known problems:

Knowledge staleness — weights are frozen after training.
Hallucination — models interpolate in weight space when memory is fuzzy.
No provenance — models cannot cite the source of a fact.
Inefficiency — storing Wikipedia (tens of GB) in trillions of parameters is extremely uneconomical.

Limited Memory Language Models (LMLMs) flip this: knowledge lives in an external repository, and the model generates queries and composes retrieved text into its answers.

Continuous queries over an unstructured knowledge base

Earlier LMLMs depended on relational knowledge bases (e.g., Wikidata) and structured query languages like SPARQL, requiring pre-curated entity-relation triples.

Co-LMLM's breakthrough: the knowledge base stores continuous vector keys + raw text values, and the model generates a vector query at runtime rather than SQL or keywords:

1. The model reads the current context. 2. It generates a vector query. 3. It retrieves the most similar keys in the knowledge base. 4. The corresponding text values are fused into generation.

Unlike a standard vector database (e.g., Pinecone, Milvus), the keys are generated dynamically by the model rather than precomputed.

Automatic annotation pipeline

Co-LMLM ships with a pipeline that ingests arbitrary text (web pages, papers, novels, chat logs), identifies factual snippets, encodes them as vector keys, and stores them. Knowledge bases can therefore be built from FineWeb-Edu, arXiv, or any text source—no manual structuring required.

Results: small model beats large models

A 360M-parameter Co-LMLM, pretrained on Wikipedia and FineWeb-Edu, achieves lower perplexity than a vanilla LLM pretrained on 40x more data.
On SimpleQA, the 360M Co-LMLM performs on par with GPT-4o-mini and above Claude Sonnet 4.5.

The takeaway: decoupling knowledge storage from reasoning may be key to efficiency—store facts in an optimized database and let the model focus on how to query and use them.

Knowledge control as a built-in capability

Because knowledge is external, Co-LMLM supports:

Updating knowledge by editing text values, no retraining.
Deleting sensitive facts by removing key-value pairs.
Restricting sources by controlling which knowledge base subsets the model accesses.
Auditing answers by inspecting which text spans were retrieved.

The authors frame this as foundational for interpretable AI and AI safety.

One-line summary

Co-LMLM teaches models to "look things up smartly" rather than memorize harder—freeing knowledge from weights makes models smaller, more accurate, more controllable, and finally source-aware.