CellOS: When LeCun's JEPA Meets a Single Cell
Vitaura (百曜科技) has introduced AURA CellOS, described as the world's first LLM-JEPA single-cell world model. The 12-billion-parameter system is pretrained on 390.5 million human single-cell transcriptomes from 47,845 sequencing samples, covering more than 40 human tissues and over 260 cell types.
Why Scaling Laws Stall in Single-Cell AI
Foundation models such as Geneformer, scGPT, scFoundation, and GeneCompass have borrowed the cloze-task training recipe from large language models, masking a subset of genes and asking the model to reconstruct them. A June 2026 *Nature Methods* study using 400 models pretrained on the 22-million-cell scTab corpus showed that performance plateaus after roughly 1% of the pretraining data, about 220,000 cells, indicating a systematic mismatch between LLM-style architectures and single-cell data rather than a data shortage.
Disease trajectories, drug responses, and differentiation are driven by weak, coordinated signals across dozens of genes whose individual expression levels are modest. A cloze-trained model memorizes static expression patterns but cannot capture state transitions.
CellOS Architecture: Three Innovations
1. Dual-View Representation Learning
- Expression view ranks genes by abundance.
- Perception view computes a Surprisal Score to surface biologically informative signals that are masked in raw counts, including regulatory and stress-response genes.
- Pipeline assets — design and optimization of CAR-T, stem-cell therapies, and synthetic-biology cell factories.
- Model-as-a-Service — target discovery, virtual drug screening, and virtual clinical prediction of cell-therapy behavior.
- CellOS is the first 12B LLM-JEPA single-cell world model, trained on 390.5 million transcriptomes across 40+ tissues and 260+ cell types.
- It replaces masked-token cloze training with cross-view JEPA embedding prediction plus surprisal-based perception.
- It achieves 0.619 Pearson_edist (66% above TranscriptFormer) and 0.792 on a biology-conservation annotation metric.
- Three-stage Dense-to-MoE training avoids catastrophic forgetting during scale-up.
- Vitaura is pursuing both internal pipeline assets and a Model-as-a-Service offering for pharma R&D.
- Vitaura (百曜科技) — model release notes for AURA CellOS.
- LeCun, Y. (2022). *Joint Embedding Predictive Architecture (JEPA)*.
- *Nature Methods* (June 2026) — systematic scaling study on the scTab corpus.
- *Cell* (December 2024) — Stanford / Genentech / CZI paper on the AI Virtual Cell era.
The two views run in parallel, filtering noise from high-dimensional transcriptomes and exposing critical transition nodes during differentiation, migration, and disease onset.
2. JEPA: From Cloze Tasks to a World Model
CellOS adopts the Joint Embedding Predictive Architecture (JEPA) proposed by Yann LeCun in 2022. Instead of reconstructing masked inputs, JEPA predicts embeddings of one view from another (expression → perception, or vice versa), forcing the model to build a unified latent representation of cell state.
This delivers strong gains on dynamic prediction tasks such as gene knockout outcomes, drug perturbations, and directed differentiation trajectories.
3. Dense-to-MoE Lossless Scaling
Training proceeds in three stages:
| Stage | Model | Training Focus | |---|---|---| | 1 | Mid-size Dense | Expression-view pretraining | | 2 | 12B MoE | Smooth expansion, preserved knowledge | | 3 | 12B MoE + dual view | JEPA alignment |
The pipeline avoids catastrophic forgetting that usually accompanies large parameter jumps.
Benchmark Results
| Model | Pearson_edist | |---|---| | CellOS | 0.619 | | TranscriptFormer | 0.373 | | scGPT | ~0.35 | | UCE | ~0.33 |
CellOS is the only model to exceed 0.6 on cell-state prediction, a 66% improvement over the next-best baseline. On a biology-conservation annotation metric it reaches 0.792, accurately resolving subtle subsets such as T-cell subtypes, immune-aging populations, and iPSC differentiation intermediates.
Parameter ablation from 0.2B → 2B → 12B shows near-log-linear scaling of annotation performance, with additional gains in rare and fine-grained cell-state discrimination after Stage-3 multi-view JEPA alignment.
Why JEPA Fits Single-Cell Data
| | Language data | Single-cell data | |---|---|---| | Structure | Token sequence | High-dimensional vector | | Underlying rule | Syntax, semantics | Regulatory networks, pathways | | Variation type | Left-to-right generation | Trajectories in continuous space | | Noise level | Low | High (batch effects, technical noise) |
JEPA assumes no sequential order, learns state-level correspondences, and supports cross-view prediction, properties aligned with the nature of single-cell transcriptomes.
Commercialization Strategy
Open Challenges
1. High-quality perturbation data remain scarce and expensive. 2. Multi-modal integration (proteomics, spatial omics, metabolomics, morphology) is unresolved. 3. Interpretability is required for scientific adoption. 4. Business value must be demonstrated through shortened R&D cycles and lower costs.