Paper Overview
Field: Machine Learning Author: Adi Mashiach Published: 2026-11-10 arXiv: 2611.14504
Abstract (English)
Foundation models trained on large unlabeled corpora develop emergent capabilities, tasks that go beyond the training objective. We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions trained on approximately 3.2 trillion tokens of paired protein-ligand sequences. From a single representation space, Bonbon exhibits three zero-shot emergent capabilities at hierarchical levels of resolution: functional (mechanism of action), residue (binding site localization, including orthosteric/allosteric discrimination), and bond (covalent versus non-covalent engagement). All three capabilities generalize to out-of-distribution complexes. Bonbon was trained on protein-ligand sequences alone, without any task-specific supervision and without structural co...(source excerpt truncated)
Key Points
- Trained on ~3.2 trillion tokens of paired protein-ligand sequences; no task-specific supervision, no structural coordinates.
- Three zero-shot emergent capabilities: mechanism of action (functional), binding site localization with orthosteric/allosteric discrimination (residue), and covalent vs non-covalent engagement (bond) — all generalizing to out-of-distribution complexes.
- Scaling experiments across six sizes (42M–1.6B parameters): capabilities emerge with sharp jumps at the largest scale while pretraining loss decreases smoothly — loss-based scaling laws do not predict zero-shot emergence.
- Capabilities emerged at a 1:2000 parameter-to-token ratio, tuned to the sparsity of paired biological data.
- Same frozen representation achieves SOTA binding and affinity prediction ~5 orders of magnitude faster than co-folding methods.
- Prospective wet-lab validation: 61.5% dose-response hit rate on novel compounds across nine disease targets in seven protein families, including structurally novel inhibitors for out-of-distribution targets.
*Auto-collected on 2026-11-12.*