Continuous Variational Synthesis: Generative Models Whose Output Is Nearly a Quadrillion Molecules

Forum topic · QianXun · 2026-09-29

Summary

A forum post discusses the arXiv preprint 'Continuous Variational Synthesis' (arXiv:2609.35083), which moves generative sampling directly into the DNA synthesis process. In variational synthesis, a solid-phase DNA synthesizer becomes a sampler: injecting nucleotide mixtures at each step makes nucleotide incorporation a random event governed by mixture ratios, so one synthesis well yields a batch of molecules drawn from a specified distribution. The paper's contribution is a continuous relaxation—parameterizing mixture ratios via softmax so SGD works—followed by post-training quantization (KL-distance rounding) onto the 15 equal-weight nucleotide mixtures hardware supports. Benchmark tasks include CYP2C9 enzymes, HLA-A*02:01 peptides, TCR-mimic antibody CDRH3s, and EF1alpha promoters, using forward-KL pretraining and reverse-KL reward fine-tuning. Reported results show a strictly dominating quality-diversity Pareto frontier over EM-based variational synthesis and PGLD; notably, prior EM methods produced clean enzyme samples at only 2e-6. In vitro, one 150 bp regulatory DNA library yielded 1.6 nM output—9.6e14 molecules—with 24.4 million full-length reads, evaluated via kernelized Stein discrepancy (KSD-B), confirming in silico performance transfers physically. The post also flags limitations: preprint status, three of four tasks being in silico only, and hard capacity limits of quantization.

Overview

A post on zhichai.net reviews *Continuous Variational Synthesis* (arXiv:2609.35083, submitted 2026-09-28 by Alan N. Amin, Mattia G. Gollub, Andrei Slabodkin, Elizabeth B. Wood, and Eli N. Weinstein), which closes the gap between what generative models can sample and what synthesizers can physically produce.

The core idea

In the usual workflow, a generative model outputs sequences that a synthesizer builds one by one. Variational synthesis instead moves sampling into the synthesis itself: in solid-phase DNA synthesis, injecting a nucleotide mixture at each coupling step makes each incorporation a random event with probabilities set by mixture concentrations. The synthesizer itself becomes a sampler, and each well yields a batch of molecules from the distribution

q_\theta(X)=\frac{1}{M}\sum_{m=1}^{M}\prod_{\ell=1}^{L}\theta^{m}_{\ell,X_\ell}

a mixture over $$M$$ wells, with $\theta$ the per-step mixture ratios. Total sample count equals synthesis yield—on the order of trillions to quadrillions of molecules.

The obstacle and the fix

Synthesizers only support a finite catalog of mixtures—typically 15 equal-weight nucleotide mixtures ( $$2^4-1$$ non-empty subsets of the four bases). This discrete constraint blocks gradient flow, forcing prior work into EM-style discrete optimization, with no real pretraining or unsupervised fine-tuning.

The paper's two-step approach:

1. Continuous-space training: parameterize $\theta = \text{softmax}(\phi)$ with real-valued $\phi$ , so $\theta$ can sit anywhere on the simplex and Adam applies directly. 2. Post-training quantization: round each $\theta$ to the nearest physically realizable mixture, using KL $(\theta\|\theta')$ as the distance (best-performing). If the synthesizer supports $$K$$ shared preset mixtures, these are optimized first to minimize total distance. Protein tasks are trained in amino-acid space with the codon-level constraints translated accordingly.

Four benchmark tasks

| Task | Length | Wells (M) | Prior | Reward | |---|---|---|---|---| | CYP2C9 enzyme | 434 aa | 16 | None (MSA data) | None; forward KL | | HLA-A*02:01 peptide | 9 aa | 32 | Uniform | MHCflurry 2.0 presentation percentile | | TCRm antibody CDRH3 | 15 aa | 32 | Autoregressive transformer | 7M-param encoder predicting pHLA binding counts | | EF1alpha promoter | 150 bp | 32 | Fine-tuned MarinDNA | BPNet-style CNN, Jurkat vs. HEK ATAC difference |

Training combines forward-KL pretraining (mode-covering) and reverse-KL reward fine-tuning (zero-avoiding), sweeping hyperparameter $\alpha$ over the tilted target $\tilde p(x)\propto \pi(x)\exp(\alpha r(x))$ to trace the quality-diversity frontier. Gradient variance is reduced via analytic entropy terms (only the last term needs REINFORCE), Rao-Blackwellization over the prior, leave-one-out control variates, and split-merge resampling—though ablations show the main gains come from the continuous relaxation plus quantization.

Results: strictly dominating Pareto frontier

CYP2C9: prior EM variational synthesis produced internally-stop-codon-free samples at only $2\times10^{-6}$ . cVS sequences scored by ESMFold reach pTM/pLDDT near evolutionary sequences, with TM-score > 0.8 to human CYP2C9.
HLA peptides: cVS improves most in the high-diversity regime; quantization cost is small.
Antibody CDRH3: cVS Pareto-dominates EM VS and PGLD on predicted binding and diversity; removing either KL stage hurts. Kernelized 2-Renyi entropy shows cVS covers broader modes while PGLD is wide but mode-narrow.
Regulatory DNA (Jurkat vs. HEK chromatin accessibility over a 150 bp design region): cVS learns structured motifs with grouped arrangements, distinct from PGLD outputs.

Scaling $$K$$ (preset mixtures) or $$M$$ (wells) monotonically improves performance. Forward-KL training takes under 5 minutes on A100/H100; in reverse-KL fine-tuning, over 99% of compute goes to the prior and reward models, not the variational synthesis model.

The wet-lab half: 1.6 nM

Only the regulatory DNA task was validated in vitro:

Synthesis yield: 1.6 nM, i.e. $9.6\times10^{14}$ molecules
Library input: 250 ng DNA
Sequencing: Illumina NextSeq 2000, 2×150 bp
Full-length reads: 24,471,996

Since only unnormalized target densities are available, the authors compare via KSD-B (kernelized Stein discrepancy for biological sequences; IMQ Hamming kernel, c=1, β=0.5, 5 neighbors per point, 1000 samples per draw, 5 independent draws). The in vitro cVS library achieves small KSD-B alongside high reward and high prior likelihood; UMAP shows in vitro, in silico, and free cVS tightly clustered, with quantization effects visible but minor. In vitro cVS significantly outperforms in silico PGLD on all metrics.

Why it matters

1. It physically connects generation and manufacturing at quadrillion-scale—physical output, not sampling estimates—directly relevant to antibody discovery, promoter engineering, and directed evolution. 2. The continuous-relaxation-then-quantize recipe transfers to any generative problem where parameters must lie in a hardware-allowed set. 3. It exposes how badly discrete optimization failed: EM variational synthesis's $2\times10^{-6}$ clean-sample rate shows it simply doesn't scale. 4. The single wet-lab validation is substantive: 1.6 nM yield with 24 million reads and KSD-B testing provides orders of magnitude more evidence than many claimed end-to-end papers.

Limitations

Preprint; three of four tasks are in silico only, with rewards based on ESMFold, MHCflurry, and custom binding predictors whose errors propagate.
Designs longer than a single oligonucleotide (e.g., full-length CYP) require assembly or enzymatic synthesis, adding constraints and errors.
$K\to\infty$ does not asymptotically reach free cVS, since not all amino-acid distributions are realizable via per-codon independent nucleotide draws—a hard quantization ceiling.
$$M$$ is only 16 or 32 (low-capacity); higher-capacity behavior is extrapolated from trends.
The $2\times10^{-6}$ figure is EM VS's failure rate, not cVS's.

References

Alan N. Amin, Mattia G. Gollub, Andrei Slabodkin, Elizabeth B. Wood, Eli N. Weinstein, *Continuous Variational Synthesis*, arXiv:2609.35083, 2026-09-28
Background: original variational synthesis; PGLD as a baseline
Tools: ESMFold, MHCflurry 2.0, MarinDNA, BPNet
Evaluation: KSD-B (Kernelized Stein Discrepancy for Biological Sequences)