PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

Forum topic · 小凯 · 2026-08-21

Summary

PGFS++ is a synthesis-aware reinforcement learning framework for input-specific molecular improvement, presented in arXiv paper 2608.19121 by Boqiao Zhang, Godbless James, Sai Krishna Gottipati, and Andrew Fitzgibbon. Building on Policy Gradient for Forward Synthesis (PGFS), the work first introduces PGFS+, which replaces indirect reactant embedding prediction with trainable embedding lookup tables for reaction templates and second reactants, combined with a more effective scoring function and RL algorithm. PGFS+ significantly improves desired molecular properties such as drug-likeness or binding affinity, but reveals a reward-hacking failure mode in which strong reactant search maps diverse input molecules to the same high-reward magnet molecules, collapsing output diversity. PGFS++ addresses this by treating each input molecule as the starting point of a forward synthesis trajectory, applying learned reaction templates with compatible in-stock building blocks to produce molecules with improved target properties, explicit synthesis routes, and structural similarity to the input. Experiments show PGFS++ improves target properties while maintaining high output diversity.

Paper Overview

Field: Machine Learning
Authors: Boqiao Zhang, Godbless James, Sai Krishna Gottipati, Andrew Fitzgibbon
Posted: 2026-08-19
arXiv: 2608.19121

Abstract (English)

Improving molecular properties, such as drug-likeness or binding affinity, is a recurring task in early-stage drug discovery. However, molecules optimized in an unconstrained chemical space have limited practical value if they cannot be synthesized. Policy Gradient for Forward Synthesis (PGFS) is a synthesis-aware reinforcement learning method for molecular improvement, but its use of reactant embedding prediction makes reactant selection indirect, which, as the authors show, limits learning effectiveness.

The authors first develop PGFS+, in which reaction templates and second reactants are represented by trainable embedding lookup tables. Combined with a more effective scoring function and RL algorithm, PGFS+ significantly improves the desired property. However, it exposes a reward-hacking failure mode: a strong reactant search can map diverse input molecules to the same high-reward "magnet" molecules, improving reward while collapsing output diversity.

To address this, the authors introduce PGFS++, a synthesis-aware reinforcement learning framework for input-specific molecular improvement. Given an input molecule, PGFS++ treats it as the starting point of a forward synthesis trajectory, applies learned reaction templates with compatible in-stock building blocks, and produces molecules with improved target properties, explicit synthesis routes, and structural similarity to the input.

Key Contributions

Identifies the limitation of indirect reactant selection via embedding prediction in PGFS
PGFS+: trainable embedding lookup tables for reaction templates and second reactants, with improved scoring function and RL algorithm
Identifies and analyzes a reward-hacking failure mode that collapses output diversity
PGFS++: input-specific molecular improvement that preserves diversity, ensures synthesizability, and maintains structural similarity to the input

Experiments on molecular improvement tasks demonstrate that PGFS++ improves target properties while maintaining high output diversity.

---

*Auto-collected on 2026-08-21*