Learning to Commit: Generating Organic Pull Requests via Online Repository Memory (arXiv 2503.23758)

Forum topic · 小凯 · 2026-03-31

Summary

This arXiv paper (2503.23758) by Mo Li, L. H. Xu, and Qitai Tan introduces Learning to Commit, a framework that helps LLM-based coding agents produce pull requests that real maintainers are more likely to accept. The authors identify the core problem as a lack of organicity: generated code ignores project-specific conventions, duplicates functionality already covered by internal APIs, and violates implicit architectural constraints. They argue that exposing agents only to the latest repository snapshot is insufficient, since snapshots reveal the final codebase state but not the repository-specific change patterns that led there. Learning to Commit addresses this with Online Repository Memory: given a strict chronological split of a repository, the agent performs supervised contrastive reflection on earlier commits—attempting each historical issue blind, comparing predictions with oracle diffs, and distilling gaps into a growing set of reusable skills capturing coding style, internal API usage, and architectural invariants. Evaluation on genuinely future merged PRs, impossible to see during skill building, spans functional correctness, style consistency, internal API reuse, and modification-area plausibility. Experiments on expert-maintained repositories with rich commit history show online repository memory effectively improves organicity scores on held-out future tasks.

Paper Overview

Field: NLP Authors: Mo Li, L. H. Xu, Qitai Tan Published: 2025-03-30 arXiv: 2503.23758

Abstract

Large language model (LLM)-based coding agents achieve impressive results on controlled benchmarks yet routinely produce pull requests that real maintainers reject. The root cause is not functional incorrectness but a lack of organicity: generated code ignores project-specific conventions, duplicates functionality already provided by internal APIs, and violates implicit architectural constraints accumulated over years of development. Simply exposing an agent to the latest repository snapshot is not enough: the snapshot reveals the final state of the codebase, but not the repository-specific change patterns by which that state was reached.

We introduce Learning to Commit, a framework that closes this gap through Online Repository Memory. Given a repository with a strict chronological split, the agent performs supervised contrastive reflection on earlier commits: it blindly attempts to solve each historical issue, compares its prediction with the oracle diff, and distills the gaps into a growing set of skills—reusable patterns capturing coding style, internal API usage, and architectural invariants. When a new pull request description arrives, the agent generates changes grounded in these accumulated skills rather than generic pretraining priors.

Evaluation is performed on genuinely future merged pull requests, impossible to see during skill construction, and spans multiple dimensions including functional correctness, code style consistency, internal API reuse rate, and modification-area plausibility. Experiments on expert-maintained repositories with rich commit histories show that Online Repository Memory effectively improves organicity scores on held-out future tasks.

Key Takeaways

LLM coding agents fail on real PRs not because of bugs but because output lacks "organicity" — it misses project conventions, internal APIs, and architectural constraints.
A code snapshot alone doesn't teach the change patterns behind the codebase; the commit history does.
The framework builds an Online Repository Memory via supervised contrastive reflection on past commits.
Gains are validated strictly on future merged PRs across correctness, style, API reuse, and modification plausibility.

*Auto-collected on 2026-03-31.*