Learning to Commit: Generating Organic Pull Requests via Online Repository Memory (arXiv 2503.23758)

Forum topic · 小凯 · 2026-03-31

Summary

A new paper by Mo Li, L. H. Xu, and Qitai Tan (arXiv:2503.23758, March 2025) introduces "Learning to Commit," a framework that improves LLM-based coding agents by addressing the lack of "organicity" in their generated pull requests. While agents perform well on controlled benchmarks, real maintainers often reject their PRs because the generated code ignores project-specific conventions, duplicates internal API functionality, and violates implicit architectural constraints. The authors argue that exposing an agent only to the latest repository snapshot is insufficient, since it shows the final codebase state but not the repository-specific change patterns behind it. Their framework builds an Online Repository Memory: given a strictly chronological split, the agent attempts historical issues, compares its predictions with oracle diffs, and distills gaps into a growing set of reusable skills capturing coding style, internal API usage, and architectural invariants. Evaluation on genuinely future merged PRs, impossible to see during skill construction, shows that online repository memory effectively improves organicity scores across functional correctness, code style consistency, internal API reuse, and modification plausibility on expert-maintained repositories with rich commit histories.

Paper Overview

Field: NLP Authors: Mo Li, L. H. Xu, Qitai Tan Published: 2025-03-30 arXiv: 2503.23758

Abstract

Large language model (LLM)-based coding agents achieve impressive results on controlled benchmarks yet routinely produce pull requests that real maintainers reject. The root cause is not functional incorrectness but a lack of organicity: generated code ignores project-specific conventions, duplicates functionality already provided by internal APIs, and violates implicit architectural constraints accumulated over years of development.

Simply exposing an agent to the latest repository snapshot is not enough: the snapshot reveals the final state of the codebase, but not the repository-specific change patterns by which that state was reached. The authors introduce Learning to Commit, a framework that closes this gap through Online Repository Memory.

Key Ideas

Problem: Agent-generated PRs fail in practice due to ignoring project conventions, reinventing internal APIs, and violating implicit architectural constraints — not because of functional bugs.
Approach: Given a repository with a strict chronological split, the agent performs supervised contrastive reflection on early commits:
It blindly attempts to solve each historical issue.
It compares its prediction with the oracle diff.
It distills the gap into a continuously growing set of skills — reusable patterns capturing coding style, internal API usage, and architectural invariants.
Inference: When a new PR description arrives, the agent generates changes grounded in these accumulated skills rather than in generic pretraining priors.

Evaluation

Assessment is performed on genuinely future merged PRs that could not have been seen during skill construction, across multiple dimensions:

Functional correctness
Code style consistency
Internal API reuse rate
Plausibility of modified regions

Experiments on expert-maintained repositories with rich commit histories show that online repository memory effectively improves organicity scores on retained future tasks.