Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

Forum topic · 小凯 · 2026-06-07

Summary

Code2LoRA is a hypernetwork framework that generates repository-specific LoRA adapters for code language models, injecting repository knowledge with zero inference-time token overhead. Unlike long-input retrieval (RAG or dependency analysis) or per-repository fine-tuning, Code2LoRA offers two modes: Code2LoRA-Static converts a repository snapshot into an adapter for stable codebase comprehension, while Code2LoRA-Evo maintains an adapter backed by a GRU hidden state updated per code diff for actively evolving codebases. Evaluation uses RepoPeftBench, a benchmark of 604 Python repositories with a static track (40K train/12K test assertion-completion tasks) and an evolution track (215K train/87K test commit tasks). Code2LoRA-Static achieves 63.8% cross-repository and 66.2% in-repository exact match, matching the per-repository LoRA upper bound; Code2LoRA-Evo reaches 60.3% cross-repository exact match, +5.2pp over a single shared LoRA. Paper: arXiv 2606.06492 by Hotsko, Li, and Deng.

Paper Overview

Field: NLP
Authors: Liliana Hotsko, Yinxi Li, Yuntian Deng
Published: 2026-06-04
arXiv: 2606.06492

Abstract

Code language models need repository-level context to resolve imports, APIs, and project conventions. Existing methods inject this knowledge as long inputs (retrieved through RAG or dependency analysis) or through per-repository fine-tuning and LoRA — costly at repository scale and brittle to evolving codebases. The authors introduce Code2LoRA, a hypernetwork framework that generates repository-specific LoRA adapters, effectively injecting repository knowledge with zero inference-time token overhead.

Code2LoRA supports two usage scenarios:

Code2LoRA-Static: converts a single repository snapshot into an adapter, suitable for comprehension of stable codebases.
Code2LoRA-Evo: maintains an adapter backed by a GRU hidden state updated per code diff, suitable for active development of evolving codebases.

Benchmark and Results

To evaluate Code2LoRA, the authors built RepoPeftBench, a benchmark spanning 604 Python repositories:

Static track: 40K training / 12K test assertion-completion tasks
Evolution track: 215K training / 87K test commit tasks

Results:

Static track: Code2LoRA-Static reaches 63.8% cross-repository and 66.2% in-repository exact match, on par with the per-repository LoRA upper bound.
Evolution track: Code2LoRA-Evo achieves 60.3% cross-repository exact match, +5.2pp better than a single shared LoRA.

Key Takeaways

Hypernetwork-generated LoRA adapters can encode repository knowledge without inference-time token overhead.
GRU-based adapter maintenance enables tracking of evolving codebases across commits.
RepoPeftBench provides a large-scale testbed for repository-level adaptation of code models.

---

*Auto-collected on 2026-06-07*