Code2LoRA: Hypernetwork-Generated Repository-Level LoRA Adapters with Zero Inference Overhead

> Paper: *Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution* > Authors: Liliana Hotsko, Yinxi Li, Yuntian Deng, Pengyu Nie (University of Waterloo) > Link: https://arxiv.org/abs/2606.06492 > Headline results: +9.9pp accuracy in static settings, +5.2pp in evolution settings, 604-repository benchmark, zero inference-time token overhead

Key points

• Core idea: Instead of feeding repository context to a code LLM at inference time, train a hypernetwork that learns to generate a dedicated LoRA adapter for any repository.
• Architecture: Frozen repository encoder → trainable hypernetwork (~720M params) → LoRA weights (rank 16) injected into Q, K, V, O, gate, up, and down projections of all layers of a frozen Qwen2.5-Coder-1.5B backbone.
• Repository encoding: Two-stage — files are chunked (512 tokens), encoded with Qwen3-Embedding-0.6B, pooled; file vectors are aggregated via weighted average + max-pooling concatenation, weighted by content distinctiveness, file size, and path priors (e.g., src/, core/).
• Two variants:
• *Code2LoRA-Static*: one-shot adapter generation from a repository snapshot.
• *Code2LoRA-Evo*: a GRU incrementally updates the adapter from commit diffs, retaining evolution history at low cost (only diffs are processed).
• Benchmark: RepoPeftBench — 604 repositories, assertion-completion tasks (40K static / 215K evolution training tasks), with Cross-Repo, In-Repo, and temporal OOD (92 repos) splits.
• Training: Two stages (LoRA generation, then GRU update for Evo); AdamW, batch size 32, single H100 80GB, ~1–2 days.

Results

Static setting (Exact Match):

| Method | Cross-Repo EM | In-Repo EM | |:---|:---|:---| | Pretrained | 45.7% | 46.8% | | RAG (k=3) | 39.7% | 42.1% | | Dep.-Resolved Context | 48.2% | 49.5% | | FFT + RAG | 53.9% | 56.8% | | Per-repo LoRA (upper bound) | — | 64.0% | | Text2LoRA (strengthened) | 45.8% | 46.7% | | Code2LoRA-Static | 63.8% | 66.2% |

• Beats the strongest baseline (FFT+RAG) by 9.9pp and even exceeds the per-repo LoRA upper bound without per-repo training.
• Notably, RAG underperforms the un-adapted baseline on Cross-Repo (39.7% vs 45.7%), showing unstable retrieval quality across repositories.
• A strengthened Text2LoRA with the same encoder and target modules only reaches 45.8%, indicating the hypernetwork architecture itself is key.

Evolution setting:

| Method | Cross-Repo EM | In-Repo EM | |:---|:---|:---| | Pretrained | 31.5% | 29.3% | | RAG (k=3) | 23.6% | 23.0% | | Code2LoRA-Static | 55.7% | 60.6% | | Code2LoRA-Evo | 60.3% | 64.5% |

• All methods degrade under software evolution; RAG collapses to 23.6%. Static adapters also go stale (63.8% → 55.7%). Evo leads with +5.2pp over Single LoRA and surpasses the per-repo LoRA upper bound (64.5% vs 64.2%).

Temporal OOD: Code2LoRA-Evo reaches 74.1% EM (vs 72.2% Static, 72.3% Single LoRA), though OOD target assertions are systematically shorter, inflating absolute numbers while preserving relative rankings.

Why it works

• Parameterized knowledge injection > context injection: repository knowledge lives in model weights — zero extra tokens, stable knowledge, deeper integration, and natural cross-repo generalization via the hypernetwork.
• The hypernetwork effectively learns "how to write an adapter for any repo" — capturing API patterns, naming conventions, and architectural style as low-rank perturbations.
• The GRU keeps adapters current: it processes only diffs, preserves historical context, and supports online updates without retraining from scratch.

Limitations

• Evaluated only on Python; other languages untested.
• Backbone is 1.5B; scaling to 7B/14B models may require much larger hypernetworks and is unverified.
• Only assertion completion is evaluated; code completion, bug fixing, and review tasks remain open.
• OOD length bias may overstate absolute numbers.
• Generated assertions may raise safety/licensing concerns requiring standard mitigations.

Conclusion

Code2LoRA's core contribution is parameterizing repository knowledge via a hypernetwork, achieving repository-level code adaptation with zero inference overhead. It addresses three long-standing pain points: RAG instability, the cost of per-repo fine-tuning, and adapter staleness under software evolution. For AI coding assistants, this suggests a future where a plugin silently generates a LoRA for the open repository in the background — completions that know your APIs, match your style, and track your changes, with no added latency or tokens.

References

• Hotsko L, Li Y, Deng Y, Nie P. Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution. arXiv:2606.06492, 2026.
• Zong Y, et al. Per-Repository LoRA for Code Adaptation. 2025.
• Charakorn R, et al. Text2LoRA: Hypernetwork for Task-Specific LoRA Generation. 2025.
• Jain N, et al. RepoBench: Repository-Level Code Evaluation. 2025.
• Ren S, et al. CodeBLEU: A Method for Automatic Evaluation of Code Synthesis. 2020.