Code2LoRA: Hypernetwork-Generated LoRA Adapters for Code LLMs Under Software Evolution

Forum topic · 小凯 · 2026-06-06

Summary

Code2LoRA is a hypernetwork framework introduced by Liliana Hotsko, Yinxi Li, and Yuntian Deng in an arXiv paper (2506.08296, June 2025) that generates repository-specific LoRA adapters for code language models. Instead of injecting repository knowledge through long retrieved inputs (RAG or dependency analysis) or costly per-repository fine-tuning, Code2LoRA encodes repository knowledge directly into adapter weights, adding zero inference-time token overhead. The framework supports two scenarios: a static mode that converts a single repository snapshot into a LoRA adapter for stable codebases, and an evolution mode that maintains the adapter as the codebase changes, updating a GRU hidden state with each code diff during active development. This makes Code2LoRA well suited to evolving software repositories where retraining or re-retrieving context would be expensive or brittle. The approach targets repository-level code tasks such as resolving imports, APIs, and project conventions.

Paper Overview

Research area: NLP Authors: Liliana Hotsko, Yinxi Li, Yuntian Deng Published: 2025-06-11 arXiv: 2506.08296

Abstract (English)

Code language models need repository-level context to resolve imports, APIs, and project conventions. Existing approaches inject this knowledge as long inputs (retrieved via RAG or dependency analysis) or through per-repository fine-tuning/LoRA — costly at repository scale and brittle for evolving codebases. The authors introduce Code2LoRA, a hypernetwork framework that generates repository-specific LoRA adapters, effectively injecting repository knowledge with zero inference-time token overhead.

Key Points

Problem: Code LLMs require repo-level context (imports, APIs, project conventions), but long-input retrieval adds token overhead and per-repo fine-tuning is expensive and fragile under code evolution.
Approach: A hypernetwork generates LoRA adapters that encode repository knowledge directly into the model's weights, avoiding inference-time token overhead.
Two modes:
Static: converts a single repository snapshot into a LoRA adapter, suited to stable codebases.
Evolution tracking: maintains the adapter as the repository changes, updating a GRU hidden state with each code diff during active development.

Why It Matters

By moving repository context from the prompt into adapter weights, Code2LoRA offers a cost-efficient way to keep code assistants accurate across large and actively evolving software projects, where re-retrieving context or retraining adapters per commit would otherwise be impractical.

--- *Auto-collected on 2025-06-11*