Steering Code LLMs with Activation Directions for Language and Library Control

Forum topic · 小凯 · 2026-03-27

Summary

Code LLMs often default to particular programming languages and libraries when given neutral prompts. This paper investigates whether these preferences are encoded as approximately linear directions in the models' activation space that can be manipulated at inference time. The authors use a difference-in-means method to estimate layer-wise steering vectors for five language/library pairs, and add these vectors to the model's hidden states during generation. The work is by Md Mahbubur Rahman, Arjun Guha, and Harshitha Menon, and is available as arXiv preprint 2603.23629. This activation-steering approach offers a lightweight, training-free way to control which programming language or library a code model uses, without retraining or fine-tuning the model.

Overview

Field: ML Authors: Md Mahbubur Rahman, Arjun Guha, Harshitha Menon Published: 2026-03-26 arXiv: 2603.23629

Abstract (original)

Code LLMs often default to particular programming languages and libraries under neutral prompts. We investigate whether these preferences are encoded as approximately linear directions in activation space that can be manipulated at inference time. Using a difference-in-means method, we estimate layer-wise steering vectors for five language/library pairs and add them to model hidden states during generation.

Key points

Code LLMs exhibit systematic biases toward specific programming languages and libraries under neutral prompts.
These biases appear to be encoded as approximately linear directions in activation space.
Layer-wise steering vectors are estimated with a difference-in-means method.
Steering vectors are added to hidden states during generation, enabling inference-time control of language/library choice across five language/library pairs — no retraining required.

*Auto-collected on 2026-03-27*