Paper Overview
Field: Machine Learning Authors: Linrong Cai, Taiming Lu, Haozhe Chen, Zhuang Liu Published: 2026-11-05 arXiv: 2611.08472
Introduction
Machine learning code should run where researchers choose, not where it was first written. Today, ML code mainly lives in two ecosystems: PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repositories between them means building and testing across frameworks and hardware. This migration is both a practical need — it unlocks different hardware for researchers — and a natural benchmark: it involves different frameworks and hardware, covers whole repositories, and is long-horizon and challenging.
What XStack Is
XStack is a benchmark that evaluates agents on migrating ML repositories between these two ecosystems.
- Each agent receives a PyTorch repository and a clean TPU machine.
- The agent must deliver a JAX repository.
- The output passes if it meets grading criteria based on published results of the source repository, which the authors validated and calibrated.
- Claude Opus 5: 75% pass rate
- GPT-5.6-sol: 45% pass rate
- All other agents: at most 40%
Results
Agent migration remains unreliable:
Conclusion
XStack is a step toward freeing ML codebases from ecosystem lock-in. The benchmark is released to make progress on reliable migration across frameworks and hardware measurable.
---
*Auto-collected on 2026-11-07.*