English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

XStack: A Benchmark for ML Repo Migration Between PyTorch and JAX

Forum topic · 小凯 · 2026-11-06

Summary

XStack (arXiv 2611.08472) is a benchmark introduced by Linrong Cai, Taiming Lu, Haozhe Chen, and Zhuang Liu that evaluates AI agents on migrating machine learning repositories between the two dominant ML ecosystems: PyTorch on NVIDIA GPUs and JAX on Google TPUs. Each agent receives a PyTorch repository and a clean TPU machine and must produce a working JAX repository. The output passes if it meets grading criteria calibrated and validated against published results of the source repository. Repo migration serves both as a practical need, unlocking different hardware for researchers, and as a natural long-horizon benchmark spanning frameworks, hardware, and entire codebases. Results show migration remains unreliable: Claude Opus 5 passes 75% of tasks, GPT-5.6-sol passes 45%, and all other agents achieve at most 40%. XStack aims to make progress toward reliable cross-framework, cross-hardware migration measurable and to free ML codebases from ecosystem lock-in.

Paper Overview

Field: Machine Learning Authors: Linrong Cai, Taiming Lu, Haozhe Chen, Zhuang Liu Published: 2026-11-05 arXiv: 2611.08472

Introduction

Machine learning code should run where researchers choose, not where it was first written. Today, ML code mainly lives in two ecosystems: PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repositories between them means building and testing across frameworks and hardware. This migration is both a practical need — it unlocks different hardware for researchers — and a natural benchmark: it involves different frameworks and hardware, covers whole repositories, and is long-horizon and challenging.

What XStack Is

XStack is a benchmark that evaluates agents on migrating ML repositories between these two ecosystems.

  • Each agent receives a PyTorch repository and a clean TPU machine.
  • The agent must deliver a JAX repository.
  • The output passes if it meets grading criteria based on published results of the source repository, which the authors validated and calibrated.
  • Results

    Agent migration remains unreliable:

  • Claude Opus 5: 75% pass rate
  • GPT-5.6-sol: 45% pass rate
  • All other agents: at most 40%

Conclusion

XStack is a step toward freeing ML codebases from ecosystem lock-in. The benchmark is released to make progress on reliable migration across frameworks and hardware measurable.

---

*Auto-collected on 2026-11-07.*

Tags

#machine-learning#benchmark#pytorch#jax#ai-agents#code-migration#tpu#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636824