On a late night in March 2025, Chen Yuan was woken by a cascade of PagerDuty alerts.
His recommendation system — an online inference service that had run for six months on an AWS c5.2xlarge — suddenly maxed out its CPU, then hit OOM, restarted, maxed out again, OOM'd again. Three machines fell like dominoes.
The root cause? Not the model. The model's Python environment broke. A certifi package update pushed overnight by the security team triggered an ABI incompatibility with numpy's dynamically linked library; libopenblas.so couldn't find its symbols, and torch.load began segfaulting. The model was fine — but the service would never start again.
The team spent four hours restoring service. Not by tuning model parameters, not by changing inference logic, but by adding one line to the Dockerfile: RUN pip install numpy==1.26.4 --force-reinstall, rebuilding a 2.7GB image, pushing, pulling, restarting.
Four hours. Because of a package dependency.
The Five Production Pains of Python ML
Pain 1: Dependency hell. A typical Python ML inference service has a dependency chain like torch → numpy → openblas → libgfortran → ... Every pip install is a bet on system reproducibility.
Pain 2: Bloated images. Model weights are only 200MB, but the Docker image is 3GB, and every docker build in CI/CD takes 12 minutes. Compare: a Go binary after go build -ldflags="-s -w" is roughly 15MB; the entire production image is <20MB.
Pain 3: Brutally slow cold starts. Python ML services take 7–15 seconds to go from zero to ready. For an elastically scaled API, a 15-second cold start makes HPA effectively useless. A Go binary cold-starts in <100ms — the startup speed the serverless era demands.
Pain 4: FFI wear and tear. Every Python call into C/C++/CUDA code involves indirection, GIL acquire/release, and reference-count maintenance. In Go, every function call is a native call — no FFI boundary, no interpreter overhead.
Pain 5: The concurrency dilemma. Python's GIL means that no matter how many cores you have, only one thread executes Python bytecode at a time. Go's goroutines have no such problem — 100 goroutines can share one copy of model weights for inference.
Why Existing Solutions Fall Short
- Go + Python sidecar: operational complexity explodes — two runtimes, two monitoring stacks, two rounds of network serialization.
- TorchScript / ONNX Runtime: still requires a C++ environment, libtorch, and CGO call overhead.
- Gorgonia / GoMLX: near-blank ecosystem compatibility — no GGUF loader, no Transformer components.
- Go version: Go 1.26+
- External dependencies: only 4 Go modules
- Binary size: ≈15MB
- Cold start: <100ms
- LLM support: GGUF loading, LLaMA/Mistral/DeepSeek, KV-Cache, Flash Attention 2
- ONNX operator coverage: 49 ops (OpSet 14–18)
- Not a PyTorch replacement — Born's battlefield isn't the lab; it's production.
- Not a "PyTorch-API-compatible" framework — the API is idiomatic Go, leveraging generics and interfaces.
- Not designed for quick experiments — it picks proven best practices and goes deep, not broad.
- Go developers: encountering ML inference requirements at work, but unwilling to dive into the Python pit.
- ML engineers: burned by Python production deployments, looking for a way out of dependency hell.
- System architects: choosing technology for a serious ML inference system.
What we actually need:
1. Pure Go implementation — no CGO; one go build produces one binary.
2. Type safety — use Go generics to catch tensor type errors at compile time.
3. GPU acceleration — without requiring a CUDA SDK installation.
4. Model interoperability — load model formats produced by PyTorch/HuggingFace.
5. Production-first — small images, fast startup, goroutine-safe.
6. Hard-core enough — run Transformers, load open-source LLMs.
This list is the reason Born exists.
What Born Is
Born is a deep learning framework written in pure Go, built from scratch — no hard dependencies on any C/C++/Python libraries.
It can run full deep learning training and inference without installing the CUDA Toolkit, Python, or any shared libraries — supporting CPU and GPU (via WebGPU), from MNIST classification to production-grade inference with LLaMA-scale LLMs.
Key metrics:
What Born Is Not
Born is a production-first framework. Its core promise: models you train in PyTorch in the lab, Born helps you deploy cleanly to production.
A Concrete Comparison
| Dimension | Python (PyTorch) | Go (Born) | |------|-----------------|-----------| | Docker image size | 2.7GB | 22MB | | Cold start time | 11s | 78ms | | Memory footprint | 1.8GB | 420MB | | Concurrent inference | 8 req/s | 120 req/s | | Cost per 1,000 inferences | $0.12 | $0.008 | | Operational complexity | Docker + pip + conda + CUDA | Single binary via go build |
How to Read This Book
Born is not "a PyTorch written in Go." Born is the Go ecosystem's own answer to ML.
---
📘 This is a chapter from the serialized technical book *Born: Mastering the Modern Go Machine Learning Framework* — 22 chapters + 4 appendices, with subsequent chapters to be published one by one.