Reverse Engineering an ASIC With Just a Laptop and z3: How Jestoph Solved Jane Street's GDS Puzzle

Forum topic · QianXun · 2026-09-05

Summary

In August, quantitative trading firm Jane Street published a challenge titled "Can you reverse engineer an ASIC?", releasing only the unlabeled GDS layout file of a chip they designed, plus sample inputs and outputs. In September, engineer Jestoph published a complete solution that required no lab equipment—just a laptop, open-source EDA tools, and a month of work. He parsed the layout with gdstk, recognized the open-source sky130 standard cell library, recovered the netlist via a polygon-overlap algorithm (~1,000 paths, 17,000 polygons), simulated the design with iverilog, and used the z3 constraint solver over 120 input bits to derive the answer. The circuit turned out to be an 11×11 Star Battle puzzle checker that outputs strings like "(* TWO STARS *)". Jestoph also found an undriven-wire bug confirmed by Jane Street. Notably, earlier solutions existed (KLayout API, Yosys + SAT), and despite an explicit no-AI rule, commenters reported AI tools solving it quickly. The post reads as an elaborate recruitment signal for Jane Street's hardware team, and demonstrates that leaked GDS files fully expose a chip's functionality.

The Challenge

On August 5, quantitative trading firm Jane Street published a blog puzzle: "Can you reverse engineer an ASIC?" Instead of physical chip analysis (decapping, electron microscopes, cleanrooms), they gave only the GDS file—the final mask layout containing all metal, routing, and transistor layers—plus a few sample inputs and outputs.

The organizers put it plainly: *"The GDS is, in a very real sense, the chip. Everything the circuit does is in there. The tricky part: nothing is labeled!"*

The prize was merchandise and an official blog feature—no cash.

One Month Down the Rabbit Hole

On September 4, an engineer going by Jestoph posted a complete writeup (427 points on Hacker News). His opening confession: *"Why do I always do things the hard way?"* He holds an EE degree but has never worked as an engineer.

His pipeline involved no acid or microscopes:

Parsed the GDS with gdstk, starting from a warm-up sub-puzzle with 27 components
Identified the sky130 standard cell library, matching pins by geometric overlap
Built a polygon-overlap graph algorithm (~1k paths, 17k polygons) to extract the netlist and generate Verilog
Simulated with iverilog, viewing waveforms in surfer
Solved with z3 over 120 input bits, yielding the answer string: (* TWO STARS *)

Along the way he built—and discarded—a sqlite3-driven circuit simulator, a custom hardware description language, and a raylib-based GDS viewer. HN commenter swiftcoder quipped about his extreme not-invented-here syndrome. On z3, he wrote: *"It's kind of magical?"*

He also found a real bug: a wire with no driver. Jane Street confirmed it by email the next day.

The Reveal: A Star Battle Checker

Multiple independent solvers reconstructed the circuit's true identity: an 11×11 Star Battle checker. Star Battle is a Sudoku-family logic puzzle—place two stars in each row, column, and region, with no stars adjacent. The chip validates puzzle solutions in hardware: 120 input bits encode the board, and it takes 121 clock cycles to produce a result.

Output strings include "(* TWO STARS *)" on success, "TRY AGAIN", "EMPTY SKY" for an empty board, and "BIG BANG" for a certain overflow—four easter eggs baked into silicon-level logic.

Jestoph wasn't first: sunaabh.com posted a solution on August 18 (KLayout API netlist extraction), and KarelPeeters published a gist on August 22 (Yosys + SAT). His unique contributions: the officially confirmed submission, the bug report, and the best storytelling—worth 427 HN points versus the 119 of the original challenge post.

The AI Ban Lasted Weeks

The rules included a bolded request: don't feed the puzzle files into AI tools or use AI to write solutions. In the solution thread, commenters reported ChatGPT reaching an answer within an hour, and one reproduced it with Codex ("Worked for 12m 36s"), though using the already-public circuit reconstruction. The ban survived from August 5 to September 4—exactly as long as the contest.

The Ladder

Jane Street has a tradition of escalating puzzles: bimonthly paper puzzles, the Electronic Trading Challenge (2017–2022, cash prizes), a neural-network reverse-engineering puzzle on mechanistic interpretability (Feb 2025), this ASIC GDS challenge (Aug 2026 per the source post), and future plans to let winners tape out their own chip designs.

HN user roadbuster saw through it early: *"This is just a job posting in disguise."* The people they're hiring sit in front of Hardcaml and FPGA trading stacks.

Security Lesson

Layout is function. Leak the GDS, and you've leaked everything the chip does—a laptop, a few open-source libraries, and one month of spare time were enough. As for whether the physical chip exists: the given layout is real, the answer strings are real; the rest, the email didn't say.