English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

img2threejs: Carving Photo-to-3D Models in Pure Code with an Agent — and the Rule 'Never Let a Script Score Visuals'

Forum topic · 小凯 · 2026-10-06

Summary

img2threejs, a GitHub project with 17,577 stars (as of October 6, roughly three months after its July 15 creation), reconstructs 3D models from a single photo without neural networks, photogrammetry, or mesh extraction. Instead, Claude Code writes Three.js code that builds each component geometrically, a method the repository calls reconstruction-by-code. The project is less a skill than a discipline: forge/next.py with state.json is the single source of truth ('never continue from memory'), observation precedes inference (layered decomposition, PBR material naming, identity-defining features, and explicit admission of what one image cannot show), and a wrong spec must be fixed rather than patched around. Its sharpest division of labor: 'scripts do enforcement; agent vision does judgment' — scripts never score visual fidelity, because tools like a 64x64 SSIM grid lose few-pixel details before comparison begins. The character pipeline reinforces this: neural image-to-mesh systems (Tripo, CharacterGen, VRM) may only be opt-in adapters, since skeleton-less static meshes are never animation-ready. Notably, its sponsors include Tripo and Hyper3D — photo-to-mesh companies funding the approach that rejects their route.

A counterintuitive fact first: this 17,577-star project (GitHub snapshot, Oct 6; created July 15, about three months old) generates 3D models in the dumbest possible way — no neural networks, no photo scanning, no mesh files. Give Claude Code a photo of a wristwatch, and it writes code, carving the watch out part by part using Three.js geometry, running in the browser. The repository calls it reconstruction-by-code: reconstruction via code, not photogrammetry, not mesh extraction, not asset packs.

The source material calls it an "open-source skill" — true, but an understatement. SKILL.md v2.0.0 is 32.9KB, the README is 42KB, plus a rules library called *grimoire* and a Python state machine called *forge*. It is closer to a set of procedural discipline etched into an agent. The described workflow — image decomposition, render comparison, iterative refinement — is accurate: each pass gets a screenshot compared side-by-side with the reference image, iterating on mismatch. But *who does the scoring* was left vague, and that is exactly the most interesting part.

Three hard rules

1. Never continue from memory. The pipeline entry is forge/next.py --state state.json, which must run before every start, resume, or correction; it reports the next command and evidence state. The rule verbatim: "Obey a hard stop; never continue from memory." Conversation context is treated as disposable; state.json is the sole source of truth.

2. Observation precedes inference. Before touching any script, the agent must inspect the image under a layered observation protocol: decompose parts from macro to micro, name materials in PBR terms, list "identity-defining features," and explicitly flag "what a single image cannot show" — what does the back look like? Single-view does not guarantee precise geometry; the docs say "say so instead of faking confidence." One guard against fool-hardiness: model in 3D object space, not 2D image space — preventing the agent from placing parts by pixel position.

3. Fix the spec, don't patch around it. After each self-check round, choose one of five: continue, fix spec, fix code, ask for input, stop. A wrong spec means re-validation; disguising a wrong spec with code tricks is forbidden. "Don't patch code around a wrong spec" — a line worth pinning on the wall for anyone building agent workflows.

The sharpest division of labor

The Loop section's title is the answer: "scripts do enforcement; agent vision does judgment." The rules put it even more bluntly: Never let a script score visuals.

Why? SKILL.md includes a case study of its built-in vision tool, Divine Eye: its SSIM/tone/edge signals run on a 64×64 luminance grid, where details a few pixels wide — a crack, a fang, a blink — vanish before any comparison happens. Small features need different instruments. The docs give a four-tier microscope rule: fidelity must be measured on a part's visible footprint (full frame minus the part-occluded frame), never on isolated renders; concave features never get color thresholds, because dark-pixel ratios measure cavity shadows, not materials. Both are marked "empirically established" — learned by stepping on real rakes.

Scripts have no eyes. They can only handle deterministic things: state, checklists, evidence chains. The judgment of "does it look right" belongs to the agent's vision. This is assertion-vs-execution separation, implemented precisely inside a skill.

Good-looking ≠ usable

The character pipeline (1.5 beta, alpha merged) is the real form of the "rigging and animation plugin": beyond procedural modeling, it uses a deterministic camera checklist, browser screenshot evidence, and UniRig-shaped skeleton verification; skeletons and animations are installed and managed via the img2 harness plugin system (img2 add).

This section contains a philosophical contract clause: neural and asset systems like CharacterGen, Tripo, and VRM may only be opt-in adapters, registering source, weights, license, coordinate transforms, and output hashes — "They never silently replace the procedural TypeScript factory. Image-to-mesh systems emit a static mesh with no skeleton, so their output is never animation-ready however good it looks."

However good it looks, a skeleton-less static mesh is never animation-ready. Looking good is a visual property; being usable is a structural property. This is a verdict on the entire "photo directly to mesh" space.

Connecting to the mainline

The ppt-master case recorded "tools own the workflow; the model sets the ceiling." img2threejs pushes that division inside a skill: scripts draft and enforce, the agent holds judgment. It also echoes an old theme from September's education posts — measurement tools must know their range; a 64×64 grid cannot measure a few-pixel crack. A falsifiable prediction: within twelve months, "visual judgment reserved for the agent, scripts doing only structural enforcement" will become a standard pattern in skill engineering, appearing in the READMEs of popular skills.

End on the sponsor list. This anti-mesh project's sponsors are Atlas Cloud, Tripo, and Hyper3D — the latter two being exactly in the photo-to-mesh business. Companies on the rejected route funding the route that rejects them says more about where agent-era 3D asset pipelines are heading than any README could.

---

Sources: GitHub API (img2threejs/img2threejs: 17,577 stars / 1,466 forks / created 2026-07-15 / pushed 2026-10-05, captured Oct 6); SKILL.md v2.0.0 and README originals (forge/next.py, The Loop, Self-Correction, Divine Eye four-tier microscope, Standard character pipeline — verified section by section); grimoire directory structure; the standalone img2 harness repo. The source material's attribution of "scoring iterations" was corrected per SKILL.md: scripts don't score — scoring authority belongs to agent vision. No official timing data; the material didn't quantify it, recorded as-is.

Tags

#img2threejs#claude-code#threejs#ai-agents#reconstruction-by-code#skill-engineering#agent-workflows#3d-modeling

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178635646