img2threejs — Image to procedural Three.js

Rebuild the object visible in a reference image as a code-only procedural Three.js model, gated by a staged sculpting pipeline and an AI-vision self-correction loop. This is reconstruction-by-code, not photogrammetry, mesh extraction, or downloaded art packs. That promise governs how the model is built — it says nothing about which file formats it can subsequently be exported to; an explicitly-selected emission target (--target <kind>) is a terminal, whole-artifact transform of the already-built model, verified to its own stated limit, never a second way to build one.

Agent-agnostic: works under Claude Code, Codex, or OpenCode. Wherever this doc says "agent vision" or "agent browser tool", use whatever the host provides — native image reading, a browser MCP (playwright/chrome-devtools), the project preview, or a user-supplied screenshot.

This file is the always-loaded router: it holds the order of operations and every hard rule as one line. The full contract behind each rule lives in the grimoire/ or docs/ file that rule names — read the named file at the moment you reach that stage, not before.

Canonical shared checkout

Keep one checkout of this repository and let every host enter it through a symlink, so Claude and Codex execute the same code instead of drifting apart:

~/.claude/skills/img2threejs -> <your checkout>
~/.codex/skills/img2threejs  -> <your checkout>

When To Use

The user attaches/points to an object image and wants a procedural Three.js model, a reconstruction/animation/destruction plan, a sculpt spec, or code. Also for material studies, action-ready props, game objects, botanical/mechanical parts, and stylized reconstructions.

Core Promise

Sculpt from a photo, in order — never one-shot a mesh:

1. Run python3 forge/next.py --state .img2threejs/state.json [<spec>] first, at every start, resume, and before every correction iteration. It reports the ordered checklist, exact next command, evidence status, and bounded correction-loop status; it never replaces the spec/pass gates. Obey a hard stop; never continue from memory.
2. Validate the image is a suitable 3D target (grimoire/intake/validation_rubric.md).
3. Assess object class + complexity, then write a qualityContract before any code.
4. Spec it: component hierarchy, materials, lighting, pivots, sockets, action anchors.
5. Build pass-by-pass from blockout → structure → form → material → lighting → interaction → optimization.
6. Verify each pass with a screenshot compared against the reference; fail a pass if an identity-defining feature is wrong even when the global score looks fine.

State explicitly when output is approximate/stylized/low-poly. A single image cannot reveal hidden sides or guarantee exact geometry — say so instead of faking confidence.

Mandatory Local State Gate

Conversation context is disposable; .img2threejs/state.json is the local checklist authority. Initialize once per reconstruction, then gate every step through it:

python3 forge/state.py init --state .img2threejs/state.json --reference <img> --profile <generic|character|installed-domain> --spec object-sculpt-spec.json
python3 forge/next.py --state .img2threejs/state.json [object-sculpt-spec.json]
python3 forge/state.py mark <step-id> --state .img2threejs/state.json --evidence <path>

• next.py prints the current step, pass, incomplete mandatory steps, exact next command, and loop/max. Exit code 3 or status=stopped is a hard stop: report the reason and request input. Never bypass it by reconstructing progress from chat history.
• Every completed step needs evidence; mark a non-applicable step skipped only with --reason — silent omission is forbidden. Loop counts derive from reviewHistory actions (refine-spec/refine-code), not agent memory. Defaults: 3 corrections per pass, 6 total.
• A domain profile's steps, gates and reference material come from the registry: in-repo modules (character) and installed plugins (cs2, animated-character from plugin-character) register identically, and forge/state.py init names what is available. A profile adds mandatory gates without changing the core order -- a domain plugin typically requires an authoritative classification, an intake manifest, and a machine-readable domain review before AI review; character requires the character contracts and landmark evidence; animated-character (requires the installed plugin-character) adds all of character plus the nine Stage R steps (grimoire/readiness/animation_contract.md). Pick it whenever the rig must MOVE — on character the Stage R gates are absent and the build completes without ever running them, which is how animation used to ship broken. Its order is load-bearing: repair the mesh, freeze it, bind additively, then verify parity. Every profile records suitability, projection applicability, and material-evidence applicability. The state file is a resumability index, not visual evidence: renders, specs, review history, and deterministic gates remain the authoritative artifacts.

Required Inputs

• one image path / screenshot / URL / attached image (if missing or unreadable, ask)
• intended use: prop, game object, hero render, playable/destructible object, animation rig (default: real-time browser prop with interactive performance)
• when a domain plugin serves the item, whatever authoritative record its intake step requires, or an explicit request for the user/vision provider to supply one; heuristic detection alone is not enough to select a geometry adapter

The Loop (scripts do enforcement; agent vision does judgment)

Run scripts from the skill root (forge/...). Pure Python 3.10+ stdlib, no pip installs. Full flags: grimoire/scripts.md. Never let a script score visuals — that is the agent's job.

1. Analyze the image first (agent vision, before any script): work the layered observation protocol in grimoire/intake/image_analysis.md — identify/classify, decompose macro→meso→micro, map part relationships, name materials in PBR terms, list identity-defining features, and flag what the single view hides. Observation before inference; controlled 3D vocabulary; 3D object-space not 2D image-space. Then probe local images: forge/stage1_intake/probe_image.py <image> (metadata only, not a visual check). 1a. Local Spec Search — after image analysis, before writing or refining a spec, pull local domain evidence (anatomy/PBR/wear/geometry/runtime/physics) rather than inventing it: python3 forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --out assessment.json (auto-runs BM25 over the core_3d collection, or the collection a declared domain contributes -- the collection is NEVER guessed from the target name; writes a localSpecSearch bundle that new_sculpt_spec.py --assessment carries into the spec). Full query-expansion recipe (bilingual terms, focused search_specs.py retrieval, cache rules): grimoire/intake/local_spec_search.md. MUST read it before retrying an incomplete or domain-specific query. 1b. Domain intake — when a domain plugin serves the item, complete its intake steps before pre-spec authoring (admission, heuristic signal, classification, family/route resolution). MUST read the contract its step names, completely, before creating the manifest or running pre-spec assessment. 1c. Optional fidelity evidence adapters — only when they improve an observed weak point; the stdlib core remains authoritative. Thin/complex masks → local SAM2; character face/pose → MediaPipe; weak front/back cues → Depth Anything V2 (forge/stage1_intake/run_vision_adapter.py <segment|landmarks|depth> ...; every adapter emits provenance; monocular depth is relative only). MCP-only scene mutations never count as implementation — write the proven change back to the spec or TypeScript, rebuild, recapture. Full adapter + MCP routing and authority boundaries: docs/integrations/reference_fidelity_tooling.md.
2. Pre-Spec Assessment Gate — classify + score complexity + write the quality contract: forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --complexity <simple|moderate|complex|ultra-complex> --out assessment.json. Rules: grimoire/intake/quality_contract.md. Set objectClass.primaryDomain (object | character | hybrid) and fill the seeded detailInventory (its targetMinDetails scales with complexity). A domain plugin may these floors through its augmentation -- the merge clamps, so a plugin can never lower one (a skin's finish/wear/hardware IS the item, so such a domain is held to the top fidelity bar. Author procedural GEOMETRY but route the FINISH through the projection path in step 2c — a procedural finish for a patterned skin (Doppler/Gamma/Marble/Fade) reads visibly wrong against the reference. A domain plugin ships its own finish rulebook and texture-acquisition guide; read what its checklist steps name. 2b. (do not skip for detailed subjects) — scan zones and enumerate every identity-defining small detail (gloss, bevel, fasteners, linework, contours, stains): . Each detail MUST map to a or entry — never prose only. Taxonomy + 3D-term recipes: . 2c. (characters AND reference-matched surfaces — painted skins, decals, painted patterns) — when the goal is matching a specific reference's surface, put the photo's own pixels on the mesh instead of approximating them procedurally. This is the single biggest fidelity lever; a procedural material for a patterned surface is the #1 reconstruction failure. Recipe ( — its two levers generalize past characters): solve the camera ( → ), the reference (, hard requirement — de-lighting is what makes projection safe), then project the de-lit crop and bake it into UVs (). For a painted skin the projected de-lit crop IS the finish — no procedural Doppler material. For characters, first capture landmarks (), fill , route . A single view cannot show hidden sides — report per-region confidence and request more views when it matters. Character sub-routes, in order — decide what parts exist before shaping any, and shape the head before the hair that sits on it:

GLB-mediated v2 render-fidelity track (1.5 alpha)

When the user supplies a GLB as an intermediate reference, the browser-rendered GLB is the structural and visual baseline for an independently authored procedural factory. The raw GLB is never pixel evidence and its topology/materials are never copied into the factory. Before any factory edit — full contract in grimoire/build/python_threejs_render_bridge.md, machine-readable schema in docs/specs/render-profile.v2.schema.json (+ example; fail-closed validation):

1. forge/stage1_intake/probe_glb.py first. A merged one-node/one-mesh asset is insufficient for semantic labels; request a multipart GLB or a browser semantic-ID pass before claiming exact regions.
2. Author ONE shared render-profile.v2 (forge/stage4_review/validate_render_profile.py) used by both the GLB and procedural routes. Region IDs are subject-specific, never inherited from the example profile; declare the required set in extensions.requiredSemanticRegions so omission is a hard validation error.
3. Capture six passes per admitted view (beauty, alpha-silhouette, semantic-id, depth, normal, roughness-material-id); score with forge/stage4_review/compare_region_passes.py. Missing semantic-ID data blocks per-region confidence rather than falling back to whole-image scores.
4. Use region-specific continuous geometry — never replace a face/head volume, cloth shell, or tail with floating primitives when the region's silhouette requires a continuous surface.
5. Run ONE correction group per loop, in order: camera → silhouette → face → clothing → accessory → materials → lighting; recapture the full pass set after each group and record the changed group, hashes and score. Never combine groups when diagnosing improvement.

Gates (do not skip)

Before any visual review or continue decision, MUST read the full gate-by-gate contract in grimoire/review/gates_reference.md (Divine Eye, VLM rescue, multi-angle, interior difference, chirality, hair, domain review, bounded correction, Divine Eye fitting, screenshot feedback, assembly, attachment, material, detail inventory, rig payload, character track). In short:

• Validate references first (grimoire/intake/validation_rubric.md, check_reference_admission.py).
• divine_eye.py is deterministic-first; the VLM (vlm_gate.py) is a gated last layer, never consulted on a hard-gate failure.
• A non-planar form must hold from ≥2 angles (diagnose_render_multi_angle.py).
• Measure INSIDE the silhouette every visual pass (interior_difference.py). Silhouette IoU reads ~11% of figure cells: a model with its face deleted scored the same 0.8803 as the finished face.
• Every -l/-r pair is a MIRROR, not a rotation — hard at spec time (validate_chirality). A pair wrong the same way on both sides still passes, and needs medial_lateral_bias vs a reference.
• Hair subjects: scalp_exposure.py is HARD and runs on geometry before any render; hair_gate.py is soft and subordinate to it. A coverage shortfall never authorises widening the masses.
• Flat colour regions with hard boundaries (blaze/bib/socks, livery stripe, painted marking) are an identity feature, so their boundaries are gated on geometry: vertex_region_gate.py. Never a texture — this pipeline emits code; the shape predicates live in _shared/vertex_paint.py.
• A curve claim ("curled into a hook, not a straight cone") needs swept_arc_gate.py: silhouette IoU passes a straight cone occupying roughly the right cells.
• Character builds validate the rig payload (stage5_rig/validate_rig_payload.py) before binding a THREE.Skeleton; it proves payload integrity only, never pose stress or likeness.
• A rig that must MOVE runs the animation gates too (grimoire/readiness/animation_contract.md; the checklist steps and gate come from the installed plugin-character -- stage5_rig/ remains in this repo as the emitter's library, not the checklist authority). A clip that exists is not a clip that plays: only G1 (maxSampledBindingDelta <= 2^-23) separates the two, and a gate whose input is missing reports unevaluated, never a pass. Bind at IDENTITY in attached mode and take the display offset from the mesh bounds alone; loop is decided by poseReturn, never by travel.
• A domain plugin's review gate also runs against its versioned scene fixture.
• Local state enforces 3 corrections per pass and 6 total by default; reaching either limit is a hard stop. correction_loop.py may stop earlier on repeated defects, oscillation, or plateau.
• continue requires a render + comparison sheet + AI-vision score ≥ threshold, every critical feature ≥ its own threshold ().

Self-Correction

After every pass, decide exactly one: continue | refine-spec | refine-code | request-input | stop. refine-spec fixes a wrong/missing/shallow spec (re-validate, don't patch code around it); refine-code fixes geometry/material/lighting that doesn't match a sound spec. Before making the decision, MUST read the root-cause guide + fidelity scale in grimoire/review/self_correction.md, record the decision, and re-run the local state gate.

Small features need a different instrument. Divine Eye's SSIM/tonal/edge signals run on a 64×64 luma grid, so a detail a few pixels wide is absent before any comparison happens. When fidelity depends on individual tears, spars, fangs or eyes, use the four-tier microscope: grimoire/review/divine_eye_microscope.md. Two empirically established rules from it: measure fidelity on a component's visible footprint (full frame minus a component-hidden frame), never on an isolation render; and never colour-gate a concave feature, where a dark ratio captures cavity shading rather than material.

Transparency and Process Debugging

Report what changed each pass with evidence (exact values/coordinates), name what still doesn't match, and never claim "done" when only "improved". A passing gate is not proof of 3D realism. Full rule + examples: grimoire/review/self_correction.md.

Left and right

A left/right pair is a reflection, never a rotation: negate the lateral axis and nothing else, (x, y, z) → (-x, y, z). With forward: +Z, Y up and a right-handed frame, the character's own left is +X. The convention lives as code in forge/_shared/chirality.py (CHARACTER_LEFT_SIGN), with two different gates for the two defects that shipped from getting it wrong: validate_chirality catches a rotation-mistaken-for-reflection at spec time, and medial_lateral_bias vs a reference catches a pair that is wrong the same way on both sides. Reflecting also inverts triangle winding — flip it back on the mirrored side or flatShading lights the limb as though lit from behind. Full write-up with the measured defects: grimoire/scripts.md ("Left and right").

Hair

Hair has its own subsystem because it has failure modes no other gate can see. Full contract, measurements and non-goals: docs/HAIR_PIPELINE.md. The hard rules:

• Roots bind to the scalp as (u, v), never absolute positions (hard validation error).
• standProud is enforced by the generator, not advisory.
• scalp_exposure.py is a HARD gate on geometry before any render; a coverage shortfall never on its own authorises widening the masses.
• Default representation tier is shell, not locks; strand impression comes from faceting and material (hair.human.code-only), since this skill emits no textures.
• plane-card is rejected for hair (needs an alpha texture this skill cannot emit).
• Hair is rigidly parented, never smooth-skinned (the geodesic field runs through the skull).

Domain plugins

A domain plugin makes a run exact where this pipeline would otherwise infer. It contributes its own checklist steps and gates, and publishes a spec-augmentation.json that this pipeline pulls at spec-authoring. With no plugin serving the item, nothing here changes: author the skeleton and infer the shape from the reference, as for any other object.

• The steps a plugin contributes appear in the checklist with their own ids and resolved paths. Read what each step names -- a plugin ships its own contract, and it governs its own domain.
• A plugin may raise this pipeline's quality floors and can never lower one; the merge clamps.
• A plugin that does not serve an item publishes no augmentation. That is not an error and not a blocked run: it is the generic path, and the reconstruction proceeds by inference.
• This pipeline names no domain. If a rule is domain-specific, it lives in that domain's plugin.

The img2 harness

Plugins are installed and managed by the img2 harness (img2threejs/img2), a separate dependency-free CLI (Node launcher, Python core). This skill never installs anything itself: when state.py init names a profile as unavailable, name the img2 add command that installs it and stop — never vendor domain logic instead.

img2 add img2threejs/plugin-<id> --ref <tag>      # install a plugin at a tag (e.g. plugin-cs2)
img2 doctor                                       # what each host resolves: base-skill path + version, plugin gates
img2 capabilities --from-kind image --to-kind glb # which installed plugin serves an edge

Setup and the full CLI reference live in the README quick-start and the harness repo's docs.

Forge Runtime Contracts

Subdivision runtime tests compile generated TypeScript against the showcase checkout. Set IMG2THREEJS_SHOWCASE_ROOT to that checkout; without it, local runtime-only tests skip with an actionable message while static contracts still run. CI should set IMG2THREEJS_REQUIRE_SHOWCASE=1 to turn a missing showcase checkout into a test failure.

IMG2THREEJS_SHOWCASE_ROOT=/path/to/img2threejs-showcase python3 forge/tests/test_subdivision.py
IMG2THREEJS_SHOWCASE_ROOT=/path/to/img2threejs-showcase python3 -m unittest discover -s forge/tests
IMG2THREEJS_SHOWCASE_ROOT=/path/to/img2threejs-showcase python3 forge/tests/test_showcase_tsc_smoke.py

Implementation Rules (brief)

TypeScript + plain Three.js unless the project uses a wrapper. Group factory createObjectNameModel(spec, options), reconstruction data kept separate from renderer objects, deterministic seeds for all procedural noise. Prefer primitives / Shape extrude / curve+tube / instancing / displacement / generated canvas textures before any external art. Full geometry & material recipes + hard-won failure patterns: grimoire/build/geometry_patterns.md.

Optional Python ↔ Three.js render bridge

When Python is requested for character rendering, use it as a deterministic job/evidence layer around the browser Three.js runtime: camera-batch manifests, source/output hashes, readiness and settle checks, screenshot persistence, masks, diagnostics, and comparison packaging. The target Three.js browser route remains the rendering authority. Do not silently replace the procedural TypeScript factory with Blender/VRM/GLB output. Full routing, manifest fields, and failure rules: grimoire/build/python_threejs_render_bridge.md.

Standard character pipeline (merged 1.5 beta + alpha)

Use grimoire/readiness/standard_character_pipeline.md for character work. Beta owns the strict sculpt/build/review gates; alpha owns deterministic camera manifests, browser screenshot evidence and UniRig-shaped rig validation. CharacterGen, Tripo, VRM and other neural/asset systems are opt-in adapters with source, checkpoint, license, coordinate conversion and output hashes. They never silently replace the procedural TypeScript factory. Image-to-mesh systems emit a static mesh with no skeleton, so their output is never animation-ready however good it looks. Executable entry points: forge/stage4_review/render_bridge.py and scripts/capture_threejs_playwright.py (init → browser capture → validate → diagnose; capture must operate on the real showcase/browser route and leave readable PNGs in the workspace).

Output

• Analysis-only: suitability verdict + scores, object extraction, macro→micro hierarchy, geometry strategy, material/lighting recipe, animation/destruction feasibility, plan + risks.
• Implementation: the above briefly, then edit code; verify with typecheck/build + a screenshot.
• Not feasible: name the blocker, ask for more views / cleaner image / accepted stylization / a narrower target. "This cannot reach the requested fidelity from this image" is a valid result.