AI-GAME-FACTORY

AI Game Factory

A tool for AI that plans and builds your game — and checks its own work.

You describe an asset as a contract. An AI agent plans it, models it in Blender, and saves it. A script measures the result against the contract. A separate critic that cannot edit anything judges it. An integrator imports it into Unreal and proves it works at runtime. Arithmetic decides whether it passed.

The agents are not trusted. They are measured.

If it fails, a fresh agent fixes exactly what failed — and the failure gets written down so it can never happen twice.

contract.yaml
     │
     ▼
 blender-builder ──► blender-validator ──► fbx-exporter ──► unreal-integrator
   (writes)            (measures)           (exports)         (proves runtime)
                            │                                       │
                            │                                       ▼
                            │                          critic-technical  critic-visual
                            │                                (judge, cannot edit)
                            │                                       │
                            ▼                                       ▼
                      fails fast ◄───────── gate: passes vs required ─────► done
                            │                       (arithmetic)
                            ▼
                      fix-builder (fresh session, fixes only what failed)

---

Why this exists

Ask a model to "make me a door" and you get something door-shaped. Ask it fifty times and you get fifty inconsistent things, none of which fit your game's grid, and no way to tell which ones are actually wrong.

This repo is the opposite bet: the model is allowed to be creative about geometry and nothing else. Everything that can be a number is a number, measured by code that has no opinion — dimensions, pivot, triangle count, UV area, material slots, collision hulls, palette, grid alignment. Everything that cannot be a number goes to a critic that has no write access and therefore nothing to protect.

Three rules carry the whole design:

1. A builder never grades its own work. Critics have no Write, no Edit, no Bash, and no 3D tools. They physically cannot fix what they are judging.
2. Every retry is a fresh session. No agent gets to accumulate a story about why its last attempt was nearly fine.
3. A failure may only happen once. Every real failure becomes a numbered flag and a matching runbook edit. The model does not get smarter; its instructions do.

---

What you get

7 agent definitions	Tool allowlists that make the separation real, not advisory
7 runbooks	The actual prompts. Numbered steps, exact commands, explicit STOP
4 frozen tools	Measure, render evidence, export FBX, bake 2K PBR — the parts that must be identical every run
factoryctl	Preflight, contract validation, artifact hashing, append-only history
A flag registry	11 starter flags: real bugs, why they happened, what fixed them
A contract template	The schema everything else keys off
A control-room web UI	Node graph of the loop, live agent transcript, asset gallery, boundary guard

Runs on Blender 5.x and Unreal Engine 5.x. Agents run on OpenCode against a local model, Claude Code, or anything that reads a markdown prompt and can call a shell.

---

Quick start

git clone <your-fork> gauntlet-factory && cd gauntlet-factory
python -m pip install pyyaml
python factory/factoryctl.py doctor

doctor tells you whether a run can actually happen on this machine — Blender found, PyYAML importable inside Blender (a different interpreter from the one you just installed into), frozen tools present, ports up, contracts parsing.

Every screenshot above is the example contract that ships with this repo. Build it yourself in about two minutes:

blender --background --python assets/source/TASK-0001/run-1/build_asset.py
blender --background assets/blender/TASK-0001/run-1/SM_Crate_Small.blend   --python factory/lib/fx_measure.py -- --obj SM_Crate_Small   --out gauntlet/evidence/TASK-0001/run-1/technical-check.json   --dims 0.6,0.6,0.6 --tris 300 --slots 1 --palette M_CrateWood=#3B2E22

That is 8 measurable checks passing on a real 0.6 m crate — geometry, pivot, UVs, collision, materials, naming, triangle budget and palette.

Then write your first contract:

cp gauntlet/contracts/CONTRACT-TEMPLATE.yaml gauntlet/contracts/TASK-0001-crate.yaml
$EDITOR gauntlet/contracts/TASK-0001-crate.yaml
python factory/factoryctl.py contracts     # catches the mistakes that cost a run

Then bring up the control room:

cd tools/factory-ui && npm install && npm run dev     # http://localhost:5173

Full walkthrough: docs/GETTING-STARTED.md.

---

The contract is the whole interface

task_id: TASK-0001
title: "Storage crate"

technical_contract:
  dimensions_m: { width: 0.6, height: 0.6, depth: 0.6 }
  pivot: "Origin at the minimum corner: X=0, Y=0, Z=0 floor."
  naming_convention: "SM_Crate_Small"
  material_slot_budget: 1
  triangle_budget: 300
  collision_required: true

checks: [dimensions, pivot, collision, uv, materials, naming, import, runtime, visual_quality]

gate:
  required_passes: 9
  gate_end: 0

required_passes must equal the number of checks — factoryctl contracts enforces it, because a gate that cannot be satisfied (or one that can be satisfied with a check outstanding) is worse than no gate.

---

The frozen tools

The single highest-leverage thing in this repo. Before they existed, every run rewrote the same measuring and export code from a prose description — thousands of lines of near-duplicates, each one a fresh chance to reintroduce a bug an earlier flag had already closed.

# measure the saved .blend against the contract → technical-check.json
blender --background asset.blend --python factory/lib/fx_measure.py -- \
  --obj SM_Crate_Small --out evidence/technical-check.json \
  --dims 0.6,0.6,0.6 --tris 300 --slots 1

# five renders plus framing, exposure and wear metrics
blender --background --python factory/lib/fx_evidence.py -- \
  --blend asset.blend --obj SM_Crate_Small --out evidence/ --albedo "#8A7A5C"

# FBX with the scale baked into the vertices, not into metadata
blender --background asset.blend --python factory/lib/fx_export.py -- \
  --out exports/SM_Crate_Small.fbx --expect-x 60

A builder writes one file: build_asset.py. Everything after that is a command from factory/lib/RECIPES.md.

That --expect-x is not decoration. Blender's FBX exporter will happily write your scale into UnitScaleFactor metadata and leave the vertices in metres. Blender re-imports that correctly, so a round-trip check passes — and Unreal, which ignores that metadata for static meshes, gets an asset 100× too small. fx_export.py refuses to ship if the vertices did not bake.

---

factoryctl

Cheap checks, so you find out in seconds instead of forty minutes in.

python factory/factoryctl.py doctor                       # can a run happen at all
python factory/factoryctl.py contracts                    # are the contracts sane
python factory/factoryctl.py manifest TASK-0001 run-1     # hash every artifact
python factory/factoryctl.py manifest TASK-0001 run-1 --check   # did anything drift
python factory/factoryctl.py status                       # every task and run
python factory/factoryctl.py ledger backfill              # rebuild history from disk

manifest is the one people underestimate. It turns "the agent says these are the run-1 renders" into something you can verify, and it catches byte-identical images — a render that did not change being read as progress is a real failure mode, not a hypothetical one.

Details: docs/factoryctl.md.

---

The control room

tools/factory-ui is a local web UI over the same files everything else reads.

• Loop — the gauntlet as a node graph, coloured by the selected run's real stage status, with a live agent transcript: every tool call with its arguments and result, per-call durations, running token count and cost, and a stop button.
• Assets — one card per run, with a drag-across slider to compare any two renders.
• Specs — the method, the runbooks that are the agent prompts, the recipes.
• Engine — port lamps for the services a run needs, and the git boundary audit.

Double-click an agent node to launch that step. Dry run is the default — it prints the exact command and spawns nothing. Every launch writes its brief to gauntlet/runs/<task>/prompts/ first, so there is always a record of exactly what an agent was told.

While an agent runs, the server re-audits git status every 3 seconds against a pre-launch snapshot, matches new dirty paths against boundary.forbidden from every contract, and SIGKILLs the agent on a hit. That is the safety net that lets this live inside a real game repo.

Assets — every run as a card, measured values read straight off its evidence	Engine — service lamps and the boundary audit, which says plainly when it is inactive

Details: tools/factory-ui/README.md.

---

The flag discipline

gauntlet/runbooks/FLAGS.md is the log of every failure a runbook could have prevented. After any run that goes wrong:

1. Write the flag — what was observed, not a theory about it.
2. Edit the runbook so it cannot recur.
3. Regenerate flags.yaml and re-sync your agents.

This is the part that makes a small local model viable. It does not get smarter between runs. Its instructions do.

The 11 flags shipped here are real. F-001 is the FBX scale trap above. F-002 is smart_project() silently emitting zero-area UVs in background mode. F-005 is an agent reporting that a measuring script "executed successfully" when the script could not even import its dependencies. You will hit these.

---

Requirements

• Blender 5.x (4.x likely works; untested here)
• Unreal Engine 5.x with the Model Context Protocol plugin, for the integrator step
• Python 3.9+ with pyyaml, for factoryctl
• An agent runner: OpenCode, Claude Code, or your own

The Unreal half is optional. Everything up to and including FBX export is useful on its own if you target a different engine — swap unreal-integrator for your own step.

---

What this is not

• Not a model. It brings its own discipline, not its own intelligence.

• Not a texture pipeline. fx_bake.py bakes procedural materials to 2K PBR maps; serious texturing belongs elsewhere.

• Not an art director. critic-visual checks a contract, not taste.

• Not push-button. You write contracts and decide when a task is done. The loop removes guesswork, not judgement.

• Not magic. It will not design your game. It builds what your contract says, and tells you plainly when that does not match what you meant.

---

License

MIT. See LICENSE.

Built while making a game. The game stayed home; the factory came here.