atmofab

atmofab

atmofab generates, validates, and certifies weather and climate compute kernels from natural-language specifications.

controlled_spec.md (physics and algorithm definition), tests.md (verification profile), and deps.yaml (dependency declaration) are authored by humans and are the canonical source. Every phase after them is executed by a deterministic conductor (tools/workflow_conductor.py), which fulfils the orchestration agent role: it launches each judgment-bearing substep as one isolated substep agent (an LLM leaf) under a fixed input/output contract, runs the deterministic gates and the build itself in its own process, and performs every build, execution, lint, and syntax check through the build-runtime library (tools/build_runtime.py).

Scope

• Generate the model (physics computation) and the runner (execution and judgment coordination) for a computation task defined by a spec, targeting CPU and GPU hardware. The material certified in this tree is Fortran on CPU. A second target profile, CUDA C++ on GPU (spec/targets/), builds its harness, and reaches Validate only at an execution site with a GPU that the operator's sites.yaml names (issue #293; docs/ORCHESTRATION.md §Execution sites).
• Manage specifications under four spec_kind values: problem (integration scenario), component (reusable operation), profile (component selection policy), and infrastructure (the shared runner harness, one node per (language, hardware) target).
• Keep the physics definition separated from execution optimization. spec.ir.yaml carries physics-affecting structure in its case / algorithm / io_contract sections and names no implementation target; execution discretion is held in the target profile (spec/targets/<target_id>.yaml) and in the generated bundle's lowering plan (docs/IMPL_PLAN_SPEC.md).
• Judge each node from its own execution evidence, and aggregate the judgment across its dependency closure.

Workflow

The core workflow is five phases. Each phase produces exactly one kind of primary artifact. docs/WORKFLOW.md is the entry point of the contract, and docs/workflow/WORKFLOW_CORE.md is the canonical source for the common invariants and the per-phase I/O contract.

Tune (implementation-variant exploration) and Promote (publication to releases/) are optional flows outside the core workflow. Neither is implemented: their contracts are deferred (docs/WORKFLOW.md §Optional flows), and no spec in this tree carries an official release.

From Generate onward, spec.ir.yaml is the sole generation and verification contract; reading controlled_spec.md is forbidden except at Generate.verify, which reads it as a secondary requirement-fidelity cross-check.

Each phase runs an ordered substep sequence, and there are exactly two launch shapes. An LLM substep runs as one pure-function leaf: the host inlines a closed context into the prompt, the model returns one JSON document, and the host validates and writes it — the leaf holds no tools, no shell and no write authority. A deterministic substep runs in the conductor's own process. Since Z4 (issue #171) there is no third shape: the agentic leaf, which ran in the checkout with tools and a hook layer around it, is retired, and a node with no bundle shape now fails closed rather than falling back to one.

The deterministic Compile.static and Generate.gate substeps route a violation back to the phase's generate substep, so the verify leaf is reached only on a deterministically-clean artifact. Validate.pre_judge blocks execution when the dependency DAG is not ready, and Validate.post_judge classifies the severity of a violation found after the judge returns.

Running a workflow

The host prerequisites — the CLI tools, the Python packages, the target spec's toolchain and static lint tool, the sandbox runtime, and the provider credentials — are one list in docs/RUNBOOK.md §0-1, which also carries the SUPPORTED VERSION RANGE of the static lint tool (a build outside it is refused at launch, because the rule set the Generate.gate applies is measured against that range), pointers to the backend documents that carry the install lines, and the reason code each absence is refused with. Every one of them is refused before the first LLM leaf is launched. This paragraph used to be a second list, and it was the longer of the two while the one an operator setting up a machine reads was the shorter (issue #109).

Create the leaf-LLM configuration once. It selects the provider, model, and reasoning effort of each LLM leaf, and is the only thing that says what a leaf launches: no flag overrides its contents, and --llm-config only selects which file is read. It is gitignored.

cp docs/examples/llm_claude.example.yaml llm.yaml

Every LLM leaf is a billed provider call. One node spends at least five of them per full Compile → Validate pass — more when a phase re-rolls — a --with-deps closure spends them per node, and a leaf that dies on a provider quota terminalizes the run unless --wait-usage-reset is passed.

Start a run:

python3 tools/run_workflow.py <spec_ref> <until_phase> --target <target_id> [--llm-config <path>]

# the dependency closure of one problem node, then the node itself
python3 tools/run_workflow.py spec/problem/dynamics/advection_diffusion/advdiff1d_linear validate --with-deps --target fortran_cpu

<spec_ref> is the spec directory of the target node and <until_phase> is one of compile / generate / build / validate. --llm-config defaults to ./llm.yaml; a missing default stops the run with llm_config_default_missing rather than being filled in. python3 tools/run_workflow.py --help is canonical for the full option set (docs/CLI_REFERENCE.md §Information-acquisition policy); the options a first run needs are:

docs/RUNBOOK.md is the canonical operational procedure: preflight requirements per backend, the minimal loop, the failure-to-phase routing table, and the recovery procedures.

Build runtime

The conductor calls the build-runtime library tools/build_runtime.py in-process for every compile, run, quality check, static lint, and syntax check; no leaf calls it, and one-off gcc / clang / gfortran builds are forbidden. Its entry points, the validation each applies, and the command_log.jsonl record each writes are specified in docs/BUILD_RUNTIME.md.

Artifacts

Trial artifacts are confined to workspace/, and a write outside it fails the phase. docs/GLOSSARY.md defines the identifiers below (node_key_safe, ir_id, pipeline_id, source_id, binary_id, run_id).

workspace/ir/<node_key_safe>/<ir_id>/                        spec.ir.yaml, ir_meta.json, dependency_graph.json
workspace/pipelines/<node_key_safe>/<target_id>/<pipeline_id>/ lineage.json
  source/<source_id>/                                        Generate output
  binary/<binary_id>/                                        Build output
  runs/<run_id>/<node_key_safe>/                             Validate output and raw evidence
workspace/orchestrations/<orchestration_id>/                 orchestration_meta.json, agent_graph.json, agent_runs.jsonl

Validate emits diagnostics.json, perf.json, verdict.json, aggregate_verdict.json, summary.json, and semantic_review.json per run_id. Promoted official-version artifacts live under releases/<spec_kind>/<domain>/<family>/<spec_id>/<target_id>/<release_id>/. docs/WORKSPACE_LAYOUT.md is the canonical layout source.

The driver prints the run's event stream on stdout (--stdout-format human, or jsonl for a caller that parses it) and mirrors it to workspace/orchestrations/<orchestration_id>/run_logs/. A run's terminal state is orchestration_meta.json#status; the node's physics judgment, including its dependencies, is aggregate_verdict.json.

Repository layout

docs/         workflow contracts, phase specifications, runbook, glossary
spec/         source specs (problem / component / profile / infrastructure) and the registry
skills/       execution procedures (SKILL.md) for the flows an operator runs
tools/        workflow driver, conductor, orchestration runtime, gates, validators, tests
.claude/      the operator's own interactive session: settings, and the development skills
releases/     the component registry, and the promoted official artifacts of the Promote flow (none yet)
workspace/    trial artifacts

In-tree specifications

spec/registry/spec_catalog.yaml is the registry of record for placement and state.

Tests

Full suite (the default):

python3 -m pytest tools/tests/ -q

Fast local loop, deselecting the wall-clock-bound slow tests:

python3 -m pytest tools/tests/ -q -m "not slow"

pytest.ini registers the slow marker, sets -p no:randomly, and points testpaths at tools/tests so a bare pytest at the repository root runs the suite. The marker is opt-out because nothing else runs the full set: deselecting it by default would leave the leaf deadline / abandon / teardown guards executed by nobody.

The review instrument under .claude/skills/ — atmofab-review-loop/scripts/mutation_check.py — carries its tests beside it in scripts/tests/. Its skill states the command that runs them (.claude/skills/atmofab-review-loop/SKILL.md), because no command in this section collects them — measured at dfeb73dc: pytest tools/tests/ and a bare pytest each collect 6266, and that directory holds 52 tests. (A second instrument, measure_claude_tool.py, measured the leaf tools that went in issue #171, and was deleted in issue #445.)

That command gets none of the guards tools/tests/conftest.py installs for the suite's own directory: not the environment strip, not the private-root redirect, not the secret-root refusal, and --keep-operator-env is an unrecognized argument there. Nothing is known to bite — neither the instrument's tests nor the script read an ATMOFAB_* name, CODEX_HOME or CLAUDE_CONFIG_DIR — and no conftest.py is placed beside them, so an instrument test that ever does need a guard has to say so itself.

The suite ignores your ATMOFAB_*, CODEX_HOME and CLAUDE_CONFIG_DIR environment variables (issue #84). They are per-run knobs that tools/run_workflow.py sets in every node's environment, so the shell most likely to have them exported is one used for this repository's own work — and a suite that inherits them answers a question about the machine instead of about the code. Measured at 165c26f: exporting every such name the tree reads took a clean run of 5280 passed to 181 failed, none of the failures belonging to any change. (That commit predates the rename, so spell the names METDSL_* to re-take it — docs/GLOSSARY.md §13.) Every run says which names it removed. To run against a knob you set on purpose, pass --keep-operator-env; expect failures that belong to the knob. tools/tests/conftest.py and tools/tests/suite_env_guard.py are canonical.

Documentation entry points

License

BSD 2-Clause. See LICENSE.