optimization-jobs

optimization-jobs

Open-source optimization for AI agents: ecosystem overview · shared solver-selection guide · how it compares with other systems.

Choose between Python/Rust black-box optimization, LP/MIP and smooth nonlinear solvers. This repository provides their shared shell/MCP job layer; it has no solver of its own.

Durable, versioned optimization jobs for agents and people: a strict job envelope, a result contract with claims and evidence, a worker SDK, a local runner, optimization-run, that schedules native solver workers against one CPU budget, and optimization-mcp, an MCP server that gives agents the same jobs as tools and tasks.

An AI that wants to optimize anything, over MCP or a shell, starts with ai-context.md: which of our solvers fits the problem, library or job, installation, the job API and the MCP server, and where the detailed documentation is. Over MCP, the tool get_guide returns it. Coding agents working on this repository start with AGENTS.md.

Part	What it is
crates/optimization-job-api	Job envelope optimization.job/0.1, result contract optimization.result/0.2, input hashes, checked journals, the worker SDK (worker::run, Worker, Prepared) and worker-declared artifacts (artifact::declare)
crates/optimization-runner	The optimization-run command: worker registration, staging, idempotency, CPU-slot admission, supervision, cancellation, deadlines, recovery and coded errors; and optimization-mcp, the MCP server over stdio
docs/job-api.md	The guide: envelope, resources, lifecycle, claims and evidence, errors, recovery, agents and MCP
docs/job-api/schemas	JSON Schemas of the job envelope and the result manifest
docs/job-api/PLAN.md	Implementation plan and acceptance gates
scripts/job-api	Shared acceptance harness, runner regression tests and a protocol fixture worker

The solvers live in their own repositories, each with a worker, a runnable tutorial and an ai-context.md for agents that model problems with it. Choosing a worker and profile maps problem classes to profiles:

Repository	Worker	Profiles
fcmaes-rust	fcmaes-job-worker	fcmaes.retry/1, fcmaes.mode/1, fcmaes.map-elites/1, fcmaes.surrogate/1 (experimental), fcmaes.evaluate/1
LP-IP-craft	lp-ip-job-worker	lp-ip.solve/1, lp-ip.evaluate/1
SQP-craft	sqp-job-worker	sqp.solve/1, sqp.evaluate/1

experiment-craft builds on this job layer. It runs reproducible campaigns of these jobs through optimization-run: parameter sweeps, dependent steps, budgets, deadlines and reports that keep each job's claim and evidence. Its baseline rebuilds the runner and the three workers at pinned revisions and tests them together. It is in early development.

Nine prompt-driven tutorials cover trebuchet design, production policies, energy sizing, curve fitting, GTOC1 route screening, surrogate optimization, LP/IP allocation, constrained SQP and synthetic rail-traffic recovery, with optional native application workers and an executable validation driver.

Install

From 0.3.0 the crates are on crates.io. On Linux (the runner supervises processes through Linux interfaces) with Rust 1.88 or newer (rustup from https://rustup.rs):

cargo install optimization-runner --locked   # optimization-run and optimization-mcp on PATH

Worker crates depend on optimization-job-api = "0.3".

From source

For changes after the latest release, build from source. You need Linux, git and Rust 1.88 or newer; Cargo fetches every other dependency. Python 3.10 or newer is needed only for the test scripts.

git clone https://github.com/dietmarwo/optimization-jobs
cd optimization-jobs
cargo build --release --locked        # target/release/optimization-run, optimization-mcp
cargo install --path crates/optimization-runner --locked   # optional: both on PATH

The runner has no workers of its own. Each worker lives in one of our solver repositories and builds there together with this runner; Cargo fetches this repository at the revision the worker pins, so no second checkout is needed:

Worker	Build from source
fcmaes	git clone https://github.com/dietmarwo/fcmaes-rust && cd fcmaes-rust && cargo build --release --locked -p optimization-runner -p fcmaes-job-worker
LP/MIP	git clone https://github.com/dietmarwo/LP-IP-craft && cd LP-IP-craft && cargo build --release --locked -p optimization-runner -p lp-ip-job-worker
Smooth NLP	git clone https://github.com/dietmarwo/SQP-craft && cd SQP-craft && cargo build --release --locked -p optimization-runner -p sqp-job-worker

To use a runner built here with workers built elsewhere, register the workers' executables by absolute path. RELEASING.md describes the crates.io release.

Quick start

Linux and Rust 1.88 or newer. In a worker repository, the runner and the worker build together:

cargo build --release --locked -p optimization-runner -p fcmaes-job-worker
./target/release/optimization-run --root /tmp/node node init --capacity 8
./target/release/optimization-run --root /tmp/node workers add \
  --id fcmaes --executable ./target/release/fcmaes-job-worker
./target/release/optimization-run --root /tmp/node run --spec JOB.json

Or build the runner and the MCP server alone here with cargo build --release --locked. One runner serves every registered worker on a node. The guide explains jobs, results and the CPU budget; the worker repositories' tutorials have complete example jobs.

Agents

The same build makes optimization-mcp. Connect an MCP client to an initialized node with registered workers, for example Claude Code:

claude mcp add optimization-jobs -- "$PWD/target/release/optimization-mcp" --root /tmp/node

Its fifteen tools serve the guide for agents (get_guide, this repository's ai-context.md), describe the node (with the CPUs and memory the host gives it) and its workers (with JSON Schemas of their payloads and runnable examples), store input files such as a new LP model (put_input), validate, submit, follow, cancel and repair jobs, and read results and artifacts, such as a running surrogate job's evaluation log in pages; with the MCP Tasks extension, a submitted job is a task that completes with the job's result. Jobs outlive the server. The server cannot register workers, change the node or run command jobs. See Agents and MCP.

Two tutorial series show the workers in use. The nine agent tutorials drive registered models through prompts. The twenty structured tutorials solve LP, MIP, SQP and black-box models with the native solvers' Python APIs, check each answer against an independent reference, and can run as trusted command jobs.

Depending on this repository

A worker repository pins a revision in its workspace manifest:

[workspace.dependencies]
optimization-job-api = { git = "https://github.com/dietmarwo/optimization-jobs", rev = "FULL_COMMIT_SHA" }
optimization-runner = { git = "https://github.com/dietmarwo/optimization-jobs", rev = "FULL_COMMIT_SHA" }

A worker crate implements optimization_job_api::worker::Worker and calls optimization_job_api::worker::run. To build optimization-run next to the worker with cargo build -p optimization-runner -p WORKER, give the worker optimization-runner as a normal dependency, even if it never uses it: with workspace members named on the command line, Cargo selects other packages only among the normal dependencies of those members (a dev-dependency is not enough). To move to a newer revision, change rev and run cargo update -p optimization-job-api -p optimization-runner.

To develop a worker and the shared crates together, patch the dependency with a local checkout:

[patch."https://github.com/dietmarwo/optimization-jobs"]
optimization-job-api = { path = "../optimization-jobs/crates/optimization-job-api" }
optimization-runner = { path = "../optimization-jobs/crates/optimization-runner" }

Trusted local commands and binary files

From 0.4.0 the runner also runs opt-in command jobs (execution.command/0.1) and moves binary files with exact size and SHA-256 checks, on the same local admission budget as optimization jobs. Commands are trusted programs under the runner's user, not a sandbox, and they are off until the node owner runs node commands --enable. They are CLI-only: the MCP server refuses them. The command and file guide covers the request and result schemas, transfer limits and the build–run–collect example.

Explicit remote execution

From 0.4.0 the runner can also place a job on a registered SSH node: fleet nodes add/probe pins a node's endpoint and identity, fleet submit --node NAME sends one optimization or command job there, and fleet reconcile/resume/cancel/repair/collect follow it. A fixed helper, optimization-run remote serve, receives every request as JSON on stdin, and persisted dispatch and launch intent keep uncertain work on its node: nothing is started twice or moved. Remote launch requires a systemd user manager with lingering on the node. Placement is explicit and CLI-only. The SSH guide covers provisioning, transfer limits, recovery rules and what the tests do not establish.

Tests

cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --release --locked
cargo build --release --locked -p optimization-runner
python3 scripts/job-api/test_review.py            # with the fake worker
python3 scripts/job-api/test_commands.py          # command jobs and file transfers
python3 scripts/job-api/test_remote.py            # helper/dispatch fault tests; simulated manager
# With all three worker binaries in JOB_API_TARGET:
python3 scripts/job-api/test_remote_ssh.py        # real OpenSSH; simulated manager
python3 scripts/job-api/test_mcp.py               # the MCP server
python3 scripts/job-api/test_integration.py \
  --fcmaes /path/to/fcmaes-job-worker \
  --lp-ip /path/to/lp-ip-job-worker --sqp /path/to/sqp-job-worker

The schema checks need pip install "jsonschema>=4.18,<5"; CI sets JOB_API_REQUIRE_SCHEMA=1 so they cannot be skipped. test_remote_ssh.py also needs pip install "asyncssh>=2.24,<3" and the OpenSSH client (ssh, ssh-keygen). CI also builds the three workers from their repositories' main and runs the harness on a mixed node. Worker repositories run the same scripts from Cargo's checkout of this repository, with JOB_API_TARGET pointing at their own target/release (guide).

License

MIT, see LICENSE.