Kratos-Agent

Kratos Agent

Kratos Studio combines the Kratos desktop and coding agent with the current OmniRoute gateway and dashboard. Kratos owns planning, tools, permissions, task state, verification, and memory for Kratos tasks. OmniRoute owns provider connections, configured model or combo routing, compression, quotas, and its native feature pages and agents.

Runtime boundaries

Component	Responsibility
Kratos SDK	Typed completions, streaming, structured tool calls, and same-endpoint retries through the supervised local OmniRoute gateway.
OmniRoute	Provider routing, fallback, compression, quotas, dashboard, APIs, and native agent features. Kratos requests opt out of OmniRoute memory and skill injection.
Agent loop and harness	Planning, tool execution, memory, audit events, permission checks, and evidence-based completion.
computer-use MCP	Desktop and browser actions, subject to Kratos tool policy.
Gemini Live	Optional text-to-speech only. Kratos sends only text enclosed by [[speak]]...[[/speak]]; Gemini receives no user audio, video, screen frames, tools, or reasoning prompts.
Local Windows speech	System.Speech recognition turns spoken requests into text for the Kratos agent. SAPI is the local speech fallback.
OpenPets	Animated desktop companion and speech bubble for approved spoken phrases.

If the gateway is unavailable, the task reports a visible model error. Kratos retries its configured endpoint and model ID; any provider fallback inside a configured OmniRoute combo remains OmniRoute's responsibility.

Chat starts with Preparing request and shows Waiting for the selected model only after Kratos assembles the model request. Workspace checkpoints start before the first approved tool that can change files; ordinary conversation skips the scan. Events and completed turns save to local JSONL/SQLite immediately. Cloudflare D1 sync runs in the background with a durable retry queue, so an unavailable cloud database does not delay the reply or Stop, and pending training records survive a restart.

The model catalog is fetched once and saved to .kratos/gateway-model-catalog.json. Opening the picker, switching models, normal requests, and restarts reuse this saved catalog. Click Refetch in Choose a model to fetch it again. The snapshot has no automatic expiry and is bound to the gateway endpoint and API key fingerprint; changing either requires its own catalog. Failed Refetch keeps the last successful list and displays a notice. Browser connection models use the metadata saved with their login.

The conversation header shows context tokens used, remaining input capacity, the output reserve, and compaction savings. Expand Context for details or use Compact context while idle. Counts are marked as estimates unless the provider reports them. The configured context window defaults to 256k; advertised model limits take precedence. Automatic compaction starts at 90% of the available input budget, preserves recent tool-call/result pairs, and saves a checkpoint without deleting the full session records.

Kratos automatically recovers from malformed tool responses and failed verification while the task remains active. Recovery changes method from schema correction to explicit JSON, one action at a time, and a revised approach. Rejected calls never execute; completed work and verification requirements remain saved. A small recovery notice disappears after five seconds, and Stop interrupts recovery. Set KRATOS_AUTONOMOUS_RECOVERY=0 to opt into bounded recovery; only that mode uses KRATOS_VERIFICATION_RETRIES (1–10, default 3) and the legacy two-correction tool limit. Network retries still use KRATOS_MAX_RETRIES; sign-in, permission, and other terminal blockers stay visible.

Each browser receives a persistent workspace profile. The first profile keeps the original project and history; additional profiles run independent Python workers in .kratos/users/<profile>/workspace, with separate sessions, model settings, context, and Stop controls. Four profiles can run concurrently by default (KRATOS_MAX_USERS, 2–32). Profiles share the configured model gateway and its provider quotas. Additional profiles save their training/session data in their own local databases; shared Cloudflare D1 and desktop, mail, microphone, and MCP connections are not inherited automatically. Browser profiles are workspace separation for users of this trusted local installation, not a replacement for authenticated accounts.

Performance roadmap: Rust API and Phase 2 browser isolation

Status: design researched on 2026-10-22; not implemented or benchmarked in this checkout. The current Python Kratos SDK, OmniRoute gateway, browser executors, and session/permission boundaries remain the supported runtime. Rust is proposed for the I/O-facing API/streaming layer, not to replace Python ML inference or rewrite OmniRoute before measurements demonstrate a benefit.

• Phase 1 — lightweight Rust API (proposed): prototype an optional local Axum + Tokio + Reqwest adapter for the existing OpenAI-compatible /v1/chat/completions request and SSE contract. Reuse a single HTTP client, forward bytes without waiting for a full reply, propagate cancellation, respect backpressure, and preserve tool-call/event ordering, request IDs, errors, authentication, and the exact configured model. For two upstream API calls, run sequentially when the second depends on the first; parallelize only truly independent calls. Python retains agent planning, tools, memory, ML integrations, and evidence checks.
• Phase 2 — fast, account-isolated browser execution (research complete; implementation pending): retain encrypted browser session import and the existing OmniRoute executors. Pool warm workers/pages for authorized accounts, reuse conversation-bound tabs and authenticated storage where valid, serialize calls that share an account, bound cross-account concurrency, and avoid per-request Chromium/process startups. Preserve strict account/tenant separation, Stop/cancellation, login/challenge handling, provider limits, and no blind retries after uncertain submission. Do not access another account's context or use rotation to evade restrictions.
• Proof before migration: compare current Python/OmniRoute against the proposed Rust path on cold/warm time-to-first-token, p50/p95 end-to-end latency, SSE correctness, CPU/RSS on a 16-GB Windows laptop, per-account contention, concurrent distinct-account requests, expiry, restart and Stop. Treat Rust's expected memory/CPU efficiency as a hypothesis for this workload, not a promised model-generation speedup.

See the detailed design and research, implementation/status tracker, and existing browser connection and session sync documentation. No Rust gateway or new browser pool is claimed to exist yet.

Setup

Requirements: Python 3.12 or 3.13, Node.js 20+ for OmniRoute and the built-in MCP servers, and the OpenPets desktop app if you use the animated companion.

1. Install dependencies with uv sync.

2. Build the bundled frontends and gateway once with uv run kratos-studio setup.

3. Start the unified app with uv run kratos-studio run --workspace .. Open http://127.0.0.1:20128/dashboard/kratos.

4. Read the initial dashboard password from .kratos/studio/gateway/studio-secrets.json. Keep that file private. For an existing OmniRoute store, use the offline import procedure in Studio operations; the imported database keeps its original login password.

5. Configure a model or combo in the OmniRoute dashboard and set its exact ID for Kratos:

   KRATOS_MODEL=<exact OmniRoute model or combo ID>
   KRATOS_API_KEY=<OmniRoute API key, if required by your endpoint>
   

The launcher supplies KRATOS_BASE_URL and starts both local services. For the terminal interface, run uv run kratos-studio run --cli --workspace . (or uv run kratos --cli --workspace .). It connects to a ready gateway on the selected port, or starts its own gateway and stops it when the CLI exits. Use uv run kratos-studio setup --cli for a gateway-only installation without building the agent web frontend. Setup prepares dependencies; run --cli opens the terminal agent.

KRATOS_MODEL is preferred. Existing .env files may keep using BRAIN_MODEL; KRATOS_MODEL takes precedence when both are set. Kratos does not guess an ID or silently choose another model when the gateway or configured model is unavailable. An OmniRoute combo may route among its configured compatible providers.

Optional speech output uses GEMINI_API_KEY or GOOGLE_API_KEY and KRATOS_LIVE_ENABLED=true. Without Gemini credentials, only marked phrases use local Windows SAPI output. Speech recognition remains local. Set KRATOS_COMPUTER_USE_MCP=false or KRATOS_OPENPETS_MCP=false to disable those built-in MCP servers.

First-party Python SDK

For ChatGPT account access, run uv run kratos-connect browser-setup, load the generated Kratos extension in your regular Chrome or Edge, and manage logins and models from its Connections panel. Connected logins appear in green and errors appear in red. This path uses the bundled chatgpt-web.ts executor in your normal signed-in tab. Terminal login is also available with uv run kratos-connect login. Official authorization is available explicitly with uv run kratos-connect login --provider chatgpt. Credentials are protected outside the workspace. Select a returned account/model ID and restart Studio; see the ChatGPT connection and browser connection guides.

from kratos_agent.sdk import KratosAgent, KratosClient

client = KratosClient()
completion = client.complete({
    "model": "your-configured-kratos-model",
    "messages": [{"role": "user", "content": "Say hello."}],
    "stream": False,
})
print(completion.text)

agent = KratosAgent()
print(agent.run("Inspect this repository and explain its entry point."))

The SDK implements the existing gateway's /v1/chat/completions contract directly. It does not import model-provider agent SDKs. Tool calls are returned as typed KratosToolCall values, then executed by the guarded Kratos agent loop.

Speech directives

Kratos displays the full answer with directive markers removed. It speaks only valid paired phrases:

The task is complete. [[speak]]I finished and verified the task.[[/speak]] The report is saved in the workspace.

Unmarked, unmatched, or malformed speech tags do not reach Gemini or local TTS. The approved phrase is also sent to the OpenPets speech bubble when its MCP server is connected.

Load the companion plugin from openpets/kratos-companion/ in OpenPets through Plugins → Developer Mode → Load unpacked plugin folder. Keep it enabled to get roaming, edge-climbing, reactions, and speech bubbles; Kratos does not install or enable the separate OpenPets desktop plugin automatically.

Safety and evidence

• Tool schemas are validated before dispatch, and tool results return to the model loop.
• Write targets are checked against the workspace and approval mode.
• External sends/posts, destructive actions, purchases/bookings, account/security changes, and system installs require a direct user request with concrete targets supplied in the call.
• A tool start or success string is not proof of a result. Observable tasks require evidence before Kratos reports completion.
• Browser assertions are checked against live browser state before they count as verification evidence.
• MCP tools use the same permissions and evidence checks as built-in tools.

Development

uv sync --extra dev
uv run pytest -q
uv run ruff check src tests

Key code areas:

• src/kratos_agent/sdk/: first-party gateway and agent APIs.
• src/kratos_agent/core/agent_loop.py: planning, model/tool cycles, and completion checks.
• src/kratos_agent/core/tool_runtime.py: tool registry, validation, and execution boundary.
• src/kratos_agent/core/risk_policy.py: significant-consequence action gate.
• src/kratos_agent/core/verification_manager.py: evidence collection and completion gate.
• src/kratos_agent/live/: local speech input, speech output, and companion UI.
• src/kratos_agent/core/mcp_runtime.py: MCP server lifecycle and discovery.

The maintained documentation index is docs/README.md. Detailed contracts live in docs/architecture.md, docs/AGENT_BACKENDS.md, docs/GEMINI_LIVE.md, and docs/tools-and-data.md.