zerollama

Zerollama

Built for agent megaprompts — not toy chat windows.

Your agent pastes a novel of tools and history every turn. Zerollama makes that feel fast, and can draw, clip, speak, and make snap yes/no calls on the same server — and it’s still backwards compatible with Ollama (same CLI, REST, and clients).

Not upstream Ollama. Build from this repo (./zerollama), not ollama.com/install.sh. If you only want “pull a model and chat” once, use upstream.

Quick start

Apple Silicon:

git clone <this-repo> zerollama && cd zerollama
./scripts/runtime/dev_bootstrap.sh
./zerollama serve
./zerollama pull llama3.2:3b
./zerollama run llama3.2:3b

CUDA / Arc → Building

./zerollama doctor
curl -s http://127.0.0.1:11434/api/tags | jq .

Default listen :11434. Point Hermes / OpenClaw / Grok Bot / Open WebUI / Continue at http://127.0.0.1:11434.

zerollama launch claude      # Claude Code, Codex, Copilot, OpenCode, …
zerollama launch openclaw
zerollama launch hermes
# Grok Bot — set its local / OpenAI-compatible provider to http://127.0.0.1:11434

Same ollama-python / ollama-js clients. Community catalog: ollama/ollama § Integrations. Cloud tags: Eliza Cloud — eliza-cloud.md.

Features

Your agent stops re-reading the novel every turn

Huge prompts get chewed up faster (~3–7× in our labs). The next message remembers what it already read. Ask several follow-ups off the same long context without paying for that trunk again and again.

→ readme-marketing-benches.md · faster-bpe-tokenize.md · gpu-profiles-l3.md · multidecode-llama-cpp.md

Draw it, clip it, say it — same place as chat

Generate images, short videos, and spoken replies from the daemon you already pointed your agent at. No second product.

→ wan-t2v.md · ltx-t2v.md · h3-cuda-port.md · media-uploads.md · music-c.md

Pull names and make snap judgments

“Who’s mentioned?” “Is this urgent?” “Which of these hits is best?” as real answers — not “please reply in JSON” against the chat model.

→ gliner-cpp.md · gliner-decide.md · laya-llama-cpp.md · clm.md

One GPU, many agents, less drama

Live chat jumps ahead of background jobs. You can see who is holding the card. If something gets bumped, you get a reason you can retry. Training waits until things go quiet. A doctor fixes “broken” models that were really bad templates.

curl -s http://127.0.0.1:11434/api/version | jq '{distribution, capabilities: .zerollama.capabilities}'

→ agent-qos-and-project-tracking.md · hermes-zerollama-gap.md · doctor-model-repair.md · t6-unified-queue.md · gpu-training.md · t10-grpo.md · t11-recipes.md

Vs upstream matrix: upstream-ollama-diff.md

CLI

Richer ls / ps than upstream — MoE size, host-safe context, PERF from bench, and which harness holds VRAM.

NAME                         ID              SIZE      PARAMS                 CTX         PERF     MODIFIED
qwen3-coder-next:6bit        ffc5c8db17e8    64 GB     15.0B MoE 512x10       80k         --       4 minutes ago
ornith-35b-optiq:latest      f4df829f8a75    22 GB     34.0B MoE 256x8        80k         54.2     12 hours ago

NAME                       PROJECT                                       SESSION                                             SIZE     PROCESSOR    UNTIL
ornith-35b-optiq:latest    hermes-lean/discord:dm:1516015052568793098    hermes:agent:main:discord:dm:…                      27 GB    100% GPU     29 minutes from now

Filters: zerollama ls image / zerollama ls video_gen · docs: bench-cache.md

Usage

Vanilla Ollama clients work. When distribution == "zerollama", send a thread id + priority so the cache and queue help:

curl http://127.0.0.1:11434/api/chat -d '{
  "model": "llama3.2:3b",
  "stream": false,
  "messages": [
    {"role": "system", "content": "You are a coding agent. (…long tools + prefs…)"},
    {"role": "user", "content": "Fix the flaky test."}
  ],
  "options": {
    "prompt_cache_key": "hermes-thread-42",
    "zerollama": {
      "qos_class": "interactive",
      "project_id": "hermes-lean",
      "project_name": "demo"
    }
  }
}'

from ollama import chat
print(chat(model='gemma3', messages=[{'role': 'user', 'content': 'Why is the sky blue?'}]).message.content)

docs.ollama.com/api · harness contract: agent-qos-and-project-tracking.md

Backends: llama.cpp (ggml Metal/CUDA or Go→llama-server), MLX safetensors, Python runtime (:8081). Optional: Flash-MoE.

Building

./scripts/build/build_zerollama_mac.sh
ZEROLLAMA_GPU_JOBS=false OLLAMA_NOPRUNE=1 ./zerollama serve

Vendor pin: f280b269 / b10615 — runtime/LLAMA_CPP_PIN.md. After patch edits: make -f Makefile.sync clean apply-patches && ./scripts/vendor/sync_vendor_llama.sh.

Documentation

docs/README.md · ROADMAP.md · CHANGELOG.md

Agent skills

33 SKILL.md packages so Cursor / Claude / Copilot agents can use zerollama without reverse-engineering the API.

npx skills add GoodSoftware-Group/zerollama
npx skills add GoodSoftware-Group/zerollama --agent cursor

→ skills/README.md · skills/skills.json

Acknowledgments

We reimplement ideas in our Go + llama.cpp + runtime shape — we do not vendor their servers as required deps.

More: open-source-shoutouts.md · @spaceodili

Issues and PRs welcome.

License: MIT (Ollama lineage).