Turn one complex goal into a live, durable, validated agent swarm.
\n
\n Plan Β· Delegate Β· Execute Β· Validate Β· Synthesize\n
\n\n
\n \n \n \n
\n\n---\n\n## 𧬠What is Loom?\n\n**Loom** is a production-oriented orchestration workspace that decomposes a single complex goal into a **Directed Acyclic Graph (DAG)** of specialized agent tasks. A planner creates the task graph, workers execute ready tasks in parallel, a mode-aware validator can request focused revisions, and a synthesizer produces the final answer β all while **React Flow** renders every event in real time.\n\nIt's not just a demo. Loom pairs a focused product experience with **production-grade identity, billing, durable workflows, streaming state, distributed infrastructure, and observability**.\n\n---\n\n## β¨ Key Features\n\n| Feature | Description |\n|---|---|\n| π§ **AI-Powered Planning** | An LLM planner decomposes goals into a typed, dependency-aware task DAG with Zod-validated structured output. |\n| β‘ **Parallel Execution** | Independent tasks in the DAG run concurrently β the swarm gets latency benefits over serial task lists. |\n| β **Mode-Aware Validation** | A validator agent reviews each task output. Rejected outputs get concrete feedback for self-correction (revision depth is mode-controlled). |\n| π¬ **Live Research** | Researcher agents use bounded Firecrawl web search, preserve source links, and state clearly when live search is unavailable. |\n| π **Real-Time Visualization** | React Flow renders the live execution graph. Zustand drives the state. WebSocket-first with SSE fallback. |\n| π **Production Auth & Billing** | Better Auth email/password sessions backed by PostgreSQL. Stripe Checkout, signed webhooks, Customer Portal, and entitlement projection. |\n| ποΈ **Durable Workflows** | Temporal moves long-running orchestration out of the HTTP process. Heartbeats, cancellation, and crash recovery. |\n| π‘ **Polyglot Telemetry** | A standalone Go service consumes Kafka events and fans them out via gRPC, WebSocket, and Prometheus metrics. |\n| πΌοΈ **Image Understanding** | Cloudinary stores attached images as authenticated assets. A vision model derives instruction-resistant text context. |\n| π€ **MCP Server** | A read-only, account-scoped Streamable HTTP MCP server lets Codex or Claude Code discover runs and retrieve final Markdown deliverables. |\n\n---\n\n## ποΈ Architecture Overview\n\n```text\nBrowser\n β authenticated POST /api/swarm { goal, mode, attachments }\n βΌ\nNext.js Route Adapter\n ββ session + origin + bounded input + infrastructure checks\n ββ PostgreSQL plan lookup β Redis user quota\n ββ text file ββββββββββββββββββββββββββββββ\n ββ public GitHub repo β fixed GitHub API ββ€β untrusted reference context\n ββ image β authenticated Cloudinary asset ββ vision-derived text\n ββ starts Temporal workflow\n β\n βΌ\nTemporal Worker β swarm Activity β planner β DAG workers β validators β synthesizer\n β β\n β ββ OpenRouter through Vercel AI SDK\n β ββ Zod structured-output validation\n βΌ\nRedis event stream ββSSE (fallback + durable replay)βββΆ Zustand βββΆ React Flow\n β β²\n ββ Kafka βββΆ Go telemetry consumer βββ¬ββΆ /metrics β (Prometheus)\n βββΆ gRPC :9090 β (StreamRunEvents)\n βββΆ wss://.../ws βββββ (primary browser transport)\n\nCodex / Claude Code ββbearer tokenβββΆ /api/mcp\n ββ list_runs\n ββ get_final_deliverable\n```\n\n> The planner, worker, validator, and synthesizer are **code roles**, not separate servers. TypeScript controls their order and data flow; the LLM supplies language reasoning.\n\n---\n\n## ποΈ Execution Modes\n\nLoom provides three execution modes that control DAG size, validation depth, and token budgets:\n\n| Mode | DAG Size | Revision Budget | Repeated-Context Policy | Best Fit |\n|---|---:|---:|---|---|\n| **Low** | 1β2 tasks | 0 revisions | smallest budgets | fast, concise work |\n| **Auto** | 2β4 tasks | up to 1 revision | balanced budgets | normal adaptive use |\n| **Max** | 4β6 tasks | up to 2 revisions | largest bounded context | deep, multi-angle work |\n\n> Max is deliberately more expensive and slower, but **never unbounded**. Every mode has deterministic caps on material repeated between model calls.\n\n### Token-Efficiency Design\n\n| Mode | Upstream Handoff | Revision Feedback | Worker β Validation | Synthesis Corpus |\n|---|---:|---:|---:|---:|\n| Low | 4,000 chars | 1,000 chars | 8,000 chars | 12,000 chars |\n| Auto | 8,000 chars | 2,000 chars | 12,000 chars | 24,000 chars |\n| Max | 16,000 chars | 4,000 chars | 24,000 chars | 48,000 chars |\n\n- Upstream blackboard outputs, validator inputs, revision feedback, and the synthesis corpus each have a **mode-specific character budget**\n- Labeled sections share a budget fairly β one oversized worker cannot crowd out every other specialist\n- Compaction preserves section labels plus opening and conclusion, with an explicit truncation marker\n- Attachment data is prepared once for planning, not copied raw into every worker prompt\n\n---\n\n## π‘οΈ Security Boundaries\n\n- **Text files** are type-checked and size-bounded before becoming untrusted planner context.\n- **GitHub attachments** accept only exact public `https://github.com/owner/repository` URLs. Loom constructs the `api.github.com` request itself β no user-controlled host fetching.\n- **Image attachments** are validated, stored with Cloudinary's `authenticated` delivery type, and described by a vision model told to treat visible instructions as data.\n- **Raw base64 image data** is never written to Redis run state, Kafka events, or Temporal input. Only the derived text description enters orchestration.\n\n---\n\n## 𧱠Tech Stack\n\n| Layer | Technology |\n|---|---|\n| **Frontend** | Next.js 16, React 19, React Flow, Zustand, Tailwind CSS |\n| **AI / LLM** | Vercel AI SDK, OpenRouter, Zod structured outputs |\n| **Auth** | Better Auth (email/password + PostgreSQL sessions) |\n| **Billing** | Stripe Checkout, webhooks, Customer Portal |\n| **Orchestration** | Temporal (durable workflows, activities, heartbeats) |\n| **State & Cache** | Redis (run state, event replay, rate limits via Lua scripts) |\n| **Event Streaming** | Kafka (versioned swarm envelopes, 6-partition topic) |\n| **Telemetry** | Go service (gRPC, WebSocket, Prometheus /metrics) |\n| **Media** | Cloudinary (authenticated image storage) |\n| **Research** | Firecrawl (bounded web search with source preservation) |\n| **Database** | PostgreSQL (users, sessions, billing projection) |\n| **Testing** | Vitest (TS), Go test with `-race` |\n\n---\n\n## π Repository Layout\n\n```\nsrc/\nβββ app/ Next.js routes and page composition\nβ βββ api/ API routes (swarm, billing, chat, MCP, auth)\nβ βββ sign-in/ Authentication page\nβββ components/ React components (SwarmGraph, GoalBar, SidePanel, etc.)\nβββ lib/\nβ βββ swarm/ Core orchestration domain + application services\nβ β βββ orchestrator.ts DAG execution engine\nβ β βββ planner.ts LLM-powered task decomposition\nβ β βββ worker.ts Specialist agent execution\nβ β βββ validator.ts Output quality gate\nβ β βββ bus.ts Event bus (Redis + Kafka)\nβ β βββ runStream.ts Client-side stream management\nβ β βββ tokenBudget.ts Mode-aware context compaction\nβ βββ billing/ Plans, Stripe gateway, entitlement repository\nβ βββ mcp/ Model Context Protocol server + token service\nβ βββ media/ Authenticated Cloudinary image storage adapter\nβ βββ research/ Firecrawl web search integration\nβ βββ auth.ts Identity and shared authentication policy\nβ βββ store.ts Zustand global store\nβββ styles/ CSS modules (foundation, workspace, auth, etc.)\nβββ temporal/ Temporal workflow/worker process boundary\n\nservices/telemetry/ Independent Go Kafka consumer β gRPC + WS + Prometheus\nproto/telemetry/v1/ Protocol Buffer contracts for the gRPC surface\nscripts/ Idempotent database migrations + build scripts\ndocs/ Architecture, design, product, and deployment docs\n```\n\n---\n\n## π Local Setup\n\n### Prerequisites\n\n- **Node.js** 20+\n- **pnpm** 11\n- **Docker** (for Redis, PostgreSQL, Kafka, Temporal)\n- **OpenRouter API key**\n- Firecrawl API key *(optional β enables live Researcher web search)*\n\n### Quick Start\n\n```bash\n# Install dependencies\npnpm install\n\n# Set up environment\ncp .env.example .env.local\n# Fill in: OPENROUTER_API_KEY, BETTER_AUTH_SECRET\n# Optional: FIRECRAWL_API_KEY, CLOUDINARY_*, STRIPE_*\n\n# Start infrastructure\npnpm infra:up\n\n# Run database migrations\npnpm db:migrate\n\n# Start the Temporal worker (in a second terminal)\npnpm temporal:worker\n\n# Start the dev server\npnpm dev\n# β http://localhost:3000\n```\n\nGenerate an auth secret: `openssl rand -base64 32`\n\n### Local Services\n\n| Service | Address | Purpose |\n|---|---|---|\n| Next.js | `http://localhost:3000` | UI, auth, API, SSE fallback + durable replay |\n| Temporal UI | `http://localhost:8233` | Workflow inspection |\n| Go Telemetry HTTP | `http://localhost:9091/metrics` | Prometheus metrics, `/healthz` |\n| Go Telemetry WS | `ws://localhost:9091/ws` | Primary live event transport for the UI |\n| Go Telemetry gRPC | `localhost:9090` | `TelemetryService`, health, reflection |\n| PostgreSQL | `localhost:5432` | Users, sessions, billing |\n| Redis | `localhost:6379` | State, replay, limits |\n| Kafka | `localhost:9092` | Event stream |\n\n---\n\n## π‘ Real-Time Event Transport\n\n### Transport Selection\n\n| `NEXT_PUBLIC_TELEMETRY_WS_URL` | Live Transport |\n|---|---|\n| **set** | WebSocket (with automatic fallback to SSE) |\n| **empty** | SSE only |\n\nThe graph header shows which transport is actually live β the choice is **observable in the product**, not asserted in a diagram.\n\n### Why WebSocket + SSE Fallback?\n\n- **WebSocket** is preferred because it pushes events straight off the Kafka fan-out (zero-latency), while SSE polls Redis at 150ms intervals.\n- **SSE** is the fallback because Vercel's serverless functions cannot hold a long-lived socket open, and corporate proxies may refuse `Upgrade`.\n- **A durable Redis log** sits underneath both. Every frame carries a monotonic `sequence`; gaps trigger replay from the Redis-backed endpoint. A replayed event is a no-op.\n\n### Fan-Out Architecture\n\nThe Go telemetry consumer reads each Kafka partition once and fans events out through an in-process hub. Each subscriber owns a **256-event buffered channel**; `hub.publish` sends non-blocking. A subscriber that fills its buffer is **evicted** (not waited on) to prevent blocking the Kafka poll loop β one frozen browser tab will never take telemetry down for everyone.\n\n---\n\n## π³ Billing & Plans\n\n| Plan | Rate Limit | Max Runs |\n|---|---|---|\n| **Free** | 3 runs/hour (at most 1 Max run/hour) | β |\n| **Pro** | 100 runs/hour | β |\n\nLimits are atomically enforced in Redis by user ID. Stripe-hosted surfaces ensure card data never enters this application.\n\n\nStripe Setup\n\n1. Create a Stripe Product and recurring Price.\n2. Set `STRIPE_SECRET_KEY` and `STRIPE_PRO_PRICE_ID`.\n3. Configure a webhook at `https://your-domain/api/billing/webhook` for:\n - `checkout.session.completed`\n - `customer.subscription.created`\n - `customer.subscription.updated`\n - `customer.subscription.deleted`\n4. Set its signing secret as `STRIPE_WEBHOOK_SECRET`.\n5. Enable the Stripe Customer Portal and set `APP_URL`.\n\nFor local webhook testing:\n```bash\nstripe listen --forward-to localhost:3000/api/billing/webhook\n```\n\n\n---\n\n## π§ͺ Commands\n\n| Command | Purpose |\n|---|---|\n| `pnpm dev` | Start development server |\n| `pnpm typecheck` | TypeScript verification |\n| `pnpm lint` | ESLint checks |\n| `pnpm test` | Unit/integration test suite |\n| `pnpm db:migrate` | Better Auth + billing + MCP schemas |\n| `pnpm infra:up` | Start local dependencies |\n| `pnpm infra:full` | Start dependencies + containerized Worker |\n| `pnpm infra:topic` | Inspect the six-partition Kafka topic |\n| `pnpm infra:logs` | Follow infrastructure logs |\n| `pnpm infra:down` | Stop containers, preserve volumes |\n| `pnpm infra:reset` | Remove containers and local data volumes |\n| `go test -race ./...` | Telemetry service tests (run in `services/telemetry`) |\n\nCI runs both halves on every push and PR: a `web` job (lint, typecheck, test against a real Redis service container) and a `telemetry` job (go vet, go build, go test -race).\n\n---\n\n## π Deployment\n\nVercel hosts the Next.js web adapter, but cannot host the Temporal Worker or Go consumer. A full deployment needs:\n\n| Component | Where |\n|---|---|\n| Next.js | Vercel |\n| Temporal Worker | Container service |\n| Go Telemetry | Container service |\n| PostgreSQL | Managed service |\n| Redis | Managed service |\n| Kafka | Managed service |\n| Temporal | Temporal Cloud or self-hosted |\n\nSee [docs/architecture.md](docs/architecture.md) and [docs/deployment.md](docs/deployment.md) for full deployment guides.\n\n---\n\n## π€ Contributing\n\nBug reports and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the local setup and CI checks. Vulnerabilities go through [SECURITY.md](SECURITY.md), privately, rather than a public issue.\n\n---\n\n## π License\n\nMIT. See [LICENSE](LICENSE).\n"}
![\"Loom\"](\"public/brand/loom-logo.svg\"){kind=logo}
\n![\"CI\"](\"https://github.com/D3S-Gaurav/Loom/actions/workflows/ci.yml/badge.svg\"){kind=icon}
\n ![\"MIT](\"https://img.shields.io/badge/license-MIT-blue.svg\"){kind=icon}
\n ![\"Contributions](\"https://img.shields.io/badge/PRs-welcome-brightgreen.svg\"){kind=icon}
\n