{"id":"94ba5456-3198-4229-98f9-a75055e10364","slug":"btech-software-agent-lab","kind":"mcp","name":"agent-lab","summary":"Build, simulate and test LLM agents. A cloud-native Python framework on FastAPI, LangGraph and PostgreSQL.","owner":"btech-software","repoUrl":"https://github.com/btech-software/agent-lab","license":"MIT","licenseTier":"permissive","trust":"community","stars":4,"match":null,"fewStars":true,"format":"mcp","createdAt":"2025-01-06T23:23:32.000Z","addedAt":"2026-10-21T14:39:05.458Z","logoUrl":"https://avatars.githubusercontent.com/u/295186131?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["databases","developer-tools"],"tags":[],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-25T23:11:22.199Z","sample":false,"url":"https://conduits.dev/tools/btech-software-agent-lab","language":"Jupyter Notebook","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"\nCloud-native framework for building, simulating and testing LLM agents
\n\n\n\n[](https://pypi.org/project/btech-agent-lab/)\n[](https://github.com/btech-software/agent-lab/actions/workflows/build.yml)\n[](https://sonarcloud.io/dashboard?id=btech-software_agent-lab)\n[](https://sonarcloud.io/component_measures?metric=coverage&selected=btech-software_agent-lab%3Aapp&id=btech-software_agent-lab)\n[](https://www.python.org/downloads/)\n[](#key-features)\n[](https://github.com/btech-software/agent-lab/blob/main/LICENSE)\n\n
\n\n---\n\n### Table of Contents\n- [What is Agent-Lab?](#what-is-agent-lab)\n- [Install](#install)\n- [Quickstart](#quickstart)\n- [How it works](#how-it-works)\n- [Project Principles](#project-principles)\n- [Key Features](#key-features)\n- [MCP Server](#mcp-server)\n- [Reference Implementation](#reference-implementation)\n- [Documentation](#documentation)\n- [Contributing](#contributing)\n- [License](#license)\n\n---\n\n## What is Agent-Lab?\n\n**Agent-Lab is a cloud-native Python framework for building, simulating and testing LLM agents on top of FastAPI, LangChain/LangGraph and PostgreSQL.**\n\nYou `pip install` it as a library, define your agents by extending its base classes, and assemble a production-ready FastAPI application with a single `create_app()` call β no need to fork or edit the framework. Everything an agent needs is included: a REST API, an MCP server, relational and vector persistence, agent memory (LangGraph checkpoints), OpenTelemetry observability, and a batteries-included simulation and testing harness with LLM-as-judge evaluation.\n\nAgent-Lab covers three things well:\n\n- **ποΈ Build** β Define agents by subclassing `WorkflowAgentBase` (LangGraph state graphs) and registering them with a decorator. `create_app()` wires up the REST API, MCP server, persistence, auth and observability around them. Extend via subclassing and configuration, never by editing framework code.\n- **π§ͺ Simulate** β Exercise agents end-to-end against live LLMs with conversational scenarios (a user simulator + judge agent) and dataset-driven experiments, scored by an LLM-as-judge and tracked over time in [Langfuse](https://langfuse.com/) and [LangWatch](https://langwatch.ai/).\n- **β
Test** β Validate every change with an integration suite that spins up real infrastructure (Postgres, Redis, Keycloak, embeddings, headless Chrome) via testcontainers.\n\nThe repository also ships a complete **reference implementation** β a standalone, deployable app with a set of ready-to-use agents (RAG, browser automation, voice memos, vision, multi-agent supervisors) β that doubles as a working example of how to consume the framework. See [Reference Implementation](#reference-implementation).\n\n---\n\n## Install\n\n```bash\npip install btech-agent-lab\n```\n\nRequires **Python 3.12+**. Agents are served through FastAPI and persist to PostgreSQL (relational data, pgvector embeddings, and LangGraph checkpoints); see the [Setup guide](https://github.com/btech-software/agent-lab/blob/main/doc/SETUP.md) for provisioning those in development or production.\n\n---\n\n## Quickstart\n\nScaffold a complete, runnable project from the bundled [project template](https://github.com/btech-software/agent-lab/tree/main/cookiecutter):\n\n```bash\nuvx cookiecutter gh:btech-software/agent-lab --directory cookiecutter\n```\n\nAnswer the prompts (project name, optional docker / GitHub Actions / testcontainers support) and you get a ready-made thin consumer of the framework:\n\n```\nmy-agent-app/\nβββ my_agent_app/\nβ βββ main.py # app = create_app(container=..., scan_packages=[...])\nβ βββ core/container.py # your DI container β add providers here\nβ βββ agents/ # example echo agent + single-node ReAct workflow agent\nβ βββ mcp/ # example MCP tool and prompt, exposed at /mcp\nβββ config-dev.yml / config-test.yml / config-docker.yml\nβββ tests/ # no-container smoke tests (+ optional testcontainers suite)\nβββ compose.yml # app + postgres (pgvector) + redis + vault\nβββ docker/, .github/, Makefile, pyproject.toml\n```\n\nRun it:\n\n```bash\ncd my-agent-app\nuv lock && uv sync --group dev --group test\nmake test_unit # boots the app with no infrastructure needed\ndocker compose up --build -d # full local stack\ncurl http://localhost:18000/status/liveness\n```\n\nYour agents are reachable over the REST API (`/agents`, `/messages`, ...), exposed through the MCP server at `/mcp`, and the interactive OpenAPI docs live at `/docs`.\n\nPrefer wiring the app by hand? Add `btech-agent-lab` to any project, subclass the agent base classes, register them with `@discoverable_agent`, and call `create_app()` β the [Developer's Guide](https://github.com/btech-software/agent-lab/blob/main/doc/DEV_GUIDE.md) walks through the manual path, configuration, the DI container, and routers.\n\n---\n\n## How it works\n\n**Tech stack:** Python 3.12, FastAPI, LangChain/LangGraph, PostgreSQL (3 databases: relational, vectors via pgvector, LangGraph checkpoints), Redis for pub/sub, Keycloak for auth, OpenTelemetry for observability.\n\n**Architecture** β Clean Architecture with Dependency Injection:\n- **Interface** β FastAPI routers + MCP server\n- **Services** β Business logic, agent implementations\n- **Domain** β SQLAlchemy models, repository interfaces\n- **Infrastructure** β DB, auth, metrics (OpenTelemetry)\n\n**The public API** β everything you need to consume Agent-Lab as a library is exported from the top-level `agent_lab` package and designed to be extended without editing framework code:\n\n| Export | Purpose |\n|---|---|\n| `create_app(...)` | Composition root β builds the FastAPI app from your agents, container, config and routers. |\n| `discoverable_agent` | Decorator that registers an `AgentBase` subclass under an `agent_type`. |\n| `AgentBase`, `WorkflowAgentBase`, `SupervisedWorkflowAgentBase`, `WebAgentBase`, `ContactSupportAgentBase` | Base classes to extend, from simple message handlers to multi-agent supervisors. |\n| `Container` | Subclass to add your own providers (services, MCP registrars, tracing backends). |\n| `ConfigSource`, `YamlConfigSource`, `VaultConfigSource` | Supply configuration from YAML, Vault, or a custom source. |\n| `RouterMount` | Mount extra FastAPI routers with their auth policy. |\n| `McpRegistrar` | Extend the MCP server with additional tools. |\n\n**Agent system** β Agents extend `WorkflowAgentBase` (which uses LangGraph state graphs). The hierarchy goes from simple (echo, adaptive RAG) to complex (coordinator-planner-supervisor multi-agent). Agents process messages, have configurable settings (Jinja2-templated prompts), and persist state in the checkpoints database. Registering an agent with `@discoverable_agent` makes it discoverable, resolvable through the DI container, and valid at the API β no manual registry or schema edits required.\n\n---\n\n## Project Principles\n\n- Give researchers and developers a framework with everything they need to build, test, and experiment with LLM agents, plus ready-to-use reference implementations.\n- Extend through the public API β subclassing, decorators and configuration β never by editing framework code.\n- Expose an MCP server for agent discovery, conversation history, and agent-to-agent communication.\n- Ship with integration and simulation test suites so every agent change is validated automatically.\n- Provide full observability through logs, metrics, and traces for explainability and evaluation.\n- Run anywhere with a cloud-native architecture built for containerized deployment and horizontal scaling.\n\n---\n\n## Key Features\n\n- **Agent Framework**: Build agents by extending base classes and assembling the app with `create_app()`; discover them via package scanning or [entry points](https://github.com/btech-software/agent-lab/blob/main/doc/DEV_GUIDE.md).\n- **Agent Simulations**: Evaluate agent behavior end-to-end with `langwatch-scenario` conversational simulations and dataset-driven experiments, scored by an LLM-as-judge and tracked in [Langfuse](https://langfuse.com/) and [LangWatch](https://langwatch.ai/). See the [testing guide](https://github.com/btech-software/agent-lab/blob/main/doc/TESTS.md).\n- **Integration Testing**: Ensure reliability and correctness with a comprehensive [integration test suite](https://github.com/btech-software/agent-lab/blob/main/doc/TESTS.md) backed by testcontainers.\n- **REST API**: Manage integrations with AI suppliers, LLM settings, agents, and conversation histories through the built-in [REST API](https://github.com/btech-software/agent-lab/blob/main/doc/REST_API.md).\n- **MCP Server**: Utilize the [Model Context Protocol (MCP) server](https://github.com/btech-software/agent-lab/blob/main/doc/MCP.md) for agent discovery, dialog history, and agent-to-agent communication.\n- **Observability**: Obtain detailed insights through logs, metrics, and traces powered by [OpenTelemetry](https://github.com/btech-software/agent-lab/blob/main/doc/OTEL.md), with reference implementations for [Grafana](https://github.com/btech-software/agent-lab/blob/main/doc/otel/GRAFANA.md) and [OpenSearch Dashboards](https://github.com/btech-software/agent-lab/blob/main/doc/otel/OPENSEARCH.md).\n- **Relational Persistence**: Store data reliably using PostgreSQL to support the [entity domain model](https://github.com/btech-software/agent-lab/blob/main/doc/DOMAIN.md) for prompts, agent-specific settings, conversations, and more.\n- **Secrets Management**: Securely store and retrieve secrets with [Vault](https://github.com/btech-software/agent-lab/blob/main/doc/VAULT.md).\n- **Vector Storage and Search**: Efficiently manage vector data using PgVector for similarity search and retrieval.\n- **Agent Memory**: Use the PostgreSQL checkpointer to store and retrieve agent memory, enabling agents to maintain context across interactions.\n- **Cloud-Native**: Optimized for cloud environments with Docker, Kubernetes, and [Terraform scripts](https://github.com/btech-software/agent-lab/blob/main/doc/SETUP.md) for streamlined deployment.\n\n---\n\n## MCP Server\n\nAgent-Lab features a MCP Server that allows agent discovery (`get_agent_list` tool), dialog history (`get_message_list` tool) and agent-to-agent communication (`post_message` tool).\n\nThe following example shows MCP Server discovering and obtaining dialog history of a [supervised coder agent](https://github.com/btech-software/agent-lab/blob/main/notebooks/05_test_agent_type-multiagent-coder.ipynb) instance:\n\n\n\n\n\n
\n\nPlease refer to [MCP guide](https://github.com/btech-software/agent-lab/blob/main/doc/MCP.md) for more details.\n\n\n**Note**: Claude Desktop is used only for demonstration purposes. This project is not affiliated with Anthropic AI.\n\n---\n\n## Reference Implementation\n\nThe repository ships a complete, deployable reference application (`agent_lab.main`) that registers every built-in agent β RAG, browser automation, voice memos, vision, and multi-agent supervisors. It's both a product you can run and the canonical example of how to consume the framework.\n\nYou can run it in two ways:\n\n```bash\n# As a standalone app (the reference implementation)\nuvicorn agent_lab.main:app --reload\n```\n\n```bash\n# Or with the bundled observability stack\ndocker compose -f compose-grafana.yml up --build # Grafana\ndocker compose -f compose-opensearch.yml up --build # OpenSearch Dashboards\n```\n\nIt ships with Helm charts (`charts/`) for Kubernetes and Terraform scripts (`terraform/`) for provisioning cloud infrastructure β databases, auth realms, and secrets. See the [Setup guide](https://github.com/btech-software/agent-lab/blob/main/doc/SETUP.md) for details.\n\n> Agent-Lab is the **generic base** for downstream reference implementations. Only generic, reusable capabilities belong in the framework; product-specific agents and services live in the apps that consume it.\n\n---\n\n## Documentation\n\nThe complete documentation lives in [`doc/`](https://github.com/btech-software/agent-lab/blob/main/doc/README.md),\norganized around how you use Agent-Lab (in the spirit of the [DiΓ‘taxis](https://diataxis.fr/) framework):\n\n- **[Your first agent](https://github.com/btech-software/agent-lab/blob/main/doc/TUTORIAL.md)** β scaffold, define, call over REST, watch the trace, expose an MCP tool: a complete 15-minute tutorial.\n- **[Developer's Guide](https://github.com/btech-software/agent-lab/blob/main/doc/DEV_GUIDE.md)** β the flagship guide: two development models, configuration, DI, building agents, and every extension point.\n- **[Researcher's Guide](https://github.com/btech-software/agent-lab/blob/main/doc/RESEARCHER_GUIDE.md)** β notebooks, experiments, and prototyping new agent behaviors.\n- **[Testing & Simulations](https://github.com/btech-software/agent-lab/blob/main/doc/TESTS.md)** β the integration suite (testcontainers) and LLM-as-judge agent simulations.\n- **Reference** β [REST API](https://github.com/btech-software/agent-lab/blob/main/doc/REST_API.md) Β· [Configuration](https://github.com/btech-software/agent-lab/blob/main/doc/CONFIGURATION.md) Β· [Entity Domain Model](https://github.com/btech-software/agent-lab/blob/main/doc/DOMAIN.md).\n- **Build & Operate** β [MCP Server](https://github.com/btech-software/agent-lab/blob/main/doc/MCP.md) Β· [Observability](https://github.com/btech-software/agent-lab/blob/main/doc/OTEL.md) Β· [Deployment](https://github.com/btech-software/agent-lab/blob/main/doc/SETUP.md) Β· [Auth with Keycloak](https://github.com/btech-software/agent-lab/blob/main/doc/KEYCLOAK.md) Β· [Secrets with Vault](https://github.com/btech-software/agent-lab/blob/main/doc/VAULT.md).\n\n---\n\n## Contributing\n\nCommunity support is greatly appreciated. If you encounter any issues or have suggestions for enhancements, please report them by creating an issue on our [GitHub Issues](https://github.com/btech-software/agent-lab/issues) page.\n\nRefer to our [Developer's Guide](https://github.com/btech-software/agent-lab/blob/main/doc/DEV_GUIDE.md) for instructions on how to contribute to the project.\n\n---\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](https://github.com/btech-software/agent-lab/blob/main/LICENSE) file for details.\n"}