{"id":"24dd53e1-7227-479d-850e-270f6aaeec36","slug":"distiller-cm5-python","kind":"mcp","name":"distiller-cm5-python","summary":"","owner":"Pamir-AI","repoUrl":"https://github.com/Pamir-AI/distiller-cm5-python","license":"Apache-2.0","licenseTier":"permissive","trust":"community","stars":7,"match":null,"fewStars":false,"format":"mcp","createdAt":"2025-04-17T04:33:01.000Z","addedAt":"2026-10-21T15:00:55.930Z","logoUrl":"https://avatars.githubusercontent.com/u/169114025?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["utilities"],"tags":[],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-26T22:52:37.709Z","sample":false,"url":"https://conduits.dev/tools/distiller-cm5-python","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Distiller CM5 Python Framework\n\nThis repository contains the Python implementation for the Distiller CM5 project, featuring a client-server architecture based on the Model Context Protocal (MCP) for interacting with Large Language Models (LLMs).\n\n## Overview\n\nThe framework consists of several key components:\n\n*   **Client (`distiller_cm5_python/client/`)**: Provides user interfaces (both GUI and CLI) to interact with an LLM backend via an MCP server. Includes features like optional local LLM server management, tool usage facilitation, streaming responses, voice input, and E-Ink display support. (See `client/README.md` for details)\n*   **LLM Server (`distiller_cm5_python/llm_server/`)**: A FastAPI server that wraps local LLMs (using `llama-cpp-python`) and exposes an OpenAI-compatible API endpoint for chat completions, model management, and caching. (See `llm_server/README.md` for details)\n*   **MCP Server (`distiller_cm5_python/mcp_server/`)**: Contains example MCP server implementations that expose specific functionalities (like Text-to-Speech or WiFi control) as tools callable by the MCP client via the protocol. (See `mcp_server/README.md` for details)\n*   **Utilities (`distiller_cm5_python/utils/`)**: Shared modules for configuration management (`config.py`), logging (`logger.py`), and custom exceptions (`distiller_exception.py`). (See `utils/README.md` for details)\n*   **SDK (`distiller_cm5_sdk`)**: An external, installable package containing reusable components like Whisper (ASR) and Piper (TTS) wrappers. Used by components like the client UI (`client/ui/App.py`) and the talk server (`mcp_server/talk_server.py`). See [Pamir-AI/distiller-cm5-sdk](https://github.com/Pamir-AI/distiller-cm5-sdk/tree/main) for details.\n\nThe primary user entry point is `main.py` in the project root.\n\n## Installation and Setup\n\n1.  **Install `uv` (Recommended Virtual Environment and Package Manager):**\n    ```bash\n    curl -LsSf https://astral.sh/uv/install.sh | sh\n    source $HOME/.cargo/env # Or add uv to your PATH manually\n    ```\n\n2.  **Create and Activate Virtual Environment:**\n    ```bash\n    # Navigate to the project root directory\n    cd /path/to/distiller-cm5-python\n    uv venv # Creates a .venv directory\n    source .venv/bin/activate # Or `.\\.venv\\Scripts\\activate` on Windows\n    ```\n\n3.  **Install Dependencies:**\n    ```bash\n    # Install this project's requirements AND the external SDK\n    uv pip install -r requirements.txt # Ensure you have a requirements.txt file\n    uv pip install distiller-cm5-sdk   # Or install from a specific source if needed\n\n    # Example manual installs if requirements.txt is missing:\n    # uv pip install fastapi uvicorn \"llama-cpp-python[server]\" jinja2 pydantic PyQt6 QScintilla PyOpenGL mcp nest_asyncio ...\n    # uv pip install distiller-cm5-sdk\n    ```\n    *Note: Installing `llama-cpp-python` might require C++ build tools. Refer to its documentation.*\n    *Note: Installing `PyQt6` might have specific system dependencies.*\n    *Note: `distiller-cm5-sdk` might have its own dependencies (like `portaudio` for `PyAudio`) - refer to its [README](https://github.com/Pamir-AI/distiller-cm5-sdk/tree/main).*\n\n4.  **Download LLM Models (for Local LLM Server):**\n    Place your desired GGUF-format models into the `distiller_cm5_python/llm_server/models/` directory.\n\n5.  **Download SDK Models/Executables (if needed):**\n    Refer to the [distiller-cm5-sdk README](https://github.com/Pamir-AI/distiller-cm5-sdk/tree/main) for instructions on downloading necessary model files (e.g., for Whisper and Piper) or executables required by the SDK components.\n\n6.  **Configuration:**\n    *   Copy `distiller_cm5_python/utils/default_config.json` to `mcp_config.json` in the project root (or your working directory) and customize settings like `PROVIDER_TYPE`, `SERVER_URL`, `MODEL_NAME`, etc.\n    *   Alternatively, set environment variables (e.g., `export SERVER_URL=http://...`). Configuration loaded by `utils/config.py` prioritizes environment variables, then `mcp_config.json`, then `default_config.json`.\n\n## Running the Application\n\n### 1. LLM Backend\n\nYou need an LLM backend accessible. Choose one:\n\n*   **Option A: Run the Local LLM Server:**\n    ```bash\n    # Make sure environment is activated\n    python -m distiller_cm5_python.llm_server.server --model-name your_model.gguf\n    ```\n    Ensure the `SERVER_URL` in your configuration points to this server (e.g., `http://127.0.0.1:8000`).\n\n*   **Option B: Use OpenRouter or other external API:**\n    Set `PROVIDER_TYPE` to `openrouter` (or similar identifier configured in `client/mid_layer/llm_client.py`) in your configuration. Ensure `OPENROUTER_API_KEY` (or equivalent environment variable) is set. The `SERVER_URL` should point to the OpenRouter API endpoint (e.g., `https://openrouter.ai/api/v1`). OpenRouter provides access to various models including OpenAI, Anthropic, Google, etc.\n\n*   **Option C: Use Llama.cpp directly (Managed by Client):**\n    Set `PROVIDER_TYPE` to `llama-cpp` in your configuration. The `main.py` script will attempt to start/stop a `llama.cpp` server automatically if one isn't detected at the configured `SERVER_URL`. This requires `llama.cpp` to be compiled and the path to its `server` executable might need configuration (check `client/llm_infra/llama_manager.py` and potentially config).\n\n### 2. MCP Tool Server (Optional)\n\nIf you want the client to use tools provided by an MCP server (like TTS or WiFi):\n\n```bash\n# Example: Run the talk server (uses distiller_cm5_sdk for TTS)\npython -m distiller_cm5_python.mcp_server.talk_server\n# Or the wifi server\n# python -m distiller_cm5_python.mcp_server.wifi_server\n```\nNote the path to the *running server script* will be needed when launching the client (`--server-script` argument).\n\n### 3. Client (UI or CLI)\n\nThe main entry point handles launching the client.\n\n*   **Run the GUI Client:**\n    ```bash\n    python main.py --gui [--server-script /path/to/running/mcp_server.py] [OTHER_OPTIONS]\n    ```\n    *   The `--gui` flag is essential.\n    *   Uses `distiller_cm5_sdk` for features like voice input.\n    *   Use `--server-script` if you want to connect to a running MCP tool server (like `talk_server.py`).\n\n*   **Run the CLI Client:**\n    ```bash\n    python main.py [--server-script /path/to/running/mcp_server.py] [OTHER_OPTIONS]\n    ```\n    *   Omit the `--gui` flag.\n    *   Use `--server-script` if needed.\n\n*   **Optional Llama.cpp Management:** If `PROVIDER_TYPE` is `llama-cpp`, `main.py` will check the `SERVER_URL` before starting the client. If no server is running, it will attempt to launch one (requires `llama.cpp` server binary available and configured). It will also attempt to shut down this managed server on exit.\n\nRefer to `python main.py --help` for all client launch options.\n\n"}