{"id":"3aeb54b9-9a33-4b86-b342-84dc181b33d7","slug":"tt-distill","kind":"mcp","name":"TT-Distill","summary":"Continuous Neuro-Symbolic Routing via O(1) Latent Algebraic Swapping","owner":"daemind","repoUrl":"https://github.com/daemind/TT-Distill","license":"Apache-2.0","licenseTier":"permissive","trust":"community","stars":0,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-03-07T22:11:29.000Z","addedAt":"2026-10-21T08:29:59.948Z","logoUrl":"https://avatars.githubusercontent.com/u/7912881?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["utilities"],"tags":["apple-silicon","dora","ggml","llama-cpp","llm","mechanistic-interpretability","metal","neuro-symbolic"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T18:22:06.882Z","sample":false,"url":"https://conduits.dev/tools/tt-distill","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# TT-Distill: Continuous Neuro-Symbolic Routing via O(1) Latent Algebraic Swapping\n\n![TT-Distill Architecture](assets/hero.png)\n\n*(State of the Art as of March 2026)*\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.18905253.svg)](https://doi.org/10.5281/zenodo.18905253)\n[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)\n\n**Repository:** [github.com/daemind/TT-Distill](https://github.com/daemind/TT-Distill)  \n**DOI:** [10.5281/zenodo.18905253](https://doi.org/10.5281/zenodo.18905253)\n\n## Abstract\n\nAs of early 2026, the AI industry faces an architectural dead-end for real-time autonomous systems: the reliance on massive Test-Time Compute (like Monte Carlo Tree Search) over monolithic models creates prohibitive latency and massive energy waste. **TT-Distill** introduces a radical paradigm shift: migrating intelligence from probabilistic token prediction to algebraic latent projection on Edge hardware.\n\nBy decoupling the reasoning process (System 2) from reflexive execution (System 1), TT-Distill acts as a continuous topological router. It discovers mathematical invariants in abstract reasoning tasks and crystallizes them into ultra-lightweight DoRA (Weight-Decomposed Low-Rank Adaptation) modules. Powered by a custom Zero-Copy Apple Metal C++ backend, the architecture achieves $O(1)$ cognitive context switching in 0.0002 milliseconds. TT-Distill transforms AI from a statistical text generator into a highly optimized, deterministic Tri-Engine (Heuristic, Pure Math, Latent Algebraic) capable of executing transductive reasoning at the microsecond scale.\n\n---\n\n## ⚡ Quick Start: The Cognitive Cockpit\n\nTo reproduce the $O(1)$ hardware swap benchmarks, you must compile the custom Metal backend:\n\n```bash\n# 1. Clone and prepare environment\ngit clone https://github.com/daemind/TT-Distill.git\ncd TT-Distill\n\n# 2. Compile the custom O(1) Metal backend\ncd llama.cpp\ncmake -S ggml -B build -DGGML_METAL=ON -DBUILD_SHARED_LIBS=ON\ncmake --build build -j $(sysctl -n hw.ncpu)\ncd ..\n\n# 3. Launch Demo 7 (O(1) Hot-Swap Benchmark)\nexport GGML_METAL_DYLIB=$(pwd)/llama.cpp/build/src/ggml-metal/libggml-metal.dylib\npython3 demos/main.py --demo 7\n```\n\n---\n\n## 1. Theoretical Foundations: The End of the Linguistic Illusion\n\nCurrent LLMs process prompts and generate reasoning step-by-step in natural language, which is computationally expensive and mathematically flawed. TT-Distill posits that language is merely a projection of deeper topological spaces.\n\n### Transductive Algebraic Learning\nInstead of prompting a model to \"think,\" TT-Distill modifies the geometry of the model's latent space on the fly. When confronting a novel problem, the system determines the underlying mathematical structure (e.g., Boolean Lattices, Affine Spaces, Homology) and injects the corresponding mathematical laws directly into the weights via DoRA. Intelligence becomes a geometric projection ($W_{new} = W_0 + \\sum g_i(B_i A_i)$) rather than a probabilistic guess.\n\n---\n\n## 2. Technical Architecture: The Tri-Engine System\n\n### Pillar 1: The Topological Synthesizer (System 2)\nInstead of textual Chain-of-Thought, this module uses Set Theory and Mathematical Morphology to parse reality.\n* It evaluates environments as discrete topological spaces.\n* It synthesizes deterministic algebraic equations (Unions, Intersections, Vectorial Translations) that describe the transformation required to solve a problem.\n* Once verified in an isolated Docker sandbox, the solution is distilled into a permanent, reusable physical reflex (DoRA).\n\n### Pillar 2: The Latent Algebraic Router (System 1)\nA highly optimized **Qwen 2 Encoder** acting as the execution engine. By leveraging a standard, robust dense Transformer architecture, TT-Distill proves that deterministic efficiency relies on the routing framework, not exotic base models.\n* **Continuous Mixture of Adapters (MoA):** The system dynamically routes input through a dictionary of distilled mathematical \"instincts\", instantly altering the Qwen 2 attention heads via DoRA.\n* **Bypassing the Tokenizer:** Utilizing direct tensor injection (`inputs_embeds`), TT-Distill communicates via pure latent representation, eliminating the discretization bottleneck of standard NLP pipelines and pushing the Qwen 2 encoder to its absolute physical limits.\n\n### Pillar 3: The Hardware Hack — Zero-Copy VRAM Swapping\nThe most significant engineering breakthrough of TT-Distill is the destruction of the VRAM memory reallocation latency wall.\n* Implemented directly within the `ggml-metal.m` C++ backend.\n* Utilizes a pre-allocated 256 MB Ring Buffer leveraging Apple Silicon's Unified Memory (`MTLResourceStorageModeShared`).\n* Eliminates Metal compute graph recreation, reducing the Hot-Swap latency of neural weights from 215 ms down to 0.0002 ms ($O(1)$ pointer swap).\n\n### Pillar 4: Cross-Platform Portability (Universal O(1))\nWhile the reference implementation is optimized for Apple Silicon Unified Memory, the TT-Distill protocol is designed for cross-platform hardware-agnostic acceleration:\n*   **Linux/Unix/Windows Staging:** The architecture supports O(1) swap on discrete GPUs via Host-Mapped Zero-Copy memory.\n*   **Technical Spec:** Detailed porting instructions for CUDA/Vulkan/DirectX can be found in the [Universal Zero-Copy Swap](plans/universal_zero_copy_swap.md) specification.\n\n---\n\n## 3. Empirical Performance and Benchmarks\n\nThe TT-Distill architecture was benchmarked against highly complex, abstract visual-spatial reasoning grids designed to resist standard LLM pattern matching. The results validate the Tri-Engine architecture on consumer-grade Apple Silicon:\n\n| Cognitive Mode | Execution Method | Median Swap Latency | Total Inference Latency |\n| :--- | :--- | :--- | :--- |\n| **Heuristic Logic** | Rule-based symbolic fallback | 0.154 ms | ~35.5 ms |\n| **Pure Mathematics** | Direct tensor math execution | N/A | ~6.35 ms |\n| **Latent Math (O(1))** | Dynamic DoRA Algebraic routing | **0.0002 ms** | **~11.64 ms** |\n\n> **Note:** The context switch between two entirely different mathematical paradigms (e.g., shifting from a Vector Space logic to a Galois Field logic) occurs in 208 nanoseconds, making the cognitive shift physically transparent to the inference loop.\n\n---\n\n## 4. Conclusion: Real-Time Neuro-Symbolic AGI\n\nTT-Distill demonstrates that the future of autonomous agents does not lie in scaling parameter counts to the trillions, nor in brute-forcing compute at test-time. It lies in algebraic compositionality and system-level memory routing.\n\nBy uniting rigorous mathematical synthesis with microsecond-level C++ hardware optimization, TT-Distill allows a lightweight Edge model to behave like a continuous dynamical system. It proves that true intelligence is the ability to instantaneously forge and swap the geometric lenses through which a neural network perceives reality.\n\n---\n\n## 5. New Features: DoRA Blending Engine (Phase 1-3)\n\nThe `dora_blender` branch introduces a complete neuro-endocrine system for automatic weight optimization:\n\n### Phase 1: Le Moteur de Fusion (DoRA Blending Engine)\n- **[`DoraBlender`](src/orchestration/dora_blender.py:63)**: CPU-based adapter blending with triple-buffered ring buffer to prevent GC-induced segfaults\n- **Blending Modes**: GEOMETRIC (weighted linear combination), TILING (spatial partitioning), HYBRID (combined)\n- **MCP Tools**: [`blend_adapters_manifold()`](src/orchestration/mcp_intelligence_manifold.py:204) for cocktail synaptique fusion\n\n### Phase 2: Le Tampon de Trajectoire Latente\n- **[`LatentTrajectoryBuffer`](src/orchestration/latent_trajectory.py:65)**: Short-term memory for tracking latent vectors with automatic tensor detachment to prevent VRAM leaks\n- **[`TrajectoryStore`](src/persistence/trajectory_store.py:22)**: JSON-based persistence for cross-session analysis\n\n### Phase 3: La Boucle d'Auto-Distillation\n- **[`AutoDistiller`](src/orchestration/auto_distiller.py:51)**: Neuro-endocrine feedback loop that adjusts weights on failure and crystallizes successful configurations\n- **MCP Tool**: [`crystallize_weights()`](src/orchestration/mcp_intelligence_manifold.py:267) for saving permanent instincts\n\n### Architecture Diagram\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                    DoRA Blending Pipeline                       │\n├─────────────────────────────────────────────────────────────────┤\n│  [Expert A, Expert B, ...] + [w1, w2, ...]                      │\n│           ↓                                                     │\n│  DoraBlender.blend_adapters()  (CPU numpy)                      │\n│           ↓                                                     │\n│  Ring Buffer Serialization (triple-buffered)                    │\n│           ↓                                                     │\n│  Metal preload() + O(1) swap()  (< 0.001ms)                     │\n│           ↓                                                     │\n│  GPU uses fused weights                                         │\n└─────────────────────────────────────────────────────────────────┘\n\n┌─────────────────────────────────────────────────────────────────┐\n│                  Auto-Distillation Loop                         │\n├─────────────────────────────────────────────────────────────────┤\n│  Solver attempts task → Success? ──No──→ Analyze trajectory     │\n│         ↓ Yes                                                    │\n│  Crystallize → Save .bin file → Register in manifold            │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n### Performance Targets\n\n| Metric | Target | Notes |\n|--------|--------|-------|\n| Metal Swap Only | < 0.001 ms | O(1) pointer swap |\n| Blend + Serialize | < 25 ms | CPU numpy operations |\n| Trajectory Buffer | O(1) append | FIFO eviction |\n| Crystallization | < 100 ms | Disk I/O bound |\n\n### Usage Example\n\n```python\nfrom src.orchestration.dora_blender import DoraBlender, BlendingMode\nfrom src.orchestration.auto_distiller import AutoDistiller\nfrom src.persistence.trajectory_store import TrajectoryStore\nfrom src.orchestration.latent_trajectory import LatentTrajectoryBuffer\n\n# Initialize blending engine\nblender = DoraBlender(blending_mode=BlendingMode.GEOMETRIC)\n\n# Blend adapters\ntimings = blender.blend_and_swap(\n    adapters=[\"expert_a.bin\", \"expert_b.bin\"],\n    gating_vector=[0.7, 0.3]\n)\n# timings = {\"merge_ms\": 12.5, \"swap_ms\": 0.0002, \"total_ms\": 12.51}\n\n# Initialize auto-distiller\ntrajectory_store = TrajectoryStore(\"data/trajectories\")\ndistiller = AutoDistiller(\n    blender=blender,\n    trajectory_buffer=LatentTrajectoryBuffer(),\n    trajectory_store=trajectory_store,\n    max_attempts=5\n)\n\n# Run distillation loop\nresult = distiller.distill(\n    task_data=arc_task,\n    initial_weights={\"DihedralGroup\": 0.5, \"ColorField\": 0.5},\n    solver_fn=my_solver\n)\n\n# Crystallize successful configuration\nif result.success:\n    print(f\"Crystallized to: {result.crystallized_path}\")\n```\n\n### Tests\n\nAll components are fully tested:\n- [`test_dora_blender.py`](tests/test_dora_blender.py): 25 tests for blending engine\n- [`test_latent_trajectory.py`](tests/test_latent_trajectory.py): 20 tests for trajectory buffer\n- [`test_auto_distiller.py`](tests/test_auto_distiller.py): 20 tests for distillation loop\n- [`test_auto_distillation_loop.py`](tests/test_auto_distillation_loop.py): 12 integration tests\n\nRun tests: `pytest tests/test_dora_blender.py tests/test_latent_trajectory.py tests/test_auto_distiller.py tests/test_auto_distillation_loop.py -v`\n"}