the-saurus

The Saurus 🦕

Feed The Saurus your papers.

A literature review pipeline that devours scientific papers and produces comprehensive, citation-backed reviews. Upload a corpus of PDFs, watch the pipeline process them in parallel, get a structured literature review with thematic analysis and traceable citations, then chat with your papers through an embedded conversational assistant.

What It Does

Pipeline

1. Upload — Drop your scientific PDFs
2. Analyze — AI agents extract themes and claims from each paper (parallel, stateless)
3. Deduplicate — Semantically equivalent themes are merged across papers
4. Review — Deep thematic analysis synthesizes findings across papers
5. Generate — Cohesive literature review with inline citations [N](p.X,§Y)

Every claim is traceable to its source: paper, page, and paragraph.

Conversational Assistant

1. Connect — Embedded chat panel served as a federated UI (Module Federation 2.0)
2. Query — Ask questions about your papers grounded in the actual pipeline outputs
3. Retrieve — MCP tools fetch themes, claims, reviews, and citations from Qdrant
4. Answer — Agno Team (GPT-4o-mini) synthesizes a response with evidence from your corpus

The assistant only knows what the pipeline extracted, keeping answers traceable and hallucination-resistant.

Both services are covered by unit tests, end-to-end test clients, LLM evaluation suites (RAGAS + DeepEval with prompt regression testing), and production observability (Langfuse tracing with automated scoring). Pipeline stages run as durable workflows via Restate, with Terraform and Helm definitions for production deployment on AWS/Kubernetes.

Prerequisites

• Python 3.13+
• Node.js 20+ with pnpm
• uv (Python package manager)
• Docker (for Qdrant, optional)
• Google AI Studio API key (get one here)

Setup

git clone https://github.com/daviguides/the-saurus.git
cd the-saurus
make setup

Copy environment files and set your API key:

cp pipeline/.env.example pipeline/.env
# Edit pipeline/.env and set PIPELINE_LLM_API_KEY

cp assistant-ws/.env.example assistant-ws/.env
# Edit assistant-ws/.env and set OPENAI_API_KEY (for the chat assistant)

Running

Core (pipeline + app)

The minimum to upload PDFs and generate a literature review:

# Terminal 1: Pipeline API (port 8002)
make dev-pipeline

# Terminal 2: React app (port 5173)
make dev-app

# Terminal 3 (optional): Restate durable execution server
make dev-restate
make register-restate  # one-time registration

Open http://localhost:5173 and upload PDFs.

Conversational assistant (optional)

Adds an embedded chat that can answer questions about your papers using MCP tools over Qdrant:

# Terminal 3: Qdrant vector database (port 6333)
make dev-qdrant

# Terminal 4: MCP server for RAG queries (port 8012)
make dev-mcp

# Terminal 5: Chat WebSocket backend (port 8001)
make dev-ws

# Terminal 6: Chat UI federated module (port 5174)
make dev-ui

All services at once (Docker Compose)

make up      # Start all services
make down    # Stop all services

Architecture

┌─────────────────────────────────────────────────────────────────┐
│  app (React 19)                              assistant-ui (MF)  │
│  Upload → Pipeline Trace → Review              Chat Drawer     │
├─────────────────────────────────────────────────────────────────┤
│  pipeline (FastAPI+Restate)  │  assistant-ws    │  papers-mcp    │
│  POST /jobs → run_pipeline   │  Socket.IO chat  │  Qdrant RAG    │
│  WS /jobs/{id}/stream        │  Agno Team       │  FastMCP       │
├─────────────────────────────────────────────────────────────────┤
│  Gemini 2.5 Flash (LLM)     │  Qdrant (vectors) │  YAML/NDJSON  │
└─────────────────────────────────────────────────────────────────┘

Pipeline

Stage	Agent	Parallelism	Description
Paper Analysis	PaperAnalyzerAgent	Per paper	Extract themes + claims in one pass
Theme Dedup	ThemeDedupAgent	Sequential	Merge semantic synonyms across papers
Theme Review	ThemeReviewerAgent	Batched (5/call)	Synthesize claims per theme
Aggregation	AggregatorAgent	Sequential	Cohesive review with citations

All agents use the Agno framework with Pydantic structured output and streaming events. LLM concurrency is controlled by a shared semaphore (configurable via PIPELINE_LLM_MAX_CONCURRENT).

Tech Stack

Layer	Technology
Frontend	React 19, TypeScript, Rsbuild, Tailwind v4, Module Federation 2.0
Pipeline	Python 3.13+, FastAPI, Restate, Agno, Gemini 2.5 Flash
Assistant	Agno Team, Socket.IO, GPT-4o-mini
RAG	Qdrant, Gemini text-embedding-004, FastMCP
Observability	Langfuse, ClickHouse, OpenLIT
Infrastructure	Terraform (AWS), Helm (Kubernetes), Docker Compose
Persistence	YAML (state), NDJSON (events), filesystem
Design	Literata + Inter + Fira Code, light/dark themes

Project Structure

the-saurus/
├── app/                    # React desktop app (upload, trace, review)
├── assistant-ui/           # Federated chat UI (Module Federation remote)
├── assistant-ws/           # Chat WebSocket backend (Agno Team)
├── papers-mcp/             # MCP server (6 tools over Qdrant)
├── pipeline/               # Core pipeline engine (multi-agent orchestrator)
├── evals/                  # Evaluation suite (RAGAS + DeepEval + Langfuse)
├── infra/
│   ├── terraform/          # AWS infrastructure (EKS, Aurora, ElastiCache, S3)
│   └── helm/the-saurus/    # Kubernetes deployment (all services + Langfuse + Restate)
├── pipeline-test-client/   # CLI for end-to-end pipeline testing
├── assistant-test-client/  # CLI for end-to-end assistant testing
├── shared/                 # Design tokens (CSS variables, light + dark)
├── Makefile                # Dev commands
└── docker-compose.yml

Configuration

Service	Env file	Key variables
Pipeline	pipeline/.env	PIPELINE_LLM_API_KEY, PIPELINE_LLM_MAX_CONCURRENT (default 2, set 10 for paid tier)
Assistant	assistant-ws/.env	OPENAI_API_KEY, AT_LLM_PROVIDER (openai/anthropic)
MCP	papers-mcp/.env	MCP_QDRANT_URL (default localhost:6333)

See each service's .env.example for all available settings.

Testing

Unit Tests

Each backend service has its own pytest suite covering agents, orchestration, persistence, API routes, and schemas. No running services or API keys required.

Service	Tests	Command
Pipeline	219	cd pipeline && uv run pytest tests/ -v
Assistant WS	56	cd assistant-ws && uv run pytest tests/ -v
Pipeline Test Client	18	cd pipeline-test-client && uv run pytest tests/ -v
Assistant Test Client	16	cd assistant-test-client && uv run pytest tests/ -v

make test    # Run all unit test suites
make lint    # Lint all Python services

End-to-End Test Clients

Two standalone CLI tools that run against live services (local or remote). They exercise the full stack: HTTP uploads, WebSocket event streaming, Socket.IO chat, LLM calls, and Qdrant queries. Useful for validating deployments, diagnosing production issues, or letting AI coding agents (like Claude Code) verify changes autonomously.

Pipeline Test Client (pipeline-test-client/): Tests the full pipeline flow over HTTP + WebSocket.

cd pipeline-test-client

# Upload a PDF and watch the pipeline process it end-to-end
uv run pipeline-test run test-cases/papers/exercise_cognitive_function_brain.pdf

# Run a YAML test case with assertions (paper count, theme count, citations)
uv run pipeline-test test-flow test-cases/cases/basic_flow.yaml

# Individual commands
uv run pipeline-test upload <pdf>        # Upload PDFs, get job_id
uv run pipeline-test stream <job_id>     # Watch live agent events
uv run pipeline-test review <job_id>     # Show generated literature review
uv run pipeline-test papers <job_id>     # Show extracted themes and claims

Assistant Test Client (assistant-test-client/): Tests the conversational assistant over Socket.IO.

cd assistant-test-client

# Ask a question about the processed papers
uv run assistant-test ask "What are the main themes found across all papers?"

# Interactive chat mode
uv run assistant-test chat

# Run a YAML test case with assertions (content, tools called, step count)
uv run assistant-test test-flow test-cases/cases/theme_query.yaml

Logged Mode (for AI agents)

Services can run in background with logs piped to files, allowing AI coding agents to start services, run test clients, and read logs to diagnose issues autonomously:

make log-pipeline    # Start pipeline, logs to logs/pipeline.log
make log-ws          # Start assistant, logs to logs/assistant-ws.log
make log-mcp         # Start MCP server, logs to logs/papers-mcp.log
make log-core        # Pipeline + app
make log-all         # All services
make log-stop        # Stop all logged services

Agno debug_mode is enabled by default, so logs include full prompts sent to the LLM, structured output schemas, streaming events, and response content.

Evaluation and Observability

Both the pipeline and assistant services have dedicated evaluation suites and observability instrumentation, housed in the evals/ workspace.

Observability (Langfuse)

LLM calls from both services are traced via Langfuse (self-hosted), capturing prompts, completions, token usage, latency, and agent lifecycle events. Agno agent calls are auto-instrumented via OpenLIT + OpenTelemetry.

make eval-langfuse       # Start Langfuse locally (Docker)
# Open http://localhost:3000, create account, copy API keys to evals/.env

Service	What is traced	LLM
Pipeline	4 agents (PaperAnalyzer, ThemeDedup, ThemeReviewer, Aggregator), structured output, streaming events	Gemini 2.5 Flash
Assistant	Agno Team coordinator, MCP tool calls to Qdrant, conversation turns	GPT-4o-mini

Evaluation (RAGAS + DeepEval)

RAGAS generates synthetic test datasets from the golden PDFs and provides RAG-specific metrics. DeepEval runs pytest-native assertions against pipeline and assistant outputs, with Gemini as the LLM judge.

Pipeline evals: faithfulness (claims cite sources), citation accuracy ([N] refs resolve to real papers), theme quality (meaningful, non-redundant), schema completeness (structured output has all fields), safety (bias, toxicity, hallucination).

Assistant evals: answer relevancy (grounded in pipeline data), tool correctness (right MCP tool selected), safety.

Prompt Regression Workflow

# 1. Run the pipeline against golden test PDFs
make eval-run-pipeline

# 2. Evaluate the output
make eval-pipeline

# 3. If scores are good after a prompt change, update the baseline
make eval-update-baseline

A CI workflow (.github/workflows/eval-regression.yml) runs evals automatically on PRs that touch prompts or agent code.

Production Scoring

Score a sample of production traces (10% by default) with RAGAS metrics, pushing results back to Langfuse for dashboarding:

make eval-score-pipeline     # Score pipeline traces
make eval-score-assistant    # Score assistant traces

Durable Execution (Restate)

Pipeline stages are orchestrated through Restate, a Rust-based durable execution engine. Each stage (paper analysis, theme dedup, theme review, aggregation) is journaled. If the server crashes mid-pipeline, Restate replays the journal on restart, skipping completed stages and resuming from where it left off.

POST /jobs → Restate ingress → PipelineWorkflow/{job_id}/run
                                  ├── paper_analysis  (journaled)
                                  ├── theme_dedup     (journaled)
                                  ├── theme_review    (journaled)
                                  └── aggregation     (journaled)

Restate is optional for local development. If the Restate server is not running, the pipeline falls back to in-process asyncio execution.

make dev-restate        # Start Restate server (Docker)
make register-restate   # Register pipeline workflow (one-time)
make stop-restate       # Stop Restate server

The Restate admin dashboard is available at http://localhost:9070.

Infrastructure

Production deployment is defined as infrastructure-as-code in the infra/ directory.

Terraform (AWS)

Provisions the cloud infrastructure across three environments (dev, staging, prod):

Module	Resource	Purpose
vpc	VPC + subnets	Network isolation (public/private, 2 AZs)
eks	EKS cluster	Kubernetes with managed node groups, KMS-encrypted secrets
aurora	Aurora Serverless v2	PostgreSQL for Langfuse (environment-aware scaling)
elasticache	Redis 7.x	Langfuse queue + Socket.IO horizontal scaling
s3	S3 buckets	Langfuse storage + pipeline artifacts

cd infra/terraform

# Deploy to an environment
terraform init -backend-config=environments/dev.tfvars
terraform plan -var-file=environments/dev.tfvars
terraform apply -var-file=environments/dev.tfvars

Helm (Kubernetes)

Deploys all services to an EKS cluster with autoscaling, health probes, and secrets management:

Component	Type	Scaling
Pipeline	Deployment	HPA 2-10 replicas (CPU 60%)
Assistant WS	Deployment	HPA 2-10 replicas (CPU 60%)
App	Deployment	2 replicas
Papers MCP	Deployment	2 replicas
Qdrant	StatefulSet	PVC (10Gi dev, 50Gi prod)
Restate	Deployment + PVC	Persistent journal (5Gi dev, 20Gi prod)
Langfuse	Web + Worker	2 replicas each
ClickHouse	StatefulSet	PVC (20Gi dev, 100Gi prod)

cd infra/helm/the-saurus

# Dev deployment
helm install the-saurus . -f values.yaml

# Production deployment
helm install the-saurus . -f values.yaml -f values-prod.yaml \
  --set secrets.pipelineLlmApiKey=$PIPELINE_LLM_API_KEY \
  --set secrets.langfuseDatabaseUrl=$LANGFUSE_DB_URL

Development

• Project Board — task tracking across initiatives
• Architecture Doc — system diagrams and design decisions

The Name

A thesaurus connects words that mean the same thing. This pipeline does exactly that for scientific themes across papers: "chronobiology" in one paper and "circadian rhythms" in another become a single canonical theme. The Saurus: part thesaurus, part dinosaur.

License

MIT