{"id":"c0c22632-55ad-4867-a12b-adf988a1d881","slug":"calibrated-copilot","kind":"mcp","name":"calibrated-copilot","summary":"Calibrated Copilot — a Gemini 3 financial agent that gives calibrated, risk-adjusted answers instead of confident guesses. Built with Google Agent Builder + MongoDB MCP. Quantifies its own confidence,","owner":"ryonzhang","repoUrl":"https://github.com/ryonzhang/calibrated-copilot","license":"MIT","licenseTier":"permissive","trust":"community","stars":1,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-06-10T12:38:59.000Z","addedAt":"2026-10-21T09:05:59.028Z","logoUrl":"https://avatars.githubusercontent.com/u/8466679?u=6fb05770f12fc98bc26f20082c5a1b358f22ed2c&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["databases","data-analytics"],"tags":["ai-agent","calibration","fintech","gemini","hackathon","mongodb","quant","risk-management"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T03:45:04.165Z","sample":false,"url":"https://conduits.dev/tools/calibrated-copilot","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Calibrated Copilot\n\n> A Gemini 3 financial agent that knows how sure it should be — calibrated, risk-adjusted answers instead of confident hallucinations.\n\n![Architecture](docs/architecture.png)\n\nBuilt for the **Google Cloud Rapid Agent Hackathon** (June 11, 2026). A **Gemini 3** agent (`gemini-3.1-pro-preview-customtools`) built with the **Google Agent Development Kit (ADK)** that integrates the **MongoDB Atlas MCP server** and gives calibrated, risk-adjusted financial guidance.\n\nBuilt by **Ruiyang Zhang** (passed all three CFA Program exams) — calibration & risk methodology is my professional focus. [github.com/ryonzhang](https://github.com/ryonzhang) · [ruiyang.co](https://ruiyang.co)\n\n---\n\n## The problem\n\nThe #1 failure of AI in finance is **overconfidence** — models state precise numbers they can't back, with the same false confidence whether the data is rich or thin. In finance, an overconfident wrong answer is worse than no answer.\n\n## The idea\n\nCalibrated Copilot does the opposite. For any portfolio question or trade idea it returns:\n\n1. **A calibrated probability with explicit confidence** — and states the basis for it.\n2. **A risk-adjusted recommendation** — position sizing proportional to confidence (fractional-Kelly intuition), with an exposure / drawdown sanity check.\n3. **An honest abstention** — *\"insufficient or conflicting evidence — I won't give a confident call\"* — when data is missing or signals disagree.\n4. **Citations for every number it used.** It never states a confident number it didn't retrieve.\n\nTrust through calibration, not bravado.\n\n---\n\n## Architecture\n\n```\n                ┌─────────────────────────────────────────────┐\n   User  ─────► │             Gemini 3 agent (ADK)            │\n   question     │   system prompt enforces calibration:       │\n                │   • retrieve before claiming                │\n                │   • probability + explicit confidence       │\n                │   • abstain on thin / conflicting data      │\n                └───────────────┬─────────────────┬───────────┘\n                                │                 │\n                  ┌─────────────▼──────┐   ┌──────▼─────────────────┐\n                  │  MongoDB MCP tool  │   │  deterministic tools   │\n                  │  (Atlas)           │   │  • risk_tool:          │\n                  │  • portfolio       │   │    fractional-Kelly    │\n                  │  • market          │   │    sizing + drawdown   │\n                  │  real data only    │   │  • risk_contribution:  │\n                  │                    │   │    vol-weighted risk   │\n                  └────────────────────┘   └────────────────────────┘\n                                │                 │\n                                └────────┬────────┘\n                                         ▼\n                              Calibrated, cited answer\n```\n\nThe deterministic `risk_tool` keeps the math **out of the LLM's head** — sizing and drawdown are computed, not hallucinated.\n\n---\n\n## Setup\n\n### Prerequisites\n- Python 3.10+\n- A **Gemini API key** ([aistudio.google.com](https://aistudio.google.com/app/apikey))\n- A **MongoDB Atlas** cluster (free tier M0 works) with a connection string\n- Node.js (the MongoDB MCP server runs via `npx`)\n\n### Install\n\n```bash\ncd calibrated-copilot\npython -m venv .venv\n# Windows: .venv\\Scripts\\activate   |   macOS/Linux: source .venv/bin/activate\npip install -r requirements.txt\n```\n\n### Configure\n\n```bash\ncp .env.example .env\n# then edit .env and fill in:\n#   GEMINI_API_KEY=...\n#   MONGODB_URI=mongodb+srv://user:pass@cluster.mongodb.net/\n```\n\n### Seed the database\n\n```bash\npython seed_data.py\n```\n\nThis loads a small sample portfolio + market snapshot into the `calibrated_copilot` database (`portfolio` and `market` collections).\n\n### Run the demo\n\n```bash\npython demo.py\n```\n\nOr talk to the agent interactively:\n\n```bash\npython -m calibrated_copilot.agent\n```\n\n---\n\n## Demo path (2-minute video)\n\nThe scripted demo runs three questions that show all three behaviours:\n\n| # | Question | What it shows |\n|---|----------|---------------|\n| a | *\"Should I add to my BTC position?\"* | Calibrated probability + confidence-scaled position size from `risk_tool` |\n| b | *\"Will the Fed cut rates next week and how should I trade it?\"* | **Abstention** — no data in Atlas, agent declines to give a confident call |\n| c | *\"What's my portfolio's risk concentration?\"* | Computes concentration from real Atlas holdings |\n\nSee [docs/demo-script.md](docs/demo-script.md) for the full 2-minute video script.\n\n---\n\n## Project layout\n\n```\ncalibrated-copilot/\n├── README.md\n├── LICENSE                 # MIT\n├── requirements.txt\n├── .env.example\n├── .gitignore\n├── seed_data.py            # load sample portfolio + market into Atlas\n├── demo.py                 # 3 scripted demo questions\n├── calibrated_copilot/\n│   ├── __init__.py\n│   ├── agent.py            # the ADK Gemini agent + MCP wiring\n│   ├── prompts.py          # the calibration system prompt (the differentiator)\n│   └── risk_tool.py        # deterministic risk_tool + risk_contribution\n├── docs/\n│   └── demo-script.md\n└── SUBMISSION.md           # Devpost submission text\n```\n\n---\n\n## How it works (the calibration contract)\n\nThe system prompt ([calibrated_copilot/prompts.py](calibrated_copilot/prompts.py)) binds the agent to a strict contract:\n\n- **Retrieve before claiming.** Any number about the user's portfolio or the market must come from the MongoDB MCP tool. No number → no claim.\n- **Quantify confidence.** Every directional/forecast statement carries a probability and a confidence band, plus the basis (\"based on 90-day realized vol of X\").\n- **Size by confidence.** Recommendations call `risk_tool`, which returns a *fractional-Kelly* position size that shrinks toward zero as confidence drops, plus a drawdown flag.\n- **Measure risk, not just weight.** Concentration questions call `risk_contribution`, which reports volatility-weighted risk shares (so a 30%-capital, 62%-vol Bitcoin sleeve is correctly flagged as ~49% of portfolio risk) plus a risk-based Herfindahl index.\n- **Abstain honestly.** If the data is missing or signals conflict, the agent says so and gives no confident call.\n\n---\n\n## What's next\n\n- More MCP data sources (filings, news, alt-data)\n- Live market feeds\n- A reliability dashboard tracking the agent's own calibration over time (Brier score on its past probabilities)\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n"}