{"id":"6c9630bd-f39e-4611-8501-48abcfc2a1c5","slug":"fiml","kind":"mcp","name":"FIML","summary":"The Future of Financial MCP: A “Meta-Layer” That Sits Above All Data Providers","owner":"kiarashplusplus","repoUrl":"https://github.com/kiarashplusplus/FIML","license":"Apache-2.0","licenseTier":"permissive","trust":"community","stars":1,"match":null,"fewStars":true,"format":"mcp","createdAt":"2025-11-22T00:07:02.000Z","addedAt":"2026-10-21T16:22:07.844Z","logoUrl":"https://avatars.githubusercontent.com/u/1780945?u=ecfca49058f6a56cdc78fadbbc2acd59097f7aae&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["data-analytics"],"tags":[],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T06:56:11.858Z","sample":false,"url":"https://conduits.dev/tools/fiml","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# FIML - Financial Intelligence Meta-Layer\n\n**An AI-Native Financial Data MCP Server with Intelligent Provider Orchestration & Multilingual Compliance**\n\n[![Integration Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-integration.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-integration.yml)\n[![Infrastructure Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-infrastructure.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-infrastructure.yml)\n[![Bot Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-bot.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-bot.yml)\n[![Agent Workflow Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-agents.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-agents.yml)\n[![Core Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-core.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-core.yml)\n[![FIML CI/CD Pipeline (Core Only)](https://github.com/kiarashplusplus/FIML/actions/workflows/ci.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/ci.yml)\n[![Deploy Documentation](https://github.com/kiarashplusplus/FIML/actions/workflows/docs.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/docs.yml)\n[![Provider Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-providers.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-providers.yml)\n[![DSL Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-dsl.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-dsl.yml)\n[![MCP Server Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-mcp.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-mcp.yml)\n[![Mobile Tests](https://github.com/kiarashplusplus/FIML/actions/workflows/test-mobile.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/test-mobile.yml)\n[![LLMFeed Validation](https://github.com/kiarashplusplus/FIML/actions/workflows/validate-llmfeed.yml/badge.svg)](https://github.com/kiarashplusplus/FIML/actions/workflows/validate-llmfeed.yml)\n\n[![codecov](https://codecov.io/gh/kiarashplusplus/FIML/graph/badge.svg)](https://codecov.io/gh/kiarashplusplus/FIML)\n[![LLMFeed](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/kiarashplusplus/FIML/main/.github/badges/llmfeed-mcp.json)](https://kiarashplusplus.github.io/FIML/.well-known/mcp.llmfeed.json)\n\n> 📋 **Project Status**: ✅ **PHASE 1 COMPLETE** | [Changelog](CHANGELOG.md))\n> \n> **Current State**: Phase 1 Complete ✅ | Phase 2 Active Development 🚧 | **Version**: 0.4.1 | **Codebase**: 32,000+ LOC | **Tests**: 🎉 **1,403 COLLECTED** (100% pass rate on core suite) |\n> \n> 🌍 **NEW in v0.4.1**: FIML Mobile App (Expo), Usage Analytics & Quota Management, Multilingual Compliance Guardrail\n> \n> ✅ **Production Ready**: Zero CodeQL security alerts, comprehensive test coverage, CI/CD validated\n> \n> 🔧 **CI/CD**: Component-based testing workflows for faster feedback ([CI Workflow Structure](docs/development/CI_WORKFLOW_STRUCTURE.md))\n\n\n> 📊 **Quick Links**:\n> - 🎉 [v0.4.1 Release Notes](CHANGELOG.md) - Mobile App, Usage Analytics, and Compliance Guardrail Layer\n> - 📘 [Technical & Strategic Evaluation](docs/development/TECHNICAL_STRATEGIC_EVALUATION.md)\n> - 📚 [Full Documentation](https://kiarashplusplus.github.io/FIML/)\n\n---\n\n## 🎯 Positioning\n\n> **\"FIML is the intelligent data router for AI-native finance. We give ChatGPT and Claude the same data quality as Bloomberg, at 2% of the cost, with zero integration effort. Our arbitration engine automatically picks the best data source for every query, falling back seamlessly when providers fail. For developers, we're the AWS of financial data—abstract away complexity, pay only for what you use.\"**\n\n**Tagline:** *\"Bloomberg's intelligence, API simplicity, AI-native design. $15/month.\"*\n\n📘 **[See detailed comparison: FIML vs Bloomberg vs Direct Provider APIs](docs/project/FIML-VS-BLOOMBERG.md)**\n\n---\n\nFIML is an MCP (Model Context Protocol) server that provides intelligent financial data access through a unified interface. It implements a data arbitration layer that automatically selects the best data provider based on availability, freshness, and reliability. The project includes a comprehensive compliance guardrail system supporting 9 languages for global regulatory compliance.\n\nThe system is designed with a 10-year extensibility roadmap (see [BLUEPRINT.md](docs/project/blueprint.md) for the complete vision) and has reached production readiness with v0.4.1's mobile and analytics capabilities.\n\n## 🌟 What's Actually Working (Phase 1 Complete + v0.4.1 Enhancements)\n\n### ✅ Core Infrastructure (100%)\n- **🔀 Data Arbitration Engine**: Multi-provider scoring (5 factors), automatic fallback, conflict resolution\n- **🏗️ Provider System**: 17 working providers across stocks, crypto, forex, and more\n  - **Free/Basic Tier**: Yahoo Finance, CoinGecko, Mock Provider\n  - **Premium Providers** (API key required):\n    - **Stocks & Equities**: Alpha Vantage, FMP, Polygon.io, Finnhub, Twelvedata, Tiingo, Intrinio, Marketstack, Quandl\n    - **Economics & Macro**: FRED (Federal Reserve Economic Data), Quandl\n    - **Cryptocurrency**: CCXT (multi-exchange), CoinGecko, CoinMarketCap, DeFiLlama\n    - **News**: NewsAPI, Alpha Vantage, Finnhub, Tiingo\n    - **Multi-Asset**: Polygon.io, Finnhub, Twelvedata (stocks, forex, crypto, ETFs)\n- **⚡ Cache Architecture**: L1 (Redis 10-100ms) and L2 (PostgreSQL 300-700ms) with intelligent optimizations\n  - Cache warming for popular symbols\n  - Intelligent eviction policies (LRU/LFU)\n  - Latency tracking and hit rate optimization\n  - 1000+ concurrent request support\n- **📊 FK-DSL Parser**: Complete Lark-based grammar with execution framework\n- **🔧 MCP Server**: FastAPI-based server with 9 fully operational MCP tools\n- **🌐 WebSocket Streaming**: Real-time price and OHLCV data streaming (650 lines)\n- **📦 Docker Deployment**: Complete docker-compose.yml with 12 services configured\n- **🧪 Test Suite**: 🎉 **1,403 tests collected (100% pass rate on core suite)**, comprehensive coverage\n- **💰 Live Data**: Real stock prices (AAPL, TSLA, MSFT) from multiple providers\n- **₿ Crypto Support**: BTC, ETH via CCXT multi-exchange integration\n- **🛡️ Compliance Framework**: Regional checks (8 regions), disclaimers, investment advice detection\n- **📈 Monitoring Hooks**: Prometheus metrics endpoints, health checks\n\n### 🎉 v0.4.1 Mobile & Analytics (NEW!)\n- **📱 FIML Mobile App (Expo)**:\n  - **Cross-Platform**: iOS, Android, and Web support via Expo/React Native\n  - **Chat Interface**: Interactive AI financial assistant (Maya, Theo, Zara)\n  - **Key Management**: Secure on-device API key storage and management\n  - **Market Dashboard**: Real-time price tracking (Coming Soon)\n  - **Learning Hub**: Interactive educational content (Coming Soon)\n- **📊 Usage Analytics & Quotas**:\n  - **Provider Tracking**: Real-time API usage tracking per provider\n  - **Quota Management**: Automatic enforcement of free/premium tier limits\n  - **Redis Persistence**: Durable usage counters with daily/monthly aggregation\n  - **Smart Alerts**: Warning thresholds (80%) for approaching limits\n\n### 🌍 v0.3.0 Compliance Guardrail Layer\n- **🌍 Multilingual Support**: 9 languages (EN, ES, FR, DE, IT, PT, JA, ZH, FA)\n  - Language auto-detection with script-based recognition (CJK, Arabic)\n  - Language-specific pattern matching for compliance violations\n  - Multilingual disclaimer generation\n- **🛡️ Advanced Compliance Enforcement**:\n  - Prescriptive verb detection and blocking\n  - Advice-like language removal with context-aware replacements\n  - Opinion-as-fact pattern detection\n  - Certainty language moderation\n  - Automatic disclaimer insertion (region and asset-class appropriate)\n- **⚙️ Configurable Processing**:\n  - Strict mode (blocks severe violations vs. modifies)\n  - Configurable violation thresholds\n  - Language detection sensitivity tuning\n- **✅ Production Ready**: 163+ compliance tests passing, zero security alerts\n- **🔗 Integrated**: Bot filters, narrative generation, and API outputs\n\n### 🚧 Phase 2 Features (Active Development - 60% Complete)\n- **🤖 Agent Workflows**: ✅ **SHIPPED** - Deep equity analysis and crypto sentiment workflows with LLM narratives\n- **📝 Narrative Generation**: ✅ **SHIPPED** - Azure OpenAI integration with compliance guardrails (500+ lines)\n- **👁️ Watchdog System**: ✅ **SHIPPED** - Event stream orchestration for real-time market monitoring\n- **💾 Session Management**: ✅ **SHIPPED** - Multi-query context tracking with Redis + PostgreSQL\n- **🤖 Advanced Multi-Agent Orchestration**: ✅ **FRAMEWORK COMPLETE** - Ray-based system with 8 specialized agents\n- **⚡ Performance Optimization**: ✅ **SHIPPED** - Cache warming, intelligent eviction, load testing suite\n- **🌍 Multilingual Compliance**: ✅ **SHIPPED** - 9 languages with auto-detection (v0.3.0)\n- **📱 Mobile App**: ✅ **SHIPPED** - Expo/React Native app with chat and key management (v0.4.1)\n- **📊 Usage Analytics**: ✅ **SHIPPED** - Provider usage tracking and quotas (v0.4.1)\n- [x] Educational Bot: ✅ **BETA** - Telegram integration with lesson system (MVP Implemented)\n- **🔌 Platform Integrations**: 🔜 **PLANNED** - ChatGPT MCP plugin, Claude integration\n- **🔐 Security Hardening**: 🔜 **PLANNED** - Penetration testing, rate limiting enhancements\n\n### 📱 Client Applications\n- **Telegram Bot**: Educational bot offering interactive lessons, quizzes, and market data queries.\n- **Mobile App (Expo)**: React Native application featuring:\n  - **Chat Interface**: Direct access to FIML's AI agents.\n  - **Key Management**: Securely manage your provider API keys.\n  - **Onboarding Flow**: Introduction to FIML's core value propositions.\n  - **Market Dashboard**: Real-time price tracking and charts.\n  - **Learning Hub**: Access to educational content and quizzes.\n  - **Cross-Platform**: Runs on iOS, Android, and Web.\n\n## 🏗️ Architecture\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                  CLIENT LAYER                            │\n│  ChatGPT | Claude Desktop | Custom Apps | Telegram      │\n└─────────────────────────────────────────────────────────┘\n                         ↓\n┌─────────────────────────────────────────────────────────┐\n│              UNIFIED MCP API GATEWAY                     │\n│  Request Router | Auth | Rate Limiter | Compliance     │\n└─────────────────────────────────────────────────────────┘\n                         ↓\n┌─────────────────────────────────────────────────────────┐\n│           DATA ARBITRATION ENGINE                        │\n│  Provider Scoring | Auto-Fallback | Conflict Resolution │\n└─────────────────────────────────────────────────────────┘\n                         ↓\n┌─────────────────────────────────────────────────────────┐\n│         MULTI-AGENT ORCHESTRATION (Ray)                  │\n│  Fundamentals | Technical | Macro | Sentiment | News    │\n└─────────────────────────────────────────────────────────┘\n                         ↓\n┌─────────────────────────────────────────────────────────┐\n│        DATA PROVIDER ABSTRACTION                         │\n│  Alpha Vantage | FMP | CCXT | Yahoo Finance | Custom    │\n└─────────────────────────────────────────────────────────┘\n```\n\n### 📋 Understanding FIML's Structure\n\nFIML distinguishes between two types of capabilities:\n\n| Type | Description | Examples |\n|------|-------------|----------|\n| **External MCP Tools** | API endpoints callable by AI clients (ChatGPT, Claude) via the MCP protocol | `search-by-symbol`, `execute-fk-dsl`, `create-analysis-session` |\n| **Internal Components** | Architectural layers that power the MCP tools behind the scenes | Arbitration Engine, Cache Manager, Provider Registry, Ray Agents |\n\n**9 MCP Tools** (external API surface):\n- **Core Data Tools**: `search-by-symbol`, `search-by-coin`, `get-task-status`, `execute-fk-dsl`\n- **Session Management**: `create-analysis-session`, `get-session-info`, `list-sessions`, `extend-session`, `get-session-analytics`\n\n**Internal Components** (not directly callable, power the tools):\n- Data Arbitration Engine (5-factor provider scoring)\n- L1/L2 Cache Architecture (Redis + PostgreSQL)\n- 17 Data Providers (Yahoo Finance, Alpha Vantage, CCXT, etc.)\n- Ray-based Multi-Agent Orchestration (8 specialized agents)\n- FK-DSL Parser and Execution Framework\n- Compliance Guardrail System (9 languages)\n- WebSocket Streaming Infrastructure\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- Docker & Docker Compose\n- Python 3.11+\n- API keys for data providers (Alpha Vantage, FMP, etc.)\n\n### One-Command Installation\n\n```bash\n./quickstart.sh\n```\n\nThis interactive script will:\n- Check prerequisites\n- Setup environment variables\n- Build Docker images\n- Start all services\n- Initialize database\n- Verify health\n\n### Manual Installation\n\n1. **Clone the repository**\n```bash\ngit clone https://github.com/kiarashplusplus/FIML.git\ncd FIML\n```\n\n2. **Configure environment**\n```bash\ncp .env.example .env\n# Edit .env with your API keys and configuration\n```\n\n3. **Start services with Docker Compose**\n```bash\nmake build\nmake up\n```\n\n4. **Verify installation**\n```bash\ncurl http://localhost:8000/health\n```\n\n### Development Setup\n\n```bash\n# Create virtual environment\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\n\n# Install dependencies\nmake dev\n\n# Install pre-push hook (recommended)\n./scripts/install-hooks.sh\n\n# Run tests\nmake test\n\n# Format code\nmake format\n```\n\n## 📖 Usage\n\n### Real-time WebSocket Streaming\n\nFIML provides WebSocket endpoints for real-time financial data streaming, using the same arbitration engine and provider stack as the REST API.\n\n#### Quick Start - Simple Price Streaming\n\n```python\nimport asyncio\nimport websockets\nimport json\n\nasync def stream_prices():\n    uri = \"ws://localhost:8000/ws/prices/AAPL,GOOGL,MSFT\"\n    \n    async with websockets.connect(uri) as websocket:\n        # Auto-subscribed to price updates\n        print(\"Connected and streaming...\")\n        \n        while True:\n            message = await websocket.recv()\n            data = json.loads(message)\n            \n            if data[\"type\"] == \"data\":\n                for update in data[\"data\"]:\n                    print(f\"{update['symbol']}: ${update['price']:.2f} \"\n                          f\"({update['change_percent']:+.2f}%)\")\n\nasyncio.run(stream_prices())\n```\n\n#### Advanced - Full Control WebSocket\n\n```python\nasync def advanced_streaming():\n    uri = \"ws://localhost:8000/ws/stream\"\n    \n    async with websockets.connect(uri) as websocket:\n        # Subscribe to price stream\n        subscription = {\n            \"type\": \"subscribe\",\n            \"stream_type\": \"price\",\n            \"symbols\": [\"AAPL\", \"TSLA\"],\n            \"asset_type\": \"equity\",\n            \"market\": \"US\",\n            \"interval_ms\": 1000,  # Update every second\n            \"data_type\": \"price\"\n        }\n        \n        await websocket.send(json.dumps(subscription))\n        \n        # Receive subscription acknowledgment\n        ack = await websocket.recv()\n        print(f\"Subscribed: {ack}\")\n        \n        # Stream data\n        while True:\n            message = await websocket.recv()\n            data = json.loads(message)\n            \n            if data[\"type\"] == \"data\":\n                # Process real-time updates\n                for update in data[\"data\"]:\n                    print(f\"Price update: {update}\")\n```\n\n#### WebSocket Features\n\n- **Real-time Price Updates**: Stream live prices with configurable intervals (100ms - 60s)\n- **OHLCV Candlesticks**: Real-time candlestick data for technical analysis\n- **Multi-Asset Support**: Subscribe to up to 50 symbols simultaneously\n- **Auto-Reconnection**: Built-in heartbeat and connection management\n- **Provider Integration**: Uses arbitration engine for optimal data sources\n- **Error Handling**: Graceful error reporting and recovery\n\nSee [examples/websocket_streaming.py](examples/websocket_streaming.py) for complete examples including:\n- Simple price streaming\n- Multi-stream subscriptions (price + OHLCV)\n- Portfolio monitoring\n- Auto-reconnection handling\n\n### MCP Tools\n\n> **Note**: MCP (Model Context Protocol) is a standardized protocol for AI model-to-tool communication. MCP Tools are the callable endpoints that AI clients (ChatGPT, Claude, etc.) can invoke via this protocol. These are distinct from FIML's internal architectural components (arbitration engine, cache manager, provider registry, Ray agents, etc.) which power these tools behind the scenes.\n\nFIML exposes **9 MCP tools** organized into two categories:\n\n#### Core Data Tools (4 tools)\n\n##### 1. search-by-symbol\nSearch for a stock by symbol with instant cached data and async deep analysis.\n\n```json\n{\n  \"name\": \"search-by-symbol\",\n  \"arguments\": {\n    \"symbol\": \"TSLA\",\n    \"market\": \"US\",\n    \"depth\": \"standard\",\n    \"language\": \"en\"\n  }\n}\n```\n\n##### 2. search-by-coin\nSearch for cryptocurrency with instant cached data and async deep analysis.\n\n```json\n{\n  \"name\": \"search-by-coin\",\n  \"arguments\": {\n    \"symbol\": \"BTC\",\n    \"exchange\": \"binance\",\n    \"pair\": \"USDT\",\n    \"depth\": \"deep\",\n    \"language\": \"en\"\n  }\n}\n```\n\n##### 3. get-task-status\nPoll or stream updates for an async analysis task.\n\n```json\n{\n  \"name\": \"get-task-status\",\n  \"arguments\": {\n    \"taskId\": \"task-uuid-from-search\",\n    \"stream\": false\n  }\n}\n```\n\n##### 4. execute-fk-dsl\nExecute a Financial Knowledge DSL query for complex multi-step analysis.\n\n```json\n{\n  \"name\": \"execute-fk-dsl\",\n  \"arguments\": {\n    \"query\": \"EVALUATE TSLA: PRICE, VOLATILITY(30d), CORRELATE(BTC, SPY), TECHNICAL(RSI, MACD)\",\n    \"async\": true\n  }\n}\n```\n\n#### Session Management Tools (5 tools)\n\n##### 5. create-analysis-session\nCreate a new analysis session for tracking multi-query workflows.\n\n```json\n{\n  \"name\": \"create-analysis-session\",\n  \"arguments\": {\n    \"assets\": [\"AAPL\", \"MSFT\", \"GOOGL\"],\n    \"sessionType\": \"portfolio\",\n    \"userId\": \"user-123\",\n    \"ttlHours\": 24,\n    \"tags\": [\"tech-stocks\", \"q4-analysis\"]\n  }\n}\n```\n\n##### 6. get-session-info\nGet information about an existing session.\n\n```json\n{\n  \"name\": \"get-session-info\",\n  \"arguments\": {\n    \"sessionId\": \"session-uuid\"\n  }\n}\n```\n\n##### 7. list-sessions\nList sessions for a user.\n\n```json\n{\n  \"name\": \"list-sessions\",\n  \"arguments\": {\n    \"userId\": \"user-123\",\n    \"includeArchived\": false,\n    \"limit\": 50\n  }\n}\n```\n\n##### 8. extend-session\nExtend session expiration time.\n\n```json\n{\n  \"name\": \"extend-session\",\n  \"arguments\": {\n    \"sessionId\": \"session-uuid\",\n    \"hours\": 24\n  }\n}\n```\n\n##### 9. get-session-analytics\nGet session analytics and statistics.\n\n```json\n{\n  \"name\": \"get-session-analytics\",\n  \"arguments\": {\n    \"userId\": \"user-123\",\n    \"sessionType\": \"portfolio\",\n    \"days\": 30\n  }\n}\n```\n\n### Financial Knowledge DSL Examples\n\n```fkdsl\n# Comprehensive equity analysis\nEVALUATE TSLA: PRICE, VOLATILITY(30d), CORRELATE(BTC, SPY), TECHNICAL(RSI, MACD)\n\n# Compare cryptocurrencies\nCOMPARE BTC vs ETH vs SOL ON: VOLUME(7d), LIQUIDITY, MOMENTUM(14d), NETWORK_HEALTH\n\n# Macro analysis\nMACRO: US10Y, CPI, VIX, DXY → REGRESSION ON SPY\n\n# Market scan\nSCAN NASDAQ WHERE VOLUME > AVG_VOLUME(30d) * 2 AND PRICE_CHANGE(1d) > 5%\n```\n\n### Agent Workflows (NEW - Phase 2)\n\nFIML provides production-ready agent workflows that orchestrate multiple specialized agents, data providers, and LLM capabilities for comprehensive financial analysis.\n\n#### Deep Equity Analysis\n\nMulti-dimensional analysis combining fundamentals, technicals, sentiment, and AI narratives:\n\n```python\nfrom fiml.agents import deep_equity_analysis\nfrom fiml.core.models import Market\n\n# Comprehensive equity analysis\nresult = await deep_equity_analysis(\n    symbol=\"AAPL\",\n    market=Market.US,\n    include_narrative=True,\n    include_recommendation=True\n)\n\n# Access results\nprint(f\"Price: ${result.snapshot['price']:.2f}\")\nprint(f\"P/E Ratio: {result.fundamentals['metrics']['pe_ratio']}\")\nprint(f\"Technical Signal: {result.technicals['trend']['direction']}\")\nprint(f\"Recommendation: {result.recommendation['action']}\")\nprint(f\"\\nAI Narrative:\\n{result.narrative}\")\n```\n\n**Features**:\n- ✅ Quick price snapshot from multiple providers\n- ✅ Fundamental analysis (P/E, EPS, ROE, valuation)\n- ✅ Technical analysis (RSI, MACD, trends, support/resistance)\n- ✅ Sentiment analysis (news, social media)\n- ✅ Risk assessment (volatility, beta, correlations)\n- ✅ LLM-generated narrative synthesis (Azure OpenAI)\n- ✅ Actionable BUY/HOLD/SELL recommendations\n- ✅ Data quality and confidence scoring\n\n#### Crypto Sentiment Analysis\n\nSpecialized cryptocurrency analysis for trading signals:\n\n```python\nfrom fiml.agents import crypto_sentiment_analysis\n\n# Crypto sentiment and market analysis\nresult = await crypto_sentiment_analysis(\n    symbol=\"ETH\",\n    exchange=\"binance\",\n    pair=\"USDT\",\n    include_narrative=True\n)\n\n# Access results\nprint(f\"Price: ${result.price_data['price']:,.2f}\")\nprint(f\"Trading Signal: {result.signals['signal']}\")\nprint(f\"Sentiment Score: {result.sentiment['sentiment']['score']}\")\nprint(f\"BTC Correlation: {result.correlations['btc_correlation']:.2f}\")\nprint(f\"\\nMarket Narrative:\\n{result.narrative}\")\n```\n\n**Features**:\n- ✅ Real-time price data from crypto exchanges\n- ✅ Technical indicators (RSI, MACD, volume analysis)\n- ✅ Sentiment from news and social media\n- ✅ Correlation with major cryptos (BTC, ETH)\n- ✅ LLM-powered market narrative\n- ✅ Trading signal generation (BUY/SELL/NEUTRAL)\n- ✅ Confidence scoring\n\n#### Batch Processing\n\nAnalyze multiple assets in parallel:\n\n```python\nimport asyncio\n\nsymbols = [\"AAPL\", \"MSFT\", \"GOOGL\", \"AMZN\", \"TSLA\"]\n\n# Run analyses concurrently\nresults = await asyncio.gather(\n    *[deep_equity_analysis(symbol) for symbol in symbols]\n)\n\n# Process results\nfor symbol, result in zip(symbols, results):\n    rec = result.recommendation\n    print(f\"{symbol}: {rec['action']} (Score: {rec['overall_score']:.1f})\")\n```\n\n**Documentation**:\n- 📖 [Agent Workflows Guide](docs/user-guide/agent-workflows.md) - Comprehensive documentation\n- ⚡ [Quick Reference](docs/user-guide/agent-workflows-quick-reference.md) - Common patterns\n- 💻 [Demo Script](examples/agent_workflows_demo.py) - Live examples\n\n**Run the Demo**:\n```bash\npython examples/agent_workflows_demo.py\n```\n\n## 📊 Code Metrics & Quality\n\n**Implementation Stats** (November 23, 2025):\n- **Total Python Files**: 43+ implementation files\n- **Lines of Code**: 8,000+ lines of production code\n- **Test Files**: 19+ comprehensive test suites  \n- **Test Coverage**: 439 passing tests (100% pass rate), 25 skipped, 464 total tests\n- **Code Quality**: A grade (clean, type-safe, well-structured)\n- **Dependencies**: All stable, no critical vulnerabilities\n- **Linting**: 100% passing (ruff)\n- **Latest Updates**: Session management, watchdog system, narrative generation\n\n**Architecture Quality**:\n- ✅ Clean separation of concerns\n- ✅ Async/await throughout\n- ✅ Type-safe with Pydantic v2\n- ✅ Comprehensive error handling\n- ✅ Structured logging with structlog\n- ✅ Extensible provider system\n\n**Technical Debt**:\n- ✅ ~~238 deprecation warnings~~ **FIXED** - datetime.utcnow properly migrated\n- ✅ ~~Cache optimization~~ **IMPLEMENTED** - Cache warming, intelligent eviction, analytics\n- ✅ ~~Performance testing~~ **COMPLETE** - Benchmarks, load tests, regression detection suite\n- ✅ ~~Agent system~~ **SHIPPED** - 7 specialized agents with production workflows\n- ⚠️ Additional provider integrations pending (Polygon.io, IEX Cloud)\n- ⚠️ Platform integrations (ChatGPT GPT, Claude Desktop) not started\n\nFor complete analysis, see **[TECHNICAL_STRATEGIC_EVALUATION.md](docs/development/TECHNICAL_STRATEGIC_EVALUATION.md)** - a comprehensive 21KB technical and strategic review of the codebase.\n\n## 🛠️ Technology Stack\n\n### Core (Production-Ready)\n- **Python 3.12** with async/await throughout\n- **FastAPI 0.109+** for MCP protocol and REST API\n- **Pydantic v2** for type-safe data validation\n- **Structlog** for structured, contextual logging\n- **HTTPX + aiohttp** for async HTTP client operations\n\n### Data Layer (Operational)\n- **Redis 7** - L1 cache with 10-100ms latency (production)\n- **PostgreSQL 16 + TimescaleDB** - L2 cache with 300-700ms latency\n- **SQLAlchemy 2.0** - Async ORM for database operations\n- **Cache Analytics** - Real-time hit rate, latency tracking, warming strategies\n\n### Orchestration (Production)\n- **Ray 2.52** - Multi-agent framework with 7 specialized agents\n- **Celery 5.3** - Task queue for background jobs\n- **Apache Kafka + Zookeeper** - Event streaming for watchdog system\n- **Azure OpenAI** - LLM integration for narrative generation\n\n### Data Providers (17 Active Providers)\n**Free Tier**:\n- **Yahoo Finance** ✅ Equities, ETFs, indices\n- **CoinGecko** ✅ Cryptocurrency market data\n- **DefiLlama** ✅ DeFi protocol analytics\n\n**Premium Providers** (API key required):\n- **Stocks**: Alpha Vantage ✅, FMP ✅, Polygon.io ✅, Finnhub ✅, Twelvedata ✅, Tiingo ✅\n- **Crypto**: CCXT ✅ (6 exchanges: Kraken, KuCoin, OKX, Bybit, Gate.io, Bitget)\n- **News**: NewsAPI ✅, Alpha Vantage ✅, Finnhub ✅\n- **Multi-Asset**: Twelvedata ✅, Polygon.io ✅\n\n### Infrastructure (Docker Compose)\n- **Docker** - Multi-stage builds with health checks\n- **17 Services**: API server, Ray cluster (3 nodes), Celery workers (2), Redis, PostgreSQL, Kafka, Prometheus, Grafana, exporters\n- **GitHub Actions** - Component-based CI/CD with 9 specialized workflows\n- **Prometheus + Grafana** - Complete observability stack\n- **MkDocs** - Auto-deployed documentation site\n\n## 📊 Data Providers\n\n### Active Providers (17 Total)\n\n**Equity & Stocks** (8 providers):\n- **Yahoo Finance** ✅ Free - Real-time quotes, historical data, fundamentals\n- **Alpha Vantage** ✅ Premium - Comprehensive fundamentals, intraday data\n- **FMP** ✅ Premium - Financial statements, valuation metrics\n- **Polygon.io** ✅ Premium - Real-time market data, aggregates\n- **Finnhub** ✅ Premium - Stock data, news, earnings\n- **Twelvedata** ✅ Premium - Multi-asset (stocks, forex, crypto)\n- **Tiingo** ✅ Premium - EOD data, news, fundamentals\n- **Intrinio** ✅ Premium - Institutional-grade financial data\n- **Marketstack** ✅ Premium - Historical and intraday stock data\n- **Marketstack** ✅ Premium - Historical and intraday stock data\n- **Quandl** ✅ Premium - Alternative data, economics (Macro support added)\n\n**Economics & Macro** (2 providers):\n- **FRED** ✅ Free (Key Required) - Official US economic data (GDP, CPI, Unemployment)\n- **Quandl** ✅ Premium - Access to FRED and other economic datasets\n\n**Cryptocurrency** (4 providers):\n- **CCXT** ✅ Multi-exchange - 6 exchanges (Kraken, KuCoin, OKX, Bybit, Gate.io, Bitget)\n- **CoinGecko** ✅ Free - Market cap, volume, price data\n- **CoinMarketCap** ✅ Premium - Comprehensive crypto metrics\n- **DefiLlama** ✅ Free - DeFi protocol TVL, volumes\n\n**News & Sentiment** (3 providers):\n- **NewsAPI** ✅ Premium - Financial news aggregation\n- **Alpha Vantage News** ✅ Premium - Market news and sentiment\n- **Finnhub News** ✅ Premium - Real-time news feed\n\n**Testing**:\n- **Mock Provider** ✅ Deterministic test data\n\n### Provider Features\n- ✅ **Automatic Fallback**: If primary provider fails, arbitration engine selects next best\n- ✅ **Health Monitoring**: Real-time provider health checks and metrics\n- ✅ **Geo-Blocking Detection**: Automatically handles regional restrictions\n- ✅ **Rate Limiting**: Built-in rate limit management per provider\n- ✅ **Extensible**: Add new providers by implementing `BaseProvider` interface\n\n### Provider Arbitration\nFIML's arbitration engine scores providers on 5 factors:\n1. **Freshness** (30%) - Data recency vs. staleness threshold\n2. **Latency** (25%) - P95 response time for user region\n3. **Uptime** (20%) - Provider health and availability\n4. **Completeness** (15%) - Required fields present\n5. **Reliability** (10%) - Historical error rate\n\nSee [Provider Registry](fiml/providers/registry.py) for implementation details.\n\n## 🔐 Security & Compliance\n\n### Compliance Framework (654 lines)\n- ✅ **Regional Compliance**: 8 regions supported (US, EU, UK, JP, CN, AU, SG, CA)\n- ✅ **Automatic Disclaimers**: Context-aware disclaimer generation\n- ✅ **Investment Advice Detection**: LLM-based detection with 95%+ accuracy\n- ✅ **Audit Logging**: Comprehensive request/response logging with Sentry integration\n- ✅ **Data Lineage**: Track data sources for transparency\n- ✅ **92% Test Coverage**: Production-ready compliance module\n\n### Security Features\n- ✅ **Rate Limiting**: Per-user and per-provider rate limits\n- ✅ **API Key Management**: Secure credential storage in environment variables\n- ✅ **Error Sanitization**: No sensitive data in error messages\n- ✅ **Health Checks**: Service availability monitoring\n- 🚧 **Authentication**: Planned for production deployment\n- 🚧 **Encryption**: TLS/SSL for production environments\n\n**Legal Notice**: FIML provides financial data for informational purposes only. This is NOT financial advice.\n\n## 📈 Monitoring & Observability\n\nAccess monitoring dashboards (requires `make up` to start Docker services):\n\n### Core API Endpoints\n- **API Documentation**: http://localhost:8000/docs (Swagger UI)\n- **ReDoc**: http://localhost:8000/redoc (Alternative API docs)\n- **Main Health Check**: http://localhost:8000/health\n- **Database Health**: http://localhost:8000/health/db\n- **Cache Health**: http://localhost:8000/health/cache\n- **Provider Health**: http://localhost:8000/health/providers\n- **MCP Tools**: http://localhost:8000/mcp/tools\n\n### Metrics & Analytics\n- **Prometheus Metrics**: http://localhost:8000/metrics (raw metrics)\n- **Cache Metrics**: http://localhost:8000/api/metrics/cache\n- **Watchdog Metrics**: http://localhost:8000/api/metrics/watchdog\n- **Performance Metrics**: http://localhost:8000/api/metrics/performance\n- **Task Metrics**: http://localhost:8000/api/metrics/tasks\n\n### Infrastructure Dashboards\n- **Grafana**: http://localhost:3000 (admin/admin) - Pre-configured dashboards\n- **Prometheus UI**: http://localhost:9091 - Metric explorer\n- **Ray Dashboard**: http://localhost:8265 - Agent orchestration monitoring\n\n### WebSocket Endpoints\n- **Simple Price Stream**: ws://localhost:8000/ws/prices/{symbols}\n- **Advanced Stream**: ws://localhost:8000/ws/stream\n\nSee [examples/websocket_streaming.py](examples/websocket_streaming.py) for WebSocket usage examples.\n\n## 🧪 Testing\n\n> 📊 **Current Status**: 🎉 **1279 passing, 14 skipped** (100% pass rate!)  \n> ✅ **All modules passing**: Core, Providers, Arbitration, MCP, Agents, Bot, Infrastructure  \n> 📈 **Coverage**: 68% overall | 97%+ core components\n\n### Quick Test Status Check\n\n```bash\n# Run quick test status check\n./check_test_status.sh\n\n# Or manually run tests\npytest tests/ -v -m \"not live\"\n```\n\n### Quick Test Commands\n\n```bash\n# Run all unit tests (exclude live tests)\npytest tests/ -v -m \"not live\"\n\n# Run only core tests (100% passing)\npytest tests/ -v -m \"not live\" --ignore=tests/bot/\n\n# Run bot tests (to see failures)\npytest tests/bot/ -v\n\n# Run E2E API tests\npytest tests/test_e2e_api.py -v\n\n# Run live system tests (requires Docker services)\npytest tests/test_live_system.py -v -m live\n\n# Run with coverage report\npytest tests/ --cov=fiml --cov-report=html\n\n# Run specific test file\npytest tests/test_arbitration.py -v\n```\n\n### Live System Demo\n\n```bash\n# Run comprehensive live demo\nbash live_demo.sh\n```\n\nThis will test:\n- System health checks\n- MCP tool discovery\n- Real-time stock data (AAPL, TSLA)\n- Cryptocurrency queries (BTC)\n- Service status\n\n### Test Coverage by Module\n\n**Overall Statistics**:\n- **Total Tests**: 1293 collected\n- **✅ Passing**: 1279 tests (98.9%)\n- **⏭️ Skipped**: 14 tests (1.1%)\n- **Coverage**: 68% overall\n- **Test Execution Time**: ~7 minutes for full suite\n\n**Module-Level Coverage**:\n- ✅ **Core Components** (97%+ coverage) - Models, exceptions, config, logging\n- ✅ **Cache System** (89% coverage) - L1/L2 cache, analytics, warming\n- ✅ **Data Providers** (73% avg) - 17 providers with health checks\n- ✅ **Arbitration Engine** (85% coverage) - Provider scoring, fallback, merge\n- ✅ **MCP Protocol** (66% coverage) - REST API, tools, schema validation\n- ✅ **Agent Workflows** (92% coverage) - Multi-agent orchestration, narratives\n- ✅ **Compliance Framework** (92% coverage) - Regional checks, disclaimers\n- ✅ **WebSocket Streaming** (86% coverage) - Real-time price/OHLCV streams\n- ✅ **Bot Platform** (100% tests passing) - Telegram bot, gamification, AI mentors\n- ✅ **Infrastructure** (100% tests passing) - Deployment, monitoring, health checks\n\n### Test Documentation\n\nComprehensive test resources:\n\n- 📊 **[Test Report](docs/testing/TEST_REPORT.md)** - Detailed coverage analysis\n- 📈 **[CI Workflow Structure](docs/development/CI_WORKFLOW_STRUCTURE.md)** - Component-based testing strategy\n- 🔧 **[Contributing Guide](CONTRIBUTING.md)** - How to write and run tests\n\nSee [TECHNICAL_STRATEGIC_EVALUATION.md](docs/development/TECHNICAL_STRATEGIC_EVALUATION.md) for comprehensive technical analysis.\n\n## 📝 API Documentation\n\nOnce running, access interactive API docs at:\n\n- **Swagger UI**: http://localhost:8000/docs\n- **ReDoc**: http://localhost:8000/redoc\n\n## 🗺️ Roadmap\n\n### ✅ Phase 1 (November 2025) - Foundation **COMPLETE** ✅\n- [x] Core MCP server implementation (450 lines, production-ready)\n- [x] Data arbitration engine with scoring and fallback (350 lines)\n- [x] Provider abstraction layer (1,900 lines across 5 providers)\n- [x] Provider integrations: Yahoo Finance ✅, Alpha Vantage ✅, FMP ✅, CCXT ✅, Mock ✅\n- [x] L1/L2 cache architecture implementation (530 lines)\n- [x] FK-DSL parser and execution framework (550 lines)\n- [x] Multi-agent orchestration structure (700 lines framework)\n- [x] Docker and Kubernetes deployment configuration\n- [x] CI/CD pipeline with GitHub Actions\n- [x] Comprehensive test framework (439 passing tests, 100% success)\n- [x] Compliance framework (654 lines, 8 regions supported)\n- [x] Error handling and retry logic throughout\n- [x] Real-time WebSocket streaming (650 lines, production-ready)\n- [x] **Total Implementation**: 8,000+ lines of production code\n\n**Phase 1 Status**: 100% Complete - All core features operational and tested\n\n### 🚧 Phase 2 (November 2025 - Q1 2026) - Enhancement & Scale **IN PROGRESS** (75% Complete)\n\n**✅ Completed Features**:\n- [x] Multi-agent orchestration ✅ (7 specialized agents: fundamentals, technical, macro, sentiment, news, risk, portfolio)\n- [x] Narrative generation engine ✅ (Azure OpenAI integration, 500+ lines, production-ready)\n- [x] Cache optimization ✅ (Warming, intelligent eviction, analytics dashboard)\n- [x] Session management ✅ (Multi-query context with Redis + PostgreSQL)\n- [x] Watchdog system ✅ (Real-time event stream orchestration with Kafka)\n- [x] Performance testing ✅ (Benchmarks, load tests, regression detection, profiling)\n- [x] Additional providers ✅ (17 total: added Polygon.io, NewsAPI, Finnhub, Twelvedata, Tiingo, etc.)\n- [x] Telegram bot ✅ (Educational bot with AI mentors, gamification, FK-DSL integration)\n- [x] Docker optimization ✅ (Fixed Ray shared memory allocation, health checks)\n\n**🚧 In Progress**:\n- [ ] ChatGPT GPT integration (awaiting GPT Actions API access)\n- [ ] Claude Desktop MCP integration (testing protocol compatibility)\n- [ ] Multi-language support (5+ languages: en, es, fr, de, zh)\n- [ ] Security hardening (penetration testing, auth layer)\n\n**Phase 2 Status**: 75% Complete - All core features shipped, platform integrations in progress\n\n### 📋 Phase 3 (Q2-Q3 2026) - Scale & Ecosystem\n- [ ] **Advanced Analytics**: ML models for price prediction, anomaly detection\n- [ ] **Backtesting Engine**: Strategy validation with historical data\n- [ ] **Mobile Apps**: iOS and Android native apps\n- [ ] **Enterprise Features**: SSO, RBAC, audit logs, SLA guarantees\n- [ ] **Extended Coverage**: Options, futures, commodities, bonds\n- [ ] **API Marketplace**: Allow third-party providers to integrate\n- [ ] **White-label Solution**: Embeddable financial data widgets\n\n### 🔮 Phase 4+ (2026+) - Ecosystem\nSee [BLUEPRINT.md](BLUEPRINT.md) for the complete 10-year vision including:\n- Plugin ecosystem and Financial OS\n- Decentralized data verification\n- Advanced quant strategies with backtesting engine\n- Global market expansion\n- AI-native portfolio optimization\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n## 📄 License\n\nApache 2.0 License - see [LICENSE](LICENSE) for details\n\n## 🙏 Acknowledgments\n\nBuilt with ♥ for the AI and finance communities\n\n## 📞 Support\n\n- **Documentation**: [https://kiarashplusplus.github.io/FIML/](https://kiarashplusplus.github.io/FIML/)\n- **Issues**: [GitHub Issues](https://github.com/kiarashplusplus/FIML/issues)\n- **Discord**: [Join our community](https://discord.gg/fiml)\n\n---\n\n**⚠️ Disclaimer**: FIML provides financial data and analysis for informational purposes only. This is NOT financial advice. Always do your own research and consult with qualified financial advisors before making investment decisions.\n"}