{"id":"93a7b3c8-690e-4b4e-b5cd-1dc06ca54bb8","slug":"data-feeder","kind":"skill","name":"data-feeder","summary":"Declarative data feeds for AI agents. YAML config → MCP server with caching, scheduling, and auth.","owner":"Ptroger","repoUrl":"https://github.com/Ptroger/data-feeder","license":"MIT","licenseTier":"permissive","trust":"community","stars":0,"match":null,"fewStars":true,"format":"skill","createdAt":"2026-04-02T14:00:27.000Z","addedAt":"2026-10-21T04:24:02.120Z","logoUrl":"https://avatars.githubusercontent.com/u/44851272?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["data-analytics","cloud-infra","memory-knowledge"],"tags":["ai-agent","data-feed","x402"],"runtime":"node","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T18:29:05.683Z","sample":false,"url":"https://conduits.dev/tools/data-feeder","language":"TypeScript","transports":["stdio"],"install":{"npm":"data-feeder"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# data-feeder\n\nDeclarative data feeds for AI agents. Config file in, MCP server out.\n\nEvery agent needs external data. Every developer writes the same fetch/cache/schedule boilerplate for each API. data-feeder replaces that with a YAML config — declare your sources, get a running MCP server with caching, scheduling, and auth.\n\n## Quickstart\n\n```bash\nnpx data-feeder init          # creates data-feeder.yaml\n# edit data-feeder.yaml\nnpx data-feeder serve          # starts MCP server on stdio\n```\n\n## Example\n\nOne feed config:\n\n```yaml\nfeeds:\n  hn_top_stories:\n    source:\n      url: https://hacker-news.firebaseio.com/v0/topstories.json\n      auth:\n        type: none\n    cache:\n      ttl: 5m\n    expose:\n      type: tool\n      name: get_hn_top_stories\n      description: \"Get the top story IDs from Hacker News\"\n```\n\nWhat the agent sees: a tool called `get_hn_top_stories` that returns cached HN data, automatically refreshed every 5 minutes.\n\n## Features\n\n- **YAML config** — declare data sources, no code required\n- **Caching** — per-source TTL, in-memory, with hit rate stats\n- **Scheduled prefetch** — cron expressions or simple intervals\n- **Auth** — API key (header or query param), Bearer token, x402 payment protocol\n- **MCP tools & resources** — auto-generated from config\n- **Hot reload** — edit config while running, feeds update without restart\n- **Budget enforcement** — daily limits and per-call caps for paid APIs\n- **Retries** — exponential/linear backoff, respects Retry-After\n- **Response extraction** — dot-notation path to pull nested data\n\n## CLI\n\n```bash\nnpx data-feeder init                     # scaffold config + .env.example\nnpx data-feeder serve                    # start MCP server (stdio)\nnpx data-feeder serve --http 3100        # start on HTTP port\nnpx data-feeder validate                 # check config for errors\nnpx data-feeder validate --config ./alt.yaml\nnpx data-feeder list                     # list all configured feeds\n```\n\n## Full Config Reference\n\n```yaml\n# data-feeder.yaml\n\nserver:\n  name: my-data-feeds              # MCP server name (default: \"data-feeder\")\n  version: 1.0.0                   # server version (default: \"1.0.0\")\n  transport: stdio                 # \"stdio\" or \"http\" (default: \"stdio\")\n  port: 3100                       # HTTP port (default: 3100)\n\ndefaults:\n  cache: 5m                        # default cache TTL\n  timeout: 10s                     # default HTTP timeout\n  retries: 3                       # default retry count\n  retry_backoff: exponential       # \"exponential\" or \"linear\"\n\nbudget:                            # optional\n  daily_max: 10.00                 # max daily spend in USD\n  alert_threshold: 0.8             # warn at 80% of budget\n\nfeeds:\n  <feed_name>:\n    source:\n      url: https://...             # base URL (required)\n      method: GET                  # GET or POST (default: GET)\n      auth:\n        type: api_key              # none | api_key | bearer | x402\n        param: apikey              # query param name (api_key only)\n        # header: X-API-Key       # OR header name (mutually exclusive with param)\n        key: ${API_KEY}            # value, supports ${ENV_VAR}\n      headers:                     # additional headers (optional)\n        X-Custom: value\n      default_params:              # always included (optional)\n        units: metric\n      params:                      # filled by tool invocation (optional)\n        lat: \"{{lat}}\"\n        lon: \"{{lon}}\"\n      response:                    # response extraction (optional)\n        path: data.results         # dot-notation path\n\n    cache:\n      ttl: 5m                      # 30s, 5m, 1h, 24h, 7d\n      key: \"{{lat}}:{{lon}}\"       # custom cache key (optional)\n\n    schedule:                      # scheduled prefetch (optional)\n      - params:\n          lat: 48.85\n          lon: 2.35\n        every: 10m                 # simple interval\n      - params:\n          symbol: EUR/USD\n        cron: \"*/1 9-17 * * 1-5\"  # cron expression\n\n    expose:\n      type: tool                   # \"tool\" or \"resource\"\n      name: get_weather\n      description: \"Get weather for a location\"\n      params:                      # tool input schema (tool type only)\n        lat:\n          type: number\n          description: \"Latitude\"\n          required: true\n\n    timeout: 15s                   # per-feed override\n    retries: 5                     # per-feed override\n```\n\n### Auth types\n\n**API Key** — added as query parameter or header:\n```yaml\nauth:\n  type: api_key\n  param: apikey        # as query param\n  key: ${MY_KEY}\n```\n```yaml\nauth:\n  type: api_key\n  header: X-API-Key    # as header\n  key: ${MY_KEY}\n```\n\n**Bearer** — Authorization header:\n```yaml\nauth:\n  type: bearer\n  token: ${MY_TOKEN}\n```\n\n**x402** — automatic payment on 402 responses (requires `viem` peer dependency):\n```yaml\nauth:\n  type: x402\n  wallet_key: ${WALLET_KEY}\n  network: base           # base or base-sepolia\n  max_per_call: 0.01      # max USD per request\n```\n\n### Environment variables\n\nUse `${VAR_NAME}` syntax in any string value. Variables are resolved from your environment or a `.env` file in the config directory.\n\n### Expose types\n\n- **tool** — callable by the agent with parameters, returns fresh or cached data\n- **resource** — readable endpoint, returns the most recent cached data (ideal with scheduled prefetch)\n\n## Templates\n\nPre-built configs in `templates/`:\n\n| Template | API | Auth |\n|----------|-----|------|\n| `openweather.yaml` | OpenWeather One Call | API key |\n| `alpha-vantage.yaml` | Alpha Vantage stocks | API key |\n| `exchangerate.yaml` | ExchangeRate.host | API key |\n| `hackernews.yaml` | Hacker News | None |\n| `generic-rest.yaml` | Any REST API | Bearer |\n\nCopy a template into your `feeds:` section and set your env vars.\n\n## Built-in utility tools\n\nEvery data-feeder server includes:\n\n- `data_feeder_status` — status of all feeds (stats, schedule info, cache metrics)\n- `data_feeder_cache_stats` — cache hit rates, entry counts, cost savings\n\n## Tool response format\n\nEvery tool response includes metadata:\n\n```json\n{\n  \"data\": { \"...\": \"...\" },\n  \"_meta\": {\n    \"feed\": \"weather\",\n    \"fromCache\": true,\n    \"fetchedAt\": \"2024-01-15T10:30:00Z\",\n    \"age\": \"2m 30s\",\n    \"latencyMs\": 0,\n    \"source\": \"https://api.openweathermap.org/...\",\n    \"cost\": 0\n  }\n}\n```\n\n## How it works\n\ndata-feeder reads your YAML config, creates a Feed instance per source (wiring together an auth handler, HTTP fetcher with retry logic, and an in-memory TTL cache), registers each as an MCP tool or resource, and starts the server on stdio or HTTP. Scheduled prefetch jobs run in the background via cron. Config file changes are detected and hot-reloaded without restart.\n\n## License\n\nMIT\n"}