invest-igator

🔍 Invest-igator

An opinionated personal investing tracker

Built with Next.js App Router, tRPC v11, Prisma/PostgreSQL, shadcn/ui, and InfluxDB for fast timeseries. Auth is powered by Better Auth (Prisma adapter) with email/password, magic links, 2FA, and Discord OAuth.

Getting Started • Features • Documentation • Contributing

---

✨ Features

• 📊 Watchlist with historical OHLCV from InfluxDB (AAPL, MSFT, etc.)
  • 📅 Date range filtering with customizable presets and max date constraints
  • 📈 Interactive charts with Recharts integration
• 💰 Corporate events: dividends, splits, capital gains
• 📈 Transactions with CSV import/export, duplicate detection, and FX-aware currencies
  • 🔄 Sortable columns with visual indicators
  • 🔍 Advanced filtering and search
• 🎯 Portfolio analytics: structure and performance calculations (TWR/MWR via tRPC)
• 🎪 Goals tracking: simple personal financial goals model
• 🎨 Modern UI: theming, toasts, and shadcn/ui + Recharts
  • ⚡ TanStack Table v8 for complex data tables
  • 🔎 Debounced search inputs (300ms) for better UX
  • 💀 Skeleton loading states for professional loading experience
  • 🎯 Active navigation indicators in sidebar
• 🔄 Auto-sync: Yahoo Finance ingestion job for OHLCV and events; FX rates also sourced from Yahoo into InfluxDB
• 🔐 Admin interface: comprehensive user management and audit logging
  • 👥 User management with sorting, filtering, and role-based permissions
  • 📋 Audit logs with date range filtering and action tracking
  • 📊 Statistics dashboard with user metrics
  • 🔍 Debounced search across users and logs
• 🔑 API Keys: programmatic access with granular permissions
  • 🎯 8 permission scopes (account, admin, apiKeys, fx, goals, portfolio, transactions, watchlist)
  • 📋 4 built-in templates (read-only, full-access, portfolio-manager, custom)
  • 🚦 Rate limiting and expiration controls
  • 🔒 SHA-256 hashed keys with secure generation
  • 📊 Usage tracking and management UI
• 🤖 MCP server: read-only Model Context Protocol endpoint (/api/mcp) for external agents — see docs/mcp.md

🛠️ Tech stack

Layer	Technology
Frontend	Next.js 15 (App Router) • React 19
API	tRPC v11 + React Query (RSC + CSR hydration)
Database	Prisma + PostgreSQL
Auth	Better Auth (email/password, magic link, 2FA, Discord)
UI	shadcn/ui + TailwindCSS 4 • TanStack Table v8
Charts	Recharts
Timeseries	InfluxDB 2.x (daily_bars + event measurements)

🏗️ Architecture (high level)

graph TB
    A[Client: React 19 + Next.js 15] --> B[tRPC v11]
    B --> C[API Routers]
    C --> D[Prisma ORM]
    C --> E[InfluxDB Client]
    D --> F[(PostgreSQL)]
    E --> G[(InfluxDB 2.x)]
    C --> H[Better Auth]
    H --> F

Key components:

• 📁 App Router under src/app/*; dashboard shell in src/app/(dashboard)/layout.tsx
• 🔌 tRPC routers in src/server/api/routers/*, composed in src/server/api/root.ts
• 🔐 Context includes db (Prisma) and session from Better Auth in src/server/api/trpc.ts
• ⚡ Influx helpers in src/server/influx.ts
• 📥 Yahoo ingestion in src/server/jobs/ingest-yahoo.ts with helpers in yahoo-lib.ts
• 🪝 Client hooks in src/trpc/react.tsx; RSC callers via src/trpc/server.ts

Example API usage:

// Server (RSC)
import { api } from "@/trpc/server";
const me = await api.account.getMe.query();

// Client (CSR)
import { api } from "@/trpc/react";
const { data } = api.watchlist.list.useQuery();

---

🚀 Getting started (local dev)

Prerequisites

• ✅ Bun 1.0+
• ✅ Node 20+ (optional, Bun runs everything)
• ✅ A PostgreSQL 16 database (use ./start-database.sh)
• ✅ An InfluxDB 2.x instance (use ./start-influxdb.sh, or bring your own local/remote)

Setup steps

1️⃣ Install dependencies

bun install

2️⃣ Create and fill .env

See the Environment variables section below. For a smoke test you can set placeholders and use SKIP_ENV_VALIDATION=1 while you iterate.

3️⃣ Start the databases (dev helpers)

./start-database.sh   # Postgres 16
./start-influxdb.sh   # InfluxDB 2.x

For local dev, make sure INFLUXDB_URL points at the local instance — INFLUXDB_URL=http://localhost:8086 in .env.local (org/bucket/token stay as in your .env). ./start-influxdb.sh reads those values and provisions the container to match.

4️⃣ Generate and apply Prisma schema

bun run db:generate

5️⃣ Start the dev server

bun run dev

🎉 Open http://localhost:3000

---

🔐 Environment variables

Validated in src/env.js via @t3-oss/env-nextjs. Server-side vars are required unless noted.

• DATABASE_URL: Postgres connection URL (required)
• BETTER_AUTH_SECRET: Secret for Better Auth JWT/cookies (required in production)
• BETTER_AUTH_URL: Public base URL of the app (default: http://localhost:3000)
• PASSWORD_PEPPER: Extra pepper for hashing local passwords (required)
• EMAIL_SERVER: Nodemailer connection string (e.g. SMTP URI) (required)
• EMAIL_FROM: From address for transactional emails (required)
• NEXT_PUBLIC_SITE_URL: Public site URL exposed to client (default: http://localhost:3000)

• AUTH_DISCORD_ID, AUTH_DISCORD_SECRET: Discord OAuth credentials (required if enabling Discord)

• INFLUXDB_URL (default http://localhost:8086)
• INFLUXDB_ORG
• INFLUXDB_BUCKET
• INFLUXDB_TOKEN
• DATABASE_URL (Postgres, repeated for clarity)

• YAHOO_API_URL (base URL, default https://query2.finance.yahoo.com/v8/finance; code appends /chart/<symbol>, so do not include /chart) — FX ingestion also sources rates from Yahoo
• POLYGON_API_URL, POLYGON_API_KEY — required by the env schema (src/env.js, z.string()), so the app fails validation on boot without them unless SKIP_ENV_VALIDATION=1; note no code path consumes them yet

• CLOUDFLARE_* (R2 image storage) — the R2 feature is optional, but CLOUDFLARE_ACCESS_KEY_ID/CLOUDFLARE_ACCOUNT_ID/CLOUDFLARE_BUCKET_NAME/CLOUDFLARE_SECRET_ACCESS_KEY are required by src/env.js and fail validation on boot unless SKIP_ENV_VALIDATION=1 (CLOUDFLARE_R2_PUBLIC_URL is the only optional one)
• NEXT_PUBLIC_* for Ads/Analytics (Umami/GA/AdSense) are optional and stubbed in E2E tests

💡 Tip: during early setup, export SKIP_ENV_VALIDATION=1 to bypass strict checks until you've filled everything in.

---

🗄️ Database (Prisma)

Command	Description
bun run db:generate	Generate/apply in dev (migrate dev)
bun run db:migrate	Deploy migrations
bun run db:push	Push schema (no migrations)
bun run db:studio	Prisma Studio (defaults to port 5000)

Schema: prisma/schema.prisma

Relevant models: User, Account, Session, WatchlistItem, Transaction, Goal, ApiKey and Better Auth support tables TwoFactor, Verification, VerificationToken. (FX rates are no longer a Postgres model — the FxRate table was dropped and rates now live in the InfluxDB fx_rates measurement.)

---

📊 Timeseries (InfluxDB)

Measurement	Fields	Tags	Purpose
daily_bars	open, high, low, close, volume	symbol	OHLCV data
dividends	amount	symbol	Dividend events
splits	numerator, denominator, ratio	symbol	Stock split events
capital_gains	amount	symbol	Capital gain distributions

Helper code: src/server/influx.ts

---

🔄 Ingestion jobs

Yahoo Finance → InfluxDB

bun run ingest:yahoo

What it does:

• ✅ Reads distinct symbols from your WatchlistItems
• ✅ Fetches full-range daily bars plus dividends, splits, capital gains
• ✅ Writes to Influx in batches with retries and gentle pacing (~2s/request)
• ✅ Also updates watchlist currency when available
• ✅ Adding a symbol to your watchlist triggers a background ingest for that symbol

FX rates (Yahoo → InfluxDB)

bun run ingest:fx

What it does:

• ✅ Sources daily FX from Yahoo bars and writes cross rates (pivoted through USD) into the InfluxDB fx_rates measurement (idempotent by measurement+tag+timestamp)

---

🧪 Running tests

E2E tests with Playwright:

# Install browsers once
bun run test:e2e:install

# Run headless
bun run test:e2e

# Run headed or with the UI
bun run test:e2e:headed
bun run test:e2e:ui

Config: playwright.config.ts (uses a built-in dev server unless PW_SKIP_WEBSERVER=1).

---

📮 API Testing with Postman

Comprehensive Postman collections are available in the postman/ directory for testing all API endpoints.

Collections include:

• 🔐 Authentication - Sign up, login, password reset (Better Auth)
• 👤 Account - Profile management, 2FA, email changes, OAuth
• 📊 Watchlist - Symbol management, price history, corporate events
• 💰 Transactions - CRUD operations, CSV import/export, filtering
• 📈 Portfolio - Structure and performance analytics (TWR/MWR)
• 🎯 Goals - Financial goals tracking
• 🔑 API Keys - Programmatic access management
• 👥 Admin - User management, audit logs (admin only)
• 💱 Currency & Theme - User preferences
• 🌐 FX - Foreign exchange rates

Quick start:

# Import into Postman
# 1. Open Postman
# 2. Import postman/Invest-igator-API.postman_collection.json
# 3. Import postman/Invest-igator.postman_environment.json
# 4. Configure environment variables (baseUrl, etc.)
# 5. Sign in via Better Auth endpoints

📚 See postman/README.md for detailed usage instructions

All endpoints are documented with:

• ✅ Request/response examples
• ✅ Input validation details
• ✅ Authentication requirements
• ✅ Environment variables for easy configuration

---

✅ Linting and typecheck

bun run check       # Biome
bun run typecheck   # TypeScript

---

🐳 Docker

Build locally

docker build -t invest-igator:local .

Run (minimum env)

docker run --rm -p 3000:3000 \
  -e SKIP_ENV_VALIDATION=1 \
  -e DATABASE_URL=postgresql://user:pass@host:5432/db \
  -e BETTER_AUTH_SECRET=change-me-to-a-random-32char-min-secret \
  -e PASSWORD_PEPPER=change-me \
  -e EMAIL_SERVER=smtp://user:pass@mail:587 \
  -e EMAIL_FROM=no-reply@example.com \
  -e INFLUXDB_URL=http://influx.example.com:8086 \
  -e INFLUXDB_ORG=... \
  -e INFLUXDB_BUCKET=... \
  -e INFLUXDB_TOKEN=... \
  invest-igator:local

SKIP_ENV_VALIDATION=1 is included because src/env.js also requires AUTH_DISCORD_ID/AUTH_DISCORD_SECRET, CLOUDFLARE_* and POLYGON_API_KEY. Provide those too and drop SKIP_ENV_VALIDATION once all vars are set.

Docker Compose (App + Postgres + Scheduler)

This repo includes a Compose file that runs:

• invest-igator: the app container (published on port 3311)
• db: internal Postgres 16
• scheduler: Ofelia to run ingestion jobs inside the app container on a cron

Quick start:

cp .env.example .env  # if you have one; otherwise create .env from the vars above
# Fill DATABASE_URL, BETTER_AUTH_SECRET, PASSWORD_PEPPER, INFLUXDB_*, EMAIL_*,
# CLOUDFLARE_*, AUTH_DISCORD_*, POLYGON_API_KEY (all required by src/env.js;
# the compose default SKIP_ENV_VALIDATION=1 lets it boot before they're all set)
docker compose up -d

Notes:

• ⚠️ InfluxDB is external by default — point INFLUXDB_URL at your own instance. To run it inside the stack, uncomment the optional influxdb service in docker-compose.yml and set INFLUXDB_URL=http://influxdb:8086.
• ⏰ Cron labels run ingest-yahoo daily at 02:15 UTC, ingest-fx at 06:00/18:00 UTC and the AI-quota sweep every 5 minutes (Ofelia schedules have a leading seconds field)
• 🔄 Migrations run automatically on container start
• 🔒 The app runs as the unprivileged bun user (uid 1000), not root; in /app only .next is writable by it (plus its own home directory and /tmp). Each Ofelia job sets user: bun too, since Ofelia's job-exec defaults to root

Stop and remove:

docker compose down -v

---

📂 Project structure

.
├── prisma/
│   ├── schema.prisma          # 🗄️  Relational models (User, WatchlistItem, Transaction, etc.)
│   └── migrations/            # 📦 Schema migrations
├── src/
│   ├── server/
│   │   ├── api/
│   │   │   ├── root.ts        # 🔗 tRPC router composition
│   │   │   ├── trpc.ts        # ⚙️  Context, middleware, procedures
│   │   │   └── routers/       # 🧩 Feature routers (watchlist, transactions, admin, etc.)
│   │   ├── jobs/
│   │   │   ├── ingest-yahoo.ts  # 📈 Yahoo Finance ingestion job
│   │   │   ├── ingest-fx.ts     # 💱 FX rates ingestion job
│   │   │   └── yahoo-lib.ts     # 🛠️  Yahoo data helpers
│   │   ├── auth/
│   │   │   └── permissions.ts # 🔐 Better Auth permissions helpers
│   │   ├── db.ts              # 🗃️  Prisma client singleton
│   │   ├── influx.ts          # 📊 InfluxDB client & helpers
│   │   ├── fx.ts              # 💰 FX rate conversion utilities
│   │   └── r2.ts              # ☁️  Cloudflare R2 storage client
│   ├── app/
│   │   ├── (dashboard)/       # 🏠 Protected dashboard routes
│   │   │   ├── watchlist/     # 📊 Watchlist with charts and date filtering
│   │   │   ├── portfolio/     # 💼 Portfolio analytics
│   │   │   ├── transactions/  # 📈 Transaction management with sorting
│   │   │   ├── account/       # 👤 Account settings with API Keys tab
│   │   │   └── admin/         # 🔐 Admin section
│   │   │       ├── users/     # 👥 User management page
│   │   │       └── audit-logs/  # 📋 Audit logs page
│   │   ├── (auth)/            # 🔑 Auth routes (login, signup, verify-request)
│   │   ├── api/               # 🌐 API endpoints (tRPC, auth, email verification)
│   │   └── layout.tsx         # 🎨 Root layout with providers
│   ├── components/
│   │   ├── ui/                # 🧱 shadcn/ui primitives (incl. DateRangePicker, Skeleton)
│   │   ├── ads/               # 📢 AdSense integration
│   │   ├── consent/           # ✅ Cookie consent provider
│   │   └── theme/             # 🌓 Theme provider
│   ├── hooks/
│   │   ├── use-debounce.ts    # ⏱️  Debounce hook (300ms)
│   │   └── use-currency.ts    # 💱 Currency utilities
│   ├── trpc/
│   │   ├── react.tsx          # ⚛️  Client-side tRPC hooks
│   │   ├── server.ts          # 🖥️  Server-side tRPC helpers
│   │   └── query-client.ts    # 📡 React Query configuration
│   ├── lib/
│   │   ├── auth.ts            # 🔒 Better Auth instance
│   │   ├── api-key-permissions.ts  # 🔑 API key permission framework
│   │   ├── api-key-utils.ts   # 🔧 API key generation & validation
│   │   └── utils.ts           # 🔧 Utility functions (cn, etc.)
│   └── env.js                 # ✔️  Environment validation (@t3-oss/env-nextjs)
├── tests/
│   └── e2e/                   # 🧪 Playwright E2E tests
├── docker/
│   └── entrypoint.sh          # 🐳 Container startup script (migrations, server)
├── Dockerfile                 # 📦 Multi-stage build (deps, builder, runner)
├── docker-compose.yml         # 🐙 Compose stack (app, db, scheduler)
├── start-database.sh          # 🗄️  Dev Postgres script
└── start-influxdb.sh          # 📊  Dev InfluxDB 2.x script

Quick pointers:

• Dashboard shell: src/app/(dashboard)/layout.tsx
• Sidebar with active navigation: src/app/(dashboard)/_components/app-sidebar.tsx
• tRPC glue: src/server/api/trpc.ts, src/trpc/react.tsx, src/trpc/server.ts
• API key system:
  • Permission framework: src/lib/api-key-permissions.ts
  • Key utilities: src/lib/api-key-utils.ts
  • Router: src/server/api/routers/api-keys.ts
  • Middleware: src/server/api/middleware/with-api-key.ts
  • UI components: src/app/(dashboard)/account/_components/api-keys-*.tsx
• Admin routers with sorting: src/server/api/routers/admin.ts
• Table components (reference): src/app/(dashboard)/admin/_components/user-management-table.tsx, audit-logs-table.tsx
• Influx helpers: src/server/influx.ts
• Ingestion: src/server/jobs/ingest-yahoo.ts
• Example router: src/server/api/routers/watchlist.ts
• Reusable components: src/components/ui/date-range-picker.tsx, src/components/ui/skeleton.tsx
• Debounce hook: src/hooks/use-debounce.ts

---

🔑 API Keys

Invest-igator supports programmatic API access via API keys with granular permission control.

Features

• 🎯 8 permission scopes: account, admin, apiKeys, fx, goals, portfolio, transactions, watchlist
• 📋 4 built-in templates:
  • read-only: Read access to all non-admin endpoints
  • full-access: Read + write + delete for all user resources
  • portfolio-manager: Portfolio, transactions, and goals management
  • custom: Build your own permission set
• 🚦 Rate limiting: Configure per-key request limits with automatic refills
• ⏰ Expiration control: Set expiry dates or create permanent keys
• 🔒 Secure by design: SHA-256 hashed keys, cryptographically secure generation
• 📊 Management UI: Create, view, and revoke keys from your account page

Quick Start

1. Create an API key: Go to Account → API Keys tab and click "Create API Key"
2. Select permissions: Choose a template or create custom permissions
3. Copy your key: Save it securely - it's only shown once!
4. Make requests: Include x-api-key header in your API calls

Example Usage

# Get your user profile
curl https://your-app.com/api/trpc/account.getMe \
  -H "x-api-key: your_api_key_here"

# Get FX rates matrix
curl https://your-app.com/api/trpc/fx.matrix \
  -H "x-api-key: your_api_key_here"

# List watchlist items
curl https://your-app.com/api/trpc/watchlist.list \
  -H "x-api-key: your_api_key_here"

Documentation

• 📖 API Key Permissions: Detailed permission system documentation
• 📖 API Key Usage Guide: Examples in cURL, JavaScript/TypeScript, and Python

---

🐛 Troubleshooting

Issue	Solution
❌ Env validation failing at boot	Set SKIP_ENV_VALIDATION=1 temporarily and fill missing vars (see src/env.js)
❌ Prisma migrate errors	Ensure Postgres is reachable; try prisma migrate reset in dev
❌ Influx writes fail	Verify INFLUXDB_URL/ORG/BUCKET/TOKEN and token has write permissions
❌ Emails not sending	Verify EMAIL_SERVER URI and that your provider allows SMTP from containers
❌ 401s in tRPC	Confirm cookies are set and BETTER_AUTH_URL matches your external origin

---

🤝 Contributing

PRs are welcome! Before submitting:

✅ Run bun run check (Biome linting)
✅ Run bun run typecheck (TypeScript)
✅ Include/update E2E tests where relevant

Quick demo dataset:

Add a few symbols to your watchlist and run the Yahoo ingest job; the watchlist add flow also triggers a background ingest for that symbol.

Star History