hermes-native

Hermes Native

Native macOS + iOS client for the Hermes Agent gateway — built with Swift + SwiftUI.

Connects directly to a Hermes Gateway via WebSocket JSON-RPC (/v1/ws), providing the full Hermes experience from a native app with no local server, CLI, or Node.js required.

Features

Core

• WebSocket JSON-RPC — 35+ gateway methods: sessions, prompts, approvals, config, skills, cron, delegation, wiki, feed. See docs/rpc-reference.md for the full catalog.
• Streaming chat — real-time message.delta, tool.start, tool.complete, reasoning.delta, and thinking.delta events, batched (collect(.byTimeOrCount)) into ~32ms flushes for smooth rendering without layout thrash
• Chat skins — pluggable renderers: TUI (terminal-style), Dark Manga, with a custom skin API (ChatSkinProviding)
• Markdown rendering — headings, lists, code blocks with syntax highlighting (Highlightr), blockquotes, inline formatting, tables, Mermaid diagrams, and LaTeX/math
• HTML / file preview — open HTML blocks and file attachments as a full-window overlay (macOS) or full-screen cover (iOS)
• Multi-gateway switching — save multiple gateways and switch the live connection between them; reuses the existing reconnect path and resets in-memory state on switch

Agent Management

• Spawn tree / Mission Control — live subagent/delegation hierarchy visualization per session with recursive tree nodes, call graph, and tool-call flow
• Session explorer — drill into any session's agents, full chat history, run timeline, and token usage
• Sessions dashboard — global session overview with filtering by status (live/ended) and source (Native/Telegram/Discord/CLI/Cron/Web)
• Session observer — watch running sessions with live event streaming
• Session playback — replay a finished session with a Gantt-style timeline chart, event list, and search/filter
• Cron dashboard + run history — view, pause, resume, remove scheduled jobs with execution timeline and charts
• Activity inbox — notifications for tool approvals, clarifications, response completions, and cron triggers, with HTML artifact preview
• Prompt breakdown — inspect the assembled system prompt: injected sections and per-section token allocation

Skills & Knowledge

• Skills browser — discover and manage agent skills with metadata, inline markdown editing, and graph visualization
• Wiki suite — Obsidian-style file browser (folder tree + markdown reader with wikilink navigation and backlinks), 2D force-directed graph, 3D SceneKit graph, and a changeset timeline of the wiki's edit history
• 3D skill graph — WebKit-powered visualization of skill relationships
• Thought graph — live DAG of the agent's tool-call chains, with heuristic reasoning summarization

Learning

• Quizzes — multiple-choice quiz player generated by the agent, persisted locally
• Flashcards + SRS — flashcard decks with SM-2 spaced repetition scheduling
• Learning dashboard — unified view of saved quiz sessions and flashcard decks with inline players

AI & Media

• Persona system — load personas from gateway + local JSON + built-in presets with Lottie/SceneKit avatars
• 3D avatar — SceneKit + SpriteKit character with state-driven expressions (idle, speaking, thinking, tool use, error)
• On-device inference — MLX-backed reasoning summarization runs off the main thread (MLX Swift LM + swift-transformers)
• TTS / voice — text-to-speech via AVSpeechSynthesizer for voice transcript events
• Video playback — digest videos play via AVPlayer from the remote gateway, with full-file caching to work around a gateway HTTP Range bug, and expand-to-fullscreen
• File attachments — image/file upload with thumbnail previews and a remote download manager
• Feed — social-media-style curated feed from the digest pipeline with source filtering and inline video
• Celebrations — confetti effects on milestone completions
• GitHub link cards — inline preview cards for GitHub URLs in chat

Infrastructure

• Cloudflare Access — built-in CF Access auth flow for enterprise gateways (per-host cookie, re-auth on host switch)
• macOS/iOS Keychain — API key + gateway URL + saved gateways stored securely via the Security framework
• Notifications — native push notifications for approvals, clarifications, and response completions
• Auto-reconnect — WebSocket keepalive with ping/pong and exponential backoff (1s → 2s → 4s → max 30s)
• Session history — local persistence of messages per session in Application Support/hermes-native/sessions/
• Cross-device sync — session metadata (titles, pins, archives, tags) synced via the gateway KV store, last-writer-wins
• Cross-platform — macOS 14+ and iOS 17+ from a single SwiftUI codebase
• Swift 6 strict concurrency — @MainActor, Sendable, no data races

Requirements

• macOS 14 (Sonoma) / iOS 17 or later
• Xcode 16+ / Swift 6.1+
• A running Hermes Gateway with API server enabled (api_server platform) — requires the researchoors/hermes-agent fork, not stock hermes-agent; see docs/gateway-setup.md
• xcodegen (for the Xcode project; brew install xcodegen)

Build

git clone https://github.com/researchoors/hermes-native.git
cd hermes-native

# SwiftPM library build (fastest; what CI lints)
swift build

# Full macOS app build (regenerates the Xcode project first)
make build

The Xcode project is generated from project.yml by xcodegen. make build runs xcodegen generate first, so newly-added source files are always picked up — avoid building a stale .xcodeproj directly.

Run

make run     # regenerate project, kill running instances, rebuild, launch (macOS)

Or open HermesNative.xcodeproj in Xcode, select a target (macOS or iOS), and hit ▶. App entry points live in App/MacApp.swift and App/IOSApp.swift.

Configuration

No gateway yet? HermesNative needs the WebSocket JSON-RPC endpoint (/v1/ws), which only exists in the researchoors/hermes-agent fork. docs/gateway-setup.md walks through installing the fork (or migrating an existing hermes-agent install), enabling the API server, and connecting the app.

On first launch, enter your gateway URL and API key:

The app converts https:// → wss:// and appends /v1/ws automatically. After onboarding, add and switch between additional gateways from Settings → Connection → Saved Gateways.

Architecture

App Entry Points (App/)
├── MacApp.swift              # macOS @main — NSApplication lifecycle, owns @StateObjects
└── IOSApp.swift              # iOS @main — App lifecycle + scene phases

HermesNativeApp.swift         # Shared app helpers (notifications, perf instrumentation, window config)
  └── ContentView             # iOS TabView (Sessions + Cron) / macOS custom split layout
      ├── SessionListView      # Sidebar session list (owned/archived/cron/other)
      ├── ChatView             # Main chat — skin-aware, streaming panel, message bubbles
      ├── SettingsView         # Gateway config, saved gateways, personas, capabilities
      ├── OnboardingView       # First-launch setup
      ├── CFAuthView           # Cloudflare Access auth (web-view login)
      ├── Wiki/                # WikiBrowserView, WikiGraphView, WikiGraph3DView, WikiTimelineView
      ├── MissionControl/      # SessionExplorer, SessionsDashboard, SessionObserver, spawn tree/graph
      ├── Playback/            # SessionPlaybackView + Gantt timeline chart
      ├── ThoughtGraph/        # Live tool-call DAG canvas + layout engine
      ├── Quiz/ + Learning/    # Quiz player, flashcards, SRS dashboard, learning hub
      └── Feed / Cron / Skills # FeedView, CronDashboardView, SkillsView

ViewModels/                   # @MainActor ObservableObjects
  ├── ChatViewModel            # Message state, streaming, tool calls, delta batching, attachments
  ├── SessionListViewModel     # Session CRUD, DB↔gateway ID mapping, cross-device meta sync
  ├── SpawnTreeStore           # Subagent event accumulation into live trees
  ├── WikiGraphViewModel       # Wiki graph/browser state, search/taxonomy filtering
  ├── WikiTimelineViewModel    # Paginated wiki changeset timeline
  ├── FeedViewModel            # Feed articles, pagination, source filtering
  ├── CronListViewModel        # Cron job management via gateway RPCs
  ├── ActivityInboxViewModel   # Tool approvals, clarifications, notifications
  ├── SettingsViewModel        # URL, API key, saved gateways, CF auth
  ├── SkillsViewModel          # Skill discovery + management
  ├── PersonaManager / CelebrationManager / HermesCapabilitiesStore
  └── Quiz/QuizViewModel       # Quiz + flashcard study modes

Services/
  ├── GatewayClient            # WebSocket JSON-RPC, events, auto-reconnect, file up/download
  ├── GatewayClient+Wiki/+Feed # Wiki and feed RPC extensions
  ├── GatewayClientWrapper     # Observable connection lifecycle (ConnectionSignature dedupe)
  ├── SkillStore / SkillCache / SkillSummaryService
  ├── ChatHistoryStore         # Message persistence to disk
  ├── FileDownloadManager / VideoCache  # Remote attachment + video caching
  ├── SRSEngine / SRSStore / QuizStore  # Spaced repetition + quiz persistence
  ├── ReasoningSummarizer / ReasoningGraphIntegrator  # Thought-graph extraction
  ├── CronPoller / NotificationService / TTSService
  ├── SessionMetaSyncService   # Cross-device metadata sync
  └── KeychainStore            # API key, gateway URL, saved gateways

See DESIGN.md for the connection/auth design and rationale, docs/rpc-reference.md for the complete RPC + event catalog, and docs/architecture-rules.md for the layer rules CI enforces (SwiftLint custom rules + architecture tests) and how to request an exception.

Key Models

Wire Protocol

JSON-RPC 2.0 over WebSocket. Requests carry an id; events are method: "event" notifications with a type, optional session_id, and a payload:

→ {"jsonrpc":"2.0","id":1,"method":"session.create","params":{"cols":120}}
← {"jsonrpc":"2.0","id":1,"result":{"session_id":"abc123","session_key":"20260501_112429_d91274"}}

← {"jsonrpc":"2.0","method":"event","params":{"type":"message.delta","session_id":"abc123","payload":{"text":"Hello"}}}
← {"jsonrpc":"2.0","method":"event","params":{"type":"tool.start","session_id":"abc123","payload":{"tool_id":"t1","name":"terminal"}}}

One persistent WebSocket per app process; sessions are multiplexed by session_id. See docs/rpc-reference.md for every method, event, and the connection/auth lifecycle.

Testing

swift build                       # library build
swift build --build-tests         # build tests
swift test --disable-sandbox      # unit tests
swiftlint --strict                # lint (0 violations required)
make build                        # macOS app build (xcodegen + xcodebuild)

CI parity for a change is: swift build, swift build --build-tests, swiftlint --strict, and xcodebuild for both macOS and iOS.

CI

Dependencies

Plus Apple system frameworks: SwiftUI, Combine, SceneKit, SpriteKit, AVFoundation, WebKit, Security.

License

MIT