smartpipe

smartpipe

Semantic pipes and queries for your terminal.

Run PDFs, images, audio, video, and text through Unix verbs that understand their input.

▶ Watch the full 86-second demo - sound, controls, no download.

Install

macOS - Homebrew is the happy path:

brew install prabal-rje/tap/smartpipe

Linux - one line (uses Homebrew if you have it, bootstraps uv otherwise; works on macOS too):

curl -LsSf https://prabal-rje.github.io/smartpipe/install.sh | sh

Windows:

powershell -ExecutionPolicy Bypass -c "irm https://prabal-rje.github.io/smartpipe/install.ps1 | iex"

Prefer your own Python tooling? Any of these works on every platform:

uv tool install smartpipe-cli           # uv
pipx install smartpipe-cli              # pipx
pip install smartpipe-cli               # pip
uvx --from smartpipe-cli smartpipe      # zero-install trial

Stay current with smartpipe update - it detects how smartpipe was installed and runs that tool's own upgrade.

Point it at a model

# log in with a ChatGPT account - covers most people
smartpipe auth login

No ChatGPT plan? Bring a cloud API key, or run local Ollama - see Models & providers.

Examples

# summarize each filing, figures included
smartpipe map "summarize the key risk" 'filings/*.pdf'

# keep only the calls that sound frustrated - audio, understood
smartpipe filter "the caller sounds frustrated" 'calls/*.mp3'

# text on stdin works the same way
echo "hello world" \
| smartpipe map "translate to Spanish"
# → hola mundo

PDFs pass through with their figures. Scanned pages route to a vision model. Audio is sent natively where the model hears it, transcribed locally otherwise. Video is sent whole where the wire supports it, or split into frames plus a transcript. Each conversion is noted per row.

It composes with grep, jq, sort, and tail -f: stdin to stdout, one item at a time.

Verbs

A verb is one operation on your data - map, filter, cluster. Each reads stdin (or named FILES) and writes stdout, so verbs pipe into each other and into ordinary Unix tools. Every verb is documented at prabal-rje.github.io/smartpipe.

Semantic verbs call a model:

Free verbs never call a model. Run them first to cut the corpus before any paid stage:

Some semantic verbs have a conditionally free mode: join --on (key equality, no prompt), distinct --exact (hash-only folding), graph --fast (local NER, on-device), map/extend --dry-run (compose without sending), and smartpipe schema with a braces/DSL expression. Each makes no chat-model calls; graph --fast is $0 by default and spends only if you configure a cloud --embed-model or a remote --stt-model (both disclosed at run time).

A one-minute tour

# 1. point smartpipe at a model (ChatGPT login, a cloud key, or local Ollama)
smartpipe use

# 2. ask one question across a folder of mixed documents
smartpipe map "What does this say about pricing?" 'docs/*.pdf'

# 3. typed extraction - braces carry names, types, and guidance
cat tickets.jsonl \
| smartpipe extend "Add {label enum(bug, feature, praise), urgency number: 0 to 1}"

# 4. group feedback by meaning, label each theme, chart it
cat feedback.txt \
| smartpipe cluster --explode members \
| smartpipe chart cluster --top 8 --save themes.svg

# 5. cut for free with `where`, then let the model judge only what is left
cat app.log \
| smartpipe where 'text has "ERROR"' \
| smartpipe filter "an actual outage"

# 6. save a whole pipeline as a file; it runs as a command
smartpipe run triage.sem --dry-run   # prints the stage graph and cost, makes zero calls

# 7. month-end close: the vision model IS the OCR; the anti-join is the worklist
smartpipe map "Extract {vendor string, invoice_number string, total number}" 'invoices/2026-06/*.pdf' \
| tee june-invoices.ndjson \
| smartpipe join "the same payment" --right ledger.jsonl --kind anti > missing-from-ledger.jsonl

# 8. video RAG, no vector database: index a folder of recordings once, ask any day
smartpipe embed 'sessions/**/*.mp4' > sessions.embeddings
smartpipe top_k 3 --near "user gives up after the coupon fails" < sessions.embeddings

# 9. the corpus as a knowledge graph - $0 by default: local NER, on-device
smartpipe graph --fast 'case-files/*.md' --save case.html

A .txt on a pipe arrives one item per line; --as file treats the whole file as one item (feeding smartpipe has the full ingestion table).

Numbers 7 and 8 are full recipes - invoice reconciliation and video RAG - two of a dozen in the cookbook.

New to this? The Learn track starts at zero and assumes nothing, including what a "model" is.

Highlight: the knowledge graph

Number 9 is new: graph --fast turns a folder of mixed files into an interactive, cited knowledge graph entirely on-device by default - a local NER model finds the entities you name, co-occurrence weights the edges, and hovering any edge shows the files behind it. A focus prompt upgrades the strongest edges to model-read relations. Full story: the graph page and the cookbook recipe.

Try it on real files

No corpus handy? smartpipe-playground ships 26 MB of CC0 / public-domain practice files - invoices, reports, photos, recordings, screen sessions, and JSONL data:

curl -L https://github.com/prabal-rje/smartpipe-playground/releases/download/v1/smartpipe-playground-v1.tar.gz | tar xz
cd smartpipe-playground

smartpipe map "Extract {vendor, invoice_number, total number}" 'invoices/*.pdf'
smartpipe filter "the customer sounds frustrated" 'recordings/*.mp3'
smartpipe top_k 3 --near "supply chain risk" 'reports/*.pdf'

For AI agents

Driving smartpipe from an agent? SKILL.md is the operating manual - invocation patterns, the cost model, machine-output contracts, and quoting gotchas, written for programmatic use.

Where your data goes

Some steps run locally no matter which chat model you pick: embeddings (fastembed) and transcription (whisper) are built in.

For chat, Ollama runs models on your machine. Any cloud model sends that run's data to its provider - gpt-5.4-mini, claude-opus-4-8, gemini-3.1-flash-lite, mistral-large-latest, openrouter/….

API keys come from environment variables, or smartpipe auth login stores one securely for you. ChatGPT subscribers can log in instead of using a key. smartpipe usage keeps local run and token totals; see Privacy & security for where credentials live and how to remove them.

Unix behavior

• stdout is data, stderr is diagnostics. Progress and receipts never touch your pipe.
• Adapts to where it runs. Readable tables at a terminal; JSONL when piped into another command.
• Order-preserving. Output order matches input order, even with parallel calls.
• Failure-tolerant. One bad item is a warning, not a crash.

Learn more

Full docs: prabal-rje.github.io/smartpipe.

• Learn track - zero to first result, six short chapters
• Install - packages and platforms
• Working with files & media - PDFs, scans, images, audio, video
• CLI reference - every flag, format, and exit code
• Models & providers - ChatGPT login, cloud keys, local Ollama
• Privacy & security

How to cite

If smartpipe is useful in your research, cite it (or run smartpipe cite):

@software{gupta_smartpipe_2026,
  author = {Gupta, Prabal},
  title = {smartpipe: semantic pipes for your terminal},
  year = {2026},
  version = {1.6.0},
  license = {Apache-2.0},
  url = {https://github.com/prabal-rje/smartpipe}
}

GitHub's "Cite this repository" button (from CITATION.cff) gives APA too.

Development

Built in the open, under Apache-2.0. Contributor setup and the quality gates are in CONTRIBUTING.md; the manual release pass lives in qa/. The CLI surface is a SemVer contract.