upupload

@vivsh1999/upupload

Client-first, multi-stage file uploader/processor with a plugin architecture for custom processing.

• Background Web Worker Threading — Offload intensive image scaling/compression to a pool of Web Workers via useWorker: true
• One-Decode Multi-Variant JPEGs — jpegCompressor.with({ variants: [...] }) encodes every size from one decode
• Progressive Memory Garbage Collection — Auto-purges file buffers and revokes object URLs on successful uploads to keep RAM clean
• Pre-built Upload Adapters — Zero-dependency, tree-shakable standard HTTP & S3/R2 presigned URL upload adapters
• Pipeline engine handles validation, original passthrough, video posters, and safe fallback
• Plugin system — every file-type-specific processor is a separate, tree-shakeable plugin
• Ships built-in plugins: rawToJpeg (RAW/HEIC/TIFF), jpegCompressor (compress/thumbnail), videoPoster
• Zero-cost imports — plugins are tree-shaken at the bundler level; pay only for what you use
• No auto-installed heavy deps — plugin dependencies are never installed unless you add them
• TypeScript-native, fully typed

---

Who Is This For?

You want to…	Start here
Use built-in plugins (or none at all) to process images/video in your app	Quick Start ↓
Write your own custom plugin for a specific file type or processing step	Custom Plugins ↓ & docs/plugins.md
Publish a plugin for the community (open-source extension)	Publishing Plugins ↓ & docs/plugins.md
Contribute to the repo itself — fix bugs, add features, improve docs	CONTRIBUTING.md

---

Agent Skills

Install the UpUpload agent skill for AI-powered guidance on plugin configuration, React hook usage, custom plugin development, and more:

npx skills add vivsh1999/upupload

Works with OpenCode, Claude Code, Cursor, Codex, and 50+ other coding agents.

---

Installation

npm add @vivsh1999/upupload

The package itself has zero image-processing dependencies on first install.

Plugin dependencies (install only what you need)

# For JPEG/PNG/WebP compression
npm add browser-image-compression

# For RAW camera files (CR3, DNG, NEF, ARW…)
npm add libraw-wasm

# Optional: for HEIC/HEIF / TIFF decode via the raw-to-jpeg plugin
npm add heic-decode heic2any utif

---

Entry Points

Path	Environment	Contents	Bundle cost
@vivsh1999/upupload	Browser	Re-exports core (pipeline engine, types, result helpers)	—
@vivsh1999/upupload/browser	Browser	Pipeline, allowlist, audio/canvas utils, plugins	8 kB
@vivsh1999/upupload/core	Universal	Generic pipeline engine, types, result helpers	1 kB
@vivsh1999/upupload/react	Browser	useFileUpload React hook	60 kB
@vivsh1999/upupload/adapters	Browser	Pre-built fetch & S3/R2 upload adapters	5 kB
@vivsh1999/upupload/server	Node	Server entry (minimal)	< 1 kB
@vivsh1999/upupload/plugins	Browser	Barrel re-export of all plugins	N/A
@vivsh1999/upupload/plugins/jpeg-compressor	Browser	JPEG/PNG/WebP compressor plugin	+4 kB
@vivsh1999/upupload/plugins/raw-to-jpeg	Browser	RAW/HEIC/TIFF decoder plugin	+12 kB
@vivsh1999/upupload/plugins/video-poster	Browser	Video poster frame plugin	+6 kB
@vivsh1999/upupload/plugins/testing	Browser	Plugin test utilities	+1 kB
@vivsh1999/upupload/preset	Browser	Zero-config upload() with auto-detected plugins	+13 kB

Only the specific plugin path you import is added to your bundle.

---

Quick Start

React (with built-in plugins)

import { useFileUpload } from "@vivsh1999/upupload/react";
import { jpegCompressor } from "@vivsh1999/upupload/plugins";

function Uploader() {
  const {
    getDropTargetProps,
    getFileInputProps,
    queue,
    startUpload,
    cancelUpload,
    isDragOver,
    isBusy,
  } = useFileUpload({
    plugins: [jpegCompressor.with({ quality: 80, maxSizeMB: 1 })],
    uploadAdapter: async (artifact, { onProgress, fileId, totalArtifacts, artifactIndex }) => {
      for (let pct = 0; pct <= 100; pct += 10) {
        await new Promise((r) => setTimeout(r, 10));
        onProgress(pct);
      }
      await fetch("/api/upload", { method: "POST", body: artifact.blob });
    },
  });
  return (
    <div {...getDropTargetProps()} style={{ border: isDragOver ? "2px dashed blue" : "" }}>
      <input {...getFileInputProps()} />
      {queue.map((item) => (
        <div key={item.id}>
          {!item.needsReselect && <img src={item.previewUrl} alt="" width={40} />}
          {item.name} — {item.status} ({item.progress}%)
          {item.status === "error" && !item.needsReselect && (
            <button onClick={() => cancelUpload(item.id)}>Cancel</button>
          )}
        </div>
      ))}
      <button onClick={() => startUpload()} disabled={isBusy}>
        Upload
      </button>
    </div>
  );
}

React (no plugins — validation + original passthrough only)

import { useFileUpload } from "@vivsh1999/upupload/react";

function Uploader() {
  const { getDropTargetProps, getFileInputProps, queue, startUpload } =
    useFileUpload();
  // No plugins passed — files pass through validation only.
  // Queue items will have 1 artifact: variant "original".
  return (/* … */);
}

Vanilla JS (with built-in plugins)

import { runDefaultBrowserPipeline } from "@vivsh1999/upupload/browser";
import { jpegCompressor } from "@vivsh1999/upupload/plugins";

const result = await runDefaultBrowserPipeline(source, opts, {
  plugins: [jpegCompressor.with({ quality: 80, maxSizeMB: 1 })],
});

Vanilla JS (no plugins)

import { runDefaultBrowserPipeline } from "@vivsh1999/upupload/browser";

const result = await runDefaultBrowserPipeline({ file, name: file.name, type: file.type }, {});
// result.artifacts has 1 item: variant "original"

Preset (zero-config)

import { upload } from "@vivsh1999/upupload/preset";

const result = await upload(file, { quality: 80 });

React Hook Options

The hook accepts a UseFileUploadOptions<TMeta, TPreload> object. Key options:

Option	Description
plugins	Processing plugins to apply
pipeline	PipelineDef[] for per-type routing
pipelineConfig	Initial BrowserPipelineOptions — every field forwarded
uploadAdapter	Function that receives each artifact and its helpers
tuning.maxConcurrency	Pipeline parallelism (default: CPU count, capped at 4)
tuning.maxUploadConcurrency	Upload adapter parallelism (defaults to maxConcurrency)
maxQueuedUploads	Backpressure limit for "uploading" state
maxFileSize	Reject files over N bytes
maxTotalBatchSize	Reject if total batch exceeds N bytes
maxNumberOfFiles	Cap on total queue items
persistence	"memory" or "indexeddb" (survives page reload)
storageKeyPrefix	Isolate IndexedDB database name
retryMode	"pipeline" (default) or "adapter-only" (skip re-compression on retry)
autoPreventTabClose	Prevent tab close during processing
autoPauseOnOffline	Auto-pause on network disconnect
autoWakeLock	Prevent screen sleep during upload
getMeta	Attach custom metadata (TMeta) to each queue item
getPipelineContextMeta	Inject values into every file's pipeline shared context
onBeforeStart	Batch pre-processing hook, returns TPreload for adapter
onFileProcessed	Fires after pipeline, before upload
onFileComplete	Fires after pipeline + upload complete
onBatchComplete	Cumulated stats when batch finishes
onBatchProgress	Live progress during batch processing/uploads
onInfo / onWarning / onError	Structured logging and error callbacks

Queue items are a discriminated union — when restored from IndexedDB after page reload, needsReselect: true and file is unavailable. Check item.needsReselect before accessing item.file.

Full reference: docs/react.md

uploadAdapter

The uploadAdapter replaces manual onFileComplete iteration and provides helper fields:

useFileUpload<{ sessionId: string }, { token: string }>({
  plugins: [jpegCompressor.with({ quality: 80 })],
  onBeforeStart: async (files) => {
    const res = await fetch("/api/bulk-init", { method: "POST" });
    return { token: await res.text() };
  },
  uploadAdapter: async (
    artifact,
    { onProgress, signal, fileId, totalArtifacts, artifactIndex, batch },
  ) => {
    // artifact: { variant, blob, filename, filetype }
    // onProgress(0-100) — updates the queue item's progress
    // signal — honour cancellation
    // fileId, totalArtifacts, artifactIndex — per-file context
    // batch.files, batch.batchId, batch.preload.token — batch context
    for (let pct = 0; pct <= 100; pct += 10) {
      await new Promise((r) => setTimeout(r, 10));
      if (signal?.aborted) return;
      onProgress(pct);
    }
    await fetch("/api/upload", { method: "POST", body: artifact.blob });
  },
});

For custom upload without the hook, use the core result helpers:

import { upload } from "@vivsh1999/upupload/preset";
const result = await upload(file, { quality: 80 });
for (const artifact of result.artifacts) {
  await fetch("/api/upload", {
    method: "POST",
    body: artifact.file,
    headers: { "Content-Type": artifact.filetype },
  });
}

---

File Processing Flow (React Hook)

When you call startUpload(), files go through three throttle-controlled stages:

Input → queue (idle)
         │
         ▼
┌──────────────────────────────────────────┐
│ 1. Pipeline Processing (maxConcurrency)  │  ← compression, transcoding
│    • Each file acquires a semaphore slot │     (0 – pipelineEndProgress%)
│    • Progress derived from completed     │
│    • stages / total stages × 90          │
│    • Multiple files processed in parallel│
│    • Plugins run sequentially per file   │
└──────────────────────────────────────────┘
         │
         ▼
┌──────────────────────────────────────────┐
│ 2. Upload Adapter (per file, sequential) │  ← your adapter sends artifacts
│    • Called once per artifact            │     (pipelineEndProgress – 99%)
│    • All artifacts of a file are sent    │
│      sequentially (one at a time)        │
│    • Adapter receives batch context      │
│      via `helpers.batch`                 │
└──────────────────────────────────────────┘
         │
         ▼
    File marked "complete"  →  onFileComplete fires

Throttles (three independent controls)

Setting	Controls	Default
tuning.maxConcurrency	How many files run the pipeline simultaneously	navigator.hardwareConcurrency (capped at 4)
tuning.maxUploadConcurrency	How many files upload artifacts simultaneously	Same as maxConcurrency
maxQueuedUploads	Backpressure: how many files can be in "uploading" state at once before new files pause	Unlimited

Important: maxConcurrency and maxUploadConcurrency are independent semaphores.

• Four files could be processing pipelines while two others are uploading artifacts.
• maxQueuedUploads is a global ceiling on the number of files in "uploading" state. When hit, files that have finished processing will not start uploading until a slot frees up.

retryUpload lifecycle

retryUpload(fileId) re-runs only the upload adapter — the pipeline (compression, transcoding, etc.) is not re-executed. The existing artifacts from the original processing are re-used. This means:

• The adapter must be idempotent: it may receive the same artifact blob across multiple retryUpload calls.
• If the pipeline failed (no artifacts), retryUpload returns early (no-op). Call retry(fileId) instead to reset the file to "idle" and re-process it through the full pipeline.

retryMode

Set retryMode: "adapter-only" on the hook so retry(fileId) skips re-compression and re-runs only the upload adapter when artifacts exist. Falls back to full re-processing if no artifacts are available.

---

Custom Plugins

Write your own plugin to handle file types or processing that the built-in plugins don't cover.

Minimal Example

import { Plugin } from "@vivsh1999/upupload/plugins";
import { artifact } from "@vivsh1999/upupload/core";

const watermark = new Plugin<{ opacity: number }>({
  id: "watermark",
  name: "Watermark Plugin",
  options: { opacity: 0.5 },
  supports: (file) => file.type?.startsWith("image/") ?? false,
  run: async (input, opts, classif, ctx) => {
    // opts.opacity is typed as number
    // classif.stemName, classif.ext — file metadata
    // ctx.shared — inter-plugin communication
    // ctx.log(level, message, extra?) — structured logging
    // ctx.signal?: AbortSignal — cancellation support
    // ctx.reportProgress(percent) — surface progress during long ops
    return artifact("watermarked", input.file, classif.stemName + ".jpg", "image/jpeg");
  },
});

Register it like any built-in plugin:

useFileUpload({ plugins: [watermark.with({ opacity: 0.3 })] });

Build a Thumbnail Plugin

A common use case is generating a smaller thumbnail variant alongside a full-size output. Here's a complete plugin that creates a 150×150 JPEG thumbnail using the Canvas API:

import { Plugin } from "@vivsh1999/upupload/plugins";
import { artifact, emptyResult } from "@vivsh1999/upupload/core";

interface ThumbnailOpts {
  /** Max width/height in pixels. Default: 150 */
  size?: number;
}

const thumbnailPlugin = new Plugin<ThumbnailOpts>({
  id: "thumbnail",
  name: "Thumbnail Generator",
  options: { size: 150 },
  supports: (file) => file.type?.startsWith("image/") ?? false,
  run: async (input, opts, classif, ctx) => {
    if (typeof OffscreenCanvas === "undefined") return emptyResult();

    const img = await createImageBitmap(input.file);
    const scale = Math.min(opts.size / img.width, opts.size / img.height, 1);
    const w = Math.round(img.width * scale);
    const h = Math.round(img.height * scale);

    ctx.reportProgress?.(50); // surface progress

    const canvas = new OffscreenCanvas(w, h);
    const ctx2d = canvas.getContext("2d")!;
    ctx2d.drawImage(img, 0, 0, w, h);
    img.close();

    const blob = await canvas.convertToBlob({ type: "image/jpeg", quality: 0.85 });
    return artifact("thumb", blob, `${classif.stemName}-thumb.jpg`, "image/jpeg");
  },
});

// Usage:
useFileUpload({
  plugins: [
    jpegCompressor.with({ variant: "full", quality: 80 }),
    thumbnailPlugin.with({ size: 150 }),
  ],
  // uploadAdapter receives both "full" and "thumb" artifacts per file
  uploadAdapter: async (artifact, helpers) => {
    if (artifact.variant === "thumb") {
      // upload to thumbnail bucket
    } else {
      // upload full-size
    }
  },
});

Tip for multi-artifact setups: Each plugin variant produces a separate artifact. The uploadAdapter receives one call per artifact with artifactIndex and totalArtifacts, letting you coordinate uploads.

Full guide: docs/plugins.md — covers createStages for multi-stage plugins, shared context patterns, after/before ordering, error handling, and testing.

Real example: examples/vanilla-html/custom-pipeline.js — a metadata-annotator plugin that reads image dimensions and writes JSON.

---

Publishing Plugins

If you've built a plugin others can use, publish it as a standalone npm package. See docs/plugins.md#publishing-a-plugin for the full checklist: naming conventions, supports() contract, shared keys, tree-shaking setup, JSR compliance, and testing requirements.

---

Documentation

Topic	File
Pipeline engine (stages, features, utilities)	docs/pipeline.md
Plugin system (using, writing, publishing, testing)	docs/plugins.md
React hook (useFileUpload, options, return value)	docs/react.md
Configuration reference (all types)	docs/configuration.md
Case study: e-commerce product photography	docs/case-studies/ecommerce-product-photography.md
Case study: wedding photography client proofing	docs/case-studies/wedding-photography-uploader.md
Case study: podcast audio publishing	docs/case-studies/podcast-audio-publishing.md

Decoder Dependencies

The rawToJpeg plugin optionally imports decoders at runtime:

Package	Format	Strategy
libraw-wasm	Camera RAW (CR3, DNG, NEF, ARW…)	Web Worker + WASM
heic-decode	HEIC/HEIF	Raw pixels, smaller bundle
heic2any	HEIC/HEIF	Fallback when heic-decode unavailable
utif	TIFF	Decodes to RGBA → JPEG

npm add libraw-wasm heic-decode utif

Examples

• examples/vanilla-html — basic pipeline + custom pipeline with a metadata-annotator plugin. Demonstrates writing a Plugin class from scratch, composing multiple plugins, and inspecting the result.
• examples/tanstack-start — TanStack Start app with TUS uploads and the React hook. Shows end-to-end upload with the useFileUpload hook.

Benchmarks

Autogenerated from vitest bench (via GitHub Actions — pushed to main).

Internal Components

Benchmark	Ops/sec	Prev Minor (v0.7.2)	Change
fileExtensionLower > .JPG → .jpg	9,615,695.44	13,372,582.93	🔴 -1.6%
fileExtensionLower > .Tar.Gz → .gz	10,184,943.51	13,614,580.97	🔴 -1.8%
fileExtensionLower > no extension → empty	11,352,943.98	13,882,037.00	🔴 -1.0%
stem > photo.jpg → photo	11,690,591.06	14,029,372.46	🔴 -1.6%
stem > archive.tar.gz → archive.tar	11,907,393.14	14,354,386.88	🔴 -1.1%
stem > noext → noext	11,707,688.00	14,669,504.53	🔴 -1.1%
toJpegName > photo.png → photo.jpg	10,713,663.49	14,692,609.71	🔴 -0.9%
toJpegName > img.heic → img.jpg	10,948,706.47	14,334,449.37	🔴 -1.6%
toThumbName > photo.png → photo.thumb.jpg	10,585,701.65	14,562,644.83	🔴 -0.9%
toThumbName > img.heic → img.thumb.jpg	10,371,815.27	14,592,224.72	🔴 -1.7%
info helper > level + message	12,687,683.59	16,873,548.41	🔴 -0.5%
info helper > level + message + code	11,091,943.31	14,994,037.43	🔴 -1.2%
audioBufferToWav > empty buffer (no samples, mono @ 44100)	421,914.63	535,655.93	🔴 -1.2%
audioBufferToWav > 1 sec mono @ 44100	1,639.94	2,048.91	🔴 -1.2%
audioBufferToWav > 5 sec stereo @ 48000	305.92	304.32	🟢 +0.5%
audioBufferToWav > 30 sec stereo @ 44100	59.9565	60.55	🔴 -1.0%
Semaphore > new Semaphore(4)	16,081,222.55	17,642,588.69	🔴 -0.8%
Semaphore > acquire + release — uncontended (concurrency=10, 1 task)	4,833,544.79	5,214,967.08	🔴 -0.9%
Semaphore > acquire — contended (concurrency=1, 2 tasks)	2,261,603.43	1,845,643.06	🟢 +22.5%
Semaphore > run() — 10 concurrent resolved promises	266,252.34	256,567.82	🟢 +3.8%
result helpers > emptyResult	15,791,180.67	18,712,871.97	🔴 -1.6%
result helpers > artifact	62,190.85	90,418.00	🔴 -1.7%
result helpers > warning	15,562,158.76	18,304,628.87	🔴 -1.4%
result helpers > infoMessage	16,059,819.04	17,560,848.56	🔴 -1.5%

Internal Composition

Benchmark	Ops/sec	Prev Minor (v0.7.2)	Change
resolvePluginRefs > 5 bare Plugin instances (identity pass-through)	11,970,347.11	15,642,920.84	🔴 -1.8%
resolvePluginRefs > 5 PluginRef with opts + .with() merging	1,675,890.96	1,553,125.11	🟢 +7.9%
resolvePluginRefs > 5 PluginRef with defaults (no registry lookup)	8,119,311.98	8,663,325.91	🔴 -1.7%
resolvePipeline > first match (image → photos)	7,396,375.67	10,996,692.92	🔴 -1.2%
resolvePipeline > nested match (video → media → videos)	6,132,217.20	9,848,416.47	🔴 -1.1%
resolvePipeline > no match (text → null)	8,532,523.88	12,852,350.97	🔴 -1.4%
validatePipeline > validatePipeline (valid)	2,556,097.84	3,511,061.24	🔴 -1.3%
validatePipeline > validatePipeline (nested, depth 4)	1,541,929.50	2,023,763.66	🔴 -1.4%
compose / stage > stage() by id+run	15,930,262.18	17,867,257.39	🔴 -1.1%
compose / stage > compose() 3 defs	5,774,856.39	5,360,573.12	🟢 +7.7%
sharedGet / sharedSet > sharedSet + sharedGet	12,142,821.68	14,727,983.94	🔴 -1.1%
createTimingMiddleware > wrap and run — no callback	1,687,287.99	1,578,290.67	🟢 +6.9%
createTimingMiddleware > wrap and run — with callback	1,704,824.85	1,589,380.42	🟢 +7.3%
Pipeline factory > Pipeline() — 3 stages	15,491,411.13	17,119,938.84	🔴 -1.6%
flattenPipeline > 10 flat stages	3,757,660.48	2,266,526.30	🟢 +65.8%
flattenPipeline > 3 nested sub-pipelines (depth 3)	5,156,515.98	3,521,964.45	🟢 +46.4%
runPipelineFrom > 3 stages via factory	711,396.84	687,971.02	🟢 +3.4%

Plugins (Individual)

Benchmark	Ops/sec	Prev Minor (v0.7.2)	Change
Plugin class > new Plugin() with run shorthand	15,855,312.86	17,113,468.39	🔴 -1.6%
Plugin class > Plugin.supports()	19,591,551.29	20,292,105.52	🔴 -0.8%
Plugin class > Plugin.with()	6,507,953.53	7,689,967.92	🔴 -1.4%
Plugin class > Plugin.with() with instanceId	5,106,420.92	5,842,252.34	🔴 -1.4%
Plugin class > Plugin.createStages()	1,304,657.42	1,809,506.82	🔴 -0.9%
PluginProvider > new PluginProvider()	1,185,039.98	1,316,399.39	🔴 -0.6%
PluginProvider > PluginProvider camelCase method	1,109,617.70	1,337,994.61	🔴 -1.4%
PluginProvider > PluginProvider.getPlugin() — found	1,170,229.16	1,344,249.90	🔴 -0.5%
PluginProvider > PluginProvider.getPlugin() — not found	1,181,385.79	1,383,136.50	🔴 -1.2%

Plugins (Pipeline Composition)

Benchmark	Ops/sec	Prev Minor (v0.7.2)	Change
runPipeline > 7 async stages (like real pipeline)	128.12	125.30	🟢 +2.3%
runPipeline > 7 stages with half skipped (when returns false)	224.26	221.78	🟢 +1.1%
runPipeline > stage error → onError fallback	298.67	293.40	🟢 +1.8%
runPipeline > stage error → onError skip	301.36	299.06	🟢 +0.8%
pipeline control flow > skipGroup	897,478.31	1,368,372.44	🔴 -1.4%
pipeline control flow > skipRemaining	1,329,881.57	1,187,327.97	🟢 +12.0%
pipeline control flow > removeFromQueue	1,329,784.83	1,198,756.09	🟢 +10.9%
parallel stages > 3 parallel stages	565,538.93	566,186.61	🔴 -0.1%
dependsOn > 2 stages with dependsOn	742,018.56	788,247.13	🔴 -0.7%