allocsight

English | 简体中文 | Español | Français | Русский | العربية

Table of Contents

• Overview
• Core Capabilities
• System Architecture
• Performance Benchmarks
• Command-Line Interface
• Query Filter Grammar
• Diagnostic Heuristics
• AI Agent & JSON Integration
• Building from Source
• License

Overview

AllocSight is a high-concurrency, zero-dependency command-line disk allocation analyzer and automated space reclamation engine for Windows. Written in native C++17 directly against the Win32 File Management APIs, it is engineered for software developers, systems administrators, and autonomous AI agents that require sub-second storage telemetry without graphical rendering overhead.

Unlike conventional directory size utilities that report only logical file lengths (nFileSizeLow / nFileSizeHigh), AllocSight computes true physical NTFS cluster allocations, accurately accounting for filesystem cluster alignment, NTFS LZNT1/XPRESS compression, sparse file regions, cloud-only recall placeholders, and alternate data streams (ADS).

Core Capabilities

1. Multi-Threaded Kernel Directory Enumeration Dispatches an 8-to-16 worker thread pool over a lock-minimized work queue (ParallelScanner). Each worker invokes FindFirstFileExW with FindExInfoBasic (omitting 8.3 short-name lookups) and FIND_FIRST_EX_LARGE_FETCH (enabling large kernel directory buffer reads).
2. True Physical Cluster Allocation Accounting
   • Aligns file allocations to the target volume's exact cluster size (GetDiskFreeSpaceW).
   • Queries actual on-disk compressed/sparse extents via GetCompressedFileSizeW when FILE_ATTRIBUTE_COMPRESSED or FILE_ATTRIBUTE_SPARSE_FILE is present.
   • Identifies cloud-only storage placeholders (FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS, FILE_ATTRIBUTE_RECALL_ON_OPEN, FILE_ATTRIBUTE_OFFLINE) as 0 local bytes to prevent inflated usage metrics on OneDrive or cloud-mounted volumes.
   • Optionally enumerates secondary NTFS Alternate Data Streams (:$DATA) using documented FindFirstStreamW / FindNextStreamW APIs (--ads).
3. Reparse Point Cycle Prevention Inspects WIN32_FIND_DATAW::dwReserved0 against <winnt.h> name-surrogate reparse macros (IsReparseTagNameSurrogate, IO_REPARSE_TAG_SYMLINK, IO_REPARSE_TAG_MOUNT_POINT) to prevent infinite directory recursion across symbolic links and volume mount junctions.
4. Heuristic Space Reclamation Engine Classifies reclaimable disk space into two actionable tiers:
   • [SAFE]: Deterministic package caches (uv, pip, conda/pkgs, npm/_cacache, pnpm-cache), shader caches (DXCache, ShaderCache), crash dumps, and orphaned staging directories.
   • [REVIEW]: Archives that have already been extracted into a sibling directory of the same stem name, duplicate backup directories, rebuildable build artifacts (target, node_modules, .venv), and stale large files unmodified for more than 180 days.
5. Direct-Navigation Markdown & JSON Reports Generates structured Markdown tables (allocsight report) and machine-readable JSON payloads (-j) where every directory and file path is accompanied by an RFC 8089 percent-encoded file:/// URI for single-click navigation in modern IDEs and Markdown viewers.

System Architecture

The codebase is organized as a header-modular C++17 pipeline under src/:

Performance Benchmarks: AllocSight vs. Mainstream AI Scanning Methods

When AI coding agents (such as Claude Code, Cursor, Codex, Antigravity, or Copilot) execute disk space diagnostics on Windows, they typically invoke shell pipelines (PowerShell Get-ChildItem), write one-off scripts (Python os.walk / Node.js fs), or call general-purpose CLI utilities.

To evaluate real-world performance with 100% rigor and zero fabrication, a deterministic test corpus of 100,000 files across 1,100 directories was generated on an NTFS volume with default 4 KB clusters (512 bytes per file: 48.8 MB logical data vs. 390.6 MB physical cluster allocation). Wall-clock latency was measured across all tools and configurations under identical warm-cache conditions (median of 5 runs; reproducible via benchmarks/run_benchmark.ps1):

Multi-Core Scaling & Legacy Device Simulation (1 ~ 16 CPU Cores)

To evaluate how AllocSight performs on older, lower-spec hardware (such as 10-year-old laptops, single-core cloud VMs, or low-power embedded industrial PCs), AllocSight provides an explicit worker thread control flag (-t <N>). Below is the empirical scaling curve across the 100,000-file NTFS test corpus:

Multi-Agent & Multi-Drive High-Concurrency Scanning (Agent Topology Guide)

When autonomous AI agent swarms (e.g. multi-agent systems scanning multiple drive partitions simultaneously via background subagents) perform whole-system disk reconnaissance, a common phenomenon is that one drive finishes very quickly while other drives appear to slow down. AllocSight's engineering diagnostic reveals four primary physical and architectural factors:

1. Physical SSD Colocation (Bus & Flash Channel Contention):
   • Multiple logical drive partitions frequently share the same underlying physical NVMe controller (for example, two logical partitions residing on the same physical SSD).
   • Running concurrent I/O scanners against two partitions residing on the same physical disk saturates the NVMe controller queue depth and destroys kernel read-ahead prefetching.
   • Best Practice: Concurrently scan partitions located on distinct physical disks. Partitions sharing the same physical disk should be scanned sequentially. Check disk topology beforehand via Get-Partition | Select DiskNumber, DriveLetter.
2. CPU Thread Oversubscription (Thread Budgeting):
   • Each AllocSight process auto-detects system CPU cores (e.g. 16 cores) and launches a matching worker pool.
   • Multiple concurrent processes without thread limits spawn dozens of threads, causing fierce contention for CPU cores and thrashing L1/L2/L3 caches.
   • Best Practice: Budget worker threads across agents with -t <cores / agents>. For example, on a 16-core workstation running 4 subagents, pass -t 4 (allocsight tree C:\ -t 4 -q), which reduces thread switching overhead by up to 70%.
3. Partition File Count Asymmetry:
   • Different volumes have drastically different file counts. For example, a small data volume may contain only tens of thousands of files (finishing in sub-second time), whereas a large volume with deep development dependencies contains millions of files (taking 10+ seconds). The difference in completion time reflects file count disparity rather than performance degradation.
4. System Volume (C:) Minifilter Overhead:
   • The OS system drive (C:\) incurs substantial latency from Windows Defender minifilter driver inspection (WdFilter.sys), active OS virtual memory paging, and NTFS access control list (ACL) security checks on protected system directories.

Performance Analysis & Architectural Trade-offs

1. Pure Scalar Addition vs. In-Memory Hierarchy Construction (dua vs. AllocSight): In raw wall-clock latency, dua finishes in 142.7 ms while AllocSight (16 cores) finishes in 168.4 ms (~25 ms difference).
   • dua's design: Optimized as an interactive TUI tool. It sums file sizes directly into CPU registers during kernel directory iteration without allocating heap memory to build a directory tree, without calculating 4 KB cluster alignment, and without file classification.
   • AllocSight's design: Built as an infrastructure tool for AI agents and disk diagnostics. Along with sub-second traversal, it constructs a complete in-memory FsNode hierarchy to compute directory percentage shares, evaluate cleanup heuristics ([SAFE] / [REVIEW]), and produce structured RFC 8089 file:/// output.
2. Physical Cluster Allocation vs. Logical Size: On NTFS volumes, files occupy full physical clusters (default 4 KB). For 100,000 files of 512 bytes:
   • Tools reporting only logical size show 48.8 MB, understating actual disk usage by 87.5%.
   • AllocSight dual-accounts both 48.8 MB logical data and 390.6 MB physical cluster footprint, giving AI agents accurate ground truth.
3. Batched Win32 Directory Streaming (FIND_FIRST_EX_LARGE_FETCH): AllocSight batches 64 KB kernel directory query buffers to minimize filesystem round-trips. Script-based approaches (Python os.walk, Node.js fs) issue individual syscalls per file, resulting in ~20-second latencies for the same corpus.
4. Cloud Recall Protection (FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS): Un-hydrated cloud placeholders (OneDrive, iCloud) report full logical file sizes while consuming 0 bytes of physical disk. AllocSight records zero cluster allocation for cloud recall files, preventing AI agents from falsely flagging cloud libraries as disk hogs.

Command-Line Interface

Synopsis

allocsight <command> [path] [options]

Commands

Options

Usage Examples

# 1. Inspect all logical volumes
allocsight drives

# 2. Render a 3-level allocation tree of D:\ showing items >= 100 MB
allocsight tree D:\ -d 3 -m 100mb -n 20

# 3. Identify Top 30 largest individual files on D:\
allocsight top D:\ --files -n 30

# 4. Run automated cleanup diagnostics on C:\ and export a Markdown report
allocsight report C:\ -n 40 -o cleanup_report.md -q

# 5. Filter for AI model weights (> 500 MB) modified more than 30 days ago
allocsight top D:\ --files -f "category:AI Models; allocated>500mb; modified>30days"

# 6. List mounted drives AND connected portable devices (smartphones/MTP)
allocsight drives --unlocaldisk

# 7. Scan a connected smartphone (e.g., photos folder in internal storage)
allocsight tree "wpd:0\DCIM" -d 2 -n 20 --unlocaldisk -q

# 8. Run smart cleanup diagnostics on a phone (APKs, large videos, downloads)
allocsight analyze "wpd:0" -n 30 --unlocaldisk -q

Smartphone & Portable Storage Inspection (MTP / WPD)

Smartphones connected via USB operate via the Media Transfer Protocol (MTP) within the Windows Portable Devices (WPD) namespace rather than standard block storage volumes with drive letters. AllocSight provides native zero-dependency WPD COM traversal:

• Explicit Safety Gating (--unlocaldisk): Because MTP devices communicate over serialized USB packet transactions without physical cluster allocation, scanning portable devices strictly requires the --unlocaldisk flag.
• Multiple Device Safety: When multiple smartphones are connected, AllocSight never initiates an unbounded multi-device sweep; it lists detected devices and requires targeting a specific index (wpd:0, wpd:1) or model name (wpd:Xiaomi).
• Subfolder Targeting: To avoid battery drain and slow USB transfers, users can directly inspect subtrees like "wpd:0\DCIM" or "wpd:Xiaomi\Download".

Query Filter Grammar

Filter expressions (-f "<expr>") consist of one or more clauses separated by semicolons (;). Prefixing any clause with ! or - negates the condition (exclusion).

Diagnostic Heuristics

The analyze and report commands evaluate the scanned FsNode hierarchy against deterministic structural rules:

AI Agent & JSON Integration

Passing -j / --json to any command emits deterministic, strictly escaped UTF-8 JSON to stdout while suppressing progress logs on stderr. Every reported entry includes both its native Windows path and an RFC 8089 fileUri:

{
  "root": "D:\\",
  "totalAllocatedBytes": 1429365116108,
  "safeReclaimableBytes": 34359738368,
  "safeReclaimableFormatted": "32.0 GB",
  "reviewReclaimableBytes": 103079215104,
  "reviewReclaimableFormatted": "96.0 GB",
  "candidates": [
    {
      "tier": "SAFE",
      "category": "safe_temp_cache",
      "reason": "Cache / temporary / crash-dump directory (>= 10 MB)",
      "type": "directory",
      "allocatedBytes": 23085449216,
      "allocatedFormatted": "21.5 GB",
      "modified": "2026/09/15",
      "path": "D:\\Cache\\pip\\http-v2",
      "fileUri": "file:///D:/Cache/pip/http-v2"
    }
  ]
}

For AI Agent integration instructions, see SKILL.md.

Building from Source

AllocSight requires a C++17 compiler targeting Windows x64 (MinGW-w64 GCC/Clang or MSVC) and links exclusively against standard operating system libraries (kernel32, advapi32).

Option 1: PowerShell Build Script (MinGW-w64)

.\build.ps1

Or compile directly in a single invocation:

g++ -O3 -s -std=c++17 -municode -static src/allocsight.cpp -o allocsight.exe

Option 2: CMake (MSVC or MinGW-w64)

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

License

Distributed under the terms of the MIT License. Copyright (c) 2026 AllocSight Contributors.