starling

Starling

Run real AI models inside your .NET app, with no Python sidecar and no cloud API, and with published evidence that they give the same answers as the upstream reference.

Starling packages open models as ordinary NuGet libraries: embeddings, entity extraction, image and video search, and chat. They run on ONNX Runtime on CPU or CUDA and plug into Microsoft.Extensions.AI (IEmbeddingGenerator, IChatClient), so code you wrote against a hosted provider can switch to a local model with a one-line change. Each model is pinned to an exact upstream commit, verified by SHA-256, and checked against the original PyTorch/Python implementation before it is released.

Using an AI coding agent? Point it at llms.txt or install the Starling skill.

Quickstart

Requires the .NET 10 SDK. Multilingual semantic search in four commands:

dotnet new console -o search && cd search
dotnet add package Starlings.Models.E5.Extensions.AI
dotnet add package Microsoft.ML.OnnxRuntime --version 1.30.0

Replace Program.cs with:

using System.Numerics.Tensors;
using Microsoft.Extensions.AI;
using Starlings.Models.E5;
using Starlings.Models.E5.Extensions.AI;

// One-time download, pinned by commit and checked by SHA-256. Inference never touches the network.
E5ModelFiles files = await E5ModelAssets.DownloadAsync();
using var embedder = E5EmbeddingGenerator.Create(files);

// e5 expects "passage: " on documents and "query: " on queries.
string[] docs = ["passage: The Eiffel Tower is in Paris.",
                 "passage: Der Rhein fließt durch Köln.",
                 "passage: 富士山は日本一高い山です。"];
var docVectors = await embedder.GenerateAsync(docs);
var query = await embedder.GenerateVectorAsync("query: Which river runs through Cologne?");

foreach (var (doc, v) in docs.Zip(docVectors))
    Console.WriteLine($"{TensorPrimitives.CosineSimilarity(query.Span, v.Vector.Span):F3}  {doc}");

dotnet run

0.680  passage: The Eiffel Tower is in Paris.
0.807  passage: Der Rhein fließt durch Köln.
0.688  passage: 富士山は日本一高い山です。

The first run downloads the model (about 487 MB) into your user cache; later runs work offline. The application chooses the ONNX Runtime native package: Microsoft.ML.OnnxRuntime for CPU, or Microsoft.ML.OnnxRuntime.Gpu.Linux for CUDA on Linux x64.

Local chat through IChatClient

dotnet add package Starlings.Models.Qwen35.Extensions.AI
dotnet add package Microsoft.ML.OnnxRuntimeGenAI --version 0.17.1

using Microsoft.Extensions.AI;
using Starlings.Models.Qwen35;
using Starlings.Models.Qwen35.Extensions.AI;

Qwen35ModelFiles files = await Qwen35ModelAssets.DownloadAsync(Qwen35Variant.Fp32x); // about 3 GB, once
using IChatClient chat = Qwen35ChatClient.Create(files);

await foreach (var update in chat.GetStreamingResponseAsync("In one sentence, why is the sky blue?"))
    Console.Write(update.Text);

Seven complete programs are in ai-dotnet/samples/: semantic search with e5 (E5Search) and with an in-memory Microsoft.Extensions.VectorData collection (E5VectorStore), text, image and video search with EmbeddingGemma 2 (EmbeddingGemma2Search), entity extraction with GLiNER2 (Gliner2Ner), streaming chat with Qwen3.5 (QwenChat), local question answering over your own files (LocalRag), and multilingual ticket triage in an ML.NET pipeline (TicketTriage).

Models

Choosing a text embedder? See choosing an embedding model.

Experimental, on nuget.org from 0.3.0: Starlings.Models.Kev (Kev-0.8B and Kev-4B decision models, CPU and CUDA) and Starlings.Models.QwenChat (Qwen3-0.6B and Qwen2.5-0.5B-Instruct text chat, CPU) have no Stable profile; their model files download from public hosting (release record). Starlings.Models.Bert builds and tests on main but is not published; build it from this repository.

Their candidate contracts are in model-pipeline/candidates/ and the BERT and Qwen chat onboarding workflow is in model-pipeline/FIVE_MODELS.md. The QwenChat sample uses the published Qwen3.5 packages, not Starlings.Models.QwenChat.

Stable means every release gate passed: provenance, numerical parity and task quality against the Python reference, a one-hour leak soak, benchmarks, a clean install from the packages, and integration. It also means regression runs continue: weekly on CPU, and on a GPU before every release for CUDA. Stable on CUDA means Linux x64 with an NVIDIA GeForce RTX 3090, where every CUDA gate and the per-release GPU run take place; other GPUs are untested. On Linux, GLiNER2 (CPU and CUDA) and EmbeddingGemma 2 video on CUDA also need MALLOC_MMAP_THRESHOLD_=65536 in the process environment; see each package README. The model catalog links each gate to its evidence, and the compatibility matrix lists each platform and provider.

Release status: 0.3.0, published on nuget.org on 2026-10-19 (GitHub release), adds the Experimental Kev and QwenChat packages, opt-in batching and the public stop-string matcher; every Stable profile of 0.2.0 is unchanged (release record). It depends on Starlings.Tokenizers 0.3.0.

How a model becomes a package

Every model goes through the same twelve stages of the delivery pipeline, drawn here as a production line. The original Python implementation runs once, from pinned code and weights, to cut a golden master: protected fixtures and tolerances that only the maintainer can change. Later stages are measured against it: the tokenizer, the exported graph in Python ONNX Runtime, then the same graph from C#. model-pipeline evaluate names the first stage that diverges. Gates G0 to G7 decide whether a profile is Stable, a maintainer approves each release, and the packages carry asset locks rather than weights. Your app downloads the weights once, verifies their hashes and then runs offline. Open the image for the full-size animated view.

Why Starling

Starling depends only on the vendor-neutral Microsoft.Extensions.AI.Abstractions. CI rejects any Azure or OpenAI SDK dependency in the libraries, so you can adopt it, or leave it, without lock-in.

Use it with your AI coding agent

Starling includes three files written for coding agents such as Claude Code, Copilot, Cursor and Codex:

• llms.txt is a short map of the repository: what Starling is, the facts that matter when writing code, and links to the rest.
• skills/starling-dotnet/SKILL.md is an Agent Skill that teaches an agent to use the packages. It covers which package fits which task, the native runtime the app must add, offline loading, verified code for every model, CUDA, and the pitfalls (e5 prefixes, EmbeddingGemma 2 prompts, input limits).
• skills/starling-add-model/SKILL.md is for contributors. It walks an agent through adding a new model in a pull request: choosing the onboarding route, writing the candidate contract, running the model-pipeline comparison against a fresh Python reference, and opening a draft PR. It also says what only the maintainer may change.

To give your agent context for a single task, paste this:

Read https://raw.githubusercontent.com/starlinghq/starling/main/llms.txt and
https://raw.githubusercontent.com/starlinghq/starling/main/skills/starling-dotnet/SKILL.md,
then use the Starlings.* NuGet packages to <what you want to build>.

To install the skill so it loads whenever it's relevant, for Claude Code:

mkdir -p .claude/skills/starling-dotnet
curl -fsSL https://raw.githubusercontent.com/starlinghq/starling/main/skills/starling-dotnet/SKILL.md \
  -o .claude/skills/starling-dotnet/SKILL.md

Use ~/.claude/skills/ instead of .claude/skills/ to install it for all your projects. Other agents that support Agent Skills take the same folder. For agents that don't, paste SKILL.md into your AGENTS.md or rules file.

Contributing

Issues and pull requests are welcome. Start with a good first issue, read CONTRIBUTING.md, and use the issue templates to request a model or report a regression. Security reports go through SECURITY.md.

Build and test

Requires the .NET 10 SDK (pinned in global.json) and Python 3.11+.

dotnet test --solution ai-dotnet/AI.slnx
python3 -I tools/check-boundary.py

cd model-pipeline
pip install -e '.[dev]'
python3 -m pytest
ruff check .
ruff format --check .
model-pipeline validate registry

Repository layout

The tokenizer is its own product, Starlings.Tokenizers from starlinghq/tokenizers. The execution plan is docs/plan/fast-faithful-ai-dotnet.md, and open decisions are in docs/kickoff.md. An experimental five-model CPU onboarding workflow covers additional Qwen and BERT models; its references are still proposals. nomic-embed-text-v1.5 joined the BERT route afterwards (#120).

License

Original code is licensed under the MIT License or the Apache License, Version 2.0, at your option (LICENSE-MIT, LICENSE-APACHE), copyright StarlingHq. Model weights, tokenizer assets, and datasets keep their own terms; the code license does not cover them.