Explore
MCP servers and skills: search, filter by where they run, and compare.
MCP servers and skills: search, filter by where they run, and compare.
1–24 of 55 tools for “benchmark-framework”
Clear allTeach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Deterministic-first, tool-agnostic.
in the skill textdead sandbox reported as CANNOT-MEASURE instead of a fail | | **A benchmark behind a public claim** | "Can we put a number in the launch post?" | `benchmark.md
SkillApache-2.0★ 19No key
TNE-SDK - The Official Python SDK for The Null Epoch - an AI-only MMO where LLMs play autonomously. Connect any model (OpenAI, Anthropic, Ollama, vLLM, etc.) via MCP, TUI launcher, or raw HTTP. Includ
PythonMIT★ 13No key
AssetOpsBench - Industry 4.0: A unified benchmark and framework for building, orchestrating, and evaluating domain-specific AI agents for Industry 4.0 asset operations and maintenance, with 460+ scena
OfficialPythonApache-2.0★ 2.4kNo key
An AI skill that provides design intelligence for building professional UI/UX across multiple platforms.
in the skill textthat provides design intelligence for building professional UI/UX across multiple platforms and frameworks. If you find this useful, consider supporting the project: Other projects NextLevelBuilder.io | GoClaw.sh
SkillMIT★ 137kNo key
Use this skill when the operator is authoring, building, loading, or debugging a custom doca-bench plug-in — a versioned shared library with DOCA_EXPERIMENTAL-marked C entry points that doca-bench loa
in the skill textFIRST and only routes to the extension framework when no built-in mode applies. - *"I want to benchmark a CUDA / GPU-side workload that drives DOCA
OfficialSkillApache-2.0★ 3.6k in collectionNo key
Evidence-gated workflow for MoE performance optimization in Megatron Bridge. Covers measurement contracts, the Three Walls framework, parallel folding, profiling, matched A/B tuning, and final validat
in the skill textMoE performance optimization in Megatron Bridge. Covers measurement contracts, the Three Walls framework, parallel folding, profiling, matched A/B tuning, and final validation. # MoE Training Optimization Workflow
OfficialSkillApache-2.0★ 3.6k in collectionNo key
Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect a
OfficialSkillApache-2.0★ 572No key
Automate your mobile devices with natural language commands - an LLM agnostic mobile Agent 🤖
in the readmeMobilerun is an open-source framework for controlling Android and iOS devices with LLM agents. It gives agents mobile-native tools to inspect UI state, understand
Hosted by authorMIT★ 9.6kNo key
Open-source, self-hosted AI penetration testing framework: maps your attack surface into a graph, autonomously exploits it from a Kali sandbox with human approval gates, and opens PRs that fix what it
in the readmeUnmask the hidden before the world does An autonomous AI framework that chains reconnaissance, exploitation, and post-exploitation into a single pipeline, then goes further
SkillMIT★ 3.1kNo key
My own collection of skills for modern Node.js development
in the skill textAvailable Skills | Skill | Description | |-------|-------------| | `documentation` | Technical writer specializing in the Diátaxis documentation framework | | `fastify` | Comprehensive best practices for Fastify development | | `init` | Creates and maintains high-signal
SkillMIT★ 2.0kNo key
Structured deep research skill for Claude Code/Open Code/Codex with human-in-the-loop control
in the skill textSkills Workflow ## Use Cases - **Academic Research**: Paper surveys, benchmark reviews, literature analysis - **Technical Research**: Technology comparison, framework evaluation, tool selection - **Market Research**: Competitor analysis, industry trends
SkillMIT★ 2.4kNo key
turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then runs tree search with parallel subagents.
in the skill textInstall ** · ** How it works ** · ** Dashboard ** · ** Upgrading ** A plugin for your agentic framework that optimizes code through experiments You give it a codebase. It discovers metrics
SkillApache-2.0★ 1.5kNo key
Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scrat
in the readmeinterviewed on it: framework-free, on a free API, from prompting all the way to serving, fine-tuning, and a red-team benchmark.** Runnable Colab notebooks
SkillMIT★ 701No key
OpenBrowser is a framework for intelligent browser automation. It combines direct CDP communication with a CodeAgent architecture, where the LLM writes Python code executed in a persistent namespace,
in the skill textbrowser automation using CodeAgent and CDP (Chrome DevTools Protocol)** OpenBrowser is a framework for intelligent browser automation. It combines direct CDP communication with a CodeAgent architecture
SkillMIT★ 276No key
Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
in the readmeopen-source, **agent-agnostic** framework for **evaluating and benchmarking AI coding agents and their skills** — built for benchmark authors, CLI builders, and skill builders — with sandboxing
PythonApache-2.0★ 151No key
OpenART is an open-source framework designed to evaluate the safety and robustness of autonomous AI agents in dynamic, long-horizon, and stateful environments. It stress-tests agent runtimes against m
in the skill textprioritizes the most usable OpenART tools for task execution, reuse, and reproducible benchmark construction. The release includes metadata, archive manifests, checksums, and a lightweight download helper
SkillAGPL-3.0★ 245No key
Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters
in the skill textmodel call in the grade path — and writes benchmark reports you can diff across variants. General eval frameworks (openai/evals, vitest-evals, viteval) score one output against
SkillMIT★ 77No key
The memory layer for production AI agents. Python + JS SDKs, 24 framework integrations, MCP server.
in the readmehosted | | This repo (Apache 2.0) | Synap Cloud (managed) | |---|---|---| | Python + JS SDKs, 24 framework packages, MCP adapter | ✅ open source | uses them | | Memory engine: ingestion, entity resolution, retrieval
Hosted by authorApache-2.0★ 77No key
Physical AI Studio is an end-to-end framework for training robots to perform tasks through imitation learning from human demonstrations.
SkillApache-2.0★ 139No key
Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external
in the skill textquery expansion to recall them later. Compared to general-purpose LLM memory frameworks like `mem0` or the MCP reference `memory` server, claude-mem-lite is purpose
SkillMIT★ 67No key
Standardized environment infrastructure for Agentic AI development.
in the readmekey environment layer technology, deeply integrated with the AReaL reinforcement learning framework, supporting large-scale Agentic RL training and agent service deployment. ### Core Philosophy: Everything
SkillApache-2.0★ 319No key
The evaluation benchmark on MCP servers
in the readmeBenchmark for Evaluating MCP Servers [![Documentation][docs-image]][docs-url] [![Package License][package-license-image]][package-license-url] 中文 | English MCPBench is an evaluation framework for
SkillApache-2.0★ 252No key
PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents
in the skill textand reference manifests ├── mock_services/ # local services used by benchmark tasks ├── agents/ # supported third-party agent frameworks ├── scripts/ # experiment and reporting scripts └── tests/ # test suite ``` ## 🙏 Third
SkillApache-2.0★ 34No key
MCP Server for Linux Scheduler Management and Auto optimization
in the readmebenchmark for evaluating the optimizations for OS! SchedCP is an `experimental` project that enables AI optimization of Linux kernel schedulers using the sched-ext framework
SkillMIT★ 119No key
Who stands behind it