medical-charts-hedis-skill

Clinical Chart HEDIS / QRS Quality Measures Extractor

An enterprise-grade agent skill and clinical evaluation framework for extracting Healthcare Effectiveness Data and Information Set (HEDIS), Quality Rating System (QRS), and CMS Star Ratings clinical quality measures, laboratory flowsheets, vital signs, and open care gaps from unstructured and semi-structured electronic health records (EHRs), narrative progress notes, and PDF medical continuity charts.

Table of Contents

• Overview & Core Value
• Key Architectural Invariants
• Repository Structure
• 7-Step Clinical Extraction Pipeline
• Supported Quality Measures & Domains
• JSON Output Schema & Specification
• Clinical Extraction Gotchas & Rules
• Testing & Validation Guide
  • 1. Running the Automated Python Test Suite
  • 2. Using the Referential Integrity & Schema CLI Validator
  • 3. Interactive Agent Testing via agentapi
  • 4. End-to-End Extraction Pipeline via Python
• Sample Clinical Test Cases & Synthetic Data Notice
• Compliance & Standards Reference
• License

Overview & Core Value

Health plans and health systems must review millions of pages of clinical documentation annually to calculate quality scores (HEDIS / CMS Stars / QRS) and close care gaps. Manual abstraction is slow, costly, and prone to error.

This skill equips AI agents and automated pipelines with rigorous medical chart review logic, combining:

1. Multi-Domain Clinical Quality Ontologies (LOINC, CPT, CPT Category II, HCPCS, ICD-10-CM).
2. Strict Referential Integrity: Zero-tolerance policy for hallucinations; every extracted metric, date, and care gap decision is anchored to an exact character-for-character quote from the chart.
3. Deterministic Mathematical & Temporal Rules: Strict enforcement of lookback windows, specimen collection dates (never execution timestamps), repeat blood pressure overrides, and post-discharge encounter timeframes.

Key Architectural Invariants

Repository Structure

medical-charts-hedis-skill/
├── SKILL.md                              # Core Agent Skill definition, instructions & invariants
├── README.md                             # Comprehensive project documentation
├── LICENSE                               # Apache License Version 2.0
├── assets/
│   └── clinical_skill_action.jpg         # Visual workflow diagram and dashboard preview
├── schemas/
│   └── hedis_extraction_schema.json      # Formal JSON Schema for extraction payloads
├── references/
│   └── hedis_measures_reference.md       # Clinical measure definitions, codes & lookback windows
├── scripts/
│   └── validate_extraction.py            # Standalone CLI validation & referential integrity script
├── tests/
│   ├── __init__.py
│   └── test_hedis_skill.py               # Unit & integration test suite (12 test cases)
└── test_charts/
    ├── chart_1_eleanor_vance.pdf         # Patient 1: Type 2 Diabetes, HTN, Preventive (PDF)
    ├── chart_1_eleanor_vance.txt         # Patient 1: Text format
    ├── chart_1_extraction.json           # Patient 1: Ground truth extracted JSON
    ├── chart_2_marcus_thorne.pdf         # Patient 2: Inpatient Post-Discharge, CVD, Fracture (PDF)
    ├── chart_2_marcus_thorne.txt         # Patient 2: Text format
    ├── chart_2_extraction.json           # Patient 2: Ground truth extracted JSON
    ├── chart_3_beatrice_soto.pdf         # Patient 3: Well-Woman Preventive, SDOH (PDF)
    ├── chart_3_beatrice_soto.txt         # Patient 3: Text format
    └── chart_3_extraction.json           # Patient 3: Ground truth extracted JSON

7-Step Clinical Extraction Pipeline

[Input Medical Chart (PDF / EHR Text)]
                 │
                 ▼
┌────────────────────────────────────────────────────────┐
│ Step 1: Patient Profile & Context Isolation            │
│  - Extract Name, DOB, Plan ID, MRN verbatim            │
│  - Verify consistent identity across all chart pages   │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 2: Temporal Anchor & Encounter Parsing            │
│  - Identify clinical Dates of Service (YYYY-MM-DD)     │
│  - Classify encounter types & attending providers      │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 3: Clinical Measurements & Vitals Extraction      │
│  - Parse Blood Pressures (triage vs repeat, lowest)    │
│  - Extract Lab Flowsheets (A1c, eGFR, Creatinine, uACR)│
│  - Standardize units, ranges, abnormal flags           │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 4: Diagnostic Reports & Procedures Parsing        │
│  - Mammograms, Colonoscopies, Eye Exams, DEXA, Pap     │
│  - Verify provider specialty & lookback compliance     │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 5: Older Adult, Care Transition & Screenings      │
│  - Advance Care Planning (ACP), Functional/Pain Status │
│  - Post-discharge Med Rec (<= 30 days of discharge)    │
│  - PHQ-9 Depression, SDOH Social Needs screenings      │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 6: Care Gap & Quality Measure Reconciliation      │
│  - Reconcile findings against NCQA / CMS Star specs    │
│  - Assign Status: CLOSED_MET, OPEN_GAP, EXCLUDED       │
│  - Generate actionable clinical follow-up items        │
└────────────────────────┬───────────────────────────────┘
                         │
                         ▼
┌────────────────────────────────────────────────────────┐
│ Step 7: Referential Integrity & Provenance Validation  │
│  - Verify exact verbatim quotes exist in source doc    │
│  - Emit validated JSON conforming to official schema   │
└────────────────────────────────────────────────────────┘

Supported Quality Measures & Domains

For detailed clinical specifications and code sets, see references/hedis_measures_reference.md.

JSON Output Schema & Specification

Every extraction produces a structured JSON document conforming to schemas/hedis_extraction_schema.json.

Key Output Sections

{
  "patient_profile": {
    "verbatim_name": "VANCE, ELEANOR V.",
    "normalized_name": "Eleanor V. Vance",
    "date_of_birth": "1954-11-14",
    "verbatim_dob": "11/14/1954",
    "gender": "Female",
    "identifiers": [
      { "type": "Member_ID", "value": "MED-8849201-A" },
      { "type": "MRN", "value": "MRN-4920188" }
    ],
    "coverage_plan": "Medicare Advantage Complete HMO"
  },
  "document_metadata": {
    "source_file": "chart_1_eleanor_vance.pdf",
    "document_type": "Comprehensive Annual Wellness Visit & Clinical Flowsheet",
    "dates_of_service_found": ["2026-03-30", "2026-03-15"],
    "facility_name": "Oakridge Health System - Internal Medicine Center",
    "primary_provider": {
      "name": "Dr. Sarah Jenkins, MD",
      "role": "Attending Physician",
      "npi": "1497820134",
      "specialty": "Internal Medicine"
    },
    "total_pages": 4
  },
  "vital_signs": [...],
  "laboratory_results": [...],
  "diagnostic_screenings": [...],
  "medications_and_reconciliation": {...},
  "assessments_and_care_planning": {...},
  "hedis_quality_measures": [...],
  "care_gaps_summary": {
    "total_measures_evaluated": 8,
    "closed_gaps_count": 6,
    "open_gaps_count": 2,
    "excluded_count": 0,
    "open_action_items": [...]
  },
  "extraction_metadata": {
    "extractor_version": "2.0.0",
    "extraction_timestamp_utc": "2026-08-15T19:20:00Z",
    "referential_integrity_verified": true,
    "provenance_accuracy_pct": 100.0,
    "notes_or_warnings": []
  }
}

Clinical Extraction Gotchas & Rules

Testing & Validation Guide

1. Running the Automated Python Test Suite

The project includes a comprehensive test suite in tests/test_hedis_skill.py using Python's standard unittest library (zero external dependencies required).

Run directly:

python3 tests/test_hedis_skill.py

Or run with standard unittest discovery:

python3 -m unittest discover tests

What the test suite verifies:

• 12 automated test cases across all three clinical chart datasets.
• Character-for-character referential integrity of all verbatim quotes against raw chart text.
• ISO YYYY-MM-DD date validation across all service and birth dates.
• Care gap mathematical accounting (total == closed + open + excluded).
• Domain-specific clinical rule execution (e.g. HbA1c control thresholds, repeat sitting BP logic, 30-day post-discharge med rec windows).

2. Using the Referential Integrity & Schema CLI Validator

Use scripts/validate_extraction.py to audit any extracted JSON against its source document:

# Validate Patient 1 (Eleanor Vance)
python3 scripts/validate_extraction.py \
  test_charts/chart_1_extraction.json \
  test_charts/chart_1_eleanor_vance.txt

# Validate Patient 2 (Marcus Thorne)
python3 scripts/validate_extraction.py \
  test_charts/chart_2_extraction.json \
  test_charts/chart_2_marcus_thorne.txt

# Validate Patient 3 (Beatrice Soto)
python3 scripts/validate_extraction.py \
  test_charts/chart_3_extraction.json \
  test_charts/chart_3_beatrice_soto.txt

3. Interactive Agent Testing via agentapi

You can drive the extraction skill via the terminal or automation scripts using the agentapi CLI:

agentapi new-conversation "Analyze test_charts/chart_1_eleanor_vance.pdf according to SKILL.md and generate a validated extraction JSON conforming to schemas/hedis_extraction_schema.json"

4. End-to-End Extraction Pipeline via Python

To automate extraction from raw charts using Gemini / Vertex AI and validate the results:

import json
import subprocess
from google import genai
from google.genai import types

# 1. Load skill instructions, schema, and chart
with open("SKILL.md", "r") as f:
    skill_instructions = f.read()

with open("schemas/hedis_extraction_schema.json", "r") as f:
    schema = json.load(f)

with open("test_charts/chart_1_eleanor_vance.txt", "r") as f:
    chart_text = f.read()

# 2. Call Gemini model
client = genai.Client()
prompt = f"""
{skill_instructions}

SOURCE MEDICAL CHART:
{chart_text}

Extract all HEDIS quality measures, vitals, labs, screenings, and care gaps conforming to the schema.
"""

response = client.models.generate_content(
    model="gemini-2.5-pro",
    contents=prompt,
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
    ),
)

# 3. Save extracted output
output_path = "test_charts/output_test.json"
with open(output_path, "w") as f:
    f.write(response.text)

# 4. Verify referential integrity with the validator script
result = subprocess.run([
    "python3", "scripts/validate_extraction.py",
    output_path, "test_charts/chart_1_eleanor_vance.txt"
], capture_output=True, text=True)

print(result.stdout)

Sample Clinical Test Cases & Synthetic Data Notice

[!NOTE] Synthetic Data & Privacy Notice: All patient names, profiles, identifiers (MRN, Plan IDs), clinical progress notes, laboratory flowsheets, diagnostic reports, and medical chart documents in test_charts/ are 100% synthetically generated for benchmarking, evaluation, and unit testing. They contain zero Protected Health Information (PHI) and do not depict or correspond to any actual individuals or patients.

Compliance & Standards Reference

• NCQA HEDIS MY2026: National Committee for Quality Assurance Technical Specifications for Health Plans.
• CMS Star Ratings: Centers for Medicare & Medicaid Services Medicare Advantage & Part D Quality Ratings.
• QRS: Quality Rating System for Health Insurance Marketplace Plans.
• HIPAA Privacy & Security: Complies with Safe Harbor de-identification and isolation standards for automated clinical pipelines.

License

This project is licensed under the Apache License, Version 2.0. See the LICENSE file for the full license text.