cloudg

---

One command collects assets from every configured provider in parallel, feeds them through a NetworkX graph for reachability and attack path analysis, fans out to Prowler, ScoutSuite, Checkov and Trivy, then merges and deduplicates all findings against 28 compliance frameworks. Out the other end come an interactive HTML report, GraphML, an RDF ontology, RAG chunks for LLM pipelines, and Terraform files that recreate the live infrastructure.

Need the map without the security tooling? cloudg map is a scanner-independent inventory mapper: it deep-collects everything deployed (or default) in an account, down to the network fabric, sweeps every service for the rest, and links it all into one asset map you can overlay with scanner findings later. See Inventory mapping.

Want an AI agent to work with all of it? cloudg mcp serve turns cloudg into an MCP server for Claude Desktop, Claude Code, Cursor or anything else that speaks the Model Context Protocol. See MCP server.

Full documentation: the rendered handbook lives at morpheuslord.github.io/cloudg, with the same content as markdown in the feature reference and release notes in the changelog. The inventory mapper has its own deep-dive documents: the inventory reference (every return structure and output file, field by field), the inventory catalog (all 156 asset types, their metadata and relationships) and the inventory internals (how it works and how to extend it). The MCP server is covered by the MCP guide, the tool reference and the privacy reference, and throttling by rate limits and throttling.

Sample report: to see what cloudg produces before running anything, open the sample report. It is the report.html of a synthetic AWS, Azure and GCP estate, so every ID in it is made up, and scripts/build_sample_report.py rebuilds it from the test fixtures.

---

Quick start

git clone https://github.com/morpheuslord/cloudg.git
cd cloudg

uv venv && source .venv/bin/activate
uv pip install -e ".[all,dev]"

# one provider, one region
cloudg run -p aws --regions us-east-1

# everything, everywhere
cloudg run -p all --regions all

# with Terraform recreation files
cloudg run -p aws --regions us-east-1 --terraform

Reports land in ./reports. Open report.html first.

Already ran the scanners yourself? Feed cloudg their native output files instead. No cloud credentials, no scanner binaries:

cloudg ingest --prowler ./prowler-output/ --checkov ./results_json.json --trivy ./trivy.json

Any combination of Prowler, ScoutSuite, Checkov and Trivy outputs works; cloudg normalises, deduplicates across scanners via the check-equivalence rulesets, maps compliance frameworks, and renders the reports.

Plain pip works too: pip install -e ".[all]". The cloud SDKs are extras, so pip install cloudg[aws] pulls only boto3/aioboto3, [azure] and [gcp] do the same for their SDKs, and [all] installs the lot. The core package with no extras still gives you the graph engine, the ontology, the normaliser and the report renderers.

The external scanners (Prowler, Checkov, Trivy, ScoutSuite) are separate executables, not Python dependencies. install.sh (Linux/macOS) and install.bat (Windows) set up everything including the scanners and the cloud CLIs.

Docker is the lazy path: the image bundles Prowler, Checkov and Trivy (ScoutSuite is not included, so run it separately and feed its report to cloudg ingest):

docker build -t cloudg:latest .
docker compose run --rm cloudg run -p aws --regions us-east-1

---

Inventory mapping

cloudg map answers a different question than cloudg run: what exists, and how is it wired together. Finding out what is wrong stays the scanners' job, and none of them runs here or even needs to be installed. What makes the map complete:

• Deep service collectors. Containers (ECR repositories, ECS services and task definitions, EKS clusters, nodegroups, Fargate profiles, addons, access entries) and the Kubernetes workloads running inside EKS; Lambda with its triggers, function URLs, images and layers; API Gateway, SQS, SNS, EventBridge, Step Functions, Kinesis; data stores; Route 53; CloudFormation stacks and the resources they manage; the whole IAM graph; and the security services and scanners watching the account (GuardDuty, Security Hub, Inspector, Macie, Config, Access Analyzer, WAF, Network Firewall, Shield, CloudTrail), including where they are not enabled.
• Breadth across every service. On AWS, a Cloud Control API sweep lists every resource type that supports listing (800+), tagged or not, next to 133 dedicated collectors. Azure is read through Azure Resource Graph across every subscription and management group, GCP through Cloud Asset Inventory across the whole organization, both with full resource properties.
• Every account at once. --org discovers the AWS Organization from the management account (OU tree, accounts, SCPs, and the Control Tower landing zone with its governed regions and enabled controls) and maps every member account.
• Typed interdependencies. Edges say what they mean: S3 INVOKES Lambda, task definition USES_IMAGE ECR repository, function ASSUMES_ROLE role, WAF PROTECTS ALB, Inspector MONITORS instance, stack MANAGES resource, SCP GOVERNS OU, plus cross-account trust to accounts you did not map. cloudg deps then answers "what does this need" and "what breaks if it goes".

# map one provider
cloudg map -p aws --regions all

# every account in the AWS Organization / Control Tower landing zone
cloudg map -p aws --org --regions all

# map everything, everywhere
cloudg map -p all --regions all

# what depends on this role, and what does it depend on?
cloudg deps arn:aws:iam::123456789012:role/app-role

# overlay scanner findings you generated earlier -> asset map + compliance map
cloudg map -p aws --regions all --findings ./reports/raw-findings.json

Outputs: inventory-map.json (assets, interconnections, collection coverage, summary), inventory-map.graphml, inventory-graph.json for viewers, inventory-dependencies.json (shared dependencies, blast radius, cross-account edges, security coverage gaps) and, with --org, inventory-organization.json. With --findings, additionally asset-map.json (each asset with its findings and severity breakdown) and compliance-map.json (framework → affected assets). The map never depends on the scanners; you can map today and merge in findings from a scan you run next week. Every file is specified field by field, with annotated examples, in the inventory reference.

All of this is also a library API, see Using it as a library:

from cloudg import CloudGConfig, CloudGEngine

engine = CloudGEngine(CloudGConfig(providers=["aws", "azure", "gcp"]))
inventory = engine.map_inventory_sync(output_dir="./reports")
print(inventory.summary["assets_by_service"])

---

MCP server

cloudg can hand its inventory, graph, findings, compliance mappings and ontology to AI agents over the Model Context Protocol: 74 tools, 13 resources, 11 resource templates and 11 prompts, from find_assets and attack_paths to blast_radius, compliance_gaps and a security_posture_review prompt.

pip install "cloudg[mcp]"     # the native server needs no extra; this adds the official SDK (mcp>=1.30,<3)

# stdio for a desktop client, with a map you already made
cloudg mcp serve --dataset prod=./reports/inventory-map.json
cloudg mcp config --client claude-desktop --dataset prod=./reports/inventory-map.json

# streamable HTTP on 127.0.0.1:8765/mcp with a bearer token
cloudg mcp serve --transport http --auth-token env:CLOUDG_MCP_TOKEN:analyst

Already running an MCP server? Mount cloudg into it (official mcp SDK 1.x or 2.x, or fastmcp) with CloudGMCPLayer(...).register_into(server), or skip the wire and await layer.call_tool(...) from Python. Every request, whichever way it arrives, goes through one policy. The default standard profile redacts secrets and fences attacker-controllable text; strict also pseudonymises account ids, ARNs, IPs and names (pseudonyms passed back as arguments are resolved) and hides the tools that touch the cloud. The live tools (map_inventory, collect_assets, run_scanners, run_pipeline) check credentials before the first API call and refuse to re-collect a scope that was collected a couple of minutes ago.

The MCP guide covers serving, client setup, transports, mounting and in-process use; the tool reference lists every primitive, and the privacy reference every profile and transform.

Rate limits

Mapping a whole AWS Organization means thousands of API calls per account, so every call cloudg makes goes through a shared rate limiter with per-service limits at or below what each provider publishes. A throttled API gets slowed down for every client at once, Retry-After is honoured, requests in flight are capped per account, retries beyond the first two of each call draw from one budget per provider, and an API that stays throttled is skipped for a while by a circuit breaker. The map does not fail over it: coverage says throttled: ... and the result carries a throttling summary. Limits are tunable under ratelimit in config.yaml; see rate limits and throttling.

---

Authentication

Every provider supports several auth methods, resolved in a fixed priority order. The same config works on a laptop, in CI, and on cloud compute. The full set of fields lives in config.yaml with comments for each method.

1. Direct keys: --aws-key / --aws-secret (plus --aws-session-token for temporary credentials), or the standard env vars.
2. OIDC web identity federation: --aws-role-arn together with --aws-web-identity-token-file. This is the GitHub Actions / GitLab CI / EKS service account pattern, no long-lived keys anywhere.
3. A named CLI profile via --profile, including SSO profiles.
4. Nothing at all: the default chain picks up env vars, cached SSO credentials, or the EC2/ECS instance role, so a scan running on cloud compute inherits its host's role.

On top of any of these you can layer STS role assumption with --aws-role-arn and, for the third-party auditor pattern, --aws-external-id. Multi-account fan-out uses accounts plus role_name in the config file, and cloudg assumes that role in each account before collecting.

1. Workload identity federation: --azure-tenant-id, --azure-client-id and --azure-federated-token-file (AKS workload identity, GitHub OIDC).
2. Service principal with a client secret: --azure-client-secret.
3. Service principal with a certificate: --azure-cert-path.
4. Managed identity: --azure-managed-identity, with managed_identity_client_id in the config for user-assigned identities.
5. The DefaultAzureCredential chain, which also covers az login sessions.

1. A credentials file via --gcp-credentials-file: either a service account key JSON or a workload identity federation (external_account) config.
2. Application default credentials: GOOGLE_APPLICATION_CREDENTIALS, gcloud user credentials, or the GCE/GKE metadata server.

--gcp-impersonate-sa layers service account impersonation on top of either, which is handy when your user account may impersonate a read-only scanner service account.

---

What the pipeline does

graph LR
    A[Collect<br/>AWS + Azure + GCP] --> B[Graph<br/>reachability, attack paths]
    A --> C[Scanners<br/>Prowler, Checkov, Trivy, ScoutSuite]
    B --> D[Ontology + RAG + Terraform]
    C --> E[Normalise<br/>dedupe, score, map to frameworks]
    D --> F[Reports]
    E --> F

Collection runs all providers concurrently with asyncio, iterating accounts and regions per provider (regions are auto-discovered when you pass --regions all). Assets and network edges go into a directed graph, where BFS from the internet node over network-flow edges finds exposed resources and blast radius scoring estimates what an attacker could reach from each node.

The same inventory feeds three other exports. The ontology module infers 64 typed relations (INTERNET_REACHABLE, ROLE_ASSUMES_ROLE, ENCRYPTED_BY_KMS, VPC_CONTAINS_SUBNET and so on) and writes RDF you can query with SPARQL. The RAG exporter chunks the graph three ways (per asset, per Louvain community, per relation domain) into JSONL for retrieval pipelines. The Terraform exporter maps 25+ asset types to .tf.json resources with an import_commands.sh to adopt them into state.

Scanner findings are deduplicated in two passes (within a scanner by scanner, check ID and resource; across scanners only when both the normalised title and the underlying check semantics in cloudg/rules/check_equivalence.yaml match), then rescored against CVSS and mapped to compliance controls.

---

Compliance rules

Findings are tagged with framework controls in four tiers, most precise first:

1. Whatever the scanner itself reports (Prowler ASFF or OCSF, Checkov check IDs).
2. Exact check-ID lookup against the shipped rulesets: 22 framework files generated from Prowler's public compliance data (Apache-2.0), with 4,166 controls and 10,236 check mappings across AWS, Azure and GCP: CIS 5.0 for each cloud, NIST 800-53 rev 5, NIST CSF 2.0, PCI DSS 4.0, SOC 2, HIPAA, GDPR, ISO 27001:2022, MITRE ATT&CK, and the AWS Foundational Security Best Practices.
3. Regex pattern rules for scanners that emit no compliance metadata: 9 hand-written files in cloudg/rules/ with 105 controls. Three of them add to the generated CIS frameworks, so the two sets together cover 28 frameworks and 4,271 controls.
4. A small built-in fallback table.

A control that a finding maps to is reported as FAIL. A control that only Prowler's passing checks map to is reported as PASS; every other control was not assessed and is left out.

The rulesets ship inside the package (cloudg/rules/). To refresh them against a newer Prowler release:

git clone --depth 1 https://github.com/prowler-cloud/prowler /tmp/prowler
python scripts/import_prowler_compliance.py /tmp/prowler

Adding your own framework is a YAML file in the rules directory:

framework: MY-FRAMEWORK
controls:
  - id: "MF-1.1"
    title: "Storage is encrypted"
    patterns: ["encrypt.*rest"]        # regex tier
    checks: ["s3_default_encryption"]  # exact tier, optional

cloudg/policies/ additionally holds Cloud Custodian policy packs (AWS governance, AWS security, Azure, GCP) you can run with custodian run independently of cloudg.

---

Reference

Flag	Meaning
-p, --provider	aws, azure, gcp or all; repeatable
--regions	all for auto-discovery, or a comma-separated list
--aws-key, --aws-secret, --aws-session-token	direct AWS credentials
--aws-role-arn, --aws-external-id	STS role assumption
--aws-web-identity-token-file	OIDC token file for web identity federation
--profile	AWS CLI profile
--subscription-id, --azure-tenant-id, --azure-client-id	Azure identity
--azure-client-secret, --azure-cert-path	service principal credentials
--azure-federated-token-file, --azure-managed-identity	federation / managed identity
--project-id, --gcp-credentials-file, --gcp-impersonate-sa	GCP identity
--scanners	comma-separated subset of prowler,scoutsuite,checkov,trivy,iam
--iac-dir	directory for Checkov to scan
--images	container images for Trivy
--ontology/--no-ontology	RDF ontology export (on by default)
--rag-export/--no-rag-export	RAG chunk export (on by default)
--terraform/--no-terraform	Terraform recreation (off by default)
-o, --output	output directory, ./reports by default

cloudg collect and cloudg scan run the individual phases; cloudg map builds the scanner-independent inventory map (--findings merges existing findings into asset/compliance maps, --no-sweep skips the catch-all sweep); cloudg ingest aggregates scanner outputs you already have (--prowler, --scoutsuite, --checkov, --trivy, each taking a file or directory and repeatable); cloudg report -i findings.json re-renders reports from a previous run.

File	What it is
report.html	interactive report, D3 topology plus findings table, works offline
findings.json	all findings, assets, edges and compliance results
topology.svg, topology.graphml	the graph as a picture and as GraphML; topology-cytoscape.json too with graph.export_cytoscape: true
ontology.ttl, ontology.jsonld	the RDF ontology
rag_chunks.jsonl, rag_metadata_index.json	retrieval-ready chunks
terraform/*.tf.json, terraform/import_commands.sh	recreation files
inventory-map.json, inventory-map.graphml, inventory-graph.json	scanner-independent inventory map (cloudg map)
asset-map.json, compliance-map.json	inventory merged with scanner findings (cloudg map --findings)

---

Using it as a library

from cloudg import CloudGConfig, CloudGEngine

config = CloudGConfig(providers=["aws"])
config.aws.role_arn = "arn:aws:iam::123456789012:role/scanner"
config.aws.external_id = "my-external-id"

engine = CloudGEngine(config)
engine.on_finding = lambda f: forward_to_siem(f)

result = engine.run_pipeline_sync()
print(result.to_summary())

The engine exposes collect(), scan() and analyze() separately if you only need part of the pipeline, map_inventory() for scanner-independent inventory mapping, ingest_reports() / run_from_reports() for working from existing scanner output files, and event hooks (on_finding, on_phase_start, on_error, on_scan_complete) for streaming integration.

Inventory mapping composes with the rest: map now, scan whenever, merge later.

from cloudg import CloudGConfig, CloudGEngine
from cloudg.inventory import InventoryMapper, RelationshipLinker

config = CloudGConfig(providers=["aws"])
engine = CloudGEngine(config)

inventory = engine.map_inventory_sync()          # no scanners involved
findings = engine.ingest_reports({"prowler": ["./prowler-out/"]})

mapper = InventoryMapper(config)
asset_map = mapper.build_asset_map(inventory, findings)        # asset -> risk
compliance = mapper.build_compliance_map(inventory, findings)  # framework -> assets

# the linker also works standalone, on any list of CloudAssets
edges = RelationshipLinker(inventory.assets).link()

Much more of cloudg is public, importable API than the CLI suggests. The Python API chapter documents the full surface, including:

API	What it gives you
cloudg.inventory.InventoryMapper / RelationshipLinker	scanner-free inventory maps and metadata-derived relationship edges
cloudg.graph.builder.GraphBuilder	NetworkX graph, attack paths, centrality/blast-radius metrics, D3/Cytoscape/GraphML export
cloudg.graph.ontology.CloudOntology	RDF ontology (64 typed relations), SPARQL-queryable, Turtle/JSON-LD
cloudg.graph.rag_export.RAGExporter	retrieval-ready JSONL chunks of the infrastructure for LLM pipelines
cloudg.renderers.terraform_export.TerraformExporter	.tf.json recreation of live infrastructure plus import_commands.sh
cloudg.ingest.parse_report and the scanner classes	every scanner's parser, usable standalone
cloudg.normaliser.FindingsNormaliser	cross-scanner dedupe, CVSS rescoring, compliance mapping
cloudg.mcp.CloudGMCPLayer	the MCP layer, in-process or mounted into your own MCP server
cloudg.resilience	shared rate limits, circuit breakers and retries (call_with_resilience, @resilient) for your own cloud calls

Custom collectors and scanners register through entry points, no core changes needed:

[project.entry-points."cloudg.collectors"]
mycloud = "my_package.collector:MyCollector"

---

Development

uv pip install -e ".[all,dev]"
pytest             # about 1,650 tests, moto-mocked AWS included
ruff check cloudg/ tests/
uv build           # wheel + sdist for PyPI

Python 3.11 or newer. The moto/aiobotocore incompatibility around async response bodies is handled in tests/conftest.py, so the suite runs against current versions of both.

---

Credits

cloudg builds on other people's work, and some of it ships inside the package. The documentation site has the same list on its credits page.

Prowler (Apache-2.0). The 22 framework files in cloudg/rules/frameworks/ are generated from Prowler's compliance JSON by scripts/import_prowler_compliance.py. That covers the control IDs, the control titles and descriptions, and the mapping from each control to Prowler check names. Every generated file says so in its header and in its source field. Prowler check names also appear in cloudg/rules/check_equivalence.yaml, and cloudg runs Prowler and parses its ASFF and OCSF output.

Scanners. Checkov (Apache-2.0), Trivy (Apache-2.0) and ScoutSuite (GPL-2.0) are run as separate programs, and their reports are parsed. Their check IDs (CKV_*, AVD-*) are used in the check-equivalence map. None of their code is part of cloudg. The Docker image installs Prowler, Checkov and Trivy, so their own licenses apply to those copies.

Cloud Custodian (Apache-2.0). The policy packs in cloudg/policies/ are written in Cloud Custodian's policy format and run with its custodian CLI.

parliament (BSD-3-Clause) from Duo Labs does the IAM policy linting.

Drawing engines. The estate map in report.html is drawn with the iso-kit from Anatomy (MIT, by Ryan), and its empty states use the kernel of Hairline 0.5.1 (MIT, by Lucas Marques). Both are copied into cloudg/templates/vendor/ and embedded in every report, so the map works offline. The iso-kit is wrapped so it can sit inside a script element, and cloudg/templates/vendor/NOTICE lists exactly what changed.

MITRE ATT&CK®. The mitre_attack_*.yaml files contain ATT&CK technique IDs and descriptions. © The MITRE Corporation. This work is reproduced and distributed with the permission of The MITRE Corporation. ATT&CK is a registered trademark of The MITRE Corporation.

Standards. The hand-written rulesets in cloudg/rules/ reference public control numbering from the CIS Benchmarks for AWS, Azure and GCP, NIST SP 800-53, NIST CSF, PCI DSS, the AICPA SOC 2 Trust Services Criteria, the HIPAA Security Rule, GDPR, ISO/IEC 27001:2022 and the AWS Foundational Security Best Practices. The standards belong to their publishers. CIS Benchmarks is a trademark of the Center for Internet Security, and cloudg is not affiliated with or endorsed by CIS or any other publisher named here.

The Python libraries cloudg depends on (NetworkX, rdflib, pydantic, Click, Rich and the rest) are listed in pyproject.toml.