Building an Automated Cloud Security Posture Management (CSPM) & CIS Benchmark Auditor in Python
How I engineered an enterprise-grade multi-cloud compliance engine to audit Azure and AWS environments against CIS Benchmarks, calculate weighted posture scores, and generate automated remediation runbooks.
Introduction: The Multi-Cloud Compliance Blind Spot
In modern enterprise environments, rapid deployment cycles and multi-cloud footprints (AWS and Microsoft Azure) have created a pervasive challenge: cloud configuration drift and silent misconfigurations.
Security operations and engineering teams routinely face critical risks:
- Cloud storage buckets and blob containers exposed with public read access.
- Overly permissive Identity and Access Management (IAM) policies granting wildcard (*) administrative privileges.
- Management ports (RDP 3389, SSH 22, and database listeners) directly accessible to the open internet (0.0.0.0/0).
- Virtual machine workloads deployed without disk encryption or with direct public IP attachments.
- Tenant-level audit logging disabled or configured below standard compliance retention thresholds.
While commercial Cloud Security Posture Management (CSPM) platforms exist, they are often expensive black boxes with opaque risk-scoring formulas, vendor lock-in, and slow adaptation to custom internal policies.
To tackle these challenges head-on, I engineered an open-source, lightweight, and deterministic Multi-Cloud CSPM & CIS Benchmark Auditor in Python. The engine audits cloud infrastructure against official Center for Internet Security (CIS) Benchmarks (v3.0.0), maps every finding directly to the MITRE ATT&CK for Cloud Matrix, computes a mathematical Cloud Security Posture Index (CSPI), and generates executable shell and PowerShell remediation runbooks.
All project code, test datasets, and audit outputs are available on my GitHub Repository: cloud-security-posture-auditor.
1. System Architecture & Pipeline Workflow
The CSPM engine follows a modular, five-stage pipeline designed for extensibility across cloud providers:
Core Architectural Components:
- Multi-Source Telemetry Collector (cloud_collector.py): Ingests live telemetry via native cloud SDKs (boto3 for AWS, Azure Resource Management SDKs) alongside standardized JSON infrastructure state payloads. It normalizes disparate provider responses into a unified CloudResource data model.
- Deterministic CIS Rule Engine (rules/): Evaluates normalized assets against modular compliance rules spanning 5 core security pillars: IAM, Storage, Network, Compute, and Logging.
- MITRE ATT&CK Cloud Correlator: Maps each detected violation to adversary tactics and techniques (e.g., T1078.004 Cloud Accounts, T1530 Cloud Storage Theft, T1190 Exploit Public Application, T1562.008 Disable Cloud Logs).
- Weighted Risk Scoring Matrix (scorer.py): Deducts calibrated penalties based on severity (Critical: 15 pts, High: 8 pts, Medium: 4 pts) to determine the overall Cloud Security Posture Index (0.0 to 100.0) and assign an enterprise letter grade (A through F).
- Remediation Hub & ChatOps Dispatcher (remediation.py, notifier.py): Generates single-click execution scripts (remediate_all.sh and remediate_all.ps1) using official Azure CLI (az) and AWS CLI (aws) commands, while broadcasting rich summary cards to Slack and Discord webhooks.
2. Normalizing Multi-Cloud Telemetry into Uniform Models
A primary difficulty in multi-cloud security is schema fragmentation: an Azure Storage Account and an AWS S3 Bucket serve similar functions but represent access policies, encryption, and logging configurations completely differently.
Using Python’s @dataclass, I created a strongly-typed schema to normalize multi-cloud assets:
With this abstraction, rule evaluators evaluate uniform properties regardless of whether the telemetry originated from a live AWS Boto3 API call, an Azure ARM REST query, or a CI/CD infrastructure definition file.
3. Auditing the 5 Security Pillars Against CIS Benchmarks
The engine implements modular rule evaluators mapped directly to CIS Benchmarks v3.0.0:
Evaluated Control Matrix:
- IAM-CIS-1.1 (Identity | CIS 1.1 | MITRE T1078.004): Multi-Factor Authentication (MFA) missing on privileged accounts.
- IAM-CIS-1.2 (Identity | CIS 1.16 | MITRE T1098): Overly permissive wildcard (*) administrative policies assigned.
- STORAGE-CIS-2.1 (Storage | CIS 2.1.5 | MITRE T1530): Unauthenticated anonymous public read access enabled on buckets/blobs.
- STORAGE-CIS-2.2 (Storage | CIS 2.1.1 | MITRE T1005): Server-Side Encryption (SSE) at rest disabled on storage.
- STORAGE-CIS-2.3 (Storage | CIS 2.1.2 | MITRE T1040): Storage endpoints accept unencrypted HTTP transport traffic.
- NET-CIS-3.1 (Network | CIS 5.2.1 | MITRE T1133): Inbound RDP (port 3389) open to the internet (0.0.0.0/0).
- NET-CIS-3.2 (Network | CIS 5.2.2 | MITRE T1133): Inbound SSH (port 22) exposed to the entire internet (0.0.0.0/0).
- NET-CIS-3.3 (Network | CIS 5.2.3 | MITRE T1190): Database ports (3306, 5432, 1433) exposed directly to public ingress.
- COMPUTE-CIS-4.1 (Compute | CIS 7.1 | MITRE T1005): Virtual machine OS or data disks lack at-rest encryption.
- COMPUTE-CIS-4.2 (Compute | CIS 7.4 | MITRE T1190): Workload virtual machines assigned direct public IP addresses.
- LOG-CIS-5.1 (Logging | CIS 3.1 | MITRE T1562.008): Tenant activity logging disabled or audit retention configured < 90 days.
Rule Logic Implementation:
Each check evaluates resource state and yields a structured Finding containing full technical rationale and deterministic CLI remediation commands:
4. The Mathematical Risk Scoring Engine: Deriving the CSPI
Binary compliance metrics (“pass” or “fail”) fail to communicate true organizational risk. An environment with 10 minor informational findings is far safer than an environment with a single unauthenticated public database port.
Get Ravishkarathnayaka’s stories in your inbox
Join Medium for free to get updates from this writer.
To provide clear risk visibility, I designed the Cloud Security Posture Index (CSPI):
Formula: CSPI = Max(0.0, 100.0 — Sum of Severity Penalties)
Penalty Deductions:
- CRITICAL Severity: Deducts 15.0 points per violation.
- HIGH Severity: Deducts 8.0 points per violation.
- MEDIUM Severity: Deducts 4.0 points per violation.
- LOW Severity: Deducts 1.0 point per violation.
This mathematical approach prevents score inflation and ensures that critical vulnerabilities immediately degrade the organization’s posture rating.
5. Automated Remediation Hub & ChatOps Integration
Identifying violations is only half the battle. To accelerate Mean Time to Remediate (MTTR), the engine synthesizes findings into ready-to-execute scripts:
- Unix Bash Script (remediate_all.sh): Contains executable az and aws CLI commands with error handling (set -e) and granular comments for every finding.
- PowerShell Playbook (remediate_all.ps1): Native Windows administrative commands utilizing the Azure Az and AWS PowerShell modules.
Simultaneously, the notifier.py module posts formatted audit summary cards to SecOps Slack and Discord channels, giving security teams immediate visibility into compliance shifts.
6. Verification Telemetry & Audit Evidence
The engine was evaluated against a realistic multi-cloud enterprise dataset comprising 12 resources across AWS and Azure:
Summary of Audit Findings:
- Resources Scanned: 12 total cloud assets (8 Azure ARM, 4 AWS Boto3).
- Rules Evaluated: 27 security controls.
- Controls Passed: 14 [✓]
- Controls Failed: 13 [✗]
- Severity Breakdown: 4 Critical (60.0 pts), 7 High (56.0 pts), 2 Medium (8.0 pts).
- Calculated Deductions: 124.0 penalty points.
- Final Posture Score: 0.0 / 100.0 (Grade F — Severe Non-Compliance).
Domain Compliance Breakdown:
- Identity (IAM): 6 Evaluated | 3 Passed | 3 Failed | 50.0% Compliant
- Storage (S3 / Blob): 9 Evaluated | 5 Passed | 4 Failed | 55.6% Compliant
- Network (NSGs / Firewalls): 9 Evaluated | 6 Passed | 3 Failed | 66.7% Compliant
- Compute (Workload VMs): 2 Evaluated | 0 Passed | 2 Failed | 0.0% Compliant
- Logging & Monitoring: 1 Evaluated | 0 Passed | 1 Failed | 0.0% Compliant
The audit output demonstrates why automated CSPM is vital: despite 14 controls passing, critical exposures in network ingress and unauthenticated public storage completely undermined the organization’s security posture.
7. Key Lessons & DevSecOps Engineering Takeaways
- Standardization is Essential: Treating multi-cloud telemetry through normalized data models decouples security policy logic from cloud provider API quirks.
- Contextual Risk Over Pass/Fail Rates: A 52% compliance rate sounds mediocre, but a 0.0 CSPI accurately reflects the existential risk of public database listeners and unencrypted storage.
- Bridge Detection with Remediation: Detection without an automated remediation path results in backlog fatigue. Generating ready-to-run CLI commands turns audit reports into immediate operational fixes.
Project Repository & Connect
The complete codebase, documentation, and sample cloud state datasets are published on GitHub:
- GitHub Repository: https://github.com/ravishkarathnayaka/cloud-security-posture-auditor
- LinkedIn: https://linkedin.com/in/ravishkarathnayaka
