AI-Driven Threat Detection Using AWS Security Agent: A Framework for On-Demand Autonomous Penetration Testing
Abstract
Modern enterprise applications face an ever-expanding threat landscape, where traditional periodic security assessments fail to keep pace with dynamic deployments and continuous integration pipelines. This paper introduces an AI-driven framework for on-demand, autonomous penetration testing using AWS Security Agent. The proposed approach enables proactive vulnerability detection by combining cloud-native infrastructure, contextual code awareness, and automated remediation workflows. By integrating domain verification, AI-assisted attack simulation, and GitHub-based code intelligence, this framework transforms penetration testing from a reactive compliance activity into a continuous, developer-aligned security practice.
1. Introduction
With the rise of microservices, APIs, and cloud-native architectures, security vulnerabilities are increasingly introduced during rapid development cycles. Traditional penetration testing models — manual, periodic, and resource-intensive — are no longer sufficient.
This paper proposes a novel approach to AI-driven threat detection, where penetration testing becomes:
- On-demand
- Context-aware
- Continuously improving via AI feedback loops
The AWS Security Agent enables this transformation by acting as an intelligent security orchestrator capable of simulating real-world attacks, analyzing application behavior, and suggesting code-level fixes.
2. Problem Statement
Enterprise systems struggle with:
- Lack of centralized inventory of attack surfaces
- Infrequent and delayed penetration testing cycles
- Limited developer integration
- Absence of automated remediation mechanisms
These gaps result in prolonged exposure to vulnerabilities such as:
- Injection attacks
- Broken authentication
- Misconfigured APIs
- Data exposure risks (including PII fields)
3. Proposed Framework: AWS Security Agent for AI-Driven Pentesting
We propose a five-stage architecture for enabling on-demand penetration testing:
3.1 Agent Space Initialization
The system begins with creating an Agent Space, which acts as an isolated security context for each application.
Begin by accessing the AWS Security Agent service through the AWS Management Console.
Step 1: Launch AWS Security Agent
From the AWS Management Console, navigate to AWS Security Agent and click “Set up AWS Security Agent.”
Step 2: Create an Agent Space
Configure a new Agent Space, which acts as an isolated environment for managing penetration testing activities.
- Provide a name and description that clearly represent the target application.
- The Agent Space name is visible to users in the web interface, so it should align with the application being tested.
- A single Agent Space can support multiple users, enabling collaborative security testing.
Step 3: Configure User Access
Under User Access Configuration, select IAM-only access.
This restricts access to users authenticated via AWS Identity and Access Management (IAM).
Step 4: Understand Access Options (Optional)
This setup uses IAM-based access and does not include Single Sign-On (SSO) via IAM Identity Center.
- IAM-only access is suitable when users already have AWS Console access.
- If you need to support users without AWS Console access (such as testers or external reviewers), integrate with IAM Identity Center to provide direct access to the AWS Security Agent web application.
Step 5: Complete Setup
Click “Set up AWS Security Agent” to finalize the configuration.
This step provisions the Agent Space and deploys the associated web application, enabling users to perform design reviews and execute penetration tests.
3.2 Enable and configure penetration testing
To begin configuring penetration testing, navigate to the Agent Spaces section from the left sidebar and select the Agent Space created in Step 3.1.
Step 1: Configure Target Domain
Enter the target domain you want to test and choose a verification method — DNS_TXT or HTTP_ROUTE.
Ensure that the domain is active and hosting the application intended for testing. Click Next to continue.
Step 2: Verify Domain Ownership
In the Target Domains table, complete domain verification to establish ownership:
- For Amazon Route 53 domains (same AWS account):
Select the domain and use One-click verification. The system will automatically create the required DNS record and complete verification. - For external DNS providers:
Copy the provided verification token, create a TXT record with your DNS provider, and then return to select Verify.
Note: Penetration tests can only be executed on verified domains.
Step 3: Configure Additional Capabilities (Optional)
Optionally, configure supporting resources such as:
- Virtual Private Clouds (VPCs)
- CloudWatch log groups
- Access credentials
The Service Access configuration is pre-enabled by default. AWS Security Agent automatically provisions a service role with the required permissions, unless you choose to supply an existing IAM role.
3.3 AI-Augmented Contextual Analysis via GitHub Integration (Optional)
Unlike traditional scanners, this system integrates with GitHub to:
- Understand application logic
- Map APIs and workflows
- Identify hidden attack paths
This introduces a key innovation:
Penetration testing driven not just by endpoints, but by source-code intelligence
3.4 Autonomous Penetration Test Execution
Users can initiate on-demand pentests via a web interface.
Get Umashankara Kalaiah’s stories in your inbox
Join Medium for free to get updates from this writer.
To initiate an autonomous penetration test, navigate to the Web App tab and select Admin Access. This launches the AWS Security Agent web application with administrator privileges.
This access mode is available only when the Agent Space is configured with IAM-only access. Alternatively, user access can be enabled through explicit user onboarding and login configuration.
From the left sidebar, select Penetration Test, then click Create your first penetration test to begin defining the test configuration.
Step 1: Define Test Scope
Select the target domain for testing or specify specific application paths. Only previously verified domains can be included in the test scope.
If the application depends on external endpoints, add them under the Accessible URLs field.
Note: Accessible URLs are intended for third-party services such as Okta, Auth0, or Stripe, which may be required for authentication or navigation during testing. These domains are not tested for vulnerabilities; they are used solely to enable authenticated access flows.
Step 2: Configure Execution Environment
Select the appropriate IAM role and CloudWatch log group for storing penetration test execution logs.
If no log group is specified, AWS Security Agent will automatically create one at runtime.
Step 3: Enable Automated Remediation (Optional but Recommended)
Enable automatic code remediation to allow the system to generate GitHub pull requests containing ready-to-apply fixes for identified vulnerabilities.
Step 4: Configure Authentication (Optional)
If the application requires authentication, provide login credentials directly within the web application.
Define the authentication flow so the AWS Security Agent can successfully access protected application areas during testing.
Step 5: Provide Supporting Context (Optional)
Upload additional artifacts such as:
- Architecture diagrams
- Threat models
- API specifications
- Design documentation
These inputs help the agent improve contextual understanding and expand test coverage.
Step 6: Execute or Save Test Configuration
Click Create and Execute to launch the penetration test. You will be redirected to the test execution dashboard.
Alternatively, select Create Penetration Test to save the configuration for later execution without running the test immediately.
3.5 AI-Driven Remediation and Feedback Loop
One of the most significant contributions is:
Automatic code remediation via pull requests
Capabilities:
- Generate secure code fixes
- Create GitHub pull requests
- Enable developer review workflows
This closes the loop between:
Detection → Analysis → Fix → Deployment
4. Architecture Overview
Below is a conceptual architecture of the proposed system:
[User / Developer]
↓
[AWS Security Agent Web App]
↓
[Agent Space (IAM Secured)]
↓
┌───────────────┬────────────────┐
↓ ↓ ↓
[Domain Verify] [GitHub] [CloudWatch Logs]
↓ ↓
└────→ [AI Threat Engine] ←────┘
↓
[Penetration Test Execution]
↓
[Findings Engine]
↓
[Auto Remediation Engine]
↓
[GitHub Pull Requests]5. Below summarizes the practical steps:
Step 1: Setup AWS Security Agent
- Create Agent Space
- Configure IAM access
Step 2: Enable Penetration Testing
- Define target domain
- Verify ownership
- Configure optional resources
Step 3: Connect GitHub (Optional but Recommended)
- Authorize repositories
- Enable remediation workflows
Step 4: Execute Pentest
- Define scope and credentials
- Run test via web application
Step 5: Analyze Findings
- Review vulnerabilities
- Track logs and attack paths
- Apply automated fixes
