This issue has been assessed as high severity. Review affected configurations immediately.
AWS Secrets Manager and Systems Manager Parameter Store are the designated homes for credentials, API keys, and other sensitive configuration values in AWS environments. They’re better than environment variables and infinitely better than hardcoded values in source code. They’re also frequently misconfigured in ways that make credential exposure trivially easy for an attacker who has compromised any adjacent IAM principal.
This guide covers the specific exposure paths that produce real incidents and what to do about each.
Secrets Manager vs. Parameter Store: The Security Tradeoffs
Both services store sensitive configuration data, but they have meaningfully different security properties.
Secrets Manager was built specifically for secrets management. It supports automatic rotation via Lambda functions, fine-grained resource-based policies, and cross-account access control. It logs all access to CloudTrail. Cost: $0.40 per secret per month.
Parameter Store is cheaper (free tier for standard parameters; $0.05 per 10,000 API interactions for advanced). Many teams use it as a cost-effective alternative for values that aren’t rotating credentials. SecureString parameters use KMS for encryption. The security model is less granular: parameter resource policies are available but rarely configured, and the default access model relies entirely on IAM policies.
The cost difference drives behavior: teams store production database passwords, API keys, and OAuth tokens in Parameter Store because it’s cheaper, then apply less rigorous access controls than they would for Secrets Manager entries.
Common Exposure Paths
1. Overly Broad IAM Policies on Secrets
The most common pattern: an IAM role (often attached to an EC2 instance, Lambda function, or ECS task) has a policy like this:
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": "*"
}
Wildcard resource on GetSecretValue means any principal assuming this role can retrieve any secret in the account. If that role is reachable via EC2 IMDS, Lambda function URL, or any other exposed mechanism, an attacker who finds the role can enumerate and exfiltrate every secret in the account.
The fix is scoping the resource to specific secret ARNs:
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": [
"arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/db/password-*",
"arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/api/stripe-*"
]
}
The -* suffix is required because Secrets Manager appends a six-character suffix to secret names.
2. Cross-Account Access Without Condition Constraints
Secrets Manager supports resource-based policies that allow cross-account access. A well-intentioned policy might look like:
{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::999888777666:root"},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
Allowing the account root means any principal in account 999888777666 with GetSecretValue permissions can access this secret. If that external account is a partner organization, a contractor, or a shared services account, the blast radius of a compromise in that account extends to your secrets.
Restrict cross-account access to specific roles:
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::999888777666:role/SpecificServiceRole"
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:shared/api-key-*",
"Condition": {
"StringEquals": {
"aws:PrincipalAccount": "999888777666"
}
}
}
3. Rotation Failures Producing Silent Stale Credentials
Secrets Manager rotation uses Lambda functions. When rotation fails — due to network connectivity issues, Lambda timeouts, permission errors, or schema changes in the target database — the secret can end up in an inconsistent state. Worse: applications may continue using a cached copy of the old secret while the rotation mechanism has partially updated it, causing authentication failures that surface as application errors rather than security events.
The operational failure mode: teams disable rotation after it causes outages, then re-enable it later without validating that the Lambda function still works. Rotation failure also means that if a secret is compromised, you don’t have the continuous rotation that limits its useful lifetime.
Monitor rotation health in CloudWatch:
secretsmanager:RotationFailedCloudTrail event indicates a rotation attempt failed- Enable the Secrets Manager console’s rotation status view and alert on “Failed” status
4. Secrets Embedded in CloudFormation Outputs
A common anti-pattern when building infrastructure: a CloudFormation stack creates a secret, retrieves it, and passes it as an output value. CloudFormation stack outputs are readable by any IAM principal with cloudformation:DescribeStacks permission — which is often broader than the intended audience for the secret.
If secret values must be passed between stacks, use Parameter Store SecureString references or dynamic references ({{resolve:secretsmanager:...}}) rather than CloudFormation outputs.
5. Lambda Environment Variables as a Secrets Anti-Pattern
Lambda environment variables are not secrets management. They’re stored in plaintext in Lambda configuration (readable via lambda:GetFunctionConfiguration) and are visible in the Lambda console to any user with console access. Despite being technically less secure than Secrets Manager, they’re widely used because they’re convenient and zero cost.
The right migration path: replace environment variable credentials with calls to GetSecretValue at Lambda initialization, cache the value in memory, and implement rotation-aware cache invalidation.
6. Parameter Store Paths With Excessive Breadth
Parameter Store paths follow a hierarchical naming convention like /prod/db/password. IAM conditions can restrict access to subtrees of this hierarchy:
{
"Effect": "Allow",
"Action": [
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:GetParametersByPath"
],
"Resource": "arn:aws:ssm:us-east-1:123456789012:parameter/prod/db/*"
}
Many environments instead use Resource: "*" or a top-level path like /prod/* that grants access across all parameter subtrees. An attacker with access to any role that has this policy can retrieve all parameters under that path.
Detection via CloudTrail
Both services emit detailed CloudTrail events. Key events to alert on:
High-priority alerts:
secretsmanager:GetSecretValuefrom a principal that doesn’t normally call it — particularly from Lambda functions, EC2 instances, or container tasks that aren’t the designated consumerssecretsmanager:GetSecretValuecalled from an IP address that isn’t your infrastructure (suggests credential compromise and external access)ssm:GetParameterwith high volume from a single principal in a short time window — suggests bulk credential exfiltration
CloudTrail detection query (Athena or CloudWatch Logs Insights):
-- Unusual GetSecretValue callers
SELECT
userIdentity.principalId,
userIdentity.arn,
eventTime,
sourceIPAddress,
requestParameters.secretId
FROM cloudtrail_logs
WHERE eventName = 'GetSecretValue'
AND eventTime > current_timestamp - INTERVAL '24' HOUR
AND sourceIPAddress NOT IN (
SELECT DISTINCT sourceIPAddress
FROM cloudtrail_logs
WHERE eventName = 'GetSecretValue'
AND eventTime BETWEEN current_timestamp - INTERVAL '30' DAY
AND current_timestamp - INTERVAL '24' HOUR
)
ORDER BY eventTime DESC;
Volume alert for bulk Parameter Store reads:
SELECT
userIdentity.arn,
COUNT(*) as call_count,
MIN(eventTime) as first_call,
MAX(eventTime) as last_call
FROM cloudtrail_logs
WHERE eventName IN ('GetParameter', 'GetParameters', 'GetParametersByPath')
AND eventTime > current_timestamp - INTERVAL '1' HOUR
GROUP BY userIdentity.arn
HAVING COUNT(*) > 50
ORDER BY call_count DESC;
Hardening Checklist
- Scope all
GetSecretValueandGetParameterIAM permissions to specific ARNs or parameter path prefixes — no wildcard resources - Enforce resource-based policies on high-value secrets that explicitly deny access from principals outside the expected set
- Enable and validate rotation for all rotating credentials; set CloudWatch alarms on rotation failure events
- Audit cross-account resource policies quarterly; restrict
Principalto specific roles, not account roots - Scan CloudFormation stacks for secret values in outputs; migrate to dynamic references
- Migrate Lambda environment variable credentials to Secrets Manager with in-memory caching
- Enable AWS Config rules
secretsmanager-rotation-enabled-checkandsecretsmanager-scheduled-rotation-success-check - Alert on
GetSecretValuecalls from source IP addresses outside your infrastructure IP ranges
VPC Endpoint Policy for Perimeter Control
If your infrastructure runs in a VPC, a Secrets Manager VPC endpoint with a restrictive endpoint policy is a strong perimeter control. An endpoint policy can restrict which secrets are accessible via the endpoint and which principals can access them:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/*",
"Condition": {
"StringEquals": {
"aws:PrincipalAccount": "123456789012"
}
}
}
]
}
Combined with VPC security groups that prevent direct internet egress from compute resources, this ensures that even if a role’s credentials are compromised and used externally, they can’t reach Secrets Manager without going through your network controls.
The combination of scoped IAM policies, resource-based access control, rotation health monitoring, and CloudTrail alerting covers the majority of real-world credential exposure incidents from these services.