Skip to content

aws: Add Config and Inspector transforms for extended protections (CDR) workflow #15230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kcreddy merged 12 commits into from
Sep 22, 2025
Merged

aws: Add Config and Inspector transforms for extended protections (CDR) workflow #15230

kcreddy merged 12 commits into from
Sep 22, 2025

Conversation

@kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Sep 8, 2025 •
edited
Loading

Proposed commit message

aws: Add transforms to Config and Inspector data streams for extended protections (CDR) workflow.

- Add latest transform to Config and Inspector data streams
to help with Cloud Native Vulnerability Management (CNVM)[1] 
and Cloud Security Posture Management (CSPM)[2] workflows.
- Add ILM policy to AWS Config as it does full sync every interval.
- Update minimum kibana version to "^8.19.0 || ^9.1.0"  to ensure 
necessary permissions for transform[3].
- Re-add 3.14.2 changelog entry as it is overwritten in VPC Flow PR[4].
- Skip system tests for securityhub* data streams to avoid fleet health 
degradation due to empty template values by httpjson. This is fixed in 
8.19.4 and 9.1.4 by beats#45810[5] and beats#46332[6]. This skip can 
be removed when the stack version is upgraded to ones containing the fix. 

[1] https://www.elastic.co/guide/en/security/current/vuln-management-overview.html
[2] https://www.elastic.co/docs/solutions/security/cloud/cloud-security-posture-management
[3] https://github.com/elastic/elasticsearch/pull/128350
[4] https://github.com/elastic/integrations/pull/15077
[5] https://github.com/elastic/beats/pull/45810
[6] https://github.com/elastic/beats/pull/46332

Note

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

@kcreddy kcreddy self-assigned this Sep 8, 2025
@kcreddy kcreddy added Integration:aws AWS Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Category: CDR enhancement New feature or request breaking change labels Sep 8, 2025
@andrewkroh andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Sep 8, 2025
@kcreddy
Copy link
Contributor Author

kcreddy commented Sep 9, 2025

/test

@kcreddy kcreddy marked this pull request as ready for review September 12, 2025 12:46
@kcreddy kcreddy requested review from a team as code owners September 12, 2025 12:46
@elasticmachine
Copy link

elasticmachine commented Sep 12, 2025

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy kcreddy requested a review from a team September 12, 2025 12:46
kcreddy
kcreddy commented Sep 12, 2025
View reviewed changes
packages/aws/elasticsearch/transform/latest_cdr_vulnerabilities_awsinspector/transform.yml
Comment on lines +12 to +14
index: "security_solution-awsinspector.vulnerability_latest-v1"
aliases:
- alias: "security_solution-awsinspector.vulnerability_latest"
Copy link
Contributor Author

@kcreddy kcreddy Sep 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxcold, the destination pattern was the only change since #14306 (comment).
Earlier it was security_solution-aws.vulnerability_latest-v1, and now security_solution-awsinspector.vulnerability_latest-v1. Notice change from aws to awsinspector.

This is to make it consistent with awsconfig. We will reserve aws when adding vulnerability data via SecurityHub which has aws pattern already for misconfigurations.

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Sep 16, 2025

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

efd6
efd6 reviewed Sep 19, 2025
View reviewed changes
@kcreddy kcreddy requested a review from efd6 September 19, 2025 05:52
@elasticmachine
Copy link

elasticmachine commented Sep 19, 2025

💚 Build Succeeded

History

cc @kcreddy

@elastic-sonarqube
Copy link

elastic-sonarqube bot commented Sep 19, 2025

Quality Gate failed Quality Gate failed

Failed conditions
1.8% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@kcreddy kcreddy merged commit 285494a into elastic:main Sep 22, 2025
8 of 9 checks passed
@kcreddy kcreddy mentioned this pull request Sep 22, 2025
Merged
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Sep 22, 2025

Package aws - 4.0.0 containing this change is available at https://epr.elastic.co/package/aws/4.0.0/

@nick-alayil nick-alayil mentioned this pull request Sep 26, 2025
Closed
1 task
tehbooom pushed a commit to tehbooom/integrations that referenced this pull request Nov 19, 2025
@kcreddy
aws: Add Config and Inspector transforms for extended protections (CD…
d087437 
…R) workflow (elastic#15230)

aws: Add transforms to Config and Inspector data streams for extended protections (CDR) workflow.

- Add latest transform to Config and Inspector data streams
to help with Cloud Native Vulnerability Management (CNVM)[1] 
and Cloud Security Posture Management (CSPM)[2] workflows.
- Add ILM policy to AWS Config as it does full sync every interval.
- Update minimum kibana version to "^8.19.0 || ^9.1.0"  to ensure 
necessary permissions for transform[3].
- Re-add 3.14.2 changelog entry as it is overwritten in VPC Flow PR[4].
- Skip system tests for securityhub* data streams to avoid fleet health 
degradation due to empty template values by httpjson. This is fixed in 
8.19.4 and 9.1.4 by beats#45810[5] and beats#46332[6]. This skip can 
be removed when the stack version is upgraded to ones containing the fix. 

[1] https://www.elastic.co/guide/en/security/current/vuln-management-overview.html
[2] https://www.elastic.co/docs/solutions/security/cloud/cloud-security-posture-management
[3] elastic/elasticsearch#128350
[4] elastic#15077
[5] elastic/beats#45810
[6] elastic/beats#46332
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maxcold maxcold maxcold approved these changes

@efd6 efd6 efd6 approved these changes

Assignees

@kcreddy kcreddy

Labels

breaking change Category: CDR documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:aws AWS Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

AWS Config: Implement transform for Cloud Security Workflows AWS Inspector: Implement transform for Cloud Security Workflows

5 participants

@kcreddy @elasticmachine @maxcold @efd6 @andrewkroh