-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reaper: Add aws-nuke integration #39
Conversation
@darkowlzz you may want to check out https://github.com/ekristen/aws-nuke |
Thanks @swade1987! @darkowlzz I've started a break away fork after getting lackluster response from the original maintainers on updates/changes/fixes and a general reluctance to bring anyone else on board to maintain. I rewrote the entire tool as a core library called libnuke and then rewrote aws-nuke. I've also written them for azure and gcp now. All this to say that embedding it into go should be a lot more simple now. The library is fairly well documented. You can look at the command structure in If you find any resources not supporting tags, let me know I'll get them added or feel free to open a PR. Technically libnuke is in 0.x I have a few more "changes" to make that could be semi-breaking but as long as you are pinning deps you'll be fine. aws-nuke is in 3.x-beta, that's only because I have a few more deprecation warnings to put into place as I've been standardizing things, it's been working very well. P.S. nice use of YAML anchor, it should still work in my version, if you find that it doesn't let me know. |
@darkowlzz let's try to move this to @ekristen's project. In case things break, we have someone to reach out to 🤗 |
Thanks @swade1987 and @ekristen . Using libnuke for reaper's backend would also be great for consistency across the providers. I have already spent a few days figuring out how to modify and extend aws-nuke to integrate here. I can spend some more time to see what needs to change to integrate the fork. I'll hold from merging this and will get back after more research and testing. |
To integrate aws-nuke in reaper, it is divided into multiple components by separating the scan and delete operations. In order to achieve this, some of the aws-nuke code is copied, modified and extended. The resource data scanned by aws-nuke is converted to the reaper's native resource type to be listed in a coherent manner with the other reaper providers and also support json output. Applying the retention period/age filter to aws-nuke resources also requires post processing of the resources after scanning and before deleting. aws-nuke doesn't provide option for custom filters. The scanned resources are passed through a custom filter, implemented as an extension of the Nuke type, which understands the custom createdat timestamp format and filters the resources accordingly. Signed-off-by: Sunny <[email protected]>
1344310
to
b5135a6
Compare
I should rev the go mod version on aws nuke so you can properly include the v3 resources. |
Question, is there a reason you don't just use a dedicated account and run the aws-nuke tool directly vs including the library and re-implementing some of the logic? If the library + resources makes more sense, I'm happy to make additional tweaks that might make things easier. Let me know. Happy to see you including it! :) |
That would be nice to have. I thought of opening an issue asking if this is intentional.
We may be able to do that in the future but at present, we have a single account with some free credits and I'm not sure if those free credits can be used with sub-accounts as well. Since I don't have root access to the account to try creating subaccounts and we wanted to start using the credits for testing soon, I have been trying to make this work within the same account such that it runs in a limited scope, not interfering with anything else in the account.
I'm still doing some testing and I plan to discuss some changes upstream that would make things easier. Just wanted to get this working before starting any discussion. |
I have azure and gcp nuke variants too. |
@ekristen Yes, I've seen those 🙂 . Haven't looked into the details yet. But excited to use them in the future. |
2014725
to
8ff1e20
Compare
libnuke is more actively developed compared to upstream aws-nuke, and also has support for GCP and Azure, which could be a nice backend for reaper in the future, all based on libnuke. A new internal package libnukemod has been added with modifications and extensions to libnuke to fit with reaper needs. It contains aws.go for all the aws-nuke related helpers. This can be further extended in the future to add GCP and Azure helpers. Signed-off-by: Sunny <[email protected]>
8ff1e20
to
aa4fb44
Compare
The existing
aws
provider in reaper uses the Resource Groups Tagging API, which doesn't support listing every resource to the extent it's needed to find and delete them. In addition, the way the status of the resources are reported to the CLI, it's difficult to reliably delete resources. These issues are better handled byaws-nuke
. aws-nuke is written in Go and can be imported and embedded in reaper, to provide a consistent and coherent test-infra resource management tooling. But the default CLI implementation of aws-nuke makes it hard to integrate with reaper. Most of the necessary scan and delete code is implemented as part of the CLI package itself. In order to modify aws-nuke, some parts of it are copied in an internal package with minimal changes and extended in a separate file. aws-nuke being MIT license, the minor modification to the copied file is explicitly noted in the copied file.Package
tools/reaper/awsnukemod
contains the copied file, modifications and extensions to it along with relevant tests and docs. This is used by the reaper main package to implementaws-nuke
provider, which would replace the existingaws
provider in the future, unless there's a good use case for it.To keep the usage simple, aws-nuke provider uses the aws CLI to infer certain details like the AWS account ID for the configured IAM principal.
aws
CLI is still required when usingaws-nuke
provider. The only required flag it adds isawsregions
, which is used by aws-nuke to decide which aws regions to scan and nuke.Example usage:
The above output is a result of translating the observed resources from aws-nuke to reaper's
resource
data type. For more detailed resource listing, for debugging needs, the JSON output can be used:The
tags
shows all the tags and other properties of the resources, which may not be accurate but a direct conversion from aws-nuke observed resource type. Correcting them is not of concern for our needs here.The
resourceGroup
is empty as unlike azure and GCP, the AWS resources don't have groups. They are grouped using the tags.aws-nuke.go
containsgetAWSNukeConfig()
which returns a Go version of the following aws-nuke configuration for the given account ID and regions:This can be used to directly use aws-nuke for testing and debugging needs in the future, if needed, as the upstream aws-nuke implementation changes.
The delete operation runs the aws-nuke delete code, printing aws-nuke delete output:
Changes to the reaper README with granular permissions needed for this will be added separately after more testing and evaluation.
Part of fluxcd/flux2#4619