Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement: Improve docs and validations #8

Merged
merged 9 commits into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 41 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ It has the following high level architecture:

![Architecture](https://raw.githubusercontent.com/schubergphilis/terraform-aws-mcaf-resource-scheduler/main/docs/architecture.png)

## Features
## :books: Features

### Supported resource types

Expand Down Expand Up @@ -52,7 +52,27 @@ Optionally a pair of webhooks can be deployed to trigger starting or stopping an

Webhooks require an API key and can be setup to only allow certain IP addresses. A POST request has to be made to one of the outputted endpoints to trigger a webhook.

## Limitations
## :wrench: Setup

To setup this module requires a composition of the resources that need to be managed. Based on that input, a state machine is generated. Be aware that the composition dictates order: the resources in the composition are controlled in that order when the composition is started. The order is reversed when the composition is stopped.

Please see the [examples](examples/) folder for code examples of how to implement this module.

### Configuration

Resource types require certain parameters in order to function. It's recommended to fill the parameters by refering to existing resources in your TF code.

| Resource | Resource Type | Required Parameters |
wvanheerde marked this conversation as resolved.
Show resolved Hide resolved
| --- | --- | --- |
| EC2 Auto-Scaling Group | auto_scaling_group | **name:** the name of the auto-scaling group to control<br>**min:** the minimal number of instances to run (used on start of composition)<br>**max:** the maximum number of instances to run, used on start of composition<br>**desired:** the desired number of instances to run, used on start of composition |
| EC2 Instance | ec2_instance | **id:** the ID of the instance to control |
| ECS Service | ecs_service | **cluster_name:** the name of the ECS cluster the task runs on<br>**desired:** the desired number of tasks, used on start of composition<br>**name:** the name of the ECS task to control |
| FSX Windows Filesystem | fsx_windows_file_system | **id:** the ID of the filesystem to control<br>**throughput_capacity:** the throughput capacity of the filesystem, used on start of composition |
| RDS Cluster | rds_cluster | **id:** the ID of the cluster to control |
| RDS Instance | rds_instance | **id:** the ID of the instance to control |
| Redshift Cluster | redshift_cluster | **id:** the ID of the cluster to control |

## :rotating_light: Limitations

### Schedule mixing

Expand All @@ -78,17 +98,29 @@ Throughput can't be changed until 6 hours after the last change was requested. A

See https://docs.aws.amazon.com/fsx/latest/WindowsGuide/managing-storage-configuration.html#managing-storage-capacity for more information.

## Setup
## :test_tube: Development

To setup this module requires a composition of the resources that need to be managed. Based on that input, a state machine is generated.
This module uses the integrated Lambda to abstract some of the more complex functionality away. For redistribution purposes, the following dependencies have been vendorized:

Please see the [examples](examples/) folder for code examples of how to implement this module.
* pyawscron 1.0.6: https://pypi.org/project/pyawscron/ - https://github.com/pitchblack408/pyawscron/tree/1.0.6

## Development
### Adding support for more resources

This module uses the integrated Lambda to abstract some of the more complex functionality away. For redistribution purposes, the following dependencies have been vendorized:
This module is extendable. To add support for more resources, follow these general steps:

* pyawscron 1.0.6: https://pypi.org/project/pyawscron/ - https://github.com/pitchblack408/pyawscron/tree/1.0.6
1. In the Lambda code
1. Add a test
1. Add the resource controller
1. Add the resource to the handler
1. Add the resource to the schema
1. Make sure tests pass

1. In the Terraform code
1. Add the resource to the resource_composition variable
1. Add validation to the resource_composition variable
1. Add an example
1. Update this README
1. Make sure validations pass

<!-- BEGIN_TF_DOCS -->
## Requirements
Expand Down Expand Up @@ -182,7 +214,7 @@ This module uses the integrated Lambda to abstract some of the more complex func
| <a name="input_stop_resources_at"></a> [stop\_resources\_at](#input\_stop\_resources\_at) | Resources stop cron expression in selected timezone | `string` | n/a | yes |
| <a name="input_tags"></a> [tags](#input\_tags) | Mapping of tags | `map(string)` | `{}` | no |
| <a name="input_timezone"></a> [timezone](#input\_timezone) | Timezone to execute schedules in | `string` | `"UTC"` | no |
| <a name="input_webhooks"></a> [webhooks](#input\_webhooks) | Deploy webhooks for external triggers | <pre>object({<br> deploy = bool<br> ip_whitelist = list(string)<br> private = optional(bool, false)<br> })</pre> | <pre>{<br> "deploy": false,<br> "ip_whitelist": [],<br> "private": false<br>}</pre> | no |
| <a name="input_webhooks"></a> [webhooks](#input\_webhooks) | Deploy webhooks for external triggers from whitelisted IP CIDR's. | <pre>object({<br> deploy = bool<br> ip_whitelist = list(string)<br> private = optional(bool, false)<br> })</pre> | <pre>{<br> "deploy": false,<br> "ip_whitelist": [],<br> "private": false<br>}</pre> | no |

## Outputs

Expand Down
2 changes: 1 addition & 1 deletion examples/office_hours/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ module "scheduler" {
{
"type" : "ecs_service",
"params" : {
"cluster_name" : "application-cluster-1"
"cluster_name" : "application-cluster-1",
"name" : "application-service-1",
"desired" : 2
}
Expand Down
2 changes: 1 addition & 1 deletion examples/on_demand/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ module "scheduler" {
{
"type" : "ecs_service",
"params" : {
"cluster_name" : "application-cluster-1"
"cluster_name" : "application-cluster-1",
"name" : "application-service-1",
"desired" : 2
}
Expand Down
2 changes: 1 addition & 1 deletion examples/webhooks/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ module "scheduler" {
{
"type" : "ecs_service",
"params" : {
"cluster_name" : "application-cluster-1"
"cluster_name" : "application-cluster-1",
"name" : "application-service-1",
"desired" : 2
}
Expand Down
8 changes: 4 additions & 4 deletions scheduler/scheduler/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@
"auto_scaling_group_params": {
"type": "object",
"properties": {
"name": {"type": "string"},
"min": {"type": "string"},
"max": {"type": "string"},
"desired": {"type": "string"},
"max": {"type": "string"},
"min": {"type": "string"},
"name": {"type": "string"},
},
"required": ["name", "min", "max", "desired"],
},
Expand All @@ -68,8 +68,8 @@
"cluster_name": {
"type": "string",
},
"name": {"type": "string"},
"desired": {"type": "string"},
"name": {"type": "string"},
},
"required": ["cluster_name", "name", "desired"],
},
Expand Down
47 changes: 46 additions & 1 deletion variables.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
variable "kms_key_arn" {
description = "The ARN of the KMS key to use with the Lambda function"
type = string

validation {
condition = can(regex("arn:aws:kms:[a-z0-9-]+:[0-9]+:key/[a-f0-9-]+", var.kms_key_arn))
wvanheerde marked this conversation as resolved.
Show resolved Hide resolved
error_message = "KMS key ARN must be in the format arn:aws:kms:<region>:<account>:key/<key-id>"
}
}

variable "resource_composition" {
Expand All @@ -14,6 +19,46 @@ variable "resource_composition" {
condition = length([for r in var.resource_composition : r if contains(["ec2_instance", "rds_instance", "rds_cluster", "auto_scaling_group", "ecs_service", "redshift_cluster", "wait", "fsx_windows_file_system"], r.type)]) == length(var.resource_composition)
error_message = "Resource type must be one of ec2_instance, rds_instance, rds_cluster, auto_scaling_group, ecs_service, redshift_cluster or fsx_windows_file_system"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "auto_scaling_group" ? keys(r.params) == tolist(["desired", "max", "min", "name"]) : true)], false)
error_message = "Auto-scaling group resources must have 'desired', 'max', 'min' and 'name' parameters"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "ec2_instance" ? keys(r.params) == tolist(["id"]) : true)], false)
error_message = "EC2 instance resources must have 'id' parameter"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "ecs_service" ? keys(r.params) == tolist(["cluster_name", "desired", "name"]) : true)], false)
error_message = "ECS Service resources must have 'cluster_name', 'desired' and 'name' parameters"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "fsx_windows_file_system" ? keys(r.params) == tolist(["id", "throughput_capacity"]) : true)], false)
error_message = "FSx Windows Filesystem resources must have 'id' and 'throughput_capacity' parameters"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "rds_cluster" ? keys(r.params) == tolist(["id"]) : true)], false)
error_message = "RDS Cluster resources must have 'id' parameter"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "rds_instance" ? keys(r.params) == tolist(["id"]) : true)], false)
error_message = "RDS Instance resources must have 'id' parameter"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "redshift_cluster" ? keys(r.params) == tolist(["id"]) : true)], false)
error_message = "Redshift Cluster resources must have 'id' parameter"
}

validation {
condition = !contains([for r in var.resource_composition : (r.type == "wait" ? keys(r.params) == tolist(["seconds"]) : true)], false)
error_message = "Wait instructions must have 'seconds' parameter"
}
}

variable "webhooks" {
Expand All @@ -27,7 +72,7 @@ variable "webhooks" {
ip_whitelist = []
private = false
}
description = "Deploy webhooks for external triggers"
description = "Deploy webhooks for external triggers from whitelisted IP CIDR's."
}

variable "composition_name" {
Expand Down
Loading