Skip to content

Commit

Permalink
Added possibility to deploy sample example with terraform
Browse files Browse the repository at this point in the history
  • Loading branch information
Anastasia D. authored and alexrashed committed Oct 19, 2024
1 parent 119f0df commit 5e156df
Show file tree
Hide file tree
Showing 14 changed files with 588 additions and 41 deletions.
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
LOCALSTACK_AUTH_TOKEN=YOUR_TOKEN
AWS_DEFAULT_REGION=us-east-1
# LOCAL_RUN=false # Default is true
27 changes: 26 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,29 @@ volume/

# lambda packages
lambdas/*/package/
lambdas/*/lambda.zip
lambdas/*/lambda.zip
terraform/*/*.zip

# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log
crash.*.log

*.tfvars
*.tfvars.json

override.tf
override.tf.json
*_override.tf
*_override.tf.json

.terraform.tfstate.lock.info

.terraformrc
terraform.rc
34 changes: 34 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
export AWS_DEFAULT_REGION=us-east-1
export AWS_ACCESS_KEY_ID ?= test
export AWS_SECRET_ACCESS_KEY ?= test
export LOCAL_RUN ?= true
SHELL := /bin/bash

include .env

build:
bin/build_lambdas.sh;

awslocal-setup:
bin/deploy.sh

terraform-setup:
$(MAKE) build
cd deployment/terraform; \
tflocal init; \
echo "Deploying Terraform configuration 🚀"; \
tflocal apply --auto-approve -var="local_run=${LOCAL_RUN}"; \
echo "Paste the function URLs above to the WebApp 🎉";

terraform-destroy:
cd deployment/terraform; \
tflocal destroy --auto-approve;

start:
LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d

stop:
@echo
localstack stop

.PHONY: build awslocal-setup terraform-setup terraform-destroy start stop
47 changes: 31 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,22 +69,37 @@ source .venv/bin/activate
pip install -r requirements-dev.txt
```

## Instructions

You can set up and deploy the sample application on LocalStack by executing the commands in our Makefile. First, create a `.env` file using the provided `.env.example` file as a template, and include your LocalStack token in it. Then, run `make -s start` to initiate LocalStack on your machine.

Next, execute `make -s terraform-setup` to provision the infrastructure on LocalStack using Terraform CLI and its scripts. Alternatively, run `make -s awslocal-setup` to set up the infrastructure with the local AWS CLI.

If you prefer, you can also follow these step-by-step instructions for a manual deployment.

### LocalStack

Start LocalStack Pro with the appropriate CORS configuration for the S3 Website:
Start LocalStack Pro with Auth Token:

```bash
LOCALSTACK_AUTH_TOKEN=... localstack start
LOCALSTACK_AUTH_TOKEN=... localstack start (-d)
```

## Instructions
### Terraform

To create the infrastructure using Terraform, run the following commands:

```shell
cd deployment/terraform
tflocal init
tflocal apply --auto-approve
```

You can create the AWS infrastructure on LocalStack by running `bin/deploy.sh`.
Make sure you have Python 3.11 activated before running the script.
We are using the `tflocal` wrapper to configure the local service endpoints, and send the API requests to LocalStack, instead of AWS.

Here are instructions to deploy it manually step-by-step.
### AWS CLI

### Create the buckets
#### Create the buckets

The names are completely configurable via SSM:

Expand All @@ -93,14 +108,14 @@ awslocal s3 mb s3://localstack-thumbnails-app-images
awslocal s3 mb s3://localstack-thumbnails-app-resized
```

### Put the bucket names into the parameter store
#### Put the bucket names into the parameter store

```bash
awslocal ssm put-parameter --name /localstack-thumbnail-app/buckets/images --type "String" --value "localstack-thumbnails-app-images"
awslocal ssm put-parameter --name /localstack-thumbnail-app/buckets/resized --type "String" --value "localstack-thumbnails-app-resized"
```

### Create the DLQ Topic for failed lambda invokes
#### Create the DLQ Topic for failed lambda invokes

```bash
awslocal sns create-topic --name failed-resize-topic
Expand All @@ -115,9 +130,9 @@ awslocal sns subscribe \
--notification-endpoint [email protected]
```

### Create the lambdas
#### Create the lambdas

#### S3 pre-signed POST URL generator
##### S3 pre-signed POST URL generator

This Lambda is responsible for generating pre-signed POST URLs to upload files to an S3 bucket.

Expand All @@ -143,7 +158,7 @@ awslocal lambda create-function-url-config \

Copy the `FunctionUrl` from the response, you will need it later to make the app work.

### Image lister lambda
#### Image lister lambda

The `list` Lambda is very similar:

Expand All @@ -166,7 +181,7 @@ awslocal lambda create-function-url-config \
--auth-type NONE
```

### Resizer Lambda
#### Resizer Lambda

```bash
(
Expand All @@ -189,23 +204,23 @@ awslocal lambda create-function \
--environment Variables="{STAGE=local}"
```

### Connect the S3 bucket to the resizer lambda
#### Connect the S3 bucket to the resizer lambda

```bash
awslocal s3api put-bucket-notification-configuration \
--bucket localstack-thumbnails-app-images \
--notification-configuration "{\"LambdaFunctionConfigurations\": [{\"LambdaFunctionArn\": \"$(awslocal lambda get-function --function-name resize | jq -r .Configuration.FunctionArn)\", \"Events\": [\"s3:ObjectCreated:*\"]}]}"
```

### Create the static s3 webapp
#### Create the static s3 webapp

```bash
awslocal s3 mb s3://webapp
awslocal s3 sync --delete ./website s3://webapp
awslocal s3 website s3://webapp --index-document index.html
```

### Using the application
#### Using the application

Once deployed, visit http://webapp.s3-website.localhost.localstack.cloud:4566

Expand Down
28 changes: 28 additions & 0 deletions bin/build_lambdas.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash

(cd lambdas/presign; rm -f lambda.zip; zip lambda.zip handler.py)

(cd lambdas/list; rm -f lambda.zip; zip lambda.zip handler.py)

os=$(uname -s)
if [ "$os" == "Darwin" ]; then
(
cd lambdas/resize
rm -rf libs lambda.zip
docker run --platform linux/x86_64 --rm -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.11" /bin/sh -c "pip3 install -r requirements.txt -t libs; exit"

cd libs && zip -r ../lambda.zip . && cd ..
zip lambda.zip handler.py
rm -rf libs
)
else
(
cd lambdas/resize
rm -rf package lambda.zip
mkdir package
pip3 install -r requirements.txt --platform manylinux2014_x86_64 --only-binary=:all: -t package
zip lambda.zip handler.py
cd package
zip -r ../lambda.zip *;
)
fi
26 changes: 2 additions & 24 deletions bin/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

export AWS_DEFAULT_REGION=us-east-1

bin/build_lambdas.sh

awslocal s3 mb s3://localstack-thumbnails-app-images
awslocal s3 mb s3://localstack-thumbnails-app-resized

Expand All @@ -14,7 +16,6 @@ awslocal sns subscribe \
--protocol email \
--notification-endpoint [email protected]

(cd lambdas/presign; rm -f lambda.zip; zip lambda.zip handler.py)
awslocal lambda create-function \
--function-name presign \
--runtime python3.11 \
Expand All @@ -30,7 +31,6 @@ awslocal lambda create-function-url-config \
--function-name presign \
--auth-type NONE

(cd lambdas/list; rm -f lambda.zip; zip lambda.zip handler.py)
awslocal lambda create-function \
--function-name list \
--runtime python3.11 \
Expand All @@ -46,28 +46,6 @@ awslocal lambda create-function-url-config \
--function-name list \
--auth-type NONE

os=$(uname -s)
if [ "$os" == "Darwin" ]; then
(
cd lambdas/resize
rm -rf libs lambda.zip
docker run --platform linux/x86_64 --rm -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.11" /bin/sh -c "pip3 install -r requirements.txt -t libs; exit"

cd libs && zip -r ../lambda.zip . && cd ..
zip lambda.zip handler.py
rm -rf libs
)
else
(
cd lambdas/resize
rm -rf package lambda.zip
mkdir package
pip3 install -r requirements.txt --platform manylinux2014_x86_64 --only-binary=:all: -t package
zip lambda.zip handler.py
cd package
zip -r ../lambda.zip *;
)
fi

awslocal lambda create-function \
--function-name resize \
Expand Down
12 changes: 12 additions & 0 deletions deployment/policies/lambda.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
}
}
]
}
14 changes: 14 additions & 0 deletions deployment/policies/lambda_ssm.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:GetParameters"
],
"Resource": [
"arn:aws:ssm:::parameter/localstack-thumbnail-app/*"
]
}
]
}
18 changes: 18 additions & 0 deletions deployment/policies/list_lambda_s3_buckets.json.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::${images_bucket}",
"arn:aws:s3:::${images_bucket}/*",
"arn:aws:s3:::${images_resized_bucket}",
"arn:aws:s3:::${images_resized_bucket}/*"
]
}
]
}
17 changes: 17 additions & 0 deletions deployment/policies/presign_lambda_s3_buckets.json.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::${images_bucket}",
"arn:aws:s3:::${images_bucket}/*"
]
}
]
}
17 changes: 17 additions & 0 deletions deployment/policies/resize_lambda_s3_buckets.json.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::${images_resized_bucket}",
"arn:aws:s3:::${images_resized_bucket}/*"
]
}
]
}
17 changes: 17 additions & 0 deletions deployment/policies/resize_lambda_sns.json.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"sns:Publish"
],
"Effect": "Allow",
"Resource": "${failure_notifications_topic_arn}"
},
{
"Action": "lambda:InvokeFunction",
"Effect": "Allow",
"Resource": "${resize_lambda_arn}"
}
]
}
13 changes: 13 additions & 0 deletions deployment/policies/website_bucket_policy.json.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "${cdn_identity_arn}"
},
"Action": "s3:GetObject",
"Resource": "${website_bucket_arn}/*"
}
]
}
Loading

0 comments on commit 5e156df

Please sign in to comment.