Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement Coraza-SPOA #103

Merged
merged 21 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions .github/workflows/codeql.yaml

This file was deleted.

55 changes: 4 additions & 51 deletions .github/workflows/container-image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,6 @@ on:
pull_request:
workflow_dispatch:


env:
HAPROXY_IMAGES: >
haproxy:2.2-alpine
haproxy:2.4-alpine
haproxy:2.5-alpine
haproxy:2.6-alpine
haproxy:2.7-alpine

jobs:
build:
runs-on: ubuntu-latest
Expand All @@ -30,14 +21,6 @@ jobs:
- name: Check out code
uses: actions/checkout@v3

- name: Run e2e tests against the example
jcchavezs marked this conversation as resolved.
Show resolved Hide resolved
shell: bash
run: >
for image in $HAPROXY_IMAGES; do
echo "Running e2e with Haproxy image $image"
HAPROXY_IMAGE=$image docker compose -f docker-compose.e2e.yaml up --abort-on-container-exit tests
done

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

Expand All @@ -56,7 +39,7 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Docker metadata - Main
- name: Docker metadata
id: meta-main
uses: docker/metadata-action@v4
with:
Expand All @@ -68,44 +51,14 @@ jobs:
type=ref,event=branch
type=ref,event=pr

- name: Image - Main
- name: Image
uses: docker/build-push-action@v3
with:
context: .
cache-from: type=gha
cache-to: type=gha,mode=max
platforms: linux/amd64
file: Dockerfile
file: example/Dockerfile
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta-main.outputs.tags }}
labels: ${{ steps.meta-main.outputs.labels }}


- name: Docker metadata - CRS4
id: meta-crs4
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ github.repository }}
flavor: |
suffix=-crs4,onlatest=true
tags: |
type=raw,value=snapshot,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=ref,event=branch
type=ref,event=pr

- name: Image - CRS4
uses: docker/build-push-action@v3
with:
context: .
cache-from: type=gha
cache-to: type=gha,mode=max
platforms: linux/amd64
file: Dockerfile
push: ${{ github.event_name != 'pull_request' }}
target: coreruleset
build-args: |
CORERULESET_VERSION=v4.0.0-rc1
tags: ${{ steps.meta-crs4.outputs.tags }}
labels: ${{ steps.meta-crs4.outputs.labels }}
labels: ${{ steps.meta-main.outputs.labels }}
9 changes: 4 additions & 5 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Lint (pre-commit)
name: Lint

on:
pull_request:
Expand All @@ -8,9 +8,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Go
uses: actions/setup-go@v3
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: v1.19.x
cache: true
go-version: '1.23'
- run: go run mage.go lint
11 changes: 3 additions & 8 deletions .github/workflows/package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,8 @@ jobs:
run: |
sudo apt update && sudo apt -y install make pkg-config rubygems && sudo gem install fpm

# Download corazawaf/coraza/coraza.conf for distribution
curl https://raw.githubusercontent.com/corazawaf/coraza/v3.0.0/coraza.conf-recommended > coraza.conf

- name: Build binary
run: VERSION=${PACKAGE_VERSION} ARCH=${{ matrix.arch }} make
run: VERSION=${PACKAGE_VERSION} GOARCH=${{ matrix.arch }} go run mage.go build

- name: Build package
run: |
Expand All @@ -62,11 +59,9 @@ jobs:
--deb-systemd ./contrib/coraza-spoa.service \
--deb-systemd-enable \
--config-files /etc/coraza-spoa/config.yaml \
./coraza-spoa_${{matrix.arch}}=/usr/bin/coraza-spoa \
./doc/config/=/usr/share/doc/coraza-spoa/haproxy-config \
./coraza-spoa=/usr/bin/coraza-spoa \
./LICENSE=/usr/share/doc/coraza-spoa/ \
./config.yaml.default=/etc/coraza-spoa/config.yaml \
./coraza.conf=/etc/coraza-spoa/coraza.conf
./example/coraza-spoa.yaml=/etc/coraza-spoa/config.yaml

## Publish to the "testing" repo
- name: Cloudsmith Push:debian/coraza-spoa-snapshots
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Test

on:
pull_request:
push:

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.23'

- name: setup environment
Copy link
Member

@jcchavezs jcchavezs Feb 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really avoid going in this direction, this assumes a haproxy is running in local which is going to be a big friction to those willing to contribute. Personally I would never install haproxy locally to debug a problem so I suggest we spin up a haproxy instance using docker with testcontainers api (see https://pkg.go.dev/github.com/testcontainers/testcontainers-go) or just docker-compose so the test is selfcontained and does not assume I have something installed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I explicitly did this without using containers (atleast inside the CI) as its faster to run. Locally and in production I would always recommend running without docker as the performance difference is huge.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Also I dont have docker on my machine since its just very slow on macos)

Copy link
Member

@jcchavezs jcchavezs Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really prefer to have something that is portable. It is OK if we want CI to run haproxy locally but we also need to provide a way to run it with docker. Again here we prioritize user experience if I have to install locally haproxy to contribute I'd rather not contribute hence we use containers. For advanced users they can definitively use local haproxy.

run: |
sudo apt-get install -y software-properties-common
sudo add-apt-repository -y ppa:vbernat/haproxy-2.8
sudo apt-get update
sudo apt-get install -y haproxy
haproxy -vv

- name: Test
run: go run mage.go test
6 changes: 2 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,5 @@
*.out
vendor/

# local files
config.yaml
logs/
rules/
# Build output
build/
81 changes: 0 additions & 81 deletions Dockerfile

This file was deleted.

41 changes: 0 additions & 41 deletions Makefile

This file was deleted.

22 changes: 9 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,23 @@ HAProxy includes a [Stream Processing Offload Engine](https://www.haproxy.com/bl

### Build

The command `make` will compile the source code and produce the executable file `coraza-spoa`.

### Clean

When you need to re-compile the source code, you can use the command `make clean` to clean the executable file.
The command `go run mage.go build` will compile the source code and produce the executable file `coraza-spoa`.

## Configuration

## Coraza SPOA

The example configuration file is [config.yaml.default](https://github.com/corazawaf/coraza-spoa/blob/main/config.yaml.default), you can copy it and modify the related configuration information. You can start the service by running the command:
The example configuration file is [examples/coraza-spoa.yaml](https://github.com/corazawaf/coraza-spoa/blob/main/examples/coraza-spoa.yaml), you can copy it and modify the related configuration information. You can start the service by running the command:

```
coraza-spoa -config /etc/coraza-spoa/coraza.yaml
coraza-spoa -f /etc/coraza-spoa/coraza-spoa.yaml
```

You will also want to download & extract the [OWASP Core Ruleset]( https://github.com/coreruleset/coreruleset/releases) (version 4+ supported) to the `/etc/coraza-spoa` directory.

## HAProxy SPOE

Configure HAProxy to exchange messages with the SPOA. The example SPOE configuration file is [coraza.cfg](https://github.com/corazawaf/coraza-spoa/blob/main/doc/config/coraza.cfg), you can copy it and modify the related configuration information. Default directory to place the config is `/etc/haproxy/coraza.cfg`.
Configure HAProxy to exchange messages with the SPOA. The example SPOE configuration file is [coraza.cfg](https://github.com/corazawaf/coraza-spoa/blob/main/examples/coraza.cfg), you can copy it and modify the related configuration information. Default directory to place the config is `/etc/haproxy/coraza.cfg`.

```ini
# /etc/haproxy/coraza.cfg
Expand All @@ -47,7 +43,7 @@ spoe-message coraza-req
event on-frontend-http-request
```

The application name from `config.yaml` must match the `app=` name, or the `default_application` will be used.
The application name from `config.yaml` must match the `app=` name.

The backend defined in `use-backend` must match a `haproxy.cfg` backend which directs requests to the SPOA daemon reachable via `127.0.0.1:9000`.

Expand All @@ -70,12 +66,12 @@ backend coraza-spoa
server s1 127.0.0.1:9000
```

A comprehensive HAProxy configuration example can be found in [docs/config/haproxy.cfg](https://github.com/corazawaf/coraza-spoa/blob/main/doc/config/coraza.cfg).
A comprehensive HAProxy configuration example can be found in [examples/haproxy.cfg](https://github.com/corazawaf/coraza-spoa/blob/main/examples/coraza.cfg).

Because, in the SPOE configuration file (coraza.cfg), we declare to use the backend [coraza-spoa](https://github.com/corazawaf/coraza-spoa/blob/88b4e54ab3ddcb58d946ed1d6389eff73745575b/doc/config/coraza.cfg#L14) to communicate with the service, so we need also to define it in the [HAProxy file](https://github.com/corazawaf/coraza-spoa/blob/dd5eb86d1e9abbdd5fe568249f36a6d85257eba7/doc/config/haproxy.cfg#L37):
Because, in the SPOE configuration file (coraza.cfg), we declare to use the backend [coraza-spoa](https://github.com/corazawaf/coraza-spoa/blob/main/examples/coraza.cfg#L14) to communicate with the service, so we need also to define it in the [HAProxy file](https://github.com/corazawaf/coraza-spoa/blob/main/examples/haproxy.cfg#L37):

## Docker

- Build the coraza-spoa image `docker-compose build`
- Run haproxy, coraza-spoa and a mock server `docker-compose up`
- Build the coraza-spoa image `docker compose build`
- Run haproxy, coraza-spoa and a mock server `docker compose up`
- Perform a request which gets blocked by the WAF: `curl http://localhost:4000/\?x\=/etc/passwd`
Loading
Loading