Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a GitHub Action to lint my Caddyfiles #949

Merged
merged 3 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/caddy_fmt.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: "Lint my Caddyfiles"

on:
push:
branches:
- main

paths:
- '.github/workflows/caddy_fmt.yml'
- 'Caddyfile'
- '**/Caddyfile'
- '**/*.Caddyfile'

pull_request:
branches:
- main

paths:
- '.github/workflows/caddy_fmt.yml'
- 'Caddyfile'
- '**/Caddyfile'
- '**/*.Caddyfile'
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: "Check out the repo"
uses: actions/checkout@v4

- name: Install Caddy
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release download \
--repo caddyserver/caddy \
--pattern 'caddy_*_linux_amd64.tar.gz' \
--output caddy.tar.gz
tar -xzf caddy.tar.gz --directory /usr/local/bin
chmod +x /usr/local/bin/caddy
which caddy

- name: Format my Caddyfiles
run: |
find . \
-name Caddyfile \
-o -name '*.Caddyfile' ! -name 'redirects.Caddyfile' | \
xargs -I '{}' --verbose caddy fmt --overwrite '{}'

- name: Print changes
run: git diff

- name: Check for changes
run: git diff --exit-code
71 changes: 71 additions & 0 deletions src/_til/2024/2024-11-22-debugging-find-and-xargs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
layout: til
title: Debugging some confusing behaviour with `find` and `xargs`
date: 2024-11-22 16:35:11 +0000
tags:
- shell scripting
summary:
Use the `--verbose` flag to see what `xargs` is doing; don't rely on `find` to return files in a consistent order.
---
I was trying to write a GitHub Actions workflow to format my Caddyfiles.
I'd written a command to find all the Caddyfiles in my repo, and pass them to `caddy fmt`:

```
find . -name Caddyfile | xargs caddy fmt --overwrite
```

Thsi command seemed to work when I ran it locally on my Mac, but it didn't do what I expected when it ran on Ubuntu in GitHub Actions.
It took me a while to work out what was going on!

I learnt two useful debugging techniques along the way:

1. **Use the `--verbose` flag to see what `xargs` is doing.**
This will print the exact command which is run by `xargs`.

```console
$ ls *.jpg | xargs echo
26704189108_9e71d865d8_b.jpg 51119071159_905212c30f_b.jpg 8076936667_c5605863a9_o.jpg

$ ls *.jpg | xargs --verbose echo
echo 26704189108_9e71d865d8_b.jpg 51119071159_905212c30f_b.jpg 8076936667_c5605863a9_o.jpg
26704189108_9e71d865d8_b.jpg 51119071159_905212c30f_b.jpg 8076936667_c5605863a9_o.jpg

$ ls *.jpg | xargs --verbose -I '{}' echo '{}'
echo 26704189108_9e71d865d8_b.jpg
26704189108_9e71d865d8_b.jpg
echo 51119071159_905212c30f_b.jpg
51119071159_905212c30f_b.jpg
echo 8076936667_c5605863a9_o.jpg
8076936667_c5605863a9_o.jpg
```

In particular, I could see whether the command I was running with `xargs` was being called with all the arguments at once, or passed them individually.

2. **Files discovered by `find` may be in a different order on different OSes.**

This was the heart of my issue -- I was passing all my filenames in one big list, and `caddy fmt` will [only format the first path it gets](https://github.com/caddyserver/caddy/issues/6702).
Because the order of files is different on different OSes, I was getting different behaviour.

Compare:

```console
$ # macOS
$ find . -name Caddyfile -o -name '*.Caddyfile'
./Caddyfile
./caddy/gone.Caddyfile
./caddy/redirects.Caddyfile

$ # Ubuntu
$ find . -name Caddyfile -o -name '*.Caddyfile'
./caddy/gone.Caddyfile
./caddy/redirects.Caddyfile
./Caddyfile
```

In hindsight, this makes sense -- there are multiple ways you might search a directory tree, and one isn't necessarily better than the other.

Don't rely on the order to be consistent!

I've had a brief glance at the man page, and `find` does have some options for controlling the ordering -- for example, you can use `-d`/`--depth` to do a depth-first search (process each directory's contents before the directory itself).

This reminds me of when I discovered `WalkDir::contents_first()` and `os.walk(topdown: bool)` while [writing `emptydir`](/2024/emptydir/#how-does-it-work).
47 changes: 47 additions & 0 deletions src/_til/2024/2024-11-22-install-from-github-releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
layout: til
title: How to install an asset from a GitHub release
date: 2024-11-22 11:35:50 +0000
tags:
- github
summary:
Use `gh release download`, which includes a pattern matcher if you want to pick a specific asset.
---
I wanted to install Caddy in GitHub Actions, and one of the ways [to install it](https://caddyserver.com/docs/install#static-binaries) is by downloading static binaries from their GitHub releases.

I wasn't sure how to download release assets in a sensible way -- I've written shell scripts for it before, but it feels messy and fragile.
Surely there must be a better way!

I stumbled upon [a GitHub Action](https://github.com/php/php-src/blob/master/.github/actions/setup-caddy/action.yml) in the PHP repository (!) which pointed me in the right direction:

```shell
gh release -R caddyserver/caddy download --pattern 'caddy_*_linux_amd64.tar.gz' -O - | sudo tar -xz -C /usr/bin caddy
sudo chmod +x /usr/bin/caddy
```

This is using [the GitHub CLI](https://cli.github.com/), which I haven't used before -- but it's pre-installed in GitHub Actions, which is very convenient for my purposes!

## In a GitHub Action

Here's how I packaged this as a step in a GitHub Action:

```yml
- name: Install Caddy
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release download \
--repo caddyserver/caddy \
--pattern 'caddy_*_linux_amd64.tar.gz' \
--output caddy.tar.gz
tar -xzf caddy.tar.gz --directory /usr/local/bin
chmod +x /usr/local/bin/caddy
which caddy
```

It's a bit more verbose and leaves a `caddy.tar.gz` file lying around, but I find this version easier to understand and debug.

## Notes

* You can specific a release to `gh release download`, or if not it will pick the latest release.
If you let it pick automatically, it skips pre-releases.