Autopause: Rootless improvements, mainly for Kubernetes

[JJGadgets]

Enhancement Type

Improve an existing feature

Describe the enhancement

Feature Request 1: AUTOPAUSE_SUDO env var

I'd like for the autopause-daemon.sh script to not use sudo to launch knockd if it is disabled by the user via an environment variable such as AUTOPAUSE_SUDO.

Feature Request 2: setcap on image build

During image builds, run setcap cap_net_raw=ep /usr/local/sbin/knockd so that rootless users can launch knockd without running it with sudo.

I'm not entirely sure if this (request 2) is possible, unlike request 1. There are some pages that suggest it's possible with BuildKit (https://wildwolf.name/multi-stage-docker-builds-and-xattrs/ and moby/buildkit#2250 (comment)), but I thought I'd put in the request anyway so it can be looked at.

Context & Details

I currently run this image in Kubernetes, and I prefer to harden as much of my selfhosted apps as I can. In particular, I'd like to run fully rootless and allow as little privileged permissions as possible.

Kubernetes allows for 2 related container securityContext configurations I'd like to use, but can't due to autopause and knockd: allowPrivilegeEscalation: true and runAsNonRoot: true.

The goal is to launch all processes in the running main container as non-root and enforce no root processes on the Kubernetes container runtime level, but autopause-daemon.sh will always launch knockd as root due to the use of sudo.

I currently use a rootful initContainer to run setcap cap_net_raw=ep /usr/local/sbin/knockd allowing all users launching knockd to capture the socket without being root, which then terminates and the actual main container with the proper entrypoint and Minecraft server runs without root. The pod itself is given the CAP_NET_RAW capability too, but Kubernetes doesn't apply that to ambient caps (kubernetes/kubernetes#56374) so non-root users can't use the capability unless the file has the cap added anyway.

Here's my botched attempt with commented out but almost working Kubernetes pod configuration for this: JJGadgets/Biohazard@e2002af
(launching knockd in another container isn't helpful because it can't see the main Java Minecraft process, but manually attaching a shell using kubectl and running knockd as non-root works just fine so the initContainers do work)

Note: The initContainer method + feature request 1 solves allowPrivilegeEscalation which is the more pressing issue, but runAsNonRoot still can't be enabled because I can't setcap as non-root even with CAP_SETFCAP added due to the ambient caps Kubernetes issue, which is what feature request 2 aims to solve (and so I can minimize the amount of jank in my deployment lol).

[itzg]

I've got it building a test image, which will be tagged as itzg/minecraft-server:test-2421-java17:

https://github.com/itzg/docker-minecraft-server/actions/runs/6448602298/job/17506052521

Let me know if that image works for your scenario.

[JJGadgets]

Sorry, I forgot one more thing: my initContainer cp sets owner to minecraft (UID 1000) but the original binary at /usr/local/sbin/knockd is still owned as root. chowning the /usr/local/sbin/knockd should allow the minecraft user to use the added caps.

[itzg]

Sorry, I forgot one more thing: my initContainer cp sets owner to minecraft (UID 1000) but the original binary at /usr/local/sbin/knockd is still owned as root. chowning the /usr/local/sbin/knockd should allow the minecraft user to use the added caps.

I'm not sure how to do that at image build time since the running UID is 1000 by default but is arbitrary. Do you have any advice?

[JJGadgets]

I don't really have a better solution to which UID is used, but it's certainly better than running anything as root IMO, and given that 1000 is already used in other places (like sudoers) for autopause, I propose we just stick to 1000 for now to close out this "run fully rootless" issue, and if you or anyone else is interested, figure out different UIDs for autopause later in a different issue.

For folks not using the same Kubernetes container runtime/securityContext hardening, and who didn't disable AUTOPAUSE_SUDO, the sudo will still run so knockd will launch as root, which bypasses the chown anyway.

To elaborate: for autopause functionality specifically, IIRC the UID 1000 is hardcoded for the sudoers as well, and maybe other areas I haven't seen, I had to change my container's runAsUser to 1000 instead of a random arbitrary number for the sudo to work (this is without additional container runtime hardening like allowPrivilegeEscalation that I mentioned in the OP).

Without autopause, the container can be launched as a different UID in the container config (not env vars) without breaking the Minecraft server itself which doesn't need root or any privileges or privileged capabilities, and from my understanding doesn't hardcode any UID/GID anywhere.

AFAIU, the UID and GID env vars don't break knockd currently if the container is launched without a UID specified in the container config itself (not the env vars), because before AUTOPAUSE_SUDO was a config option, knockd will launch as root using sudo before the other processes run as the UID/GID specified in the env vars.

[JJGadgets]

As for how to implement, I think COPY --chown Dockerfile directive is the way to go. Though, from what I see, that will mean you might have to separate out the knockd curling and other steps from the install-packages.sh script? Or maybe just the part where it's placed into /usr/local/sbin?

[itzg]

chown'ing an executable via COPY or within the install-packages.sh script is trivial. This doesn't sound secure to me since a non-root user would have ownership of an executable that has elevated capabilities that they could modify and in turn exploit those elevated capabilities. I thought the point of privilege escalation was to allow a non-root user to selectively and temporarily gain elevated access to what is typically a root-level capability.

Several attack vectors would need to exploited before getting to the point of running inside the container, arbitrarily modifying the executable, and exploiting that -- so maybe my concern is not significant.

Making a note for myself and/or future users that report an issue because a UID other than 1000 doesn't work, the startup will alter the minecraft user's UID to whatever is provided:

docker-minecraft-server/scripts/start

Line 28 in df497ce

usermod -u $UID minecraft

As such, I am not going to research/confirm if altering the minecraft user's UID from 1000 or setting spec.securityContext.runAsUser to a value other than 1000 will break auto-pause / knockd.

[itzg]

Pushed a change to test branch/image with knockd chown'ed to minecraft. Let me know if that that resolves that part.

[JJGadgets]

Apologies, I've been busy with other stuff.

The setcap just needs to be after the chown line for it to work (else chown overrides the setcap).

I've tested this commit with an image built with my fork, and it's been working for a while now.

JJGadgets@deb40d1

[itzg]

Is the chown still necessary? Thats the part that bothers me.

[JJGadgets]

Yes, the chown dictates which UID/user gets to execute with the additional capabilities.

[itzg]

Yes, the chown dictates which UID/user gets to execute with the additional capabilities.

Can you point to documentation regarding that assertion? I want to make sure you're not conflating setuid bit with executable ownership.

[TheAceMan]

I had been struggling with auto pause in kubernetes as well. is there a configuration to get auto pause to work with itzg.github.io/minecraft-server-charts?

Looks like @JJGadgets got it working with the changes he mentioned and his image is working for me as well with the itzg minecraft-server-charts adding the extra env values needed.

[itzg]

I will work on merging the setcap line but I will not include the chown step. I do now see that the chown in my branch was reverting the setcap but both should not be needed anyway.

[JJGadgets]

I will retest if the chown is needed after the setcap PR is in place with the latest commits and report back.

From my understanding (though I need to re-find the source of this info, I reached this conclusion through testing and searching up during testing), it's needed because setcap gives the owner of the file the permission to use the capability set with setcap, not everyone.

[antoncuranz]

Hey, I am also very interested in these improvements.
Is there a reason why the AUTOPAUSE_SUDO environment variable was not added?
I am currently running the image with allowPrivilegeEscalation: true (not really sure how to avoid this) but had to patch the autopause-daemon.sh script to not use sudo.
(runAsNonRoot: true works fine for me)

[itzg]

Just wanted to note that there's an undocumented env var SKIP_SUDO that can be set to "true" that will let the startup script just run as the container's initial user.

[antoncuranz]

Thank you for the info. It seems that setting this variable has no effect in my setup, as sudo is still hardcoded in autopause-daemon.sh.
Maybe we can add a condition checking for SKIP_SUDO in autopause-daemon.sh as well?