Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move supervisor API request handlers into dedicated package and add unit tests #11471

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

brandond
Copy link
Member

@brandond brandond commented Dec 16, 2024

Proposed Changes

  • Properly handle creation of new etcd cluster containing only the current node
    Handles a corner case revealed by testing supervisor APIs
  • Reorganize supervisor APIs used by joining nodes
    Moves request handlers and bootstrap password validation code out of the server package into dedicated packages.
  • Misc cleanup / removal of unused fields.
  • Allow clients to generate their own keys, instead of requiring them to use key generated by the server
    Improves overall security posture by no longer reusing private keys for agent certificates across multiple nodes in the cluster. Both agent and server contain backwards-compatibility code to allow interop with other version that do not support certificate signing requests when acquiring agent certs.
    After upgrading, existing nodes can be re-keyed to use unique keys by deleting their existing certs and keys from disk or running k3s certificate rotate, and restarting the node.
    This also removes the requirement to rotate certs on servers before agents, since they no longer reuse certs+keys pre-generated by the server. Agent certs will in fact be renewed (with the same key) on every startup. Rotation will work as it currently does, where the keys are removed to force creation of new keys and certs.
  • Collect mocks from various tests into tests/mock package for reuse
  • Add extensive tests for supervisor API request handlers

Types of Changes

tech debt / enhancement

Verification

  • Verify that node joins work as before
  • Verify that agents do not share private keys (/var/lib/rancher/k3s/agent/*.key)

Testing

Yes

Linked Issues

User-Facing Change


Further Comments

@brandond brandond requested a review from a team as a code owner December 16, 2024 04:47
Copy link

codecov bot commented Dec 16, 2024

Codecov Report

Attention: Patch coverage is 61.37841% with 297 lines in your changes missing coverage. Please review.

Project coverage is 45.03%. Comparing base (68fbd1a) to head (eb8144a).

Files with missing lines Patch % Lines
pkg/server/handlers/handlers.go 74.21% 49 Missing and 17 partials ⚠️
pkg/server/handlers/secrets-encrypt.go 7.81% 59 Missing ⚠️
pkg/agent/config/config.go 38.02% 25 Missing and 19 partials ⚠️
pkg/nodepassword/validate.go 67.69% 34 Missing and 8 partials ⚠️
tests/mock/core.go 68.99% 34 Missing and 6 partials ⚠️
pkg/server/handlers/cert.go 5.26% 18 Missing ⚠️
pkg/server/handlers/token.go 20.00% 14 Missing and 2 partials ⚠️
pkg/etcd/etcd.go 54.54% 3 Missing and 2 partials ⚠️
pkg/cli/secretsencrypt/secrets_encrypt.go 71.42% 2 Missing ⚠️
pkg/daemons/control/deps/deps.go 0.00% 0 Missing and 2 partials ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #11471      +/-   ##
==========================================
- Coverage   48.44%   45.03%   -3.41%     
==========================================
  Files         181      184       +3     
  Lines       18809    18996     +187     
==========================================
- Hits         9112     8555     -557     
- Misses       8348     9225     +877     
+ Partials     1349     1216     -133     
Flag Coverage Δ
e2etests 35.78% <48.73%> (-7.63%) ⬇️
inttests 35.07% <47.31%> (+0.02%) ⬆️
unittests 17.05% <47.85%> (+2.79%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@brandond brandond marked this pull request as draft December 16, 2024 08:32
@brandond
Copy link
Member Author

Moving to draft for addition of tests

@brandond brandond force-pushed the refactor-join-api branch 2 times, most recently from d086dd9 to c973c3d Compare December 18, 2024 02:35
@brandond brandond marked this pull request as ready for review December 18, 2024 02:36
@manuelbuil manuelbuil self-requested a review December 18, 2024 15:57
@brandond brandond changed the title Refactor cluster join APIs Move supervisor API request handlers into dedicated package and add unit tests Dec 18, 2024
manuelbuil
manuelbuil previously approved these changes Dec 19, 2024
Copy link
Contributor

@manuelbuil manuelbuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a big security improvement (and testing!)

func NewHandler(ctx context.Context, control *config.Control, cfg *cmds.Server) http.Handler {
nodeAuth := nodepassword.GetNodeAuthValidator(ctx, control)

prefix := "/v1-{program}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious: How is this {program} var populated?

Copy link
Member Author

@brandond brandond Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this format it is just a placeholder for mux to match the request path. It is used as a var in the request handlers and must match the program name for RBAC purposes.

dereknola
dereknola previously approved these changes Dec 20, 2024
@brandond brandond dismissed stale reviews from dereknola and manuelbuil via 65cec0a December 21, 2024 05:19
@brandond brandond force-pushed the refactor-join-api branch 2 times, most recently from 65cec0a to 93f5e42 Compare December 21, 2024 05:24
The servers package, and router.go in particular, had become quite
large. Address this by moving some things out to separate packages:
* http request handlers all move to pkg/server/handlers.
* node password bootstrap auth handler goes into pkg/nodepassword with
  the other nodepassword code.

While we're at it, also be more consistent about calling variables that
hold a config.Control struct or reference `control` instead of `config` or `server`.

Signed-off-by: Brad Davidson <[email protected]>
Clients now generate keys client-side and send CSRs. If the server is down-level and sends a cert+key instead of just responding with a cert signed with the client's public key, we use the key from the server instead.

Signed-off-by: Brad Davidson <[email protected]>
Convert nodepassword tests to use shared mocks

Signed-off-by: Brad Davidson <[email protected]>
Make this field an interface instead of pointer to allow mocking. Not sure why wrangler has a type that returns an interface instead of just making it an interface itself. Wrangler in general is hard to mock for testing.

Signed-off-by: Brad Davidson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants