gep: add GEP-3388 HTTPRoute Retry Budget

[ericdbishop]

What type of PR is this?

/kind gep

What this PR does / why we need it:

To seek consensus on the ideal configuration of a "retry budget" in HTTPRoute, allowing application developers to dynamically limit the rate of client-side retries to their service based on a percentage of the active request volume.

Extends #1731

Which issue(s) this PR fixes:

Fixes #3388

Does this PR introduce a user-facing change?:

TBD. GEP is marked "Provisional" while implementation details are sorted out, and goals are reviewed.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ericdbishop
Once this PR has been reviewed and has the lgtm label, please assign danwinship for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @ericdbishop!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @ericdbishop. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mikemorris]

/ok-to-test

[ericdbishop]

For the API implementation, I've been comparing approaches between introducing retry budgets as a part of HTTPRoute, or implementation via policy attachments. Retry budgets are the default retry policy for Linkerd, and are highly recommended by Envoy when configuring cluster circuit breaker thresholds, so simplicity will be a priority here.

HTTPRoute retry stanza

• The UX for retry budgets we'd like to match is applying the policy to a service as a whole, as opposed to individual routes. If you have multiple HTTPRoute objects pointing to the same service, you would have to configure the retry budget in multiple places.

• While Linkerd implements a retry budget alongside individual route configuration within the ServiceProfile CRD, the retry budget is configured once across all routes in a single set of fields within the CRD. In practice, this works similarly to the Envoy CircuitBreaker threshold.

• If we wanted retry budgets to be configured on a per-route basis (as opposed to at the service level), it would require a change to be made in Envoy Route. I believe similar changes would need to be made for Linkerd.

Policy Attachment

• May be more confusing from a UX perspective to configure retries in two different places (HTTPRoute for static retries, versus a policy attachment for a dynamic retry threshold).

• Ideally, we could define a single policy, targeting a service, that would dynamically configure a retry threshold based on the percentage of active requests across all routes destined for that service's backends.

• This may allow for a dynamic threshold of retries to be configured at the service level, alongside a static max number of retries on the route level. Application developers would then be allowed more granular control of which requests should be retried. For example, maybe an application developer will not want to perform retries on a specific route where requests are not idempotent, and can disable retries for that route.

---

Please let me know if my assessment of the functionality between different data plane implementations, and my understanding of how we could use policy attachments, are correct here. I would lean towards the policy attachment implementation, as it seems like the more straightforward approach, both for data plane implementations, as well as for the UX we offer to application developers. Would appreciate your thoughts here, @kflynn @mikemorris!