feat: core service ready/liveness probes to be configurable

[dylanhitt]

Recently ran into an issue on our cluster that has a lot of churn (we use karpenter and leverage spot instances pretty heavily). For some reason there are points when a new node in the cluster comes up and the AWS VPC CNI resyncs we see a bit of latency. I couldn't quite figure out why. The 1 second timeout on the readiness probe procs fails quite a bit causing noise in our events monitoring as well as sometimes restarting the container when it seems otherwise healthy.

The only way I found to solve the issue is to increase the timeoutSeconds.

Let me know what you all think.

Cheers

[MinerYang]

Hi @dylanhitt ,

Thanks for your contribution!
We understood you may have some latency from your Kubernetes infra level. However it is not a common level, could you try to maintain this config first at your side, and we will see if more users from community will needs this.

Best,
Miner