Load Testing using 3 replicas

Specifications:

• Baseline 3 worker nodes of 16GB each assigned on Jetstream
• 3 replica-pods spawned for each microservice using kubeadm

I. Let's try first with 3000 requests per microservice:

• Regular sustained load of 3000 requests is handled well for a period of 30 minutes.
• Load balancing ensures great throughput and each pod is being utilized optimally.
• Minimal error rate observed of 0.02% and that too just for the Registration microservice requests that are Database-heavy operations.
• Overall, average-throughput is approximately 50 requests/second.

Aggregate highlights for Response-time per microservice:

All Response-times per microservice over time

Above-par throughput and low error rate 💯

II. Next, let's test at a higher load of 6000 requests at a time:

• Regular sustained load of 6000 requests is handled well for a period of 30 minutes.
• Load balancing ensures great throughput and each pod is being utilized optimally.
• Minimal error rate observed of 0.02% and that too just for the Registration microservice requests that are Database-heavy operations.
• Overall, average-throughput is approximately 50 requests/second.

Aggregate highlights for Response-time per microservice:

All Response-times per microservice over time

Above-par throughput and low error rate 💯

Key-takeaways

• The system handles 3000 requests and 6000 requests comfortably, when we have 3 replicas set up for each microservice.
• The requests aren't handled at the same time, rather there's some kind of sequential processing behavior.
• One potential improvement for throughput could be to exploit async-await functionality for concurrent requests at the Gateway.

Load balancing in action 😎

Note how the RAM usage for the weather-reporter (computation heavy microservice) is equally split over 3 replicas.

II. Spike Testing :

• Our system can handle a maximum of 4300 requests in a span of 60 secs with an acceptable error rate of 0.01% and a throughput of 48.6 requests/sec.

Aggregate highlights for Response-time per microservice:

All Response-times per microservice over time

Key-takeaways

• We observed that our system was able to handle a load of 4300 requests/minute for a span of 60 secs with 0.01% error rate.
• When we increased the load, we were getting a significant error rate.
• From the graphs, we can infer that our python API and registration API are taking maximum time to execute requests. For python API, we have used in-built libraries to implement our logic that is causing significant delays in processing requests. The registration API involved writing a chinook of user information to the database which is time consuming. Also, the synchronous behavior of REST API’s implemented for inter microservice communication is causing delays in our system and limiting the capacity of our system.

Next up: Testing with 5 replicas (Link)