Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mitigate 429 errors in cloud function execution for validation reports #863

Open
cka-y opened this issue Dec 18, 2024 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@cka-y
Copy link
Contributor

cka-y commented Dec 18, 2024

Describe the Problem

While testing changes related to issue #826, I observed that running multiple validation reports simultaneously can result in 429 Too Many Requests errors. This happens when no additional instances of the Cloud Function responsible for updating the database are available. An example can be seen in this execution log: Cloud Console Execution Summary

Proposed Solution

To prevent these errors and improve reliability, consider one of the following solutions:

  1. Cloud Task Implementation:
    Replace the direct function call in the workflow with a Cloud Task. Cloud Tasks provide automatic retries in case instances are unavailable, reducing the likelihood of 429 errors.

  2. Pub/Sub Triggered Function:
    Transition the database update function from an HTTP-triggered function to a Pub/Sub-triggered function. This would decouple the function execution and ensure scalability during high workloads.

By implementing one of these solutions, the system can handle high concurrent requests more gracefully without impacting database updates.

@cka-y cka-y added the enhancement New feature or request label Dec 18, 2024
@cka-y cka-y changed the title -- Mitigate 429 errors in cloud function execution for validation reports Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant