Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Better rate limit handling #76

Open
ReubenFrankel opened this issue Oct 28, 2024 · 1 comment
Open

feat: Better rate limit handling #76

ReubenFrankel opened this issue Oct 28, 2024 · 1 comment
Assignees
@ReubenFrankel
Labels
enhancement New feature or request

Comments

@ReubenFrankel
Copy link
Contributor

ReubenFrankel commented Oct 28, 2024 •
edited
Loading

There have been a number of times the tap has hit a 429 Too Many Requests error and the inbuilt back-off strategy has not been able to recover the sync. We should investigate if there is a better way to handle these situations, such as

  1. respecting a remaining rate limit value, if available (typically a header)
  2. simply increasing the max attempts (override backoff_max_tries)
Error logs 
2024-10-28T18:04:32.667482Z [info     ] 2024-10-28 18:04:32,667 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.006067, "tags": {"stream": "sellables", "endpoint": "/sellables", "http_status_code": 429, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:32.667813Z [info     ] 2024-10-28 18:04:32,667 | INFO     | backoff              | Backing off _request(...) for 2.2s (singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:32.668060Z [info     ] 2024-10-28 18:04:32,667 | ERROR    | root                 | Backing off 2.24 seconds after 1 tries calling function <bound method RESTStream._request of <tap_veeqo.streams.SellablesStream object at 0x7f80ffe6e970>> with args (<PreparedRequest [GET]>, None) and kwargs {} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:34.915149Z [info     ] 2024-10-28 18:04:34,914 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.006491, "tags": {"stream": "sellables", "endpoint": "/sellables", "http_status_code": 429, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:34.915625Z [info     ] 2024-10-28 18:04:34,914 | INFO     | backoff              | Backing off _request(...) for 4.4s (singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:34.915871Z [info     ] 2024-10-28 18:04:34,914 | ERROR    | root                 | Backing off 4.43 seconds after 2 tries calling function <bound method RESTStream._request of <tap_veeqo.streams.SellablesStream object at 0x7f80ffe6e970>> with args (<PreparedRequest [GET]>, None) and kwargs {} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:39.352907Z [info     ] 2024-10-28 18:04:39,352 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.006668, "tags": {"stream": "sellables", "endpoint": "/sellables", "http_status_code": 429, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:39.354010Z [info     ] 2024-10-28 18:04:39,352 | INFO     | backoff              | Backing off _request(...) for 8.5s (singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:39.354263Z [info     ] 2024-10-28 18:04:39,352 | ERROR    | root                 | Backing off 8.50 seconds after 3 tries calling function <bound method RESTStream._request of <tap_veeqo.streams.SellablesStream object at 0x7f80ffe6e970>> with args (<PreparedRequest [GET]>, None) and kwargs {} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:47.866144Z [info     ] 2024-10-28 18:04:47,865 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.008413, "tags": {"stream": "sellables", "endpoint": "/sellables", "http_status_code": 429, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:47.866557Z [info     ] 2024-10-28 18:04:47,865 | INFO     | backoff              | Backing off _request(...) for 16.7s (singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:04:47.866747Z [info     ] 2024-10-28 18:04:47,865 | ERROR    | root                 | Backing off 16.70 seconds after 4 tries calling function <bound method RESTStream._request of <tap_veeqo.streams.SellablesStream object at 0x7f80ffe6e970>> with args (<PreparedRequest [GET]>, None) and kwargs {} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.590515Z [info     ] 2024-10-28 18:05:04,589 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 0.007571, "tags": {"stream": "sellables", "endpoint": "/sellables", "http_status_code": 429, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.591069Z [info     ] 2024-10-28 18:05:04,590 | ERROR    | backoff              | Giving up _request(...) after 5 tries (singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.591557Z [info     ] 2024-10-28 18:05:04,590 | INFO     | singer_sdk.metrics   | METRIC: {"type": "counter", "metric": "http_request_count", "value": 61, "tags": {"stream": "sellables", "endpoint": "/sellables", "pid": 258}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.591870Z [info     ] 2024-10-28 18:05:04,590 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "sync_duration", "value": 52.638946533203125, "tags": {"stream": "sellables", "pid": 258, "context": {}, "status": "failed"}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.592199Z [info     ] 2024-10-28 18:05:04,590 | INFO     | singer_sdk.metrics   | METRIC: {"type": "counter", "metric": "record_count", "value": 671, "tags": {"stream": "sellables", "pid": 258, "context": {}}} cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.592414Z [info     ] 2024-10-28 18:05:04,590 | ERROR    | tap-veeqo.sellables  | An unhandled error occurred while syncing 'sellables' cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.592638Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.592901Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1231, in sync cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.593101Z [info     ]     for _ in self._sync_records(context=context): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.593335Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1128, in _sync_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.593573Z [info     ]     for idx, record_result in enumerate(self.get_records(current_context)): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.593833Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 597, in get_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.594049Z [info     ]     for record in self.request_records(context): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.594276Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 409, in request_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.594486Z [info     ]     resp = decorated_request(prepared_request, context) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.594685Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/backoff/_sync.py", line 105, in retry cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.594937Z [info     ]     ret = target(*args, **kwargs) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.595193Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 294, in _request cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.595415Z [info     ]     self.validate_response(response) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.595726Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 201, in validate_response cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.595945Z [info     ]     raise RetriableAPIError(msg, response) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.596202Z [info     ] singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.596385Z [info     ] Traceback (most recent call last): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.596584Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/bin/tap-veeqo", line 8, in <module> cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.596905Z [info     ]     sys.exit(TapVeeqo.cli())   cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.597108Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/click/core.py", line 1157, in __call__ cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.597360Z [info     ]     return self.main(*args, **kwargs) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.597549Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/click/core.py", line 1078, in main cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.597751Z [info     ]     rv = self.invoke(ctx)      cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.597989Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/plugin_base.py", line 82, in invoke cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.598151Z [info     ]     return super().invoke(ctx) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.598360Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/click/core.py", line 1434, in invoke cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.598561Z [info     ]     return ctx.invoke(self.callback, **ctx.params) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.598791Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/click/core.py", line 783, in invoke cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.598974Z [info     ]     return __callback(*args, **kwargs) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.599174Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 507, in invoke cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.599384Z [info     ]     tap.sync_all()             cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.599586Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/tap_base.py", line 466, in sync_all cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.599748Z [info     ]     stream.sync()              cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.599918Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1231, in sync cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.600233Z [info     ]     for _ in self._sync_records(context=context): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.600504Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/core.py", line 1128, in _sync_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.600711Z [info     ]     for idx, record_result in enumerate(self.get_records(current_context)): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.600991Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 597, in get_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.601231Z [info     ]     for record in self.request_records(context): cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.601499Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 409, in request_records cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.601677Z [info     ]     resp = decorated_request(prepared_request, context) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.601866Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/backoff/_sync.py", line 105, in retry cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.602038Z [info     ]     ret = target(*args, **kwargs) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.602212Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 294, in _request cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.602404Z [info     ]     self.validate_response(response) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.602627Z [info     ]   File "/tmp/shelltaskscripts3219743586439785816/.meltano/extractors/tap-veeqo/venv/lib/python3.9/site-packages/singer_sdk/streams/rest.py", line 201, in validate_response cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.602884Z [info     ]     raise RetriableAPIError(msg, response) cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
2024-10-28T18:05:04.603132Z [info     ] singer_sdk.exceptions.RetriableAPIError: 429 Client Error: Too Many Requests for path: /sellables cmd_type=elb consumer=False job_name=dev:tap-veeqo-to-target-postgres:54b5b47e-efc1-4169-a278-8aae9a590cb6 name=tap-veeqo producer=True run_id=38201f4f-e882-448d-a7d6-cbece59e66a5 stdio=stderr string_id=tap-veeqo
@ReubenFrankel ReubenFrankel added the enhancement New feature or request label Oct 28, 2024
@ReubenFrankel
Copy link
Contributor Author

  1. simply increasing the max attempts (override backoff_max_tries)

5dd84d9

@ReubenFrankel ReubenFrankel self-assigned this Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ReubenFrankel ReubenFrankel

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ReubenFrankel