You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Its container will fetch data in 100 day intervals by default and signal has_more: true to kamu, so that pulling it in CLI results in multiple blocks committed as it pulls container until it's exhausted:
When I add this dataset to kamu node and run the ingest flow, however, it only executes one iteration and produces one AddData block. Thus I have to repeatedly trigger the flow until the source is exhausted.
Expectation:
Ingest flow should loop until the source is exhausted has_more == true
In the short term this looping could be done inside the ingest task, but ideally we would want each iteration to be in a separate task so that each of them went through the scheduling process and quotas checks independently
The text was updated successfully, but these errors were encountered:
Consider
com.defillama.tokens.prices
dataset.Its container will fetch data in 100 day intervals by default and signal
has_more: true
tokamu
, so that pulling it in CLI results in multiple blocks committed as it pulls container until it's exhausted:When I add this dataset to kamu node and run the ingest flow, however, it only executes one iteration and produces one
AddData
block. Thus I have to repeatedly trigger the flow until the source is exhausted.Expectation:
has_more == true
The text was updated successfully, but these errors were encountered: