Add pagination support for completed items and projects

[dltn]

Hey @darekkay, this tool is wonderful! Thanks for open sourcing this.

Currently, the export only contains 30 each of completed items and projects (the API default limit):

% cat ~/Downloads/todoist.json | jq .completed.items | jq length
30

% cat ~/Downloads/todoist.json | jq .completed.projects | jq length
30

This PR adds pagination support so all previously completed items / projects are exported:

% cat ~/Downloads/todoist.json | jq .completed.items | jq length
2702

% cat ~/Downloads/todoist.json | jq .completed.projects | jq length
220

[darekkay]

Hey @dltn
thanks for your contribution! The change itself looks good and works like a charm.

However, I've got over 10.000 archived tasks, and it takes almost 1 minute for the tool to finish. I believe, some people will have even more tasks. Additionally, the tool doesn't have any loading feedback when using the download button (it does work when clicking the direct link, though). I've created #10 to track this issue, as it's not directly related to this PR.

My proposal is to add a disclaimer, e.g. "Exporting large amounts of archived tasks may take some time".

Some users might not care about archived tasks, so an additional improvement would be to add a checkbox like "Download all archived tasks (this will take more time to complete)" - default = "off" for better performance.

Last but not least, did you find any information about the default quota? Todoist might not be happy about getting so many requests per download (50+ requests for me).

[dltn]

Wow, that's a while! I'll address the feedback and test larger archives. The limit is 50 requests per minute, so about ~10K tasks/minute at 200 items per request. Behavior is likely undefined if we exceed that - so we'd better to add a timeout to be safe.

[dltn]

(Haven't forgotten about this, just busy in the new year. Looping back soon.)

[darekkay]

(Haven't forgotten about this, just busy in the new year. Looping back soon.)

There's no hurry :)

[dltn]

Added optional checkbox (default off) control and disclaimers:

I learned the way format is currently passed through the OAuth state parameter is non-standard. In our case, this complicates passing more than one piece of state (before: format, after: format & archive option).

This PR encodes the archive option in the existing "format string" – which isn't ideal, but minimizes the changes needed to keep compatibility. I don't want to bloat this PR further, so I created #11 to track this.

[darekkay]

Looks good, thanks a lot! 😊🚀

[darekkay]

I've deployed the change to production