Finalize "Pending" zarr if all upload URLs must have timed out by now

[yarikoptic]

Instigator:

• --mode verify detects lots of unexpected diffs in metadata without server timestamp boost backups2datalad#49 (comment)

which summarizes as - we do not adjust "modified" time stamp of the dandiset even if zarr is modified (some files uploaded/changed) whenever it is never finalized. I think we should have some periodic job which would go through Pending zarrs, and if it was put into Pending state longer before the time window we allow for uploads/changes to happen in zarr -- "forcefully" finalize it in current state

NB in principle, whenever we have versioning implemented, we could revert back by removing versions of modifications to the keys and DeleteMarkers possibly added. But since we do not track to that level ATM, we cannot revert AFAIK. Also reversal might introduce the need to add similar reversal in backups2datalad (or just to not update Zarr until finalized into specific version), thus the easiest is just to finalize.

I think it might relate to

• Dandiset "modified" timestamp not updated when entries deleted from Zarr #1871

[jjnesbitt]

Another way to go about this is to send recurring emails to dandiset owners if zarrs/blobs have been lingering in an unfinished state. This would encourage them to take action on those stale files.

[satra]

@jjnesbitt - i think this is where the ui thought process may have come, and it came up in the context of unembargoing blobs. we perhaps need something to finalize or cancel blobs or zarrs on the ui side in addition to reminders. i don't know if the user has any option to cancel an unfinished upload now or to find it.

[yarikoptic]

instead of finalizing it we might want to actually "roll back" (delete uploaded since prior finalized state versions and deletemarkers)... something to keep in mind for redesign (#1892)

edit:

• that was the idea behind Do we have enough information to GC abandoned Zarr upload attempt? #1842