This is a good idea, but you probably want to run git lfs prune to avoid the cache growing indefinitely.

That way (some version of) old objects will be downloaded from cache, git lfs pull updates them (the partial hit will save bandwidth), then the cache is pruned of older stuff with git lfs prune, and the cache action only saves the pruned version for the next time.

FYI: I've validated this locally in my repo.