Use LFS for all new files added/changed

[tracker1]

This will use LFS for tracking new Zip files added/changed in the repository.

closes #27

[fcambus]

Thanks for relaunching the discussion on this topic. I'm not very knowledgable about Git LFS at the moment, so I was hoping someone would jump in, but so far it hasn't happened :-)

When I first saw your issue, I noticed that there was still a 1GB limit, but I can't find any references about the limit anymore, did GitHub lift it?

[matthewpkemp]

Thanks for relaunching the discussion on this topic. I'm not very knowledgable about Git LFS at the moment, so I was hoping someone would jump in, but so far it hasn't happened :-)

When I first saw your issue, I noticed that there was still a 1GB limit, but I can't find any references about the limit anymore, did GitHub lift it?

References are here in respect to repo limits: https://help.github.com/en/github/managing-large-files/versioning-large-files

And here re lfs: https://help.github.com/en/github/managing-large-files/about-git-large-file-storage

[tracker1]

It probably doesn't really matter since these files very rarely change in practice... Also, git lfs requires end-users to install/enable it git lfs install after checkout at times.

Also, not sure if the 1GB limit also applies to LFS, and it wouldn't move the existing files without rewriting the history, I think there's a script/command for that.

[lordscarlet]

I forgot this discussion ever happened, and I did the same. 😆

#44

If all it means is that I have to pay $5/mo if it goes over, I'm not too worried about that. although, now I can't find where I saw that $5 number.

The fact that people may have to install LFS worries me slightly more. Does anyone know how big of a deal that is?

[sairuk]

as long as you have git >=1.8.3.1 you can install the git-lfs package (linux) and run git lfs install in the repo working tree to initialize it, the install may only have to be run once? i haven't played with lfs in a while now and there is no content on lfs yet in this commit to test further with.

decent tutorial on lfs here: https://github.com/git-lfs/git-lfs/wiki/Tutorial

doesn't look like the intent is to migrate the existing archive to lfs, probably worth considering having one process for pulling bins because after this is done you'll get some bins with pull but new stuff wont come down locally without a git lfs fetch iirc

just a note that git lfs migrate will need to be used and pushed to migrate existing bins to lfs. if you run git lfs ls-files on this branch atm there is nothing on lfs

after migrate you can see the lfs content

$ git lfs ls-files
495112ee0b - 2018/impure71.zip
da4584e41a - 2018/mist0218.zip
c6e0ef0e7f - 2018/mistergirls-2-perling-aint-easy.zip
a5fc4da32d - 2018/ninjapenguin2018.zip
3bbf9039f3 - 2019/blocktronics-67rpm.zip
9b55aad043 - 2019/blocktronics-blocky-horror.zip
31e1fdfda2 - 2019/blocktronics-dsotb.zip

if the entire archive is moving to lfs, .gitattributes should be updated to include rar files capture ./2011/cph.artpack.24.back.in.business.2011.rar

i used git lfs migrate import --no-rewrite -m "test migrate to lfs" */*.zip to test

[lordscarlet]

@sairuk what if you do the same for #44 ?

[sairuk]

@lordscarlet i have added notes to #44 with the result of trying to do the same, in short lfs is already over quota trying to checkout that branch, which was always going to be a risk with a full migration.