A few optimizations that speedup training

[hvoss-techfak]

Thanks for creating such a great project.

I'm currently training on a rather large dataset (500+ GB) of audio data and have found a few optimizations that increase the pre-processing speed in the balance sampler from 10+ minutes down to ~15 seconds and changes to the sampling function to remove the for loop and use only torch functions, that made some pretty big changes for me (from 2 iterations/s to 3.5 iterations/s).

Both changes produce exactly the same results as before.

[AlekseySh]

Hey, @hvoss-techfak
thank you for the suggested changes and your interest in OML!

First, let's see if CI is green.

[hvoss-techfak]

I forgot to add the imports. One Second.

[AlekseySh]

Unfortunately, CI is not green, so, please, run tests locally (you will find the commands in the Makefile, you can also check the Contributing guide)

[AlekseySh]

@hvoss-techfak double check linters, please

[hvoss-techfak]

The imports are now fixed, but the tests still give me an error with the "labels = torch.IntTensor(labels)", as apparently not all triplet inputs are given as integers? I'll run the tests locally and check if I can fix this

[hvoss-techfak]

I relaxed the IntTensor to a normal tensor and now all the short tests pass.

[AlekseySh]

let's try one more time

[AlekseySh]

double check linters please

[hvoss-techfak]

That linters seems to be a formatting issue in the inbatch_hard_tri.py. It is fixed now.

[AlekseySh]

by the way, you can run command "make run_precommit", it will help to fix the linters

[AlekseySh]

@hvoss-techfak seems like the tests are fine now, so I will check the code soon

could you run all_tests locally please? some my fail (because of environment, OS and so on), but it should be easy to understand if the are related to your changes or not

[hvoss-techfak]

All tests are now done. I had a few that failed because of my ddp setup and some import errors, but apart from that all other tests passed.

[DaloroAT]

Hey, @hvoss-techfak

Thank you for your contribution!

I left a couple of comments removing redundant code.

[AlekseySh]

@hvoss-techfak check out the comments please

[hvoss-techfak]

all changes are done

[AlekseySh]

@DaloroAT thank you for review it!

[AlekseySh]

@hvoss-techfak welcome to the contributors, feel free to work on any issue you like :)