- Lecture: pdf version, odp version
- Seminar & homework: here
- Video (both lecture and seminar)
Note: while you can do the entire thing in Colab, we recommend using a node with at least 4 cpus to better "feel" the difference :)
Most modern PCs already have 4+ cores, but you can also find free cloud alternatives here. The easiest version is to use Kaggle kernels.
Note 2: The practice notebook was tested in Linux and MacOS. Running in Windows may cause problems due to inability to fork processes. When in doubt, try WSL or docker(kitematic). Run linux inside a VM will also do the trick.
More stuff:
- Numba parallel - a way to develop threaded parallel code in python without GIL
- joblib - a library of multiprocessing primitives similar to mp.Pool, but with some extra conveniences
- BytePS paper - https://www.usenix.org/system/files/osdi20-jiang.pdf
- Alternative lecture: Parameter servers from CMU 10-605 - here
- Alternative seminar: python multiprocessing - playlist