update_batch_size in arch config

[thomashirtz]

Hello,
I wanted to ask some information about an implementation detail
Is it possible to give an insights about the reason of the parameter update_batch_size in the config ?
I am using the ddqn system and it seems like it just split the batch size in chunk before mapping them, which seems equivalent to just not vmap it to me. Would it be possible to give explanation about why is it done this way ?
thanks !

[EdanToledo]

Hi, great question. Basically, the point of the update batch size is to control how many duplications of the algorithm are performed on a single device. It is the single device equivalent of pmapping the learn function over each device. Since it's on one device, we can choose how many duplications we want to perform. The reason for this control is twofold, 1. To match the design outlined in the actual podracer paper that created this system design and 2. To allow flexibility over how you scale the algorithm. Certain things can change depending on the algorithm and network and environment you are using. Like maybe the environment is the bottleneck, maybe the network or the update function etc.

Does that make sense? There is also an illustration of the update_batch_size in the anakin diagram in the readme.

[thomashirtz]

But I thought that when you do vmap you don't "duplicate" but you simply distribute computation before aggregating them ? So there is only one "algorithm" no ?

I'll read the paper to try to understand better

Thank you again for your help :)

[EdanToledo]

You are correct, in my code you are distributing it, I probably shouldn't be using the word "duplicate", I just thought it might be easier to explain. A better word is "replicate" as what is happening is that the code calculates the minimal unit of computation given the configs and then replicates it multiple times for one device and then over the multiple devices to achieve the specified config. Give the original anakin podracer architecture paper a read and I'll try answer any remaining questions.