Networking Troubles

Currently minimesos has a few bugs related to networking

Because the Agent runs in BRIDGE mode, it uses a different network stack than Docker executors that run in HOST mode. This causes some subtle bugs like https://github.com/ContainerSolutions/minimesos/issues/401

There are several ways to solve this problem:

1. Run the Agent in HOST mode
2. Go back to Docker-in-Docker mode, so that the inner Docker running in HOST mode uses the same network stack as the Agent container
3. Add IP per container

Let's look into each option

1 Run the Agent in HOST mode

This looks simple enough but the problem is it goes the against the idea of isolation. We now have to manage the resources of the host and add a lot of code to prevent port collisions.

2 Docker-in-Docker mimics production very closely. However the problem here is that it is difficult to share images from the host to the inner Docker engine since /var/lib/docker, where images are stored, is not supposed to be shared.

3 IP per container is the future of networking in Mesos. It is not blocked by anything. The problem with this is that very few frameworks support this and we have to make changes to existing frameworks.