Docker-in-docker scenarios are not supported by docker-compose setup
More and more functionality in LAVA allows to use docker containers for various features/operations (one of a few examples is "qemu from docker image" feature). Given that setup maintained in this repository (pkg/docker-compose) itself runs in Docker, that effectively means running Docker-in-Docker.
The canonical reference on this matter is http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ . It goes thru the option of running "true" Docker-in-Docker, and concludes with:
Simply put, when you start your CI container (Jenkins or other), instead of hacking something together with Docker-in-Docker, start it with:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
Now this container will have access to the Docker socket, and will therefore be able to start containers. Except that instead of starting “child” containers, it will start “sibling” containers.
In other words, instead of running a docker container truly inside another docker container, any request to start a new container gets pushed "up", to the host which runs the primary docker daemon, and that's what runs the sub-container, and also where it runs (in a container on the host, not in the container inside another container).
This obviously leads to security concerns, but usage of Docker itself is a known security concern (separation which Docker offers is not at the level as e.g. a true VM offers, so any software or hardware (Meltdown, etc.) bug in these separation means will allow a container to escape).
Again, the usage above is effectively the canonical one, based on the fact that it has been advertised by Docker developers for a while, very easy to achieve, and any other alternative is considerably harder and has various drawbacks (as elaborated in the blog post above).
So, I don't think that we can stay away from this usage, as again, Docker features in LAVA appear and grow in their usefulness and importance.
The question then is how far we would like to go with it. Possible choices are:
- Enabled it (== /var/run/docker.sock mounting) by default.
- Add it to docker-compose.yaml, but have it commented out, so interested parties uncomment it themselves if they need it.
- Add a kind of option, which uses can set in the environment or separate config file, to enable or disable this feature.