TLDR; I’ve decided to interface with docker from generic-worker via its command line interface rather than via the native docker go library

The obvious way to call docker from generic-worker would be to import docker client library, and call the library APIs. This seems like a no-brainer at first, since it avoids the overhead of spawning processes, monitoring process output and exit codes, and avoids the issue of having generic-worker talk potentially to an incompatible docker cli version.

However, there are some major disadvantages with this approach, namely:

1. if you want to pick up fixes from a newer docker client, you need to roll a new worker binary
2. the internet is full of resources documenting the reasonably well-understood command line interface. the native library APIs are generally less well documented
3. it is harder to troubleshoot issues when things go wrong, if you can’t reproduce the behaviour by executing docker commands directly in the docker CLI
4. it is harder for operations teams to understand what is going on if they can’t see docker commands in the logs
5. reporting bugs back to docker is harder if you can’t just copy/paste some docker-commands
6. build times will be longer and build binaries bigger
7. it would make it easier to switch worker implementation in the future, if we ever wanted to, also to a non-go worker, if it is clear which docker commands are getting called behind the scenes

Since the docker CLI is the de facto interface to docker, I think I’d like to make the switch early to using it, rather than importing the docker client library, as we currently do.

Note, I may write to you in a couple of months and tell you it was a disaster, and switch back - but for now, this is the approach I’d like to try.

Also note, this decision is also based on the very positive experience with calling out to taskcluster-proxy via an external process, which had many of the same benefits over embedding the proxy directly into the worker.

Miles's PR

Released in generic-worker 15.1.3.