Create an official drupal-vm image on Docker Hub

[geerlingguy]

Issue Type

• Feature Idea

Summary

I think I may be able to get this working pretty simply once I merge in the support for Docker I'm working on in #1206

Basically, I'll need to:

• Create a new GitHub repository for the image maintenance.
• Build the image with a Dockerfile (default to all the standard Drupal VM defaults).
• Add the image to Docker Hub
• Find a way to get the Docker image build runs triggered by commits to Drupal VM (or maybe just tagged releases?). Could be manual for now, but I hate manual processes!

For later optimization, it would be good to explore things like:

• Building the image using ansible-container so that each build step can be cached as a separate FS layer. It'll probably be a fairly substantial delta on each Drupal VM image build :/
• Making different branches/tags for different things (e.g. Drupal core development, sprints, patch testing, contrib module testing... who knows).

[thom8]

@geerlingguy if you keep the Dockerfile in the main repo Docker Hub will auto build releases.

eg. https://hub.docker.com/r/beet/box/tags/

You can also specify multiple Dockerfiles in the hub configuration

[geerlingguy]

@thom8 - Hmm, good to know.

I might consider setting the official image to have a default share location (from the docker-compose file), and not have Drupal installed; then I could add a composer docker-quickstart or something like that, which would copy the compose file, grab the docker image, then run the playbook using current local settings against it to provision it with all the local configuration changes.

[thom8]

Have you considered using https://github.com/leighmcculloch/vagrant-docker-compose ?

At least in the short-term it might be easier to continue to use vagrant if it can continue to handle hosts file / networking etc.

Also it allows using env vars in docker-compose.yml which can be passed in via the Vagrantfile --
https://github.com/leighmcculloch/vagrant-docker-compose/blob/master/example/Vagrantfile#L20

[geerlingguy]

@thom8 - heh, I like how in https://hub.docker.com/r/beet/box/tags/ the size of the image keeps going up a few MB per tag :D

I would like to stick with all-Docker or no Docker for now, but will take a peek at vagrant-docker-compose. It could be that Drupal VM transitions to that for some time, then fully jumps the shark if I'm ever more comfortable sticking my flag firmly in the Docker camp than in the Vagrant-for-local-environment camp.

[geerlingguy]

I've added a basic image that will likely be a wee bit massive, since it's currently installing Drupal 8 via the Composer drupal-project, and includes the database and everything: https://hub.docker.com/r/geerlingguy/drupal-vm/

I am considering maybe maintaining two or three Dockerfiles—one with D8 installed and ready to go (for core hacking, patching, or demos, like Simplytest.me, and one without a Drupal site included, that's intended to be used for project work.

Finally, maybe a third that has all the installed_extras? Or maybe a more common set of them?

Some of my roles don't play as nicely as others in a container, though... so that's my hesitation in going too far too fast!

[geerlingguy]

The builds were failing during the Varnish handlers, on the systemd reload task:

RUNNING HANDLER [geerlingguy.varnish : reload systemd] *************************

�fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["systemctl", "daemon-reload"], "delta": "0:00:00.213274", "end": "2017-05-13 05:25:28.171839", "failed": true, "rc": 1, "start": "2017-05-13 05:25:27.958565", "stderr": "Failed to get D-Bus connection: Unknown error -1", "stderr_lines": ["Failed to get D-Bus connection: Unknown error -1"], "stdout": "", "stdout_lines": []}�

Ah, the infamous Failed to get D-Bus connection: Unknown error -1 :P

See the failed build log: https://hub.docker.com/r/geerlingguy/drupal-vm/builds/bquhnptr39lkim7yr7uutdy/

Also, working out some size comparisons locally to see if using Debian 8 for the Docker image is the best option (noting, of course, that until I get things more optimized, the bulk of the weight is in the top layer... meaning updates to the image will require people download ~800+ MB of filesystem 😕:

Distro	Drupal preinstalled?	Size (expanded)	Size (on Docker Hub)
Debian 8	yes	1.28 GB	324 MB
Debian 8	no	863 MB	267 MB
Ubuntu 16.04	yes	1.47 GB	TODO
CentOS 7	yes	TODO	TODO

[geerlingguy]

Yay, there's a :latest image built on Docker Hub: https://hub.docker.com/r/geerlingguy/drupal-vm/

Going to test with my docker-compose.yml file to see what I can do quickly/easily. Then I may work on a second Dockerfile with no Drupal installation (empty database and empty codebase directories).

[geerlingguy]

Hmm... I can't put the Dockerfile in a subdirectory and access the parent directory for resources during the build. See: http://stackoverflow.com/q/24537340/100134

[geerlingguy]

After playing around with builds for a couple hours, I've determined it's close to impossible to get Docker Hub to run two automated builds on the same project—one with one Dockerfile, another with another Dockerfile in a subdirectory (since it switches build contexts and you can't do anything in parent directories).

It might be possible to get something set up through the Docker Cloud interface (instead of Hub), but that's a bit murky, as until about 30 minutes ago I had never even heard of Docker Cloud... and apparently there's some third registry out there like 'Docker Enterprise' or something that is even more involved.

Why they couldn't just have one central thing (like Hashicorp's Atlas) is beyond me. Seems like a lot of wasted/duplicate effort. Of course, what do I know... they also changed the entire underlying project's name to moby after 2,000+ issues in the docker namespace 😭

[geerlingguy]

The image is up, and working, moving concrete action items over to #1366

[derimagia]

Don't know about Hub to know if this helps, in regards to build contexts, docker build now (recently) accepts -f - which lets docker participate in pipelines and pipe from stdin (See moby/moby#31236 for some more info on it)

[geerlingguy]

@derimagia - Yeah, unfortunately Hub/Cloud doesn't work the same way yet :(

I was testing with Dockerfiles in various locations and trying to get the Hub and/or Cloud configs to work with a Dockerfile outside of the repository root... and it works, but it then requires all the other config to be in subdirectories of the directory (context) where the Dockerfile resides.

[thom8]

@geerlingguy you could try using Travis / Circle for the image builds then push them to Docker hub.

That way you can move the provisioning scripts into the build context before running it.

Would recommend Circle as you can set it all up manually via SSH into the build container then just script it. I remember you mentioned Travis allows this too? couldn't find anything in the docs.

[geerlingguy]

@thom8 - Maybe so... for now I want to keep it pretty low key until I find exactly how I want to make things tweakable. For selfish reasons, I'm working on making the base image something I can easily plug into all my personal sites I work on maybe a day or two every six months. Since it takes 10-20 minutes to bring up Drupal VM, I tend to batch my work on personal sites.

If I can get the Docker base image to work nicely with all of them and come up in a second or two (plus time to drush sql-sync the database from prod), that's a huge productivity boon and might allow me to actually maintain personal Drupal sites instead of only applying module and core updates :)

[thom8]

RE: #1356 (comment)

docker compose support has been added to Vagrant 1.9.5

Could be a good bridge to a 100% docker stack.

[geerlingguy]

Link to the PR in question:

• https://github.com/mitchellh/vagrant/pull/8576/files
• Docs: https://www.vagrantup.com/docs/docker/configuration.html

It definitely looks interesting!