Using GitHub Actions instead of Drone

[arschles]

We've discussed moving from drone to GH actions in the past, and we've also had some PRs in to move wholesale to GH actions from drone. Here are issues and PRs that encompass most of the work:

• Add a GH action to publish a tweet to the @gomodsio Twitter account when we publish a new GitHub release #1460 primarily discusses ideas for GH actions we could install to make some lives easier. This issue isn't about replacing drone
• Is it worth using testcontainers to set up our unit, integration or e2e tests? #1500 (issue) and \#1500 - prototype for using testcontainers-go for dependencies #1544 (corresponding PR) is for unrelated work, but ended up requiring us to move to GH actions because of drone.io limitations
• Adding YAML for a GH Action (try 2) #1463 is a try to move to add a GH action to do much of what drone already does. I opened this, my goal was to start a discussion. Highlights of the discussion: Adding YAML for a GH Action (try 2) #1463 (comment) is where @marwan-at-work asks what value actions provides beyond things we already use, and the discussion continues from there.

Personally, I think we should move to GH actions wholesale for two major reasons:

• We get a VM for each job we run (see here for details). This allows us to isolate jobs from each other, and is closer to the model we were used to when we were on Travis CI
• You can run background containers by running the equivalent of docker run -d ..., instead of using services in drone or circle CI
• We can expand to windows VMs in the future, if we ever decide to get off of appveyor. I'm not advocating for that, only illustrating flexibility

Since we've discussed actions v. drone/others in a few places, I'd like to centralize that discussion here.

cc/ @marwan-at-work @robjloranger @twexler @chriscoffee

[marwan-at-work]

@arschles

This allows us to isolate jobs from each other

Can you expand more on this? Drone as far as I know treats each step in the pipeline in an isolated container, so not sure GitHub Actions has an added value there.

You can run background containers by running the equivalent of docker run -d ..., instead of using services in drone or circle CI

Is that a pro, a con, or just an alternative way to do the same thing?

My 2cents is that, docker run -d looks a bit hacky to manage in comparison to writing a declarative service container that automatically runs in the background, no?

We can expand to windows VMs in the future

I think consolidating our CIs into 1 that runs for both Linux and Windows sounds like a really good reason to be honest. So I'm for that. However, I don't think this is enough just yet given that GitHub Actions is still difficult to work with in my opinion.

Here are my Pros and Cons

DroneCI Pros

1. It's fully open source and written in Go, supporting the community always feels good :)
2. It's really easy to work with, you just have one yaml file, and everything is a container image. No VMS, no docker-sock.
3. Its API as well as its runtime has been stable for a while and no one has opened an issue to say that it's not working for them.
4. No volumes to manage!! Each step is a container that automatically gets a volume and CWD into the Codebase, so there's no need to juggle things around. In GitHub actions, you have to manually link things and look for the environment variable ("GITHUB_ACTION_DIRECTORY" or something) where your code is living.
5. No tooling set up required. With GitHub actions you have to install Go and all the tooling in a "pre-step" here: https://github.com/actions/setup-go -- this seems like a step backwards.
6. You can directly run Go scripts to do whatever you want (liveness probe, parsing, diffing etc). Here's an example: https://github.com/gomods/athens/blob/master/.drone.yml#L10:L12

DroneCI Cons

1. No windows support
2. Docs are ok, but not great.

GitHub Actions Pros:

1. Windows support out of the box, which would consolidate our CIs.
2. Since it's Github, it will probably be widely used and supported pretty soon.

Github Actions Cons:

See the DroneCI pros above 😄

[arschles]

(The GH discussions on https://github.com/gh-community/discussions-feedback/discussions has threading, I would love that here!)

This allows us to isolate jobs from each other

@marwan-at-work I should be more specific, one-VM-per-job would allow us to isolate at the hypervisor level. we would not have to worry about port collisions, etc... This could also be a con too

You can run background containers by running the equivalent of docker run -d ..., instead of using services in drone or circle CI

Yea, this is an alternate way of doing things. Pro vs. con is definitely up to opinion. For me, I like it because you can build and run any arbitrary image. For example, it would solve #1481 for us.

You can also run services if we want, which is more similar to what drone does.

given that GitHub Actions is still difficult to work with in my opinion.

Any specific things I can point to that are difficult to work with? I can go tell the actions team.

I'll comment on some of the drone pros/cons:

It's fully open source and written in Go, supporting the community always feels good :)

Yup, I'm with you here

It's really easy to work with, you just have one yaml file, and everything is a container image. No VMS, no docker-sock.

GH actions give you a single yaml file also. And in each yaml file you can have 1 or more jobs. You can decide to expand to multiple yaml files if you want, though. In GH actions, the default is running directly on a VM, but it's a one-liner to run inside a container. In that case, you're right on that you do have to mount the docker sock & volumes etc...

Its API as well as its runtime has been stable for a while and no one has opened an issue to say that it's not working for them.

I don't have platform uptime or API stability numbers for GitHub, but yes, I bet it's not great since the very first beta of actions. However, actions is becoming widely used. For example, the node js project is using actions for a few things. It would be really hard for GH to change the API now, and very damaging to the product if there was significant downtime.

No volumes to manage!! Each step is a container that automatically gets a volume and CWD into the Codebase, so there's no need to juggle things around. In GitHub actions, you have to manually link things and look for the environment variable ("GITHUB_ACTION_DIRECTORY" or something) where your code is living.

Good point and definitely true for containers. GH actions is certainly more oriented to a single VM per job, so a container per build step isn't "first class" as much. The go build step, for example, is done inside the VM, not the container.

No tooling set up required. With GitHub actions you have to install Go and all the tooling in a "pre-step" here: https://github.com/actions/setup-go -- this seems like a step backwards.

Definitely true, another good point. This opens up for more customizability, but also for more upfront setup cost.

You can directly run Go scripts to do whatever you want (liveness probe, parsing, diffing etc). Here's an example: https://github.com/gomods/athens/blob/master/.drone.yml#L10:L12

This is possible with actions too with the run entry. One difference is that you pass it a full shell script instead of a YAML list of commands.

I hope everything here was constructive! Obviously I'm pro-actions, mostly because everything is in one place, not even for a data-driven or feature-driven reason.

If you want to stick with drone, would you be up for running actions and drone side-by-side for now?

[marwan-at-work]

For me, I like it because you can build and run any arbitrary image

Doing this for #1481 would be a security leak because someone will be able to print out the storage credentials. So I'd keep that as is.

That said, building the current PR image would be great for when someone updates our Dockerfiles and pushes PR for it. It'd be good to look into if Drone can do that. Otherwise, that's a good plus for GitHub Actions.

If you want to stick with drone, would you be up for running actions and drone side-by-side for now?

Probably not because then we'd have like 5 CIs (appveyor, azure pipelines, CircleCI, DroneCI, and Github actions) lmao...contributors will be very confused and intimidated to make any CI changes.

I think this is a good discussion: so far I see 3 really good benefits of GitHub Actions:

1. Being able to get rid of appveyor/azurepipelines/circleci to consolidate windows as well as pushing to Helm?
2. Being able to build a dockerfile during PR review, but only if dockerfile changed which GA has that feature which is pretty awesome.
3. Being able to build/run dockerfile but only if we make a bot that only runs that step if a maintainer asks GitHub Actions to do so.

However, I still see a big overhead of complexity with GitHub Actions (such as setting up Go, setting up service containers, trying to figure out whether you're in a container or in a VM, voluming etc.)

I think what we can do one of two hacking sessions:

1. A hack session where we move everything to GitHub Actions and see how we feel about it, knowing that all of that work could be discarded.
2. A hack session where we just do the Docker-image build thingy that integrates with a bot that verifies you're a maintainer.

What do peeps think? I'd be down to pair programming over the weekend or a week night if anyone's interested.

[arschles]

I'm totally up for pair programming! We could continue on #1463 or from scratch. Up to you @marwan-at-work

[WillAbides]

This thread is now five months old, but I want to weigh in as an athens noob who struggled with getting CI working on my first PR. Seeing checks on my PR from drone, circleci, and appveyor is intimidating. Making my tests pass required making essentially the same change in multiple places and reading docs for the ones I'm not familiar with.

GitHub Actions deserves an entry in the pro column that it could reduce the number of build systems and make onboarding new developers easier.

Actions has also improved since this discussion was started. The uptime is getting closer to that of GitHub's front end, and features like manually starting and rerunning actions have made it less fiddly than it used to be.

Looking through .drone.yml and .circleci, it appears that both of them can be entirely replaced with GitHub actions. Probably appveyor too, but I'm less confident in the Windows side of things.

[DrPsychick]

With the recent changes and switch to GitHub actions (https://github.com/gomods/athens/tree/main/.github), are there any open concerns in this discussion?

Since the last reply is almost 3 years old, I'd close this discussion and open up a new one for specific concerns about CI (for example Windows builds).