Real examples of the action usage

[machulav]

Hey, welcome!

In this discussion, I will collect all the real examples of the ec2-github-runner action usage I'm aware of.

Feel free to add your story as well 🙌

If you have any questions, please create a new discussion or issue.

[machulav]

@lielran shared in the comment:

here is the final result, we are already using it in our CD pipeline.

You're the first one I know who started using the action!
Thank you for sharing and for all the support! 🙌

[lielran]

@lielran shared in the comment:

here is the final result, we are already using it in our CD pipeline.

You're the first one I know who started using the action!
Thank you for sharing and for all the support! 🙌

I'll add to that.
Run Flyway in order to maintain the DB schema as part of the CD pipeline, just before the deployment it self.

[machulav]

And even more from this comment 🙂

we are using flyway to align our DB schema (rds,pg). the db is running in isolated subnet without(direct) internet access. the easist thing to do is run the migration from within the vpc on a public subnet(application subnet).

[snussik]

As I found this action extremely useful and glad, that @machulav created it before I have to )) - here is my first test workflow on it:

[machulav]

@snussik, many thanks for the feedback and information about your workflow!

May I ask you why do you need to run your workflow on the self-hosted EC2 runner? Why don't the GitHub-hosted runners work for you in this case?

[snussik]

Sure! The issue is in nodejs problems with memory and cpu usage. On quite big datasets github runner specs are not enough.

[machulav]

An example of usage from @winterpixelgames 🙌

https://twitter.com/winterpixelco/status/1371869394318491648

[machulav]

@osquery uses the action in one of their automation: https://github.com/osquery/osquery/blob/master/.github/workflows/build_aarch64.yml 😍

Example:

[M-JobPixel]

At Jobpixel we've got a test workflow running:

Thanks for the plugin

[machulav]

An interesting example from @stas00 and BigScience Workshop team, where they try to create an instance in 3 different AZs as the capacity of such a powerful instance is limited and may not be available in one AZ in the required time.

https://github.com/bigscience-workshop/Megatron-DeepSpeed/blob/main/.github/workflows/main.yml

[stas00]

Might be a good idea to link to a specific SHA of the current config, as this is all new and may change.
i.e.: https://github.com/bigscience-workshop/Megatron-DeepSpeed/blob/9e14c02a1dd22e4d36e2ee9a33e44d33774b8de7/.github/workflows/main.yml

plus all credits for this solution go to @jpalomaki - I just adapted his proposal.

[metola]

Hi @machulav , thanks for plugin

[jappyjan]

at nifty.store (by elbstack GmbH) we use self-hosted runners to be able to use more powerfull runners so that our build time decreases significantly.

[pradeeptalwar]

Hello Volodymyr ,

Will you be ok for me to fork off this repo to see if we can include MarketType=spot ?

Kind regards,
Pradeep

[Dmitry1987]

[ADMIN] permission to fork was granted ✔️

Enjoy! 😁

[nilesingalls]

works very nicely, thank you!

[elgehelge]

Anybody who have managed to put these jobs into a reusable component that can be used many times throughout a workflow?
Eg. (job that uses a small instance) -> (job that uses a GPU) -> (job that uses a large CPU instance)
Would love to see some examples workflows of this kind.

[Dmitry1987]

that's easy imho, you just do this:

create a folder near your main workflow and name it lets say "runners", add file "runners/action.yaml" with

# Sub action for the ec2 spawns, parameterized, to avoid duplicating the code for different types of ec2 instances
name: make a new runner
description: new ec2 runner of any type
inputs:
  # parameter provided by main pipeline which calls this action
  server-type: 
    description: server of some kind, like gpu or windows
    required: true
    default: windows

runs:
  using: "composite"
  steps:
      - name: Start runner with GPU support
        id: start-ec2-runner-gpu-flavor
        uses: machulav/ec2-github-runner@v2
        if: ${{ inputs.server-type == 'gpu' }}
        with:
          mode: start
          ec2-image-id: ami-123123123123
          ec2-instance-type: the gpu type you need
          subnet-id:.....
          security-group-id: .....

      - name: Start runner with Windows support
        id: start-ec2-runner-windows-flavor
        uses: machulav/ec2-github-runner@v2
        if: ${{ inputs.server-type == 'windows' }}
        with:
          mode: start
          ec2-image-id: ami-678789789789
          ec2-instance-type: the windows type
          subnet-id:.....
          security-group-id: .....

then in the main pipeline you can call it like this

    - id: start a runner
      uses: ./.github/workflows/runners
      with:
        server-type: gpu

this avoids some duplication and you can manage all runner definitions in one place let's say and even import them as submodules to share with many repos and teams 🔥

[harishpothugantisolibri]

Hello everyone, Any one tried github self-hosted ec2 runners with dependabot. Is this action support that

[kemalgoekhan]

once it creates ec2, it does not assign public ip address. how can I enable Auto-assign public IP option? @machulav do you have any idea?

[Dmitry1987]

you can do in your subnet level, so that anything in the subnet will get public ip. or for example to make a new subnet and then enable it in there. as a workaround. but yeah those options should be available. my guess by the number of forks of the repo, is that many simply fork it as a base and then add all new required options afterwards, including maybe some internal ci related goodies 😅 i'm so lazy to do that so i'm using this one, but i desperately need the spot instances support and the windows mode, so these two PRs which were not merged yet, will probably be ingredients in my forked salad made of pieces 😁, and on top of it i need to use "cpu_options: threads_per_core=1" which doesn't exist in PRs so i'll just hardcode it to my version. that's the way to go with all the more specific flags and features that not many people need (for example i might be the only user in the thousands, who needs that option, so why overload this 'general public' action by such small specifics. the module will consist of all aws sdk options then, if we do that 🤷‍♂️ . btw the maintainer is looking for new maintainers if anyone interested to join efforts 💪 )

[Dmitry1987]

oh btw i'm posting all this simply for people who land to the thread from google etc' ✅ i know it's 2 years late.

[akhilgautam30]

Is there any plan to extend this extension for windows lastest images as well?

[Dmitry1987]

Is there any plan to extend this extension for windows lastest images as well?

there's a PR here #104 and it was released here https://github.com/crunchy234/ec2-github-runner/releases/tag/windows-support-7