Customisation script and files in /home/runner

[eliandoran]

Hi,

I'm trying to set up a cluster using GitHub Runner Controller (gha-runner-scale-set), using Kubernetes mode.

By default GitHub Runner Controller uses the official ghcr.io/actions/actions-runner:latest image, but I find it quite lacking (not even git is available on it), so I tried this image instead.

It's working just fine, except when I try to run a GitHub Docker action where it will fail with:

##[debug]System.Exception: Executing the custom container implementation failed. Please contact your self hosted runner administrator.
##[debug] ---> System.IO.FileNotFoundException: File not found at '/home/runner/k8s/index.js'. Set ACTIONS_RUNNER_CONTAINER_HOOKS to the path of an existing file.
##[debug]   at GitHub.Runner.Worker.Container.ContainerHooks.ContainerHookManager.ValidateHookExecutable()
##[debug]   at GitHub.Runner.Worker.Container.ContainerHooks.ContainerHookManager.ExecuteHookScript[T](IExecutionContext context, HookInput input, ActionRunStage stage, String prependPath)
##[debug]   --- End of inner exception stack trace ---
##[debug]   at GitHub.Runner.Worker.Container.ContainerHooks.ContainerHookManager.ExecuteHookScript[T](IExecutionContext context, HookInput input, ActionRunStage stage, String prependPath)
##[debug]   at GitHub.Runner.Worker.Container.ContainerHooks.ContainerHookManager.RunContainerStepAsync(IExecutionContext context, ContainerInfo container, String dockerFile)
##[debug]   at GitHub.Runner.Worker.Handlers.ContainerActionHandler.RunAsync(ActionRunStage stage)
##[debug]   at GitHub.Runner.Worker.ActionRunner.RunAsync()
##[debug]   at GitHub.Runner.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)

I've tracked this error a bit and I found in the GitHub Actions docs there is a description of this:

The custom script must be located on the runner, but should not be stored in the self-hosted runner application directory (that is, the directory into which you downloaded and unpacked the runner software). The scripts are executed in the security context of the service account that's running the runner service.

Note: The triggered script is processed synchronously, so it will block job execution while running.

The script is automatically executed when the runner has the following environment variable containing an absolute path to the script:

ACTIONS_RUNNER_CONTAINER_HOOKS: The script defined in this environment variable is triggered when a job has been assigned to a runner, but before the job starts running.

I managed to look at the official image using docker run -it --entrypoint="/bin/bash" ghcr.io/actions/actions-runner:latest and I found out that /home/runner is populated with a few files, including the /home/runner/k8s/index.js that I presumably need in order to run containers in Kubernetes mode.

What is your opinion, should this k8s container hook be part of this Docker image? What about the other files that are available in /home/runner?

[myoung34]

If people find that this image would suit the actions scaler then that's fine but it's not built that way

I'm happy to review a PR to see a gut check against it, but I dont currently have any opinion other than my previously stated: I dont wish to directly tie this to any orchestrator (k8s, nomad, etc)

I don't mind adding in the functionality but don't with to start that work myself. I'd think that a new set of workflow scripts that build off the current artifacts and add a /k8s or similar might be a decent middle ground

[eliandoran]

@myoung34 ,

Thank you for your prompt response and I think it's a fair take.

I'll have a look to see what I can do and if I can find more information about this script.

[VincentVerweij]

Hi @eliandoran, rougly 4 months have passed. Did you get this beautiful image to work with the actions runner controller?

I tried to use it myself where I used this as my base image. In my Dockerfile I did add the runner-container-hooks binary.
I used the installation that is used in the official image:
https://github.com/actions/runner/blob/main/images%2FDockerfile#L20

Then I did set the group to docker with chgrp -R, as is the case with the official image too.
And then in the values.yaml file of the runner-scale-set, I point to the myoung image that I built upon, and there I set all the environment variables that are required for myoung.
But I also set the ACTIONS_RUNNER_CONTAINER_HOOKS which in my case points to /actions-runner/k8s/index.js
This is also done in the original chart's values file.

And as you mentioned, I too use containerMode.type: "kubernetes".
And my error is not the same as yours, it is equal to the one mentioned in this issue #349.
As said in that issue, that's related with Docker stuff, but with the official image it does work (it spins up a separate container suffixed with workflow).
Also, the image works for all none container jobs.

I also tried setting the securityContext.fsgroup to be equal to 121, the one defined in here https://github.com/myoung34/docker-github-actions-runner/blob/master/Dockerfile.base#L90.

And then I also tried adding volumes for the working directory but without success 😢

So, still curious how you solved it.

Maybe we can set an env var ACTIONS_RUNNER_CONTROLLER so that these are installed only when people need it and for backwards compatibility.

[eliandoran]

@VincentVerweij , on my side I ended up trying a fork of the official image which is compatible with gha-runner-scale-set: eliandoran/runner@cba275f

It worked to bypass the issue I was having, but I ended up abandoning it since it did not work for my use case:

Error: Error: Building container actions is not currently supported
Error: Process completed with exit code 1.
Error: Executing the custom container implementation failed. Please contact your self hosted runner administrator.