How to get the $uid / $gid variables in custom docker compose services?

[deviantintegral]

We've been extending a project with additional ddev services, most recently with ZAP. In this case, we want to bind-mount in a directory to save reports generated by the scan. By default, this image (and others I've worked with) have UIDs set to something like 1000.

I thought we could fix this with:

  zap-bare:
    container_name: ddev-${DDEV_SITENAME}-zap-bare

    build:
      context: zap-bare-build
...

And then:

FROM owasp/zap2docker-bare:2.12.0

USER root
RUN apk add -u shadow
RUN usermod -u $uid zap
RUN groupmod -g $gid zap
RUN find / -uid 1000 -exec chown -h $uid {} +
RUN find / -gid 1000 -exec chgrp -h $gid {} +
USER zap

However, looking at .ddev/.ddev-docker-compose-full.yaml, only the web and database containers have uid / gid args passed in:

    db:
        build:
            args:
                BASE_IMAGE: drud/ddev-dbserver-mariadb-10.6:v1.21.5
                gid: "20"
                uid: "501"
                username: andrew

Since these values are per-host, is there any way to get ddev to include these three values in all compose services?

[rfay]

DDEV doesn't add extra layers to your custom services. You can add them in your docker-compose.*.yaml though, just (as you saw) by adding a build stage as you saw. And I know you'd rather have this done for you, because you can't put all of your team's info in there.

I'm not sure how I know how to do what you need, because DDEV would likely break lots of services by inserting extra layers.

At this point, DDEV doesn't know that your project's Dockerfile exists. It's not even in the scope of DDEV's world. So I guess you have a build stage for zap-bare... but DDEV doesn't know that, and couldn't know it without inspecting your docker-compose.zap.yaml.

This sounds like a hard problem.

[deviantintegral]

This sounds like a hard problem.

That makes me feel a lot better that I hacked in user: root!

We also ran into this with Playwright, until we started stacking it into the web service.

I'm thinking this must not be that common of a challenge, and we're really extending ddev past typical uses. Perhaps it's worth waiting to see if others post before putting any significant thought or dev effort into this.

[Biont]

@rfay Are you sure you can't just export the username, uid & gid as environment variables before handing over to docker-compose? That would make them available for external services. Being able to use the same filesystem permissions as the host user is essential for any addon that needs filesystem access.

Is there a technical reason this information could not just be exported together with the other environment vars? Do you see any other solution to this problem?

Ironically, I am just struck by this as I have tried to pull of a Playwright addon without stacking it into the web service like @deviantintegral (which I now understand might be the only way to get around the permissions issue).

Another - more complex- solution could be to add that extra layer (I assume you are referring to the Go templating like here?) right into the API of DDEV addons' install.yaml:

If there was support for pre_start_actions and if these had access to the {{ .Username }} stuff when doing templating, we could dynamically place the docker-compose dropin.
This would be a more controlled approach, but I guess the whole install.yml is completely gone after an addon is installed. So this is probably impossible right now. Still, I thought I could throw it out there.

[rfay]

You're right, DDEV already does provide DDEV_UID and DDEV_GID in

ddev/pkg/ddevapp/ddevapp.go

Lines 1882 to 1975 in f4acf1e

	func (app *DdevApp) DockerEnv() {
	uidStr, gidStr, _ := util.GetContainerUIDGid()
	// Warn about running as root if we're not on Windows.
	if uidStr == "0" || gidStr == "0" {
	util.Warning("Warning: containers will run as root. This could be a security risk on Linux.")
	}
	// For gitpod, codespaces
	// * provide default host-side port bindings, assuming only one project running,
	// as is usual on gitpod, but if more than one project, can override with normal
	// config.yaml settings.
	if nodeps.IsGitpod() || nodeps.IsCodespaces() {
	if app.HostWebserverPort == "" {
	app.HostWebserverPort = "8080"
	}
	if app.HostHTTPSPort == "" {
	app.HostHTTPSPort = "8443"
	}
	if app.HostDBPort == "" {
	app.HostDBPort = "3306"
	}
	if app.HostMailhogPort == "" {
	app.HostMailhogPort = "8027"
	}
	if app.HostPHPMyAdminPort == "" {
	app.HostPHPMyAdminPort = "8036"
	}
	app.BindAllInterfaces = true
	}
	isWSL2 := "false"
	if dockerutil.IsWSL2() {
	isWSL2 = "true"
	}
	// DDEV_HOST_DB_PORT is actually used for 2 things.
	// 1. To specify via base docker-compose file the value of host_db_port config. And it's expected to be empty
	// there if the host_db_port is empty.
	// 2. To tell custom commands the db port. And it's expected always to be populated for them.
	dbPort, err := app.GetPublishedPort("db")
	dbPortStr := strconv.Itoa(dbPort)
	if dbPortStr == "-1" || err != nil {
	dbPortStr = ""
	}
	if app.HostDBPort != "" {
	dbPortStr = app.HostDBPort
	}
	envVars := map[string]string{
	// The compose project name can no longer contain dots; must be lower-case
	"COMPOSE_PROJECT_NAME": strings.ToLower("ddev-" + strings.Replace(app.Name, `.`, "", -1)),
	"COMPOSE_CONVERT_WINDOWS_PATHS": "true",
	"COMPOSER_EXIT_ON_PATCH_FAILURE": "1",
	"DDEV_SITENAME": app.Name,
	"DDEV_TLD": app.ProjectTLD,
	"DDEV_DBIMAGE": app.GetDBImage(),
	"DDEV_DBAIMAGE": versionconstants.GetDBAImage(),
	"DDEV_PROJECT": app.Name,
	"DDEV_WEBIMAGE": app.WebImage,
	"DDEV_APPROOT": app.AppRoot,
	"DDEV_COMPOSER_ROOT": app.GetComposerRoot(true, false),
	"DDEV_DATABASE": app.Database.Type + ":" + app.Database.Version,
	"DDEV_FILES_DIR": app.GetContainerUploadDirFullPath(),
	"DDEV_HOST_DB_PORT": dbPortStr,
	"DDEV_HOST_MAILHOG_PORT": app.HostMailhogPort,
	"DDEV_HOST_WEBSERVER_PORT": app.HostWebserverPort,
	"DDEV_HOST_HTTPS_PORT": app.HostHTTPSPort,
	"DDEV_PHPMYADMIN_PORT": app.PHPMyAdminPort,
	"DDEV_PHPMYADMIN_HTTPS_PORT": app.PHPMyAdminHTTPSPort,
	"DDEV_MAILHOG_PORT": app.MailhogPort,
	"DDEV_MAILHOG_HTTPS_PORT": app.MailhogHTTPSPort,
	"DDEV_DOCROOT": app.Docroot,
	"DDEV_HOSTNAME": app.HostName(),
	"DDEV_UID": uidStr,
	"DDEV_GID": gidStr,
	"DDEV_MUTAGEN_ENABLED": strconv.FormatBool(app.IsMutagenEnabled()),
	"DDEV_PHP_VERSION": app.PHPVersion,
	"DDEV_WEBSERVER_TYPE": app.WebserverType,
	"DDEV_PROJECT_TYPE": app.Type,
	"DDEV_ROUTER_HTTP_PORT": app.RouterHTTPPort,
	"DDEV_ROUTER_HTTPS_PORT": app.RouterHTTPSPort,
	"DDEV_XDEBUG_ENABLED": strconv.FormatBool(app.XdebugEnabled),
	"DDEV_PRIMARY_URL": app.GetPrimaryURL(),
	"DDEV_VERSION": versionconstants.DdevVersion,
	"DOCKER_SCAN_SUGGEST": "false",
	"GOOS": runtime.GOOS,
	"GOARCH": runtime.GOARCH,
	"IS_DDEV_PROJECT": "true",
	"IS_GITPOD": strconv.FormatBool(nodeps.IsGitpod()),
	"IS_CODESPACES": strconv.FormatBool(nodeps.IsCodespaces()),
	"IS_WSL2": isWSL2,
	}

- it does not grab the username, but could easily (3rd return value from util.GetContainerUIDGid() ).

So if you can get by with DDEV_UID and DDEV_GID, that's great; a PR to add DDEV_USERNAME would be easy. However, you still have the problem of actually adding the user into your Docker container, which isn't completely easy. DDEV works hard to get your username with correct uid/gid added using a layer in the container, see

ddev/pkg/ddevapp/config.go

Lines 1002 to 1024 in 0dd6015

	func WriteBuildDockerfile(fullpath string, userDockerfilePath string, extraPackages []string, composerVersion string, extraContent string) error {
	// Start with user-built dockerfile if there is one.
	err := os.MkdirAll(filepath.Dir(fullpath), 0755)
	if err != nil {
	return err
	}
	// Normal starting content is just the arg and base image
	contents := `
	#ddev-generated - Do not modify this file; your modifications will be overwritten.
	### DDEV-injected base Dockerfile contents
	ARG BASE_IMAGE
	FROM $BASE_IMAGE
	`
	contents = contents + `
	ARG username
	ARG uid
	ARG gid
	ARG DDEV_PHP_VERSION
	RUN (groupadd --gid $gid "$username" || groupadd "$username" || true) && (useradd -l -m -s "/bin/bash" --gid "$username" --comment '' --uid $uid "$username" || useradd -l -m -s "/bin/bash" --gid "$username" --comment '' "$username" || useradd -l -m -s "/bin/bash" --gid "$gid" --comment '' "$username" || useradd -l -m -s "/bin/bash" --comment '' $username )
	`

- you'd have to do something like that, or consider using a built-in user and altering its userid.

[Biont]

@rfay You're a lifesaver! Thank. I think I have even seen $DDEV_UID/$DDEV_GID in the wild before, but wasn't able to make use of them. It seems they are only exported when docker-compose starts, so you can not leverage them as your_service.build.args, which -oh boy- would make life a LOT easier. I have not been able to spot this in the code, but is this a two-step process where the vars are only avalable during container run time and not at build time? It would be great for addons if you could use them as build args!

However, I was able to craft something with an entrypoint.sh and following your suggestion to alter the in-container uid/gid.
It's a few more roundtrips that I'd have liked but there is a working result now. Thank you again.