database.sqlite Permission denied

[phoenix1184]

Hey

I trie to use docker rootless and docker compose.
With normal docker, everything works fine.

Since then I got error message when docker compose up:
2fauth | touch: /2fauth/database.sqlite: Permission denied

I've a folder $DOCKERDIR/APPDATA/2fauth. Inside is file database.sqlite (file), installed (file), storage (folder)

Here is my compose file:
2fauth:
image: 2fauth/2fauth
restart: always
networks:
- default
container_name: 2fauth
volumes:
- $DOCKERDIR/appdata/2fauth:/2fauth
- /etc/localtime:/etc/localtime:ro
ports:
- 8300:8000/tcp
environment:
- ASSET_URL=$fauth_URL
- TZ=$TZ
- APP_NAME=$fauth_APP_NAME
- APP_ENV=local
- APP_DEBUG=true
- SITE_OWNER=$fauth_SITE_OWNER
- APP_KEY=$fauth_APP_KEY
- APP_URL=$fauth_URL
- IS_DEMO_APP=false
- LOG_CHANNEL=daily
- LOG_LEVEL=debug
- DB_CONNECTION=sqlite
- DB_DATABASE="/srv/database/database.sqlite"
- CACHE_DRIVER=file
- SESSION_DRIVER=file
- MAIL_DRIVER=smtp
- MAIL_HOST=$fauth_MAIL_HOST
- MAIL_PORT=$fauth_MAIL_PORT
- MAIL_FROM=$fauth_SITE_OWNER
- MAIL_USERNAME=$fauth_SITE_OWNER
- MAIL_PASSWORD=$fauth_MAIL_PASSWORD
- MAIL_ENCRYPTION=tls
- MAIL_FROM_NAME=$fauth_MAIL_FROM_NAME
- MAIL_FROM_ADDRESS=$fauth_SITE_OWNER
- AUTHENTICATION_GUARD=web-guard
- AUTH_PROXY_HEADER_FOR_USER=null
- AUTH_PROXY_HEADER_FOR_EMAIL=null
- PROXY_LOGOUT_URL=null
# WebAuthn settings
- WEBAUTHN_NAME=$fauth_WEBAUTHN_NAME
- WEBAUTHN_ID=null
- WEBAUTHN_ICON=null
- WEBAUTHN_USER_VERIFICATION=preferred
- TRUSTED_PROXIES=$fauth_TRUSTED_PROXIES
- BROADCAST_DRIVER=log
- QUEUE_DRIVER=sync
- SESSION_LIFETIME=120
- REDIS_HOST=$fauth_REDIS_HOST
- REDIS_PASSWORD=$fauth_REDIS_PASSWORD
- REDIS_PORT=6379
- PUSHER_APP_ID=$fauth_PUSHER_APP_ID
- PUSHER_APP_KEY=$fauth_PUSHER_APP_KEY
- PUSHER_APP_SECRET=$fauth_PUSHER_APP_SECRET
- PUSHER_APP_CLUSTER=$fauth_PUSHER_APP_CLUSTER
- MIX_PUSHER_APP_KEY="${PUSHER_APP_KEY}"
- MIX_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}"
- MIX_ENV=local

Any tips why it is not working?

database.sqlite is permission rwx

[sahinakkaya]

UPDATE: Problem is solved after doing the following:

sudo rm -rf 2fauth
mkdir 2fauth
docker compose up

getting the same error. Here is my docker-compose.yml:

version: "3"
services:
  2fauth:
    image: 2fauth/2fauth
    container_name: 2fauth
    volumes:
      - ./2fauth:/2fauth
    ports:
      - 9090:8000/tcp
    environment:
      # You can change the name of the app
      - APP_NAME=2FAuth
      # You can leave this on "local". If you change it to production most console commands will ask for extra confirmation.
      # Never set it to "testing".
      - APP_ENV=local
      # Set to true if you want to see debug information in error screens.
      - APP_DEBUG=false
      # This should be your email address
      - [email protected]
      # The encryption key for  our database and sessions. Keep this very secure.
      # If you generate a new one all existing data must be considered LOST.
      # Change it to a string of exactly 32 chars or use command `php artisan key:generate` to generate it
      - APP_KEY=SomeRandomStringOf32CharsExactly
      # This variable must match your installation's external address but keep in mind that
      # it's only used on the command line as a fallback value.
      - APP_URL=https://2fauth.myserver.com
      # Turn this to true if you want your app to react like a demo.
      # The Demo mode reset the app content every hours and set a generic demo user.
      - IS_DEMO_APP=false
      # The log channel defines where your log entries go to.
      # 'daily' is the default logging mode giving you 5 daily rotated log files in /storage/logs/.
      # Several other options exist. You can use 'single' for one big fat error log (not recommended).
      # Also available are 'syslog', 'errorlog' and 'stdout' which will log to the system itself.
      - LOG_CHANNEL=daily
      # Log level. You can set this from least severe to most severe:
      # debug, info, notice, warning, error, critical, alert, emergency
      # If you set it to debug your logs will grow large, and fast. If you set it to emergency probably
      # nothing will get logged, ever.
      - LOG_LEVEL=notice
      # Database config (can only be sqlite)
      - DB_DATABASE="/srv/database/database.sqlite"
      # If you're looking for performance improvements, you could install memcached.
      - CACHE_DRIVER=file
      - SESSION_DRIVER=file
      # Mail settings
      # Refer your email provider documentation to configure your mail settings
      # Set a value for every available setting to avoid issue
      - MAIL_DRIVER=smtp
      - MAIL_HOST=mail.myserver.com
      - MAIL_PORT=587
      - [email protected]
      - MAIL_USERNAME=sahin
      - MAIL_PASSWORD=my_mail_password
      - MAIL_ENCRYPTION=tls
      - MAIL_FROM_NAME=sahin
      - [email protected]
      # Authentication settings
      # The default authentication guard
      # Supported:
      #   'web-guard' : The Laravel built-in auth system (default if nulled)
      #   'reverse-proxy-guard' : When 2FAuth is deployed behind a reverse-proxy that handle authentication
      # WARNING
      # When using 'reverse-proxy-guard' 2FAuth only look for the dedicated headers and skip all other built-in
      # authentication checks. That means your proxy is fully responsible of the authentication process, 2FAuth will
      # trust him as long as headers are presents.
      - AUTHENTICATION_GUARD=web-guard
      # Name of the HTTP headers sent by the reverse proxy that identifies the authenticated user at proxy level.
      # Check your proxy documentation to find out how these headers are named (i.e 'REMOTE_USER', 'REMOTE_EMAIL', etc...)
      # (only relevant when AUTHENTICATION_GUARD is set to 'reverse-proxy-guard')
      - AUTH_PROXY_HEADER_FOR_USER=null
      - AUTH_PROXY_HEADER_FOR_EMAIL=null
      # Custom logout URL to open when using an auth proxy.
      - PROXY_LOGOUT_URL=null
      # WebAuthn settings
      # Relying Party name, aka the name of the application. If null, defaults to APP_NAME
      - WEBAUTHN_NAME=2FAuth
      # Relying Party ID. If null, the device will fill it internally.
      # See https://webauthn-doc.spomky-labs.com/pre-requisites/the-relying-party#how-to-determine-the-relying-party-id
      - WEBAUTHN_ID=null
      # Optional image data in BASE64 (128 bytes maximum) or an image url
      # See https://webauthn-doc.spomky-labs.com/pre-requisites/the-relying-party#relying-party-icon
      - WEBAUTHN_ICON=null
      # Use this setting to control how user verification behave during the
      # WebAuthn authentication flow.
      #
      # Most authenticators and smartphones will ask the user to actively verify
      # themselves for log in. For example, through a touch plus pin code,
      # password entry, or biometric recognition (e.g., presenting a fingerprint).
      # The intent is to distinguish one user from any other.
      #
      # Supported:
      #   'required': Will ALWAYS ask for user verification
      #   'preferred' (default) : Will ask for user verification IF POSSIBLE
      #   'discouraged' : Will NOT ask for user verification (for example, to minimize disruption to the user interaction flow)
      - WEBAUTHN_USER_VERIFICATION=preferred
      # Use this setting to declare trusted proxied.
      # Supported:
      #   '*': to trust any proxy
      #   A comma separated IP list: The list of proxies IP to trust
      - TRUSTED_PROXIES=null
      # Leave the following configuration vars as is.
      # Unless you like to tinker and know what you're doing.
      - BROADCAST_DRIVER=log
      - QUEUE_DRIVER=sync
      - SESSION_LIFETIME=120
      - REDIS_HOST=127.0.0.1
      - REDIS_PASSWORD=null
      - REDIS_PORT=6379
      - PUSHER_APP_ID=
      - PUSHER_APP_KEY=
      - PUSHER_APP_SECRET=
      - PUSHER_APP_CLUSTER=mt1
      - MIX_PUSHER_APP_KEY="${PUSHER_APP_KEY}"
      - MIX_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}"
      - MIX_ENV=local

[VaultSoldier]

I have been experiencing the same problem. My solution was sudo chown "$USER":"$USER" 2fauth.

[yannduran]

I'm using Portainer & I've just run into this problem as well. Who should be the owner of the folder that we map to? I have a separate user called docker which normally owns all of my appdata folders. I normally just include

PUID=1028 # user: docker
PGID=100  # group: users

in my docker compose file's volume section, but I've just realised that there was no mention of those two environment variables.

What user/group should own the folder that the volume maps to?

@phoenix1184 how did you end up with database.sqlite (file), installed (file), storage (folder) in your 2fauth folder? Did they get created when you were somehow able to successfully start the container? Every time I try to start the container I get the same error message that you mentioned.

If I manually create the file/folders that are indicated in the error messages, I still end up with a "permission denied" for the log file it's trying to create.

Running version 3.8 commit a1e0d4c built on 2023-07-07T12:52:44Z
supervisord version: v0.6.8
PHP 8.1.20 (fpm-fcgi) (built: Jun 14 2023 20:21:16)
nginx version: nginx/1.22.1
In StreamHandler.php line 146:
                                                                               
  The stream or file "/srv/storage/logs/laravel-2023-08-01.log" could not be   
  opened in append mode: Failed to open stream: Permission denied              
  The exception occurred while attempting to log: The stream or file "/srv/st  
  orage/logs/laravel-2023-08-01.log" could not be opened in append mode: Fail  
  ed to open stream: Permission denied                                         
  The exception occurred while attempting to log: Please provide a valid cach  
  e path.                                                                      
  Context: {"exception":{}}                                                    
  Context: {"exception":{}}                                                    

I wonder if this could be related to issue 192?

[yllekz]

Running into similar problems after making some config changes on my server. Are there environment variables for PUID and PGID? There really should be.

I was able to work around it by doing a brute force chmod 700 and chown 1000 with the -R switch on my 2fauth appdata folder to fix it, just as a side note.

[Bubka]

Running into similar problems after making some config changes on my server. Are there environment variables for PUID and PGID? There really should be.

No there is not. What would be the purpose ? Getting their values at run time? Setting them?

[yllekz]

The idea is such that it is not running as the godmode/root user, as the openmediavault guide suggests. It's a security best practice.

[Bubka]

The image runs as 1000:1000. But without env vars, this is hard coded in the Dockerfile.

[yannduran]

That should be able to be overidden by env variables. I'm installing this on a Synology NAS & there is no user or group whose id is 1000. Mine are 1028:100. I don't want to have to use chown to force the folder be be owned by a user/group that doesn't exist.

Not all systems have 1000:1000. Would it be that difficult to allow them to be set by PIUD & GUID env variables? Unless you have 1000:1000 hard coded throughout your docker image it shouldn't cause any problems for your code.

[Bubka]

Please have a look at the Dockerfile. All commands are run as 1000:1000. Is this what you called "Unless you have 1000:1000 hard coded throughout your docker image..."?

I'm not an advanced Docker user, the Docker stuff for 2FAuth is not my contribution, so I need to understand how it works.

To me, using specific uid:gid in the Dockerfile only impacts the build. It gives an image which will run with these ids. The only way to change something at runtime is by applying some changes through the entrypoint file. Is that correct?

[yannduran]

I had a look at the docker file in the link you provided. I'm a developer with almost 40 years of software development experience, but my docker experience is limited to consuming docker images, I've never put one together myself. I've never needed to.

First off yes, sort of, that's what I meant for the 1000:1000, but I also agree with your statement that being in the dockerfile they should only be applicable to the build process, not being relevant at runtime. The problem with doing it like that in the dockerfile is that it assumes that 1000:1000 is going to match any volumes that we add to persist data between restarts etc (like $DOCKERDIR/appdata/2fauth:/2fauth). I don't think doing it that way's a recommended/best practice, that's what environment variables are there for, to supply values that match the consumer's permissions etc. TZ, for timezone, is a very common environment variable, so that log files can be formatted for the timezone of the user at runtime, instead of all of the dates being some arbitrary format that doesn't correspond to values that make sense to the user. Are the dates in the log file GMT? US dates? UTC dates? Are the times Daylight Savings or Standard times?

1000:1000 is pretty common for Open Media Vault users (or there's another OS) and I see it mentioned in youtube tutorials, so for those consumers of your image it would "just work". Then there are other systems, like Synology, where those user/group ids just don't exist, and that's why some people are having the permissions problem. The permissions for commands to use or for mapped volumes should be set/overridden at runtime. I don't think I even have the option of creating a user/group with an id of 1000. As users are added you just end up with the next number after the last one AFAIK.

From what I understand, many, if not most, docker images have some sort of user/group id env variable (I've seen PUID/PGID, PID/GID, USER_ID/GROUP_ID) through which the application can use those values so they match the permissions of the user running it. It doesn't matter what they're called, we just have to use whatever the creator of the image has decided to make available.

I didn't manage to find a quick example to show you how those env variables are then used for the permissions at runtime (as you rightly assumed) but what then happens is the application can run code that uses the values that are provided to it by the consumer of the image instead of assuming anything, like the 1000:1000 set in the dockerfile. So whoever wrote the application code might be the person who would know what to do to override 1000:1000 at runtime. I find it a little unusual to have the puid/guid variables declared multiple times in the dockerfile. Usually you'd declare something like that at the beginning of the file. But there's nothing "wrong" about that per se, it's just unnecessary extra lines. It could be a style thing, or an experience thing, but it just seemed odd to me.

I don't think I've been terribly helpful. But having said that I just did a quick search online because I wanted to understand it a bit more myself (even though I can't imagine a scenario where I'd need to create a docker image myself, but never say never). Here's a short explanation Understanding PUID and PGID.

[Bubka]

Thx @yannduran, this was really helpful. It's now clear to me why you expect UID/GID vars and thanks to more searches how this permissions issue is commonly addressed in the docker world.

To all
For now I can't provide a quick fix because of the way the 2FAuth image is defined, but I will work on it.
By then, if you can't/don't want to change the ownership of the mounted drive, you can build your own 2FAuth image with specific UID/GID to match your setup: https://docs.2fauth.app/getting-started/installation/docker/docker-cli/#build-the-image-with-build-arguments

[yannduran]

@Bubka I'm glad you found what I wrote helpful.

However, we absolutely shouldn't need to have to build our own images. The 2 changes that I describe below can be implemented without any disruption to existing users, or even future users.

The UID & GID (ARG) variables are used for setting permissions on objects INSIDE the container.

All folders/files etc INSIDE the container have nothing to do with the permissions used on the docker host, so apart from some questionable practices in the dockerfile itself, only ONE thing needs to be changed there. Then some modifications will need to be made in the application itself, to use the environment variable values instead of assuming 1000:1000.

I found that the environment variables near the bottom of the dockerfile, (see the code below). These exist for consumers of the container to be able to pass to the container any values that they need/want to change from the default values.

ENV \
    # You can change the name of the app
    APP_NAME=2FAuth \
    # You can leave this on "local". If you change it to production most console commands will ask for extra confirmation.
    # Never set it to "testing".
    APP_ENV=local \
    # Set to true if you want to see debug information in error screens.
    APP_DEBUG=false \

    #... all the way down to

    MIX_PUSHER_APP_KEY="${PUSHER_APP_KEY}" \
    MIX_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}" \
    MIX_ENV=local

Change #1 (dockerfile):

To enable the application to have the correct permissions to access our mapped folder, simply add the following lines between the ENV \ command and the APP_NAME definition:

    PUID=1000 \
    GUID=1000 \

So it would end up looking like this:

ENV \
    PUID=1000 \
    GUID=1000 \

    # You can change the name of the app
    APP_NAME=2FAuth \
    # You can leave this on "local". If you change it to production most console commands will ask for extra confirmation.
    # Never set it to "testing".
    APP_ENV=local \
    # Set to true if you want to see debug information in error screens.
    APP_DEBUG=false \

    #... all the way down to

    MIX_PUSHER_APP_KEY="${PUSHER_APP_KEY}" \
    MIX_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}" \
    MIX_ENV=local

These PUID/GUID variables are only for accessing our mapped folder, at runtime. So they have nothing to do with the existing PID/GID variables already being used for doing compile-time stuff. You could also call the new variables USER_ID & GROUP_ID etc, it really doesn't matter as long as they're documented along with the other env variables. So far I've seen PUID/GUID used more than anything else. We just need to know the name of the variables so we can use them to pass the correct permission values for the application to be able to access our mapped folder.

Change #2 (application):

The endpoint application needs to be changed to use these environment variable values to add/edit/update folders/files inside our mapped folder on the docker host.

In the dockerfile the defaults get to 1000:1000.

• if no env variable values are specified by the consumer of the container, these default values would be used exactly the same as before
• if the consumer does provide values for these variables, they'll be used instead of the defaults

So if our PUID & GUID value are not 1000:1000 we simply supply the correct values that match the permissions of our mapped folder. They're passed to the docker engine either through docker commands or docker compose files.

I'm guessing that these env variables then get somehow magically passed through to the endpoint application.

---

Recommendations:

This is not required for fixing our issue, it's just a best practice recommendation. It will help in understanding/maintaining this dockerfile in the future.

1. Initialise the UID/GID variables to 1000:1000 near the TOP of the dockerfile only once (you'll see this code is there already)

ARG UID=1000
ARG GID=1000

2. Remove all other occurrences of the same declarations above, as they only need to be defined once. Defining ARG variables multiple with the same values times is bad practice & makes it harder to understand & maintain the code in the future.
3. Replace any hard-coded 1000:1000 instances throughout the file with ${UID}:${GID}. You'll see that this is already used in some places in the file. But each time they're used, the variables are defined (again) just above the line that uses the values. There should be no hard-coded 1000:1000 anywhere in the file, only ${UID}:${GID}.

---

I hope that some of this may be helpful. I haven't looked at the application code yet, but I'll see if I can determine what needs to be changed to use the newly added environment variables. The language is PHP, which I've never used. But I'm hoping to be able to follow it well enough to see what might need to be changed.

A quick look (searching for /2fauth which is the name of the data folder inside the container) has discovered some more bad advice when using your own SQLite database file.

chown 1000:1000 /yourpath/2fauth/database.sqlite
chmod 700 /yourpath/2fauth/database.sqlite

I'll say it again. NOTHING should be changing the owner or permissions of ANY object in our mapped folder, or assuming permissions of 1000:1000.

So instead of using chown & chmod it should be using the environment variables to use the correct permissions to access the mapped folder & edit its contents. This is a very common scenario. Any container that provides a config or data folder to map to would have to handle this. I know the "what", but not the "how" at this stage.

Maybe you can discuss this with the author of the application code? They should be able to find any places where the folder is being accessed easier/quicker than I could. They should be familiar with using the environment variable values as there are quite a few env variables already provided by the container.

[Bubka]

I'm the author of 2FAuth. What I can't figure out is how I can have 2FAuth (so php code) to have permission over the /2fauth mapped folder at runtime. This scenario seems to be commonly addressed in the docker world by changing the exec user via an entrypoint script. But this is only possible if the container is ran as root, which is not the case of the 2FAuth image. See https://stackoverflow.com/questions/39397548/how-to-give-non-root-user-in-docker-container-access-to-a-volume-mounted-on-the for a similar situation.

Regarding your recommendations:

Remove all other occurrences of the same declarations above, as they only need to be defined once. Defining ARG variables multiple with the same values times is bad practice & makes it harder to understand & maintain the code in the future.

The Dockerfile uses multiple build stages (the FROM ... AS ...), which mean that any of them can be directly build with the docker build --target xxx command. I think this is why the original image creator choose to repeat the ARG UID=1000 \ ARG GID=1000 for each of them. See https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact