bug report: getmail not set up

[rhyst]

📝 Preliminary Checks

• I tried searching for an existing issue and followed the debugging docs advice, but still need assistance.

👀 What Happened?

I set ENABLE_GETMAIL to 1 in my config file and create a getmail-example.cf file in my dms/config/ folder. Getmail is not enabled and instead I recieve the following error to the postmaster address:

Error: Specified config/data dir "/var/lib/getmail" does not exist - create or specify alternate directory with --getmaildir option

If I examine the file structure I can indeed see that the /var/lib/getmail symlink links to /var/mail-state/lib-getmail which does not exist.

👟 Reproduction Steps

I tried recreating this with a fresh local instance. First I ran the commands to get the compose file and env file.

DMS_GITHUB_URL="https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master"
wget "${DMS_GITHUB_URL}/compose.yaml"
wget "${DMS_GITHUB_URL}/mailserver.env"

I ran the container and I created a user and and alias:

docker compose up -d
docker exec -ti mailserver setup email add [email protected]
docker exec -ti mailserver setup alias add [email protected] [email protected]

I then edited the mailserver.env file to set

LOG_LEVEL=debug
ENABLE_GETMAIL=1

I ran recreate the container with:

docker-compose down
docker-compose up

I can now see in /var/mail/example.com/user/new/ the error emails related to the getmail folder not existing.

🐋 DMS Version

v13.3.1

💻 Operating System and Architecture

Ubuntu 20.04.6 LTS

⚙️ Container configuration files

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest
    container_name: mailserver
    # Provide the FQDN of your mail server here (Your DNS MX record should point to this value)
    hostname: mail.example.com
    env_file: mailserver.env
    # More information about the mail-server ports:
    # https://docker-mailserver.github.io/docker-mailserver/latest/config/security/understanding-the-ports/
    ports:
      - "25:25"    # SMTP  (explicit TLS => STARTTLS, Authentication is DISABLED => use port 465/587 instead)
      - "143:143"  # IMAP4 (explicit TLS => STARTTLS)
      - "465:465"  # ESMTP (implicit TLS)
      - "587:587"  # ESMTP (explicit TLS => STARTTLS)
      - "993:993"  # IMAP4 (implicit TLS)
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
    restart: always
    stop_grace_period: 1m
    # Uncomment if using `ENABLE_FAIL2BAN=1`:
    # cap_add:
    #   - NET_ADMIN
    healthcheck:
      test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
      timeout: 3s
      retries: 0

📜 Relevant log output

[  DEBUG  ]  Handling general environment variable setup
[   INF   ]  Welcome to docker-mailserver v13.3.1
[  DEBUG  ]  Registering functions
[   INF   ]  Checking configuration
[  DEBUG  ]  Checking for improper restart
[  DEBUG  ]  Checking that hostname/domainname is provided or overridden
[  DEBUG  ]  Domain has been set to example.com
[  DEBUG  ]  Hostname has been set to mail.example.com
[   INF   ]  Configuring mail server
[  DEBUG  ]  Setting up general log files
[  DEBUG  ]  Setting up Dovecot
[  DEBUG  ]  Enabling IMAP services
[  DEBUG  ]  Setting up Dovecot dhparam
[  DEBUG  ]  Setting up Dovecot quota
[  DEBUG  ]  Spam emails will be moved to the Junk folder
[  DEBUG  ]  Spam emails will not be marked as read
[  DEBUG  ]  Setting up Dovecot Local User
[  DEBUG  ]  Creating user 'user' for domain 'example.com'
[  DEBUG  ]  Configuring DKIM
[  DEBUG  ]  OpenDKIM enabled but no DKIM key(s) provided
[  DEBUG  ]  Configuring OpenDMARC
[  DEBUG  ]  Configuring policyd-spf
[  DEBUG  ]  Setting up Security Stack
[  DEBUG  ]  Postgrey is disabled
[  DEBUG  ]  Configuring Postscreen
[  DEBUG  ]  Disabling Postscreen DNSBLs
[  DEBUG  ]  SpamAssassin is disabled
[  DEBUG  ]  Disabling ClamAV
[  DEBUG  ]  Fail2Ban is disabled
[  DEBUG  ]  Configuring Amavis
[  DEBUG  ]  Rspamd is disabled
[  DEBUG  ]  Setting up SSL
[  DEBUG  ]  TLS configured with 'modern' ciphers
[ WARNING ]  !! INSECURE !! SSL configured with plain text access - DO NOT USE FOR PRODUCTION DEPLOYMENT
[  DEBUG  ]  Setting up PERMIT_DOCKER option
[  DEBUG  ]  Setting up mailname and creating '/etc/mailname'
[  DEBUG  ]  Applying hostname to Dovecot
[  DEBUG  ]  Configuring Postfix (early setup)
[  DEBUG  ]  '/tmp/docker-mailserver/postfix-virtual.cf' not provided - no mail alias/forward created
[  DEBUG  ]  Setting up Postfix dhparam
[  DEBUG  ]  Spoof protection is disabled
[  DEBUG  ]  Configuring Postfix (late setup)
[  DEBUG  ]  Setting up Postfix Relay Hosts
[  DEBUG  ]  (Postfix setup) Overriding / adjusting configuration with user-supplied values
[  DEBUG  ]  Fetchmail is disabled
[  DEBUG  ]  Fetchmail parallel is disabled
[  DEBUG  ]  Setting up logrotate
[  DEBUG  ]  Postfix log summary reports disabled
[  DEBUG  ]  Logwatch reports disabled.
[  DEBUG  ]  Consolidating all state onto /var/mail-state
[  DEBUG  ]  Checking /var/mail permissions
[  DEBUG  ]  Removing files and directories from older versions
[  DEBUG  ]  Ensuring  is owned by '_rspamd:_rspamd'
chown: cannot access '': No such file or directory
[  DEBUG  ]  Exporting environment variables now (creating '/etc/dms-settings')
[  DEBUG  ]  Setting up configuration checksum file
[  DEBUG  ]  Printing environment variables. Make sure no sensitive data is copied.
ACCOUNT_PROVISIONER='FILE'
AMAVIS_LOGLEVEL='0'
CLAMAV_MESSAGE_SIZE_LIMIT='25M'
DEFAULT_RELAY_HOST=''
DMS_VMAIL_GID='5000'
DMS_VMAIL_UID='5000'
DOVECOT_INET_PROTOCOLS='all'
DOVECOT_MAILBOX_FORMAT='maildir'
DOVECOT_TLS='no'
ENABLE_AMAVIS='1'
ENABLE_CLAMAV='0'
ENABLE_DNSBL='0'
ENABLE_FAIL2BAN='0'
ENABLE_FETCHMAIL='0'
ENABLE_GETMAIL='1'
ENABLE_IMAP='1'
ENABLE_MANAGESIEVE='0'
ENABLE_OAUTH2='0'
ENABLE_OPENDKIM='1'
ENABLE_OPENDMARC='1'
ENABLE_POLICYD_SPF='1'
ENABLE_POP3='0'
ENABLE_POSTGREY='0'
ENABLE_QUOTAS='1'
ENABLE_RSPAMD='0'
ENABLE_RSPAMD_REDIS='0'
ENABLE_SASLAUTHD='0'
ENABLE_SPAMASSASSIN='0'
ENABLE_SPAMASSASSIN_KAM='0'
ENABLE_SRS='0'
ENABLE_UPDATE_CHECK='1'
FAIL2BAN_BLOCKTYPE='drop'
FETCHMAIL_PARALLEL='0'
FETCHMAIL_POLL='300'
GETMAIL_POLL='5'
LOGROTATE_INTERVAL='weekly'
LOGWATCH_INTERVAL='none'
LOGWATCH_RECIPIENT='[email protected]'
LOGWATCH_SENDER='[email protected]'
LOG_LEVEL='debug'
MARK_SPAM_AS_READ='0'
MOVE_SPAM_TO_JUNK='1'
NETWORK_INTERFACE='eth0'
ONE_DIR='1'
OVERRIDE_HOSTNAME=''
PERMIT_DOCKER='none'
PFLOGSUMM_RECIPIENT='[email protected]'
PFLOGSUMM_SENDER='[email protected]'
PFLOGSUMM_TRIGGER='none'
POSTFIX_DAGENT=''
POSTFIX_INET_PROTOCOLS='all'
POSTFIX_MAILBOX_SIZE_LIMIT='0'
POSTFIX_MESSAGE_SIZE_LIMIT='10240000'
POSTFIX_REJECT_UNKNOWN_CLIENT_HOSTNAME='0'
POSTGREY_AUTO_WHITELIST_CLIENTS='5'
POSTGREY_DELAY='300'
POSTGREY_MAX_AGE='35'
POSTGREY_TEXT='Delayed by Postgrey'
POSTMASTER_ADDRESS='[email protected]'
POSTSCREEN_ACTION='enforce'
RELAY_HOST=''
REPORT_RECIPIENT='[email protected]'
REPORT_SENDER='[email protected]'
RSPAMD_CHECK_AUTHENTICATED='0'
RSPAMD_GREYLISTING='0'
RSPAMD_HFILTER='1'
RSPAMD_HFILTER_HOSTNAME_UNKNOWN_SCORE='6'
RSPAMD_LEARN='0'
SA_KILL='10.0'
SA_SPAM_SUBJECT='***SPAM*** '
SA_TAG2='6.31'
SA_TAG='2.0'
SMTP_ONLY='0'
SPAMASSASSIN_SPAM_TO_INBOX='1'
SPOOF_PROTECTION='0'
SRS_DOMAINNAME='example.com'
SRS_EXCLUDE_DOMAINS=''
SRS_SECRET=''
SRS_SENDER_CLASSES='envelope_sender'
SSL_TYPE=''
SUPERVISOR_LOGLEVEL='warn'
TLS_LEVEL='modern'
TZ=''
UPDATE_CHECK_INTERVAL='1d'
VIRUSMAILS_DELETE_DELAY='7'
[   INF   ]  Starting daemons
[  DEBUG  ]  Starting cron
[  DEBUG  ]  Starting rsyslog
[  DEBUG  ]  Starting dovecot
[  DEBUG  ]  Starting update-check
[  DEBUG  ]  Starting opendkim
[  DEBUG  ]  Starting opendmarc
[  DEBUG  ]  Starting postfix
[  DEBUG  ]  Starting amavis
[  DEBUG  ]  Starting changedetector
[   INF   ]  mail.example.com is up and running

Improvements to this form?

N/A

[glime-ai]

Please note that this response was created using generative AI, leveraging previously available information from the repository. This information may be helpful. However, it is our strong recommendation that you carefully consider the information before blindly applying it.

---

It appears that the error you're encountering with Getmail is due to the specified directory /var/lib/getmail not existing in your Docker container. This directory is crucial for Getmail's operation as it expects to find its configuration and data there. The symlink /var/lib/getmail pointing to /var/mail-state/lib-getmail suggests that the directory should be created and managed by the Docker Mailserver setup, but it seems it wasn't created properly.

To resolve this issue, you can manually create the directory /var/mail-state/lib-getmail in your Docker container. Here are the steps you can follow:

1. Ensure your Docker container is running.
2. Access the container's shell:

   docker exec -it mailserver /bin/bash

3. Create the necessary directory:

   mkdir -p /var/mail-state/lib-getmail

4. Exit the container:

   exit

5. Restart the container to ensure all configurations are properly reloaded:

   docker-compose down
   docker-compose up -d

This should resolve the error by ensuring that the required directory exists and is accessible to Getmail. If the issue persists, you might need to check the permissions of the newly created directory or consider any additional configuration that might be affecting Getmail's operation.

This comment was generated by Glime.

[rhyst]

That did actually work. I guess there may need to be a note in the docs or otherwise it should create the folder automatically.

[polarathene]

docker-mailserver/target/scripts/startup/setup.d/mail_state.sh

Line 26 in 5bd8df6

[[ ${ENABLE_GETMAIL} -eq 1 ]] && SERVICEDIRS+=('lib/getmail')

No /var/mail-state volume mount:

$ ls -lah /var/lib

total 96K
drwxr-xr-x 1 root         root         4.0K Jan 23 10:51 .
drwxr-xr-x 1 root         root         4.0K Jan 23 10:51 ..
drwxr-x--- 1 amavis       amavis       4.0K Jan 23 18:12 amavis
drwxr-xr-x 1 root         root         4.0K Jan 13 08:39 apt
drwxr-xr-x 1 clamav       clamav       4.0K Jan 22 02:06 clamav
drwxr-xr-x 1 root         root         4.0K May 15 11:42 dovecot
drwxr-xr-x 1 root         root         4.0K Jan 23 10:52 dpkg
drwxr-xr-x 2 root         root         4.0K Nov  9  2022 fail2ban
drwx------ 2 fetchmail    nogroup      4.0K Jan 13 08:38 fetchmail
drwxr-xr-x 2 root         root         4.0K Jun 30  2023 logrotate
drwxr-xr-x 2 root         root         4.0K Sep 29  2023 misc
drwxr-xr-x 2 _mta-sts     _mta-sts     4.0K Jan 13 08:38 mta-sts
drwxr-xr-x 2 root         root         4.0K Jan 10 00:00 pam
drwxr-xr-x 1 postfix      postfix      4.0K May 15 11:42 postfix
drwx------ 2 postgrey     postgrey     4.0K Jan 13 08:38 postgrey
drwxr-xr-x 2 root         root         4.0K Jul 14  2021 postsrsd
drwxr-xr-x 2 root         root         4.0K Jan 13 08:38 python
drwxr-x--- 2 redis        redis        4.0K Jan 13 08:39 redis
drwxr-x--- 2 _rspamd      _rspamd      4.0K Dec 15 15:52 rspamd
drwxr-xr-x 3 debian-spamd debian-spamd 4.0K Jan 13 08:38 spamassassin
drwxr-xr-x 1 root         root         4.0K Jan 13 08:38 systemd
drwxr-xr-x 3 root         root         4.0K Jan 13 08:39 ucf

Now with a /var/mail-state mount:

$ ls -lah /var/lib

total 80K
drwxr-xr-x 1 root         root         4.0K May 15 11:35 .
drwxr-xr-x 1 root         root         4.0K May 15 11:34 ..
lrwxrwxrwx 1 root         root           26 May 15 11:35 amavis -> /var/mail-state/lib-amavis
drwxr-xr-x 1 root         root         4.0K Jan 13 08:39 apt
drwxr-xr-x 1 clamav       clamav       4.0K Jan 22 02:06 clamav
lrwxrwxrwx 1 root         root           27 May 15 11:35 dovecot -> /var/mail-state/lib-dovecot
drwxr-xr-x 1 root         root         4.0K Jan 23 10:52 dpkg
drwxr-xr-x 2 root         root         4.0K Nov  9  2022 fail2ban
drwx------ 2 fetchmail    nogroup      4.0K Jan 13 08:38 fetchmail
lrwxrwxrwx 1 root         root           27 May 15 11:35 getmail -> /var/mail-state/lib-getmail
lrwxrwxrwx 1 root         root           29 May 15 11:35 logrotate -> /var/mail-state/lib-logrotate
drwxr-xr-x 2 root         root         4.0K Sep 29  2023 misc
drwxr-xr-x 2 _mta-sts     _mta-sts     4.0K Jan 13 08:38 mta-sts
drwxr-xr-x 2 root         root         4.0K Jan 10 00:00 pam
lrwxrwxrwx 1 root         root           27 May 15 11:35 postfix -> /var/mail-state/lib-postfix
drwx------ 2 postgrey     postgrey     4.0K Jan 13 08:38 postgrey
drwxr-xr-x 2 root         root         4.0K Jul 14  2021 postsrsd
drwxr-xr-x 2 root         root         4.0K Jan 13 08:38 python
drwxr-x--- 2 redis        redis        4.0K Jan 13 08:39 redis
drwxr-x--- 2 _rspamd      _rspamd      4.0K Dec 15 15:52 rspamd
drwxr-xr-x 3 debian-spamd debian-spamd 4.0K Jan 13 08:38 spamassassin
drwxr-xr-x 1 root         root         4.0K Jan 13 08:38 systemd
drwxr-xr-x 3 root         root         4.0K Jan 13 08:39 ucf

Symlink is there like expected but...

$ ls /var/mail-state
lib-amavis  lib-dovecot  lib-logrotate  lib-postfix  spool-postfix

The actual mount location being symlinked to lacks the actual directory.

When the feature was implemented in Sep 2022 (but merged May 2023), it added a debug command and cron job that rely on /var/lib/getmail, and schedules that cron job at startup to run the getmail command to get new mail at this fixed location, but there was no logic to ensure the directory already existed 😕

We don't run the test suite with /var/mail-state, so this sort of failure wouldn't be caught either. The /var/lib/getmail exists just not it's counterpart. (due to symlink, doesn't exist when /var/mail-state is not mounted, so no content is actually moved via mv)

---

This is a bug, so it should be addressed for v14, creating the folder at /var/mail-state/lib-getmail/ with mkdir -p. Seems like something our mail-state startup script should take care of.