MySQL version is not compatible with MySQL v8's default password hashing mechanism

[timwis]

Subject of the issue

In January 2018, MySQL announced they were changing the default authentication plugin for MySQL server from mysql_native_password to caching_sha2_password in MySQL v8.0.4. This required mysql clients to support the caching_sha2_password method in order to connect.

The bitwardenrs/msyql-server image does not appear to support this method, which results in an error when attempting to connect to a modern MySQL (v8.0.4+) database:

Can't connect to DB: BadConnection("Plugin caching_sha2_password could not be loaded: /usr//usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory

This can be worked around by using a user account that uses the mysql_native_password authentication method (as described here), but ideally we'd be able to take advantage of the latest (and most secure) authentication method.

I assume this is a problem with the runtime image's libmariadbclient-dev supporting this authentication plugin, or perhaps how it's locating the plugin file, but I haven't been able to figure it out. MariaDB's docs suggest it should be supported (I assume it's the C library).

Your environment

• Bitwarden_rs version: 1.13.1-325039c3

• Install method: Docker image
• Clients used: N/A
• Reverse proxy and version: Caddy, but error is shown at bitwardenrs docker image startup
• Version of mysql/postgresql: MySQL 8.0.17 (digital ocean managed cluster)
• Other relevant information: N/A

Steps to reproduce

Docker-compose file

  bitwarden:
    container_name: bitwarden
    image: bitwardenrs/server-mysql
    volumes:
      - ./bitwarden-data/:/data/
    environment:
      DATABASE_URL: mysql://$BITWARDEN_DB_USERNAME:$BITWARDEN_DB_PASSWORD@$BITWARDEN_DB_HOSTNAME:$BITWARDEN_DB_PORT/$BITWARDEN_DB_DATABASE
      ADMIN_TOKEN: $BITWARDEN_ADMIN_TOKEN
      ENABLE_DB_WAL: 'false'
      RUST_BACKTRACE: 1 # detailed crash logs

Expected behaviour

Bitwarden should connect to the MySQL database

Actual behaviour

Error thrown, as described in subject section above.

Relevant logs

See subject section above.

[Ornias1993]

Mariadb is not mysql, those are two seperate projects.
If you want to use mysql, you should use their client and not mariadb-client.

"Let me mix-and-match different sql clients and servers" seems pretty out-of-scope.

[timwis]

@Ornias1993 then why, I wonder, is the image called bitwardenrs/server-mysql, and the bitwarden_rs documentation page called Using the MySQL Backend (which makes no mention of MariaDB)?

Because MariaDB is not just some other project; it's an open source "drop-in replacement for MySQL". If you read the links I provided, you can see that they maintain feature and API compatibility with MySQL.

[Ornias1993]

@timwis Why? No idea, I didn't write the documentation.
I can only say this project does NOT use mysql as a dependency for the binary so it isn't relevant what MySQL is or is not doing.

If you look at the binary, you would've known it's mariadb based and not mysql based:
https://github.com/dani-garcia/bitwarden_rs/wiki/Building-binary

I think it is just a documentation issue, it should be clearly stated that the MySQL is based on MariaDB and not on MySQL.

I'm running the binary with the lastest variant of the MariaDB client on the latest version of the MariaDB Server and have no such issue.

To be clear:
MariaDB is a "Drop in REPLACEMENT", that means you replace both the server AND client with mariadb and it should work using the same MySQL syntax.

It's might work to mix match client and server, but such a thing is not guaranteed. If you run a project on MySQL it's highly advicable to run it on both client and server and the same goes for mariadb.

I don't appreciate your passive agressive attitude however.

edit
I've spend some more time reading on the docker part of this problem.
The problem is in the documentation indeed...

Even the docker build script uses mariadb and I think it's wrong advice to use a MySQL server docker container with a mariadb-client frontend...

Certainly because Mysql and mariadb are not 100% compatible:
https://mariadb.com/kb/en/mariadb-vs-mysql-compatibility/

edit2
@dani-garcia Maybe you could look into replacing MySQL docker container with the MariaDB one in the documentation? They have an official docker available and it might prevent some future compatibility issues:
https://hub.docker.com/_/mariadb?tab=tags

[demize]

Just commented on #1348 but I'm going to mention it here too: blaming the server is not the answer here, blaming the documentation is not the answer here, this is absolutely a client issue. MariaDB is supposed to support this, and it's not supposed to fail the way it does in #1348, and #1348 shows the complete failure with MySQL is a regression. The fix here needs to be fixing the client to either properly support the new authentication method or at the very least not failing to use the older one.

[BlackDex]

I have made this comment already on an other report and i will copy paste it here.

---

TL;DR:
For x86_64:
Use the Alpine version which should have this plugin activated and enabled, the Debian version does not!
All other arch:
Debian does not have this plugin build within there packages, we can't (and will not) try to fix there packages.

If you ensist on using MySQL8 then create a user using ... IDENTIFIED WITH mysql_native_password BY ...

---

To be blunt, we use the packages provided by the underlying OS, which in this case is Debian.
Debian does NOT support/build this new caching_sha2_password method! It is not something we will just build in our self by recompiling the default provided packages by Debian, this feature wasn't in there before, but it will be in there eventually.

I think you forget that every distro has there own building flags and which features they add or remove and which versions of those tools they support or not.

As debian doesn't support the latest version of mariadb-client we also can not support this.

I will update the documentation to mention that we do not support MySQL 8.x using it's default password caching and that you need to use the mysql_native_password instead!

Again, this is not something we can and will fix on our side, if Debian releases a new stable version of there Distro we will update that accordingly and we just have to hope it adds this feature.

Also, our builds use the MariaDB libraries since that is what Debian, Ubuntu and Alpine support, there are no packages in there main repo's which hold the Orcale MySQL libraries anymore.

[BlackDex]

I will convert this ticket to a discussion also since there is nothing we can do to fix the default packages from Debian.
We want to have a stable running docker image, and we do that with the default packages from the OS.