update db connect mongo connection test

[lukepatrick]

the isMaster MongoDB command is depreciated in future versions of mongo. hello is in the updated patch versions across a few major versions: 4.4.2: (and 4.2.10, 4.0.21, and 3.6.21)

https://docs.mongodb.com/v4.4/reference/command/hello/#mongodb-dbcommand-dbcmd.hello

This minor update supports future MongoDB development

[CLAassistant]

All committers have signed the CLA.

[arm4b]

Thanks for the contribution!

Could you please include the changelog record for this change?

[Kami]

LGTM, but it would be good to add a (direct) test for this functionality if we don't have one already.

[lukepatrick]

added changelog record

[arm4b]

Thanks for the contribution!

[blag]

I think this change broke a huge number of the ST2 Exchange pack tests:

==================== packs-resource-register ====================

. ~/virtualenv/bin/activate; \
if [ "${FORCE_CHECK_ALL_FILES}" = "true" ] || [ 0 -gt 0 ]; then \
	st2-check-register-pack-resources /tmp/packs/reamaze || exit 1; \
else \
	echo "No files have changed, skipping run..."; \
fi;
Registering content from pack reamaze
Running script: /home/circleci/virtualenv/bin/python /tmp/st2/st2common/bin/st2-register-content --register-fail-on-failure --register-runner-dir=/tmp/st2/contrib/runners --config-file=/home/circleci/ci/conf/st2.tests.conf --register-all --register-pack=/tmp/packs/reamaze
2021-08-21 01:09:21,219 INFO [-] Connecting to database "st2" @ "127.0.0.1:27017" as user "None".
Traceback (most recent call last):
  File "/tmp/st2/st2common/bin/st2-register-content", line 22, in <module>
    sys.exit(content_loader.main(sys.argv[1:]))
  File "/tmp/st2/st2common/st2common/content/bootstrap.py", line 449, in main
    setup(argv)
  File "/tmp/st2/st2common/st2common/content/bootstrap.py", line 440, in setup
    ignore_register_config_opts_errors=True,
  File "/tmp/st2/st2common/st2common/script_setup.py", line 105, in setup
    db_setup()
  File "/tmp/st2/st2common/st2common/database_setup.py", line 55, in db_setup
    connection = db_init.db_setup_with_retry(**db_cfg)
  File "/tmp/st2/st2common/st2common/persistence/db_init.py", line 93, in db_setup_with_retry
    ssl_match_hostname=ssl_match_hostname,
  File "/tmp/st2/st2common/st2common/persistence/db_init.py", line 58, in db_func_with_retry
    return retrying_obj.call(db_func, *args, **kwargs)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/retrying.py", line 206, in call
    return attempt.get(self._wrap_exception)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/tmp/st2/st2common/st2common/models/db/__init__.py", line 251, in db_setup
    ssl_match_hostname=ssl_match_hostname,
  File "/tmp/st2/st2common/st2common/models/db/__init__.py", line 204, in _db_connect
    connection.admin.command("hello")
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/pymongo/database.py", line 740, in command
    codec_options, session=session, **kwargs)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/pymongo/database.py", line 637, in _command
    client=self.__client)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/pymongo/pool.py", line 694, in command
    exhaust_allowed=exhaust_allowed)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/pymongo/network.py", line 161, in command
    parse_write_concern_error=parse_write_concern_error)
  File "/home/circleci/virtualenv/lib/python3.6/site-packages/pymongo/helpers.py", line 164, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: no such command: 'hello', bad cmd: '{ hello: 1 }', full error: {'ok': 0.0, 'errmsg': "no such command: 'hello', bad cmd: '{ hello: 1 }'", 'code': 59, 'codeName': 'CommandNotFound'}
Registering resources for pack reamaze failed

All of the failing packs have this error.

I can't find any place in the PyMongo documentation where the hello command is called in the connection.admin.command('hello') fashion, but I haven't dug extensively through these code search results:
https://github.com/mongodb/mongo-python-driver/search?q=hello

However, I did find this section of the documentation, which suggests using the ping command, with code that is suspiciously similar to what we currently have:

Note: Starting with version 3.0 the MongoClient constructor no longer blocks while connecting to the server or servers, and it no longer raises ConnectionFailure if they are unavailable, nor ConfigurationError if the user’s credentials are wrong. Instead, the constructor returns immediately and launches the connection process on background threads. You can check if the server is available like this:

from pymongo.errors import ConnectionFailure
client = MongoClient()
try:
    # The ping command is cheap and does not require auth.
    client.admin.command('ping')
except ConnectionFailure:
    print("Server not available")

@lukepatrick Can I get your opinion on this? You probably know the PyMongo API better than I do, so can you tell from that traceback if StackStorm Exchange errors caused by something else? Or should we tweak this code to use the ping command instead of the hello command?

[nzlosh]

I hit this error on my ubuntu 18.04 dev server. Turns out it was running MongoDB 3.2. hello was added to Mongo 3.6, 4.0, 4.2 and 4.4. Mongo 3.2/3.4 are already end of support so we shouldn't be attempting to maintain backward compatibility for them if we can avoid it. https://www.mongodb.com/support-policy/lifecycles

We should encourage users to upgrade to version 4.x of MongoDB. Our currently targetted platforms have packages available from MongoDB repositories.

The Exchange is using Mongodb version v3.4.24, we should upgrade to a 4.x version to resolve the hello compatibility.

[arm4b]

Good find about the ping command!

I guess that's a discussion that led to the doc update above:
https://www.mongodb.com/community/forums/t/how-to-use-the-new-hello-interface-for-availability/116748/
Exactly the same situation that we have here.

Looks like it allows us to get the best from the 2 worlds.

[arm4b]

Thanks everyone, created #5341 PR to workaround that.

Also documented an issue StackStorm-Exchange/exchange-incubator#168 for the CircleCI config update in StackStorm Exchange. We definitely should update the old mongo v3.4 in there.

So you're both right.

[blag]

Good work @nzlosh and @armab!

The fast and easy fix is to switch to using ping as per the mongodb.com forum link.

But the long term fix is to update all ST2 Exchange packs to use MongoDB v4.0 at least, since all OSes that we support (Ubuntu 18.04, Ubuntu 20.04, CentOS 7, and CentOS 8) all install with MongoDB 4.0.

[nzlosh]

The other question I had is, why do we test packs in the exchange against St2 dev packages and not the current stable packages?

[blag]

It's a good early warning system for anything that would break packs. Every now and then there's a change that would break one or more packs. Testing against dev packages helps us identify issues (on a weekly basis, on the weekend) and fix them before a major ST2 version is released.

It might be good to test against both dev and the latest release, but in practice I don't think we've had any issues with that. People are usually developing packs against a released version so that gets intrinsically tested.