"Key provided but not found in metadata" is misleading in many cases

[themonk911]

When running the unit tests, I can see a lot of messages that say (e.g.)

{"timestamp": "2020-08-26T14:22:58.2411", "classname": "CzechRepublicAgeSexDataSource", "record": {"key": "CZ_31_311", "date": "2020-03-13", "age": "20-29", "sex": "male", "new_confirmed": 1, "_vec": "CZ_31_311"}, "message": "Key provided but not found in metadata"}

However, looking in the metadata, those keys are definitely present. So it appears that the error message is incorrect in this case.

[owahltinez]

This is tricky, because the unit tests intentionally run on a subset of the metadata (to make things reasonably fast). However, the data being pulled for the test is real data which means that it will have regions which do not exist in the metadata.

Something we could do is to disable certain warnings during unit testing, since it's only meant for basic validation, whereas things like mismatched keys can be tracked via monitoring (although it looks like only errors are being reported so far, and not warnings).

[themonk911]

Yeah the tests don't fail on these warnings, it's just noise. I'm pro disabling them during testing, since the unit tests do not fail on them anyway.