1. I think the original comment here might not make sense in the future
2. The example NWB files are all generated using the version of NWB being tested, and because pynwb caches the spec by default, there should be no difference between using cached namespaces and not.
3. Since the pynwb.validate and pynwb-validate should be the same now, we don't really need this test anymore since we have the pynwb-validate test below. But since this validate_nwbs() function is super conservative in its testing of every combination, then for consistency, I suggest we validate with both use_cached_namespaces=True and use_cached_namespaces=False.

If I set use_cached_namespaces=True here, I get several errors when running the validate_nwb() section of test.py. I had added use_cached_namespaces=False so that it matched the previous test behavior, but if it is expected that there should be no difference between using the cached namespaces and not in this particular case, then maybe these errors are indicative of another issue?

You can replicate by running test.py, but the errors look like this below. Maybe the mylab extension generation needs to be updated?:

2024-12-23 10:12:02,969 - INFO - Validating with pynwb.validate method.
Traceback (most recent call last):
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/validate/validator.py", line 280, in get_validator
    return self.__validators[dt]
           ~~~~~~~~~~~~~~~~~^^^^
KeyError: 'NWBFile'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/smprince/Documents/code/pynwb/test.py", line 172, in validate_nwbs
    errors = validate(io, use_cached_namespaces=True)  # previously io did not validate against cached namespaces
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/utils.py", line 672, in func_call
    return func(**pargs)
           ^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/pynwb/src/pynwb/validation.py", line 191, in validate
    validation_errors += _validate_helper(io=io, namespace=validation_namespace)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/Documents/code/pynwb/src/pynwb/validation.py", line 19, in _validate_helper
    return validator.validate(builder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/validate/validator.py", line 298, in validate
    validator = self.get_validator(dt)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/utils.py", line 668, in func_call
    return func(args[0], **pargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/smprince/anaconda3/envs/pynwb/lib/python3.11/site-packages/hdmf/validate/validator.py", line 283, in get_validator
    raise ValueError(msg)
ValueError: data type 'NWBFile' not found in namespace mylab

I think these lines:

could be updated to:
ns_builder.include_namespace("core")

to fix these errors. I think these changes more closely match the latest version of ndx-template create_extension_spec.py file. However, should ns_builder.include_type still work without including the entire namespace?

Interesting. For now, let's update that to ns_builder.include_namespace("core").

We need to think through what it means to run the pynwb validator on a file, validating against a particular non-core namespace. Should the core namespace always be included during validation, regardless of whether the extension includes the core namespace, and the choices are either use the core namespace cached in the file or the one installed with pynwb? I think so...

I think we eventually want to move toward not even allowing validation against a particular namespace. Either the file is valid given its cached namespaces (or the namespaces installed by pynwb and loaded by the user), or the file is not. Otherwise, we run into weird issues with such as:
hdmf-dev/hdmf#608 and hdmf-dev/hdmf#525

I would say let's make the above change for now, and iterate on these other ideas in a separate PR which does not need to make it in pynwb 3.0.0rc1.

Should the core namespace always be included during validation, regardless of whether the extension includes the core namespace, and the choices are either use the core namespace cached in the file or the one installed with pynwb? I think so...

I think so too? But agreed it would definitely be helpful to compile and discuss all these different scenarios to get a better idea of the end goal validation behavior

I would say let's make the above change for now, and iterate on these other ideas in a separate PR which does not need to make it in pynwb 3.0.0rc1

Sounds good!