Make concatenation signature public?

[bblay]

✨ Feature Request

Could we consider making _concatenate._CubeSignature a public symbol?

Motivation

I'm stitching thousands of octants into global fields. I group the cubes by their concatenation signature so I can loop through them and sanity check, for each each group, that:

• there are 8 octants or two hemispheres in the group, and warn otherwise
• they were all stitched into a single cube, and warn otherwise

These warnings are useful because they highlight problems, such as:

• insufficient loading callback, which adds metadata for merge and concatenate to work
• bugs in the trimming code, which removes duplicate rows and columns
• Merge dimension mashup #5768

The warnings are able to report on the cubes which need investigating. Without this, I found it was much harder to tell what's been stitched properly and what hasn't.

Also, for further consideration: I had to do something similar in the #5768 workaround, but with the merge signature. That's a simple named tuple so I didn't have to use the actual symbol, I just used the params I would have passed in. This is liable to break if its params change.

[trexfeathers]

Hi @bblay. We are certainly aware that it is very difficult to understand compatibility issues. Whether we could act on your suggestion is super uncertain at the moment. We currently have at least 3 modules that inspect what is in Cubes, and make decisions about their compatibilities:

• iris._concatenate
• iris._merge
• iris.common.resolve

This causes problems for developers and users alike when trying to understand Cube compatibility. iris.common.resolve was imagined to be a potential common solution (hence the namespace) - we could make ONE thing that is all singing, all dancing, and designed for user interaction - but recent experiments have shown that this will be more awkward than hoped ( #5394 (comment) ).

While the merge and concatenate mechanisms are private we are free to think about various ideas for refactoring. If made public, with users starting to rely on certain objects existing and behaving in a certain way, we will loose a lot of that freedom.

[bblay]

Thanks for the response, @trexfeathers . Some good points.

I wasn't aware of the all-singing attempt. I'll ignore that here.

You're right about exposing the merge and concatenate structures. That could cause problems down the line. So the question then evolves to this:

"I had to write a function which groups cubes by their concatenation compatibility, but it uses an Iris internal structure. Could/should we add such a function to Iris?"

groups = iris.concatenation.group_by_compatibilty(cubes)
for group in groups:
    my_code.sanity_check_group(group)
    new_cubes = my_code.process(group)
    my_code.sanity_check_result(new_cubes)

That would seem to work equally well for merge.

[trexfeathers]

Thanks, good to have this issue in the backlog