Custom Vacuum rules that want resolved inputs not always resolved before running

[TristanSpeakEasy]

We have run into an issue in custom vacuum rules we write that if we set Resolved: true we expect the yaml we are walking through to be fully resolved (ie no $ref nodes).

And this does happen sometimes but is non-deterministic.

Ie somethings it seems the yaml is resolved before the rule is run other times it isn't.

[daveshanley]

How are you using vacuum in this use case? there are multiple code paths, based on if you're using by passing in a document, or raw bytes. A snippet of how you're applying the rules would be helpful.

[TristanSpeakEasy]

We are passing in a document but will try and get a small reproducible together

[daveshanley]

There is also another mechanism used by the bundle command that will perform the same thing as resolving, but it uses a different technique to the resolver. There is a new feature in libopenapi that allows references to be resolved sequentially. Can you try setting the ExtractRefsSequemtially property to true and try again?

This invokes a different codepath and runs the index sequentially when extracting refs, vs async.

You can see it here: https://github.com/pb33f/libopenapi/blob/main/datamodel/document_config.go#L109

[TristanSpeakEasy]

@daveshanley that does indeed seem to fix the issue, but is this just a work around for something that should be fixed elsewhere?

For example I would expect setting Resolved: true in a rule to ensure that I am always getting a resolved document into the rule? Should the rule wait to be executed until the document is fully resolved?

[TristanSpeakEasy]

Also should I be setting ExtractRefsSequentially to true always? To make sure that for both validation and iterating through a document model that I am not going to start visiting something to early? Or is there some sort of channel etc I should be waiting on before the document is ready to traverse?

[daveshanley]

The cause of the issue of why sometimes async extraction of refs are fully resolved, and sometimes they are missing is currently unknown. It happens with super large, deeply nested specs. Which is why the sync option was made available to always ensure resolved nodes.

It's a bug honestly, so I'll keep this issue open until we figure it out.