Message versioning and ROS 2 message translation

[GuillaumeLaine]

This documents the newly versioned subset of PX4 messages, and the ROS 2 translation node that enables seamless communication between PX4 and ROS 2 applications that may be using different message definition versions.

Related Work

• [PX4-Autopilot] Implementing ROS 2 translation node and CI checks: Message versioning and translation for ROS PX4-Autopilot#24113
• [PX4-Autopilot] Introducing versioned PX4 messages: msg: introduce subset of versioned messages  PX4-Autopilot#23850
• [px4_msgs] Updating build paths for versioned messages: CMakeLists: include versioned msg path px4_msgs#46

To Do

• Complete TODOs in the markdown of this PR

[GuillaumeLaine]

@hamishwillee I think the PR is in a review ready state. I left a few TODOs in the markdown to resolve before merging. Mostly things that I need to go back to once the last related PR is merged

[hamishwillee]

My thinking here is that we're going to want users to get used to this, and to almost do it by default - right?

If so I think you're going to need to add instructions to install and use the versioned messages in this guide - probably in a section of https://docs.px4.io/main/en/ros2/user_guide.html#build-ros-2-workspace

Note that when I say "install and use", I really mean "install and run" - you would link to your instructions in the translation node doc for how to update versions and so on. There would be some duplication in that doc.

[hamishwillee]

This is a little messy and perhaps can be simplified. In particular

• ROS2 Message Translation node supports versioning, not the versioning supporting the translation node.

Perhaps something like:

Suggested change

	Message versioning was introduced to address the challenges of maintaining compatibility across systems where different versions of message definitions may be in use, such as between PX4 and external systems like ROS 2 applications.
	This versioning mechanism supports the [ROS 2 Message Translation Node](../ros2/px4_ros2_msg_translation_node.md), which enables seamless communication between PX4 and ROS 2 applications; when different versions of message definitions are in use, the ROS 2 translation node ensures that messages can be converted and exchanged correctly.
	Message versioning has been in introduced in PX4 v1.16 to make it easier to maintain compatibility between PX4 and ROS 2 versions.
	This versioning mechanism uses the [ROS 2 Message Translation Node](../ros2/px4_ros2_msg_translation_node.md) to convert between different versions of messages, allowing seamless communication between PX4 and ROS 2 applications; when different versions of message definitions are in use.

Perhaps add a diagram showing where the magic happens?

It would also be good "somewhere here" to mention any forward thinking about the ROS 2 native translation system that they have been talking about for a couple of years. Perhaps as a note at the end.

[hamishwillee]

I think it is worth saying when you should do this. I presume for every workspace?

So something like: "For every workspace you create, it is recommended to install and run the node:", followed by the steps.

[GuillaumeLaine]

I don't think this needs to be done for every PX4/ROS workspace. I think it's sufficient to have one workspace to build and run the translation node, and separate workspaces containing their respective apps. The translation node and those apps will then build separately, but they can still be run in parallel.

That said I don't think it's wrong to have the translation as part of multiple larger workspaces, I'm just not sure we should prescribe a specific workflow to the developer.

[GuillaumeLaine]

I've added a few sentences maybe it helps clarify

[hamishwillee]

Copy the versioned message file (.msg/.srv)`

• what is .srv here?
• what is px4_msgs_old? How do they get that folder/repo?
• "and add the" to "and append the current version information"

[GuillaumeLaine]

1. .srv are the service definitions files (e.g. VehicleCommand.srv), as opposed to the .msg message definition files. It is indeed confusing because the px4_msgs ROS2 package actually contains both the .msg and .srv definitions, and we use the term "message" loosely to designate specifically a message file, as well as either a message or service file. To clarify, I will change to:

Suggested change

	- Copy the versioned message file (`.msg`/`.srv`) to `px4_msgs_old/msg/` (or `px4_msgs_old/srv/`) and add the current version to the file name.
	- Copy the versioned `.msg` message file (or versioned `.srv` service file) to `px4_msgs_old/msg/` (or `px4_msgs_old/srv/`) and append the current version information to the file name.

2. px4_msgs_old is the ROS2 package that stores the versioned history of PX4 messages (and services), used by the translation node package. That folder is added to the PX4 repo in the final open PR, here. Currently it is empty, but in practice if px4_msgs has a message MyMessage.msg which is at MESSAGE_VERSION = 3, then px4_msgs_old will contain two definitions MyMessageV1.msg and MyMessageV2.msg. I will try to clarify

[hamishwillee]

What do I do if I have custom messages I want to version?

[hamishwillee]

Update the existing translations that use the current topic version to the now old version.

1. What's a translation? Needs a definition.
2. Am I replacing or adding a new one of these "whatever they are". I guess I'm asking if I make 3 versions of my message do I have three files? Or one that is updated three times?

[GuillaumeLaine]

Good points, I've elaborated on this.

I guess I'm asking if I make 3 versions of my message do I have three files?

Exactly, 3 new versions of a message means adding three new translation files: each translation file bridging one version to the next

[hamishwillee]

This example doesn't help answer my questions above - in what file would it be?

[hamishwillee]

I don't know anything about nested message definitions. What are they?

Either way, perhaps a warning or tip?

[GuillaumeLaine]

Nested messages are message definitions that include fields referencing other message types instead of using only the classic data types. For instance PositionSetpointTriplet.

Thanks I've added the example and made the paragraph into a warning

[hamishwillee]

I haven't looked at the example, but it seems to me that this might require some worked examples of each type. If the headers are specific to each type, perhaps move them under the following sections?

[GuillaumeLaine]

You're right, I've gone and added a worked example also to make the steps clearer

[hamishwillee]

So what versions will it work for?

[hamishwillee]

Do you mean services as in something different than message/topics? Perhaps this is what you mean by .svr above? Is there information on what those mean anywhere?

Either way, so are you saying that services require a linear history, but you can split or merge messages as you like?

[GuillaumeLaine]

Correct, ROS distinguishes between services and topics like so: https://docs.ros.org/en/foxy/How-To-Guides/Topics-Services-Actions.html. In PX4, we have the srv/ services folder vs the msg/ topics folder.

Messages define the data format used to communicate over either topics or via services. The confusing part is that .srv files define service messages, and .msg files define topic messages (the file extension introduces confusion and make them seem more generic)

And finally yes, currently services require a linear history to translate while topics (.msg) can be split and merged

Thanks for the comment, I realized I was also confused by this too. I'll do my best to make this clear in the docs

[hamishwillee]

So a ROS application must compile against just one version of a topic? What if there are two ROS applications running on the network compiled against different versions - is this the problem case?

[hamishwillee]

@GuillaumeLaine

OK, lots of comments here. They don't reflect that this is pretty good structure. Just I'm using myself as the straw man, and I don't know much ROS 2.

When you've had a look at the comments and tried address them, can you loop in Beat and Benja to have a look? I ask because I probably will be on holiday when you do this. Back around week of Jan 20th.

[mrpollo]

Maybe worth adding to the 1.16 release wip page? @hamishwillee

[github-actions]

No flaws found