Track additional Wayland seats

[Lachlan-Frawley]

This allows chromium-ozone-wayland to track more than one seat. In turn, this lets VNC/RDP pass keyboard and mouse inputs to chromium.

[rakuco]

Hi. Could you clarify the upstreaming status of this patch? Has this been discussed upstream somewhere?

[lukasz-czechowski]

Hi, the issue with missing inputs in VNC/RDP sessions still seem to be the case (meta-browser master branch as for 06.2024). What is the status of this patch?

[Lachlan-Frawley]

Honestly I completely forgot about this. I haven't attempted to upstream the patch and probably will never try. If there is interest I will rebase this patch onto the current master, although I strongly suspect the actual patch itself still works fine and could probably be added to existing projects with a bbappend.

[lukasz-czechowski]

Can we at least merge it into master?
I've verified that the patch itself works, but some issues still remain (in particular, the pointer is not visible) - I can provide fixes for those, but they depend on the original patch.

[Lachlan-Frawley]

I have yet to test my rebase of the patch, but I've pushed it to my fork for now anyway. I'll make sure to test this locally on my imx8 system at least and comment on the results when I'm done.

[Lachlan-Frawley]

I've finally managed to get VNC working on my imx8 device and verified the patch works as expected. What steps need to be taken to get this merged?

Also interestingly enough, I can see the cursor.

[MaxIhlenfeldt]

To be honest, I'd prefer this patch to be upstream, I think the chances of it being accepted are very high. If you upload a CL (instructions) I can send it to the right reviewer and take care of all the other organizational stuff :)

[Lachlan-Frawley]

Sure, I'll try to get to that soon when I have some time.

[Lachlan-Frawley]

I've submitted the work here: https://chromium-review.googlesource.com/c/chromium/src/+/5652465

[lukasz-czechowski]

I've finally managed to get VNC working on my imx8 device and verified the patch works as expected. What steps need to be taken to get this merged?

Also interestingly enough, I can see the cursor.

Thanks!
As for missing cursor, I'm observing this on my setup that uses Weston compositor. I found that assigning a cursor in void WaylandConnection::UpdateCursor() in wayland_connection.cc solves the problem.

[Lachlan-Frawley]

Interesting, I'm also using Weston. I'm using Scarthgap for my test distribution. What about you?

(I'm also using a Toradex im8qm as my test platform)

[Jino26]

I've finally managed to get VNC working on my imx8 device and verified the patch works as expected. What steps need to be taken to get this merged?
Also interestingly enough, I can see the cursor.

Thanks! As for missing cursor, I'm observing this on my setup that uses Weston compositor. I found that assigning a cursor in void WaylandConnection::UpdateCursor() in wayland_connection.cc solves the problem.

I'm also facing this issue of cursor not visible on Weston when I'm using Windows RDP tool. Can you give me more details about this fix that you've mentioned here?

[lukasz-czechowski]

I've finally managed to get VNC working on my imx8 device and verified the patch works as expected. What steps need to be taken to get this merged?
Also interestingly enough, I can see the cursor.

Thanks! As for missing cursor, I'm observing this on my setup that uses Weston compositor. I found that assigning a cursor in void WaylandConnection::UpdateCursor() in wayland_connection.cc solves the problem.

I'm also facing this issue of cursor not visible on Weston when I'm using Windows RDP tool. Can you give me more details about this fix that you've mentioned here?

Below I'm attaching the patch that helps in my case. I'm not sure if it's a proper solution, but it works in Scarthgap with Weston. Patch was tested on version 126.0.6478.126

The patch adds mouse, keyboard and touchscreen devices from additional seats to device list,
and considers additional seats when updating the cursor.
Because only one cursor seem to be supported in WaylandConnection, first available
cursor from additional seats list will be used in case there is no cursor in primary seat.

diff --git a/ui/ozone/platform/wayland/host/wayland_connection.cc b/ui/ozone/platform/wayland/host/wayland_connection.cc
index faddaf941f5..bfcce9272bc 100644
--- a/ui/ozone/platform/wayland/host/wayland_connection.cc
+++ b/ui/ozone/platform/wayland/host/wayland_connection.cc
@@ -390,6 +390,12 @@ std::vector<InputDevice> WaylandConnection::CreateMouseDevices() const {
     devices.emplace_back(pointer->id(), InputDeviceType::INPUT_DEVICE_UNKNOWN,
                          "pointer");
   }
+  for (auto& seat : extra_seats_) {
+    if (const auto* pointer = seat.second->pointer()) {
+      devices.emplace_back(pointer->id(), InputDeviceType::INPUT_DEVICE_UNKNOWN,
+                           "pointer");
+    }
+  }
   return devices;
 }
 
@@ -399,6 +405,12 @@ std::vector<KeyboardDevice> WaylandConnection::CreateKeyboardDevices() const {
     devices.emplace_back(keyboard->id(), InputDeviceType::INPUT_DEVICE_UNKNOWN,
                          "keyboard");
   }
+  for (auto& seat : extra_seats_) {
+    if (const auto* keyboard = seat.second->keyboard()) {
+      devices.emplace_back(keyboard->id(),
+                           InputDeviceType::INPUT_DEVICE_UNKNOWN, "keyboard");
+    }
+  }
   return devices;
 }
 
@@ -414,12 +426,31 @@ std::vector<TouchscreenDevice> WaylandConnection::CreateTouchscreenDevices()
                          "touch", gfx::Size(),
                          /*touch_points=*/10);
   }
+  for (auto& seat : extra_seats_) {
+    if (const auto* touch = seat.second->touch()) {
+      devices.emplace_back(touch->id(), InputDeviceType::INPUT_DEVICE_UNKNOWN,
+                           "touch", gfx::Size(),
+                           /*touch_points=*/10);
+    }
+  }
   return devices;
 }
 
 void WaylandConnection::UpdateCursor() {
+  std::unique_ptr<WaylandCursor> cursor;
   if (auto* pointer = seat_->pointer()) {
-    cursor_ = std::make_unique<WaylandCursor>(pointer, this);
+    cursor = std::make_unique<WaylandCursor>(pointer, this);
+  } else {
+    for (auto& seat : extra_seats_) {
+      if (auto* pointer = seat.second->pointer()) {
+        cursor = std::make_unique<WaylandCursor>(pointer, this);
+        break;
+      }
+    }
+  }
+
+  if (cursor) {
+    cursor_ = std::move(cursor);
     cursor_->set_listener(listener_);
     cursor_position_ = std::make_unique<WaylandCursorPosition>();