Add scrcpy window without video playback #4868

rom1v · 2024-04-20T16:37:20Z

Add the possibility to solely control the device without screen mirroring:

scrcpy --no-video --no-audio

This is different from OTG mode, which does not require USB debugging at all. Here, it is just the standard mode but with the possibility to disable video playback.

By default, always open a window (even without video playback), and add an option --no-window.

This impacts the behavior of some usages.

For example, the following command used to only play audio without video or controls:

scrcpy --no-video

Now, it opens a window, and allows to control it using the keyboard and mouse (by default, mouse mode is switched to UHID if video mirroring is disabled, because a relative mouse mode is required).

The controls must be disabled explicitly if necessary to play audio only:

scrcpy --no-video --no-control

To get the same behavior as before (play audio without window), disable the window:

scrcpy --no-window

(this implicitly set --no-control and --no-video-playback, which in turn set --no-video if there is no recording or V4L2 sink)

Fixes #4727
Fixes #4793

Here is a binary for Windows:

scrcpy-pr4868.4.zip _{SHA-256: b769c9bc403d91d78fae3efa87856dbe2af5905c8d3a33392beaf5fa7103c30}

old

scrcpy-pr4868.zip _{SHA-256: cf100b210e75c19d58f50b3c704d193a6ddbd45520928b7f83acc40a8246815}

There is no frame rate to count.

Add the possibility to only control the device with any keyboard and mouse mode without screen mirroring. This is different from OTG mode, which does not require USB debugging at all. Here, the standard mode is used but with the possibility to disable video playback. By default, always open a window (even without video playback), and add an option --no-window. Fixes #4727 <#4727> Fixes #4793 <#4793> PR #4868 <#4868>

PR #4868 <#4868>

AnshulJ999 · 2024-04-28T12:52:03Z

-KM --no-video --no-audio -d
scrcpy 2.4 https://github.com/Genymobile/scrcpy
INFO: ADB device found:
INFO: --> (usb) JZU4CURGLJA6P74H device Realme_Pad_LTE
INFO: (tcpip) 192.168.1.204:5555 device AFTSSS
C:\scrcpy\scrcpy-server: 1 file pushed, 0 skipped. 23.1 MB/s (69243 bytes in 0.003s)
[server] INFO: Device: [Realme] Realme Realme Pad LTE (Android 13)
INFO: Renderer: directed

Hey, so this is what I got when using the command you mentioned on Reddit. It seems the command works, and there's a window with an Android logo (same as OTG mode), but it quickly disappears and exits the command.

Also I'm not sure if this is possible yet via the --no-window or --no-video option, but just wanted to add this feature request from my Reddit comment:

A windowless approach to OTG mode (meaning it wouldn't be a window, it could exist in the system tray maybe), where a user-defined hotkey can be used to toggle between the host screen and the android device. This would make it very similar to deskdock.

I believe the rest of the things work already: ADB wifi works, UHID keyboards have gestures and hotkeys (unlike OTG mode).

Thanks.

rom1v · 2024-04-28T13:13:57Z

Thank you for your report.

but it quickly disappears and exits the command.

Without any error message?

I cannot reproduce the problem. Maybe because the binary was from an old version of this PR. Please test this new one:

scrcpy-pr4868.2.zip _{SHA-256: 2d0827e876485d5afb5476577953efc2d3df7fb81ee6e30092f5d94b41b125c}

Also, do you have the same issue with another device?

A windowless approach to OTG mode (meaning it wouldn't be a window, it could exist in the system tray maybe), where a user-defined hotkey can be used to toggle between the host screen and the android device. This would make it very similar to deskdock.

Technically, scrcpy receives events from the scrcpy window. If there is no window, I don't think it is possible to receive the events with SDL. It would only be possible by platform-specific means I guess.

rom1v · 2024-04-28T13:22:34Z

I cannot reproduce the problem.

OK, on Windows, I can reproduce. I will investigate.

Add the possibility to only control the device with any keyboard and mouse mode without screen mirroring: scrcpy -KM --no-video --no-audio This is different from OTG mode, which does not require USB debugging at all. Here, the standard mode is used but with the possibility to disable video playback. By default, always open a window (even without video playback), and add an option --no-window. Fixes #4727 <#4727> Fixes #4793 <#4793> PR #4868 <#4868>

PR #4868 <#4868>

rom1v · 2024-04-28T14:09:52Z

OK, with these two changes it should be fixed:

fix 1

diff --git a/app/src/screen.c b/app/src/screen.c
index cda562468..9ee61383c 100644
--- a/app/src/screen.c
+++ b/app/src/screen.c
@@ -272,6 +272,8 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
 static int
 event_watcher(void *data, SDL_Event *event) {
     struct sc_screen *screen = data;
+    assert(screen->video);
+
     if (event->type == SDL_WINDOWEVENT
             && event->window.event == SDL_WINDOWEVENT_RESIZED) {
         // In practice, it seems to always be called from the same thread in
@@ -477,7 +479,9 @@ sc_screen_init(struct sc_screen *screen,
     sc_input_manager_init(&screen->im, &im_params);
 
 #ifdef CONTINUOUS_RESIZING_WORKAROUND
-    SDL_AddEventWatch(event_watcher, screen);
+    if (screen->video) {
+        SDL_AddEventWatch(event_watcher, screen);
+    }
 #endif
 
     static const struct sc_frame_sink_ops ops = {

fix 2

diff --git a/app/src/input_manager.c b/app/src/input_manager.c
index cd2706559..5154d474e 100644
--- a/app/src/input_manager.c
+++ b/app/src/input_manager.c
@@ -626,6 +626,23 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     im->kp->ops->process_key(im->kp, &evt, ack_to_wait);
 }
 
+static struct sc_position
+sc_input_manager_get_position(struct sc_input_manager *im, int32_t x,
+                                                           int32_t y) {
+    if (im->mp->relative_mode) {
+        // No absolute position
+        return (struct sc_position) {
+            .screen_size = {0, 0},
+            .point = {0, 0},
+        };
+    }
+
+    return (struct sc_position) {
+        .screen_size = im->screen->frame_size,
+        .point = sc_screen_convert_window_to_frame_coords(im->screen, x, y),
+    };
+}
+
 static void
 sc_input_manager_process_mouse_motion(struct sc_input_manager *im,
                                       const SDL_MouseMotionEvent *event) {
@@ -635,12 +652,7 @@ sc_input_manager_process_mouse_motion(struct sc_input_manager *im,
     }
 
     struct sc_mouse_motion_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              event->x,
-                                                              event->y),
-        },
+        .position = sc_input_manager_get_position(im, event->x, event->y),
         .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE
                                              : POINTER_ID_GENERIC_FINGER,
         .xrel = event->xrel,
@@ -760,12 +772,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
     uint32_t sdl_buttons_state = SDL_GetMouseState(NULL, NULL);
 
     struct sc_mouse_click_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              event->x,
-                                                              event->y),
-        },
+        .position = sc_input_manager_get_position(im, event->x, event->y),
         .action = sc_action_from_sdl_mousebutton_type(event->type),
         .button = sc_mouse_button_from_sdl(event->button),
         .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE
@@ -840,11 +847,7 @@ sc_input_manager_process_mouse_wheel(struct sc_input_manager *im,
     uint32_t buttons = SDL_GetMouseState(&mouse_x, &mouse_y);
 
     struct sc_mouse_scroll_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              mouse_x, mouse_y),
-        },
+        .position = sc_input_manager_get_position(im, mouse_x, mouse_y),
 #if SDL_VERSION_ATLEAST(2, 0, 18)
         .hscroll = CLAMP(event->preciseX, -1.0f, 1.0f),
         .vscroll = CLAMP(event->preciseY, -1.0f, 1.0f),
diff --git a/app/src/screen.c b/app/src/screen.c
index 9ee61383c..11b4f58a1 100644
--- a/app/src/screen.c
+++ b/app/src/screen.c
@@ -962,6 +962,8 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
 struct sc_point
 sc_screen_convert_drawable_to_frame_coords(struct sc_screen *screen,
                                            int32_t x, int32_t y) {
+    assert(screen->video);
+
     enum sc_orientation orientation = screen->orientation;
 
     int32_t w = screen->content_size.width;

Please test this new binary:

scrcpy-pr4868.3.zip _{SHA-256: 1c1dea45e87ea101c29a314dd4e4994b12ce33aecbb95317bbfcd4ff642d365}

AnshulJ999 · 2024-04-28T14:25:15Z

Thank you! It's working now. And yes I'm on Windows 10.

Just wanted to clarify a few things:

So the scrcpy -KM --no-video --no-audio command basically is like OTG mode, but with the UHID keyboard/mouse.

The --no-window option removes the scrcpy window, so it is only used for audio playback. Nothing else seems to work.

But as you mentioned, the control events are not possible without a window? So --no-window can't be combined with -KM
it seems.

Anyway thank you, I will check if I can assign a hotkey with AHK or EventGhost and make this suit my workflow a bit more.

rom1v · 2024-04-28T14:33:53Z

So the scrcpy -KM --no-video --no-audio command basically is like OTG mode, but with the UHID keyboard/mouse.

The main purpose of OTG mode is to work without adb (USB debugging), not to only control the device.

With this PR, if you want to only control the device and have USB debugging enabled, you can.

OTG uses AOA, so the most similar command would be:

scrcpy --keyboard=aoa --mouse=aoa --no-video --no-audio

But it is better to use UHID:

scrcpy --keyboard=uhid --mouse=uhid --no-video --no-audio
scrcpy -KM --no-video --no-audio

The --no-window option removes the scrcpy window, so it is only used for audio playback.

In the current stable version, several features do not open a window at all: audio-playback only, recording, v4L2.
With this PR, these features now open a window, but there's a new option --no-window to restore the previous behavior. No window implies no control.

ristomatti · 2024-04-28T21:40:16Z

The options --window-height doesn't seem to work with this option, at least not with -KM --no-video --no-audio. My use case involves moving scrcpy's window off screen with just 1px shown. This creates an indicator where my phone is physically located next to the my monitor. 🙂

It works if I start with --keyboard aoa --mouse aoa --no-video --no-audio --otg --window-height=600. For me on Linux, this gives almost the same end result, except --legacy-paste only works when running with -KM --no-video --no-audio. Copy paste doesn't work without --legacy-paste on my setup. My phone is a Samsung Galaxy S23 / Android 14 (non-rooted).

ristomatti · 2024-04-28T21:44:51Z

there's a new option --no-window to restore the previous behavior. No window implies no control.

Is there a reason why this mode has to imply --no-control? If it didn't, it would be possible to run scrcpy from a keyboard shortcut to control the Android device and then kill the process using an OS shortcut. 😬 (although it might be that scrcpy would not then grab the keyboard and mouse, defeating the intended use)

rom1v · 2024-04-29T07:32:30Z

The options --window-height doesn't seem to work with this option

Thank you for the report. Fixed:

scrcpy-pr4868.4.zip _{SHA-256: b769c9bc403d91d78fae3efa87856dbe2af5905c8d3a33392beaf5fa7103c30}

except --legacy-paste only works when running with …

Yes, legacy paste processing is performed on the server side, and there is no server in OTG mode.

(although it might be that scrcpy would not then grab the keyboard and mouse, defeating the intended use)

That's it.

ristomatti · 2024-04-29T08:07:24Z

@rom1v Confirm, fixed. Thank you!

Add the possibility to solely control the device with any keyboard and mouse mode without screen mirroring: scrcpy -KM --no-video --no-audio This is different from OTG mode, which does not require USB debugging at all. Here, the standard mode is used but with the possibility to disable video playback. By default, always open a window (even without video playback), and add an option --no-window. Fixes #4727 <#4727> Fixes #4793 <#4793> PR #4868 <#4868>

PR #4868 <#4868>

This is particularly important to react to server socket disconnection since video and audio may be disabled. PR #4868 <#4868>

Add the possibility to solely control the device without screen mirroring: scrcpy --no-video --no-audio This is different from OTG mode, which does not require USB debugging at all. Here, the standard mode is used but with the possibility to disable video playback. By default, always open a window (even without video playback), and add an option --no-window. Fixes #4727 <#4727> Fixes #4793 <#4793> PR #4868 <#4868>

PR #4868 <#4868>

This is particularly important to react to server socket disconnection since video and audio may be disabled. PR #4868 <#4868>

rom1v · 2024-05-11T15:16:28Z

I fixed some bugs and merged into dev.

Disable FPS counter when no video playback

cca2c9f

There is no frame rate to count.

This was referenced Apr 20, 2024

UHID does not work the same as OTG #4727

Open

Wireless OTG mode #4793

Open

rom1v changed the title ~~Add --no-window~~ Add scrcpy window without video playback Apr 20, 2024

rom1v added a commit that referenced this pull request Apr 23, 2024

Update documentation for --no-window

d6e001a

PR #4868 <#4868>

rom1v force-pushed the window branch from b0bfc9d to d6e001a Compare April 23, 2024 07:54

rom1v mentioned this pull request Apr 28, 2024

Input only via tcpip #4891

Closed

rom1v added a commit that referenced this pull request Apr 28, 2024

Update documentation for --no-window

2f41c47

PR #4868 <#4868>

rom1v force-pushed the window branch from d6e001a to 2f41c47 Compare April 28, 2024 08:23

rom1v force-pushed the window branch from 2f41c47 to be9bf7d Compare April 28, 2024 14:07

rom1v added a commit that referenced this pull request Apr 28, 2024

Update documentation for --no-window

64f7c72

PR #4868 <#4868>

rom1v force-pushed the window branch from be9bf7d to e250628 Compare April 29, 2024 07:29

rom1v mentioned this pull request May 2, 2024

OBS unable capture scrcpy audio when --no-video option is used #4897

Closed

2 tasks

rom1v added a commit that referenced this pull request May 11, 2024

Update documentation for --no-window

b5ed834

PR #4868 <#4868>

rom1v added a commit that referenced this pull request May 11, 2024

Terminate on controller error

c875465

This is particularly important to react to server socket disconnection since video and audio may be disabled. PR #4868 <#4868>

rom1v force-pushed the window branch 2 times, most recently from c875465 to b5ed834 Compare May 11, 2024 14:53

rom1v added a commit that referenced this pull request May 11, 2024

Terminate on controller error

14348f4

This is particularly important to react to server socket disconnection since video and audio may be disabled. PR #4868 <#4868>

rom1v added 3 commits May 11, 2024 17:06

Update documentation for --no-window

b5c8de0

PR #4868 <#4868>

Terminate on controller error

063a833

This is particularly important to react to server socket disconnection since video and audio may be disabled. PR #4868 <#4868>

rom1v force-pushed the window branch from 14348f4 to 063a833 Compare May 11, 2024 15:13

rom1v merged commit 063a833 into dev May 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add scrcpy window without video playback #4868

Add scrcpy window without video playback #4868

rom1v commented Apr 20, 2024 •

edited

AnshulJ999 commented Apr 28, 2024

rom1v commented Apr 28, 2024

rom1v commented Apr 28, 2024

rom1v commented Apr 28, 2024 •

edited

AnshulJ999 commented Apr 28, 2024

rom1v commented Apr 28, 2024

ristomatti commented Apr 28, 2024

ristomatti commented Apr 28, 2024

rom1v commented Apr 29, 2024

ristomatti commented Apr 29, 2024

rom1v commented May 11, 2024

Add scrcpy window without video playback #4868

Add scrcpy window without video playback #4868

Conversation

rom1v commented Apr 20, 2024 • edited

AnshulJ999 commented Apr 28, 2024

rom1v commented Apr 28, 2024

rom1v commented Apr 28, 2024

rom1v commented Apr 28, 2024 • edited

AnshulJ999 commented Apr 28, 2024

rom1v commented Apr 28, 2024

ristomatti commented Apr 28, 2024

ristomatti commented Apr 28, 2024

rom1v commented Apr 29, 2024

ristomatti commented Apr 29, 2024

rom1v commented May 11, 2024

rom1v commented Apr 20, 2024 •

edited

rom1v commented Apr 28, 2024 •

edited