Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sets the policy for how the display should show IME. #5703

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

Mr-JingShi
Copy link

Hi,bro.
Scrcpy is very good, I like it.

I want to display IME in virtualdisplay。

Possible values are "local", "fallback_display" and "hide".
Copy link
Collaborator

@rom1v rom1v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your PR. After a quick glance, it looks very good and clean (with all details implemented) 👍

As a French speaker, I have never needed to use an IME, so I'm not very familiar with IME policies. I'd like to get opinions from IME users 🙂

There are 3 options (local, fallback, hide), is it absolutely necessary to expose them? (Maybe yes, I'm just asking.)
Or isn't there an "obvious" default option to set without asking the user?

For example, are there cases where you need to use an IME on your main display if you're mirroring a virtual display?

Wouldn't it be sufficient to call unconditionnally in NewDisplayCapture:

ServiceManager.getWindowManager().setDisplayImePolicy(virtualDisplayId, DISPLAY_IME_POLICY_LOCAL)`

@@ -100,15 +101,27 @@ private void runCleanUp(Options options) {
boolean powerOffScreen = options.getPowerOffScreenOnClose();
int displayId = options.getDisplayId();

int restoreDisplayImePolicy = -1;
if (displayId > 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only for secondary displays? (not the main display?)

Copy link
Author

@Mr-JingShi Mr-JingShi Dec 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the main display must be able to display IME normally.
I think We shouldn't modify the main display settings。

if (setDisplayImePolicyMethod == null) {
try {
setDisplayImePolicyMethod = manager.getClass().getMethod("setDisplayImePolicy", int.class, int.class);
} catch (NoSuchMethodException e) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change between shouldShowIme() and getDisplayImePolicy() was performed by this commit introduced in Android 12, so it can be checked directly with Build.VERSION.SDK_INT < AndroidVersions.API_31_ANDROID_12.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

@Mr-JingShi
Copy link
Author

Mr-JingShi commented Dec 27, 2024

Thank you for your PR. After a quick glance, it looks very good and clean (with all details implemented) 👍

As a French speaker, I have never needed to use an IME, so I'm not very familiar with IME policies. I'd like to get opinions from IME users 🙂

There are 3 options (local, fallback, hide), is it absolutely necessary to expose them? (Maybe yes, I'm just asking.) Or isn't there an "obvious" default option to set without asking the user?

For example, are there cases where you need to use an IME on your main display if you're mirroring a virtual display?

Wouldn't it be sufficient to call unconditionnally in NewDisplayCapture:

ServiceManager.getWindowManager().setDisplayImePolicy(virtualDisplayId, DISPLAY_IME_POLICY_LOCAL)`

Thank you for your replay.

I have a project about Huawei electric car,the project sometimes needs to input text then search on secondary display, and then notify the main driver(main display).Huawei electric cars are very expensive, and not every developer can have one.
so we developed an app named secondaryscreen instead of Huawei electric cars.

The secondary display is TYPE_HDMI on Huawei electric car, IME show in secondary display.
Android Studio-Add seconday display is TYPE_VIRTUAL, IME show in secondary display.
Developer options-Simulate secondary displays is TYPE_OVERLAY, IME show in main display.
we create a virtualdisplay in my app, the virtualdisplay is TYPE_VIRTUAL, But IME show in main display.

I'm very confused, then I find Developer options-Force desktop mode, it can show IME in secondary display.
But Some phones don't have this feature.

We know,the main display must be able to display IME normally,so,I just want to display IME in secondary display when I touch in secondary display such as Huawei electric car and Android Studio‘s secondary display。

/**
 * Virtual display flag: Indicates that the display should support system decorations. Virtual
 * displays without this flag shouldn't show home, navigation bar or wallpaper.
 * <p>This flag doesn't work without {@link #VIRTUAL_DISPLAY_FLAG_TRUSTED}</p>
 *
 * @see #createVirtualDisplay
 * @see #VIRTUAL_DISPLAY_FLAG_TRUSTED
 * @hide
 */
@TestApi
public static final int VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS = 1 << 9;

/**
 * Virtual display flags: Indicates that the display is trusted to show system decorations and
 * receive inputs without users' touch.
 *
 * @see #createVirtualDisplay
 * @see #VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
 * @hide
 */
@SystemApi
public static final int VIRTUAL_DISPLAY_FLAG_TRUSTED = 1 << 10;

If the secondary screen wants to display IME, it must has the VIRTUAL_DISPLAY_FLAG_TRUSTED and VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS flag, otherwise it will report a permission problem.

java.lang.SecurityException: Attempted to set IME flag to an untrusted virtual display: 1

The virtualdisplay that created by my app does not have VIRTUAL_DISPLAY_FLAG_TRUSTED in Android 11 、12.

/**
 * @return {@code true} if the display is non-system created virtual display.
 */
boolean isUntrustedVirtualDisplay() {
    return mDisplay.getType() == Display.TYPE_VIRTUAL
            && mDisplay.getOwnerUid() != Process.SYSTEM_UID;
}

Although Android 10 does not have a VIRTUAL_DISPLAY_FLAG_TRUSTED flag, But Android 10 will check OwnerUid.
So, Not all virtual display can be set up successfully. then the default value will be meaningless(local、fallback or hide).
I think it is better to let users choose the IME display strategy, If virtualdisplay don't have permission, report an error to the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants