adb shell cmd list packages fails while creating driver session

[itkhanz]

I have recently upgraded my Appium UiAutomator2 driver version from 2.42.1 to 3.5.2, and I encountered following error when I try to create the instance of AndroidDriver with UiAutomator2.

USB debugging under Developer options is already enabled and everything was working fine before upgrading driver version as follows:-

appium driver uninstall uiautomator2
appium driver install [email protected]

I have even tried by upgrading the Appium UiAutomator2 driver to latest version i.e. 3.7.5 but it still does not work and throws the same error. When I downgrade back to the UiAutomator2 version 2.42.1 then it starts working. Also I do not get this error on emulator or when using cloud provider real devices.

I have gone through the similiar issue raised here appium/appium#20324 but I do not have the same developer mode settings, and i have Samsung device and the problem only happend locally on my real device when I use driver version 3.5.2 or higher.

Client side error

java.lang.RuntimeException: org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: An unknown server-side error occurred while processing the command. Original error: Error executing adbExec. Original error: 'Command '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages' exited with code 255'; Command output: 
Exception occurred while executing 'list':
java.lang.SecurityException: Shell does not have permission to access user 150
 com.android.server.am.ActivityManagerService.handleIncomingUser:15012 android.app.ActivityManager.handleIncomingUser:4805 com.android.server.pm.PackageManagerShellCommand.translateUserId:3517 
	at com.android.server.am.UserController.handleIncomingUser(UserController.java:2680)
	at com.android.server.am.ActivityManagerService.handleIncomingUser(ActivityManagerService.java:15012)
	at android.app.ActivityManager.handleIncomingUser(ActivityManager.java:4805)
	at com.android.server.pm.PackageManagerShellCommand.translateUserId(PackageManagerShellCommand.java:3517)
	at com.android.server.pm.PackageManagerShellCommand.runListPackages(PackageManagerShellCommand.java:969)
	at com.android.server.pm.PackageManagerShellCommand.runListPackages(PackageManagerShellCommand.java:865)
	at com.android.server.pm.PackageManagerShellCommand.runList(PackageManagerShellCommand.java:724)
	at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:226)
	at com.android.modules.utils.BasicShellCommandHandler.exec(BasicShellCommandHandler.java:97)
	at android.os.ShellCommand.exec(ShellCommand.java:38)
	at com.android.server.pm.PackageManagerService$IPackageManagerImpl.onShellCommand(PackageManagerService.java:7071)
	at android.os.Binder.shellCommand(Binder.java:1085)
	at android.os.Binder.onTransact(Binder.java:903)
	at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4962)
	at com.android.server.pm.PackageManagerService$IPackageManagerImpl.onTransact(PackageManagerService.java:7055)
	at android.os.Binder.execTransactInternal(Binder.jav

Appium Server logs

[ADB] Getting install status for com.digitalchargingsolutions.b2c.integ
[ADB] Running '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages'
[Logcat] Stopping logcat capture
[AppiumDriver@cccc] Event 'newSessionStarted' logged at 1722943684073 (13:28:04 GMT+0200 (Central European Summer Time))
[AppiumDriver@cccc] Encountered internal error running command: Error executing adbExec. Original error: 'Command '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages' exited with code 255'; Command output: 
Exception occurred while executing 'list':
java.lang.SecurityException: Shell does not have permission to access user 150
 com.android.server.am.ActivityManagerService.handleIncomingUser:15012 android.app.ActivityManager.handleIncomingUser:4805 com.android.server.pm.PackageManagerShellCommand.translateUserId:3517 
	at com.android.server.am.UserController.handleIncomingUser(UserController.java:2680)
	at com.android.server.am.ActivityManagerService.handleIncomingUser(ActivityManagerService.java:15012)
	at android.app.ActivityManager.handleIncomingUser(ActivityManager.java:4805)
	at com.android.server.pm.PackageManagerShellCommand.translateUserId(PackageManagerShellCommand.java:3517)
	at com.android.server.pm.PackageManagerShellCommand.runListPackages(PackageManagerShellCommand.java:969)
	at com.android.server.pm.PackageManagerShellCommand.runListPackages(PackageManagerShellCommand.java:865)
	at com.android.server.pm.PackageManagerShellCommand.runList(PackageManagerShellCommand.java:724)
	at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:226)
	at com.android.modules.utils.BasicShellCommandHandler.exec(BasicShellCommandHandler.java:97)
	at android.os.ShellCommand.exec(ShellCommand.java:38)
	at com.android.server.pm.PackageManagerService$IPackageManagerImpl.onShellCommand(PackageManagerService.java:7071)
	at android.os.Binder.shellCommand(Binder.java:1085)
	at android.os.Binder.onTransact(Binder.java:903)
	at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4962)
	at com.android.server.pm.PackageManagerService$IPackageManagerImpl.onTransact(PackageManagerService.java:7055)
	at android.os.Binder.execTransactInternal(Binder.java:1321)
	at android.os.Binder.execTransact(Binder.java:1280)

Error: Command '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages' exited with code 255
    at ChildProcess.<anonymous> (/Users/ibtisam.khan/.appium/node_modules/appium-uiautomator2-driver/node_modules/teen_process/lib/exec.js:110:19)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:305:5)
[HTTP] <-- POST /session 500 755 ms - 4614 

When I try to manually execute following command directly with user 0 then it works
adb shell cmd package list packages --user 0
but otherwise it does not work. I do not understand where this problem has come from after updating driver version despite having same device and device settings.

Environment:

Appium server 2.6.0
Appium UiAutomator driver 3.5.2
Appium Java client 9.3.0
Samsung Galaxy S20 - Android 13.0 (real device)

[mykola-mokhnach]

I assume the issue might have to do with the fact the device is included into an MDM profile.

[mykola-mokhnach]

I've added the workaround to uia2 driver v3.7.6
Please check the above scenario with the this version

[itkhanz]

Thank you for the swift resolution.

In the meantime I already did some googling and uninstalled Secure Folder from apps as mentioned here (although in the beginning I was hesitant to try this solution because there was no apparent link):
https://android.stackexchange.com/a/250553

It resolved the issue for me, and now it is working with UiAutomator2 driver version 3.5.2.
Although I still do not understand the underlying reason for any possible connection of this error with uninstalling secure folder, or having called the adb command with user 150.

I will check with the driver version 3.7.6 while having Secure Folder installed and see it works.

[mykola-mokhnach]

This issue definitely has to do with some Android internals, in particular the package manager service logic and permissions. UIA2 driver is a simple/dumb CLI client in that scenario.
Let us know if you have any additional findings there.

[itkhanz]

Works with version 3.7.6 when Seucre Folder is installed on Samsung devices.
It successfully launched the app session even when Seucre Folder is installed on Samsung device.

Here are appium server logs where it apparently retries with appending --user 0 which has administrative/root privileges of all system apps and core functions.

[ADB] There is no need to install/upgrade '/Users/ibtisam.khan/dcs/dev/dcs-app-automated-tests/apps/android/app-brandB2C-envIntegration-beta.apk'
[AndroidUiautomator2Driver@d7fc] Performing fast reset on 'com.digitalchargingsolutions.b2c.integ'
[ADB] Getting install status for com.digitalchargingsolutions.b2c.integ
[ADB] Running '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages'
[ADB] Running '/Users/ibtisam.khan/Library/Android/sdk/platform-tools/adb -P 5037 -s RF8N90E8ZBR shell cmd package list packages --user 0'
[ADB] 'com.digitalchargingsolutions.b2c.integ' is installed

For driver versions below than 3.7.6, I tested with uninstalling Secure Folder and it also works fine. When Secure Folder is installed it gives the same error with incremented user i.e.

Exception occurred while executing 'list':
java.lang.SecurityException: Shell does not have permission to access user 151

Since I want to keep the driver versions in sync with the driver versions of cloud provider so I do not encounter any issues when testing locally due to driver version mismatch, I will keep using 3.5.2 and uninstall Secure Folder as I do not need it. Not sure if there is an easy way to have multiple driver versions installed, and use any specific version by providing the version as command line argument when starting appium.

To uninstall the seucre folder on Samsung, here are the steps
Settings -> security and privacy -> Secure Folder -> More settings -> Uninstall

Big Thanks.

[itkhanz]

This issue definitely has to do with some Android internals, in particular the package manager service logic and permissions. UIA2 driver is a simple/dumb CLI client in that scenario. Let us know if you have any additional findings there.

Here are some more findings as to why having Secure Folder causes this issue which I came across while reading:
expo/expo#22473 (comment)
expo/expo#20636

Quoting the answer from below StackOverFlow response:-
https://stackoverflow.com/a/74960143

This error may be caused by the fact that some devices support multiple user accounts and adb attempts installing the build on all accounts. However, one of the users (user 150) cannot be manipulated from adb due to restricted privileges from the device manufacturers. This happened to me recently after updating my Samsung S21+ to OneUI 5, I could no longer run expo dev-client builds.

I also updated my Samsung device recently (can't remember when) and am using Samsung One UI version 5.1 which may have caused this issue for me because the way newer version of UiAutomator2 driver executed ADB commands.

[itkhanz]

Also in a comment to StackOverFlow, another user has pointed similar cause for this issue. Quoting Firelord's response to original question:

Samsung has restricted privileges of adb (shell) user from making changes to packages installed under user 150. There is nothing we can do. This is a design choice (probably to ensure bloat continues to float in the device). Rooting is the only solution to remove bloatware in such situations.

https://android.stackexchange.com/q/243104