1
votes

I'm getting the xml dump from different apps running the uiautomator dump command within a service of an app i built, in a rooted phone. Everything works fine when Chrome or Android System WebView are set as WebView implementation with a version different to 84. When version is 84, only in WebView apps, I can't get the views, the xml only shows that there's a WebView but doesn't show it's child elements.

I tried the same with Appium (just for testing purposes, in case of me missing something, because I need to get the xml within my app without being plugged to a pc), and the behavior is the same (as far as I know, Appium also uses uiautomator to dump the views).

I have tried in Android 8 with Chrome 81.0.4044.138, and in Android 9 with Android System WebView 77.0.3865.92, and works fine. When I update the Android 9 phone Android System WebView to most recent version (84.0.4147.125), cant't get elements from WebView apps, neither with Chrome, just get the WebView element.

What I want to know is if the last Android WebView versions (both Chrome and Android System, version 84) have something new that I'm missing, that don't let uiautomator to dump the xml file correctly from WebView-based apps. Maybe if I have to do something else, or if it's a bug. Thank you!

1
did you get the solution for this issue? - Pramod Nikule
No, only "solution" for now is to change Chrome version. I'll try to get the window dump using accessibility service and see if this way works. - Gabriel BA
I am getting the same issue on multiple device with Chrome version 83. I downgraded the chrome version and it is working fine for most of the devices. But for Android-11 device, the default version of chrome is 83 which can't be downgraded. Did you find any issue already reported for it? - Pramod Nikule
Didn't find any issue, but found a way of getting better results using AccessibilityService, i'll post an answer. - Gabriel BA

1 Answers

0
votes

I found a faster and non-root way of getting the view hierarchy that also works fine in most Chrome versions (in case of webview-based apps), even in Chrome 80+, and it was ussing AccesibilityService.

So, i first created a service which extends AccessibilityService:

public class BasicAccessibilityService extends AccessibilityService {

    private static BasicAccessibilityService instance;

    public static BasicAccessibilityService getInstance(){
        return instance;
    }

   
    @Override
    protected void onServiceConnected() {
        instance = this;
        super.onServiceConnected();
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        return super.onStartCommand(intent, flags, startId);
    }

    @Override
    public void onAccessibilityEvent(AccessibilityEvent event) {
    }

    @Override
    public void onInterrupt() {
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
    }

    @Override
    public void onCreate() {
        super.onCreate();
    }

    @Override
    public AccessibilityNodeInfo getRootInActiveWindow() {
        try {
            return super.getRootInActiveWindow();
        } catch (Throwable ignored) {
            returenter code heren null;
        }
    }
}

In Manifest:

<service
    android:name="BasicAccessibilityService"
    android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
    <intent-filter>
        <action android:name="android.accessibilityservice.AccessibilityService" 
    />
    </intent-filter>
        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/service_config" />
</service>

In service_config.xml:

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:description="@string/accessibility_service_description"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFlags="flagReportViewIds|flagDefault|flagRequestFilterKeyEvents|flagRetrieveInteractiveWindows|flagEnableAccessibilityVolume"
    android:canRetrieveWindowContent="true"
    android:canRequestTouchExplorationMode="true"
    android:accessibilityFeedbackType="feedbackGeneric"
    android:canPerformGestures="true"
    android:canRequestFilterKeyEvents="true"
    android:notificationTimeout="0"
    />

Now the Accessibility Service is set up, and can be called whenever we want to get the view hierarchy. Remember that for it to work the app must have the accessibility permission granted. When the permission is granted the onServiceConnected method is automatically called, so we can call our service from it instance and get the views like:

BasicAccessibilityService.getInstance().getRootInActiveWindow()

This will return an AccessibilityNodeInfo which contains the current views information.

If you need to get the view hierarchy as an XML, you can use the AccessibilityNodeInfoDumper class from the uiautomator source code, which can be found here:

https://android.googlesource.com/platform/frameworks/uiautomator/+/5fe6e7f321f02b08224fad72374ed041f459b411/core/com/android/uiautomator/core/AccessibilityNodeInfoDumper.java

Also you will have to add to your code the AccessibilityNodeInfoHelper found in here:

https://android.googlesource.com/platform/frameworks/uiautomator/+/5fe6e7f321f02b08224fad72374ed041f459b411/core/com/android/uiautomator/core/AccessibilityNodeInfoHelper.java

When i implemented the above code to parse the AccessibilityNodeInfo to XML, i got almost the same problem i had before when getting the views with uiautomator dump. I found that the problem was in the isVisibleToUser method within dumpNodeRec in the AccessibilityNodeInfoDumper class.

For some reason, with some apps the isVisibleToUser method returns false with views that are visible inside webview-based application, and this behaviour changes depending on Chrome or System Web View version. So a possible solution is to not use that method and obtain all views, even those who are "invisible", or another option is to make your own method to determine if some view is visible or not.

For me was enought commenting the line that use the isVisibleToUser method. Hope this helps anyone who needs to do something similar.