Firebase Robo Test doesn't follow the Robo Script created in Android Studio

3
votes

I create the roboscript from android studio and upload the created json file into Robo Test lab along with the latest apk file for negative testing (using wrong credentials). But all test in the Firebase Robo Test failed and when I check the videos, the Robo Test just click randomly instead, not as guided. Anyone can please help on this issue? Thank you so much!

Below are the roboscript used :

[ { "eventType": "VIEW_CLICKED", "timestamp": 1604457186524, "replacementText": "Skip", "actionCode": -1, "delayTime": 0, "canScrollTo": false, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatTextView", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 2, "resourceId": "com.kim.themobile.dev:id/tvSkipBtn", "contentDescription": "", "text": "Skip" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "", "contentDescription": "", "text": "" }, { "className": "androidx.appcompat.widget.ContentFrameLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "android:id/content", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_TEXT_CHANGED", "timestamp": 1604457315709, "replacementText": "(my email : correct email)", "actionCode": -1, "delayTime": 0, "canScrollTo": true, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatEditText", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/etEmail", "contentDescription": "", "text": "" }, { "className": "android.widget.LinearLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/llEdittext", "contentDescription": "", "text": "" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_TEXT_CHANGED", "timestamp": 1604457570870, "replacementText": "(my password-wrong password for invalid login)", "actionCode": -1, "delayTime": 0, "canScrollTo": false, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatEditText", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/etPassword", "contentDescription": "", "text": "" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/rlPassword", "contentDescription": "", "text": "" }, { "className": "android.widget.LinearLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/llEdittext", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_CLICKED", "timestamp": 1604457800428, "replacementText": "Log In", "actionCode": -1, "delayTime": 0, "canScrollTo": true, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatButton", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/btnSubmitLogin", "contentDescription": "", "text": "Log In" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 4, "resourceId": "com.kim.themobile.dev:id/rlbuttons", "contentDescription": "", "text": "" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_CLICKED", "timestamp": 1604457812228, "replacementText": "", "actionCode": -1, "delayTime": 0, "canScrollTo": false, "elementDescriptors": [ { "className": "android.widget.ImageView", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "com.kim.themobile.dev:id/ivRevealPw", "contentDescription": "Show Password", "text": "" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/rlPassword", "contentDescription": "", "text": "" }, { "className": "android.widget.LinearLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/llEdittext", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_TEXT_CHANGED", "timestamp": 1604457818109, "replacementText": "(my password)", "actionCode": -1, "delayTime": 0, "canScrollTo": false, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatEditText", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/etPassword", "contentDescription": "", "text": "Hnm234!@" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/rlPassword", "contentDescription": "", "text": "" }, { "className": "android.widget.LinearLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 3, "resourceId": "com.kim.themobile.dev:id/llEdittext", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_CLICKED", "timestamp": 1604457894026, "replacementText": "Log In", "actionCode": -1, "delayTime": 0, "canScrollTo": true, "elementDescriptors": [ { "className": "androidx.appcompat.widget.AppCompatButton", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/btnSubmitLogin", "contentDescription": "", "text": "Log In" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 4, "resourceId": "com.kim.themobile.dev:id/rlbuttons", "contentDescription": "", "text": "" }, { "className": "android.widget.RelativeLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "", "contentDescription": "", "text": "" } ] }, { "eventType": "VIEW_CLICKED", "timestamp": 1604457961623, "replacementText": "", "actionCode": -1, "delayTime": 0, "canScrollTo": false, "elementDescriptors": [ { "className": "com.google.android.material.tabs.TabLayout$TabView", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 1, "resourceId": "", "contentDescription": "Top Stories", "text": "" }, { "className": "com.google.android.material.tabs.TabLayout$SlidingTabIndicator", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "", "contentDescription": "", "text": "" }, { "className": "com.google.android.material.tabs.TabLayout", "recyclerViewChildPosition": -1, "adapterViewChildPosition": -1, "groupViewChildPosition": 0, "resourceId": "com.kim.themobile.dev:id/tlTab", "contentDescription": "", "text": "" } ] } ]

1
androidjsonfirebasefirebase-test-lab

1 Answers

0
votes

What action does your Roboscript fail at? Your Robo run results in Firebase Test Lab mention how many Roboscript actions were performed successfully. Knowing that, you could truncate your Roboscript accordingly and experiment with the failing action (e.g., change its elementDescriptors).

Another way to debug this is to record an equivalent Espresso test in Android Studio using Espresso Test Recorder (from the main menu, Run -> Record Espresso Test). Running this test in Android Studio should produce a similar outcome to running the corresponding Roboscript with Robo in Firebase Test Lab.

Also, make sure that your app looks and behaves the same way during recording and replaying phases. In particular, your app should have the same app package name for debug and release APKs and show the same screens during recording and replaying phases.