Regression Testing — JSON Scenario Guide
Describe your app's critical flow once, run it on a real device any time.
How it works
On your app's detail page, there's a Regression Testing card. Paste or upload a JSON file describing the steps of a critical flow — login, search, checkout, whatever matters most — and we run it step-by-step on a real device that already has your app installed. You get back a pass/fail result for every step, plus a screenshot at each point, within a couple of minutes.
This does not accept code (no Selenium scripts, no arbitrary programs) — only the declarative JSON format described below. That's intentional: it keeps things safe and predictable, and it's usually faster to write than a real test script anyway.
Basic structure
A scenario is a JSON object with a name and a list of steps:
{
"name": "Login flow",
"steps": [
{ "action": "...", ... },
{ "action": "...", ... }
]
}
package field — we always
run the scenario against your app's own package automatically, so there's no way to
accidentally target a different app.
Actions
Each step's action must be one of these seven:
| Action | Required fields | What it does |
|---|---|---|
assert_visible |
selector |
Fails the step if the element isn't on screen. Good for "did the right screen load". |
tap |
selector |
Taps the first element matching the selector. |
type_active |
text |
Types into whatever field currently has focus (e.g. right after tapping it). |
type |
selector, text |
Finds the element first, then types into it. |
screenshot |
name (optional) |
Captures the current screen. Shows up in your results regardless of pass/fail. |
wait |
seconds (optional, default 2, max 10) |
Pauses — useful after a tap that triggers loading/animation. |
back |
— | Presses the device back button. |
description is optional on every step — add it to make your results easier to read (it shows up next to the pass/fail icon).
Selectors
For assert_visible, tap and type, you point at an element with a selector object — use exactly one of these four:
| Selector | Example | Matches |
|---|---|---|
text_contains |
{"text_contains": "Log In"} |
Any element whose visible text contains this substring. The one you'll use most. |
text_exact |
{"text_exact": "Log In"} |
An element whose text matches exactly — use when text_contains is too loose (e.g. "Log In" vs "Log In with Google"). |
resource_id |
{"resource_id": "com.yourapp:id/login_button"} |
An element by its Android resource ID, if you know it (most reliable, but requires access to your app's layout IDs). |
xpath |
{"xpath": "//android.widget.Button[2]"} |
Raw XPath, for anything the other three can't express. |
Full example — a login flow
{
"name": "Login screen — phone number flow",
"steps": [
{
"action": "assert_visible",
"selector": { "text_contains": "Log In" },
"description": "Login screen loaded"
},
{
"action": "tap",
"selector": { "text_contains": "Phone Number" },
"description": "Tapped the phone number field"
},
{
"action": "type_active",
"text": "5551234567",
"description": "Typed a test phone number"
},
{
"action": "screenshot",
"name": "number_entered"
},
{
"action": "assert_visible",
"selector": { "text_contains": "Send Code" },
"description": "'Send Code' button appeared"
},
{
"action": "back",
"description": "Went back"
},
{
"action": "screenshot",
"name": "after_back"
}
]
}
Limits
- Up to 30 steps per scenario — keep it to one focused flow, not your whole app.
waitis capped at 10 seconds per step.- One scenario = one run = one credit. If the device happens to be busy with a scheduled test right when you submit, we automatically retry on another device and don't charge you a credit for it.