I built a mobile test automation practice for a team that had no automated tests at all. It now covers 90+ customer journeys, driving a real build unattended in about twelve minutes, in two languages, and every run ends in a computed verdict backed by a screenshot at every checkpoint. Its job is simple to state: make "can we ship?" a reading rather than a feeling.
The production side of that system, regression runs that end in GO/NO-GO release decisions backed by screenshots and real-app evidence, is written up on the Akkodis blog. This post is the layer underneath it: the methodology. What to automate, to what standard, and how to keep a suite alive as the app changes. Examples are invented and generic, and they use Flutter's integration_test package; the patterns port to any E2E driver.
Automate journeys, not screens
The first decision that shapes everything else: the unit of automation is a customer journey, not a screen and not a widget. A journey is something a user came to the app to do, end to end: sign in, find a product, pay, see the confirmation. If it fails, a real person did not get what they came for. That is the definition of a release blocker, so it is also the definition of a test.
Starting from zero, do not inventory screens. Inventory journeys, then order them by what a failure costs. Money paths first: sign-in, search, purchase, anything that touches payment or entitlements. The first ten journeys on that list protect more of the business than the next hundred.
// integration_test/journeys/checkout_test.dart
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
testWidgets('guest can search, add to cart and pay', (tester) async {
final app = await AppRobot.launch(tester, signedIn: false);
await app.search.query('notebook');
await app.catalog.openFirstResult();
await app.product.addToCart();
await app.cart.checkout();
await app.payment.payWithCard(TestCards.valid);
await app.orders.expectConfirmation();
});
}
The first journey I automated was login and OTP. Everything else in the app sits behind it, and it carries negative paths from day one: a wrong code entered more than once, a resend, a retry. Getting that one journey to run reliably taught the harness most of what it needed before checkout ever came into the picture.
A journey test reads like the user story it protects. That is deliberate, and it matters later: when this fails at 3am, the failure name alone tells the morning triage what is broken for users.
The coverage standard
The trap most young suites fall into: a hundred tests, all of them happy paths. The demo works and production still burns, because production users mistype passwords, lose signal in checkout and get their cards declined. So the practice defines coverage per journey, in three lanes, and a journey only counts as covered when all three are automated:
- Happy path: valid input, healthy backend. Necessary, and the least likely place to find a release blocker.
- Negative paths: the backend or the user says no. Declined payment, wrong password, expired session, empty results. The app must explain what happened, offer a way forward, and lose nothing the user built up.
- Edge cases: the world misbehaves. Process death mid-flow, airplane mode at checkout, a slow network, first launch on a clean device. This lane is where real-world crash reports actually come from.
A negative-path test asserts recovery, not just error text:
testWidgets('declined card keeps the cart intact', (tester) async {
final app = await AppRobot.launch(
tester,
backend: FakeBackend()..declineNextPayment(),
);
await app.cart.fillWith(items: 2);
await app.cart.checkout();
await app.payment.payWithCard(TestCards.valid);
// The user is told what happened, in words, on screen.
await app.payment.expectDeclineExplanation();
// And loses nothing: the cart survives, retry is one tap away.
await app.cart.expectItemCount(2);
});
The edge-case lane earned its place early. One resilience epic does nothing but misuse the app: rapid repeated taps, the wrong OTP entered again and again, interrupting the flow and relaunching, dropping the network in the middle of a step. One of those scenarios surfaced a reproducible crash on both platforms, the kind nobody writes a happy-path test for and every real user eventually triggers.
Holding this standard is the practice. Anyone can automate a happy path in an afternoon; a team that automates the decline, the timeout and the relaunch has decided what quality means and written it down as executable truth.
Tests that survive the app changing
E2E suites die of maintenance, not of technology. The killer is coupling: ninety tests that each know how the payment screen is built become ninety edits every time it changes, and six months later someone quietly deletes the suite. The countermeasure is old and boring: every screen gets one robot (call it a page object if you prefer), and only robots touch finders.
// integration_test/robots/payment_robot.dart
class PaymentRobot {
PaymentRobot(this.tester);
final WidgetTester tester;
// One place knows how the payment screen is built. When the
// UI changes, one file changes, not ninety tests.
Future<void> payWithCard(TestCard card) async {
await tester.tap(find.byKey(const Key('payment-saved-card')));
await tester.tap(find.byKey(const Key('payment-pay-button')));
await tester.pumpAndSettle();
}
Future<void> expectDeclineExplanation() async {
expect(find.textContaining('declined'), findsOneWidget);
}
}
The flakiness I actually fought came from five directions at once: different screen sizes shifting where things sat, taps aimed at coordinates that moved whenever a banner or a list changed, fixed waits that failed on a slow device or a slow API, Arabic layouts mirroring every position that passed in English, and dynamic content, promos and stock, changing under the test. The fix that removed most of it was in the app rather than the tool: a semantics ID on every interactive widget, so the runner addresses what a thing is, not where it is. The rules that followed are the ones below.
Two companion rules keep the suite honest. Finders target stable keys, not display text that copywriting will change next sprint. And there are no blind waits: a test that sleeps for three seconds is a test that flakes on a slow emulator and wastes two seconds on a fast one; wait for conditions instead. Flakiness gets treated as a defect with a root cause, never rerun until green, because the first time the team stops trusting a red suite, the suite is dead.
Off the pull-request path, on a schedule
A full journey suite on real builds is too slow for a pull-request gate, and pretending otherwise ruins both. So the practice splits the work honestly: pull requests run the fast lanes (analysis, unit and widget tests, minutes at most) plus a device-free lint that catches a broken test before it reaches a device, and the complete journey suite runs unattended against a release build, on demand today and on a schedule as the pipeline lands:
# nightly-regression.yml (pattern; any CI provider works)
on:
schedule:
- cron: "0 1 * * *" # while the team sleeps
workflow_dispatch: # and on demand before a release
jobs:
regression:
runs-on: macos-latest
strategy:
matrix:
platform: [android-emulator, ios-simulator]
steps:
- run: flutter test integration_test
- uses: actions/upload-artifact@v4
if: always()
with:
name: evidence-${{ matrix.platform }}
path: build/evidence/
The morning ritual is the other half of the design: results come with screenshots, failures get triaged before new work starts, and every failure is either a real regression (fix today), a legitimate product change (update the journey), or suite debt (fix the test properly). Nothing gets muted.
Scale with QA, not around QA
Automation efforts fail socially more often than technically, usually by treating QA as the thing being replaced. This practice scaled precisely because it was built with QA: their journey knowledge is the suite's backbone. QA owns the journey inventory, the priority order and the definition of done for each lane, because nobody knows better how users actually break an app. Engineering owns the harness, the robots and the CI wiring. Both look at the morning results together.
In practice the split was clean. I built the engine: the runner, the report and the verifier. QA wrote the flows, because they already knew the journeys, and they start the runs themselves from the picker, choosing epics, a user profile and a language, then read the report. I get pulled in when the engine needs a change or a new screen needs its semantics IDs, not to run regression.
That split also changes what QA's day looks like. The repetitive regression pass that used to eat every release cycle now runs without them, started from a picker that needs no command line, and their time moves to the work automation cannot do: exploratory testing, new-feature risk, and feeding the next journeys into the inventory.
Releasable is the metric
None of this exists to make a dashboard green. The point is that "can we ship today?" stops being a feeling and becomes a reading: on the last run, the 90+ journeys that define the product either passed on a release build or they did not, with screenshots either way. Better still, the report grades itself against thresholds set before the run, and a separate verifier recomputes every number, so the verdict is never the opinion of the thing being graded. Release decisions turn into GO/NO-GO calls made on evidence.
The release side of this, what the GO/NO-GO output looks like and what it changed about shipping, is the subject of the companion piece on the company blog: Automating Mobile App Regression: Overnight Testing and Evidence-Based Releases. Akkodis Middle East also shared it on LinkedIn.
Where to start on Monday
If your team has zero automated E2E tests, the whole practice compresses into a first month that looks like this: list your journeys and sort them by the cost of failure. Automate the top three, happy path only, behind robots from day one. Then, before adding journey number four, take those three to the full standard: negative paths and edge cases. Wire the suite into a scheduled job with artifacts, and hold the triage without exceptions. Ten journeys held to a real standard beat a hundred happy paths every time, and unlike the hundred, they will still be running in a year.