v0.27.0 · free to use · source-available CLI (BSL 1.1), MIT agent

Flutter end-to-end tests in plain English

Write what a user does. FlutterProbe runs it on your emulator, simulator or phone, through direct access to the widget tree, and shows what broke and why.

Or apply by email: the form opens a public GitHub issue.

$ brew install AlphaWaveSystems/tap/probe
~2x
Faster median suite time than Maestro (one app, 20 runs; method in the post)
0 of 20
Flaky FlutterProbe runs in that benchmark
3
Device kinds: emulators, simulators, physical
7
Migration formats

A real test next to its real report

Excerpts from real runs of the repository's own fixture app on an Android emulator.

webview.probe
test "type into a field, tap a button, wait for the result"
  wait until "Page loaded" appears
  type "hello" into css:"#message" in webview
  store text of css:"#message" in webview as typed
  type "<typed>" into #echo_input
  see "Echo: hello"
  tap "Contact support" in webview
  wait until "Message sent" appears in webview
  see css:"#result" in webview
probe test
✓  the page is shown and its title is found (2.808s)
✓  type into a field, tap a button, wait for the result (3.569s)
✓  the result text is read, not only found (3.864s)
✓  stored text has its runs of spaces collapsed, and can be typed into a Flutter field (4.709s)
✓  stored text drops the characters that show nothing (4.323s)
✓  a Flutter button reloads the page and the page starts again (4.232s)

6 passed  (23.505s)

✗  a title the page does not have
     line 15: see "Getting Stopped" in webview: No element in the
     WebView page "Fixture page" matches "Getting Stopped"

The left listing is a whole test of the file, and the six passing lines are one run of that file: webview.probe (Flutter 3.44.8, emulator, Android 15). The failing block is not part of that run: it is a test written to fail, from a separate run of a variant file of the same app (webview.probe.expect-fail), to show the message you get when a step cannot find what it looks for; one repeated prefix is trimmed from that message. The WebView steps are Android only; every other step works on iOS too.

Three ways in

Only the first one needs nothing from us. Say which one fits, and here is what happens next.

Available now

Free

Install the CLI, add the agent to your app and run tests on your own devices in about ten minutes.

  • No account, no sign-up.
  • Unlimited local runs, unlimited seats.
Open now, by hand

Design partner

For the first 10 Flutter teams who want Probe Cloud and will tell us what is missing. Free during the beta, 50% off for life on any paid plan when paid plans launch. The beta may change or end.

  • You send an application: a GitHub issue form (public) or an email.
  • We answer by hand and tell you what access you get and when.
Planned, by invitation

Team or Scale

Paid plans for teams that want the shared history. There is no checkout yet and nothing to buy today.

  • Design partners get 50% off for life when paid plans launch.
  • Prices and allowances are planned, not final.

Probe Cloud: your test history, shared with the team

A design-partner preview. The free CLI never needs it.

Trend

The pass rate for each day, so a slow slide shows before a release.

Failing tests

Tests that failed, newest failure first, with the file they live in.

Unstable tests

Tests that passed and also failed or needed a retry, with how often.

Pull requests

The newest commit of each pull request, compared with the base branch.

The hosted service is not live yet. Uploads are built for GitHub Actions; for any other CI, the JUnit and JSON reports work as before.

Plans

The CLI and agent are free. The paid plans are planned and by invitation: nothing can be bought yet.

Available now

Free

$0

Local runs, unlimited seats. 1,000 results a month in Probe Cloud (preview).

Planned, by invitation

Team

$49 / month

Unlimited seats, no overage. 100,000 results a month.

Planned, by invitation

Scale

$149 / month

Unlimited seats, no overage. 500,000 results a month.

Planned prices in US dollars, not offers, and not final. A pay-what-you-use account is planned too. All details on the pricing page.

Three steps, with the exact commands

01

Install

One binary, no runtime dependencies. Then add flutter_probe_agent to your app as a dev dependency.

brew install AlphaWaveSystems/tap/probe
02

Write and run

Scaffold with probe init, write plain English in a .probe file, run it on a connected device.

probe test tests/ -v
03

See what a report holds

Works today and sends nothing: it prints exactly what Probe Cloud would receive. Design partners get access to send it.

probe test tests/ --upload-dry-run

Everything you need

No plugins to install. No extra packages. Batteries included.

Natural language

ProbeScript reads like English. No locators, no async/await. QA, product, and engineering can all read the tests.

Direct widget-tree access

The Dart agent in your app reads the live widget tree. No XPath, no accessibility bridge, and widget keys work as selectors.

Cross-platform

Same tests on Android emulators, iOS simulators, and physical devices. WiFi mode for physical iOS devices. Permissions, lifecycle, and platform quirks handled automatically.

Test recording

Record taps, swipes, and text input on a real device. FlutterProbe writes the .probe file as you interact. probe test --record also keeps a step-by-step recording of each test: screenshots, widget trees and the app's Dart HTTP calls, viewable offline.

Visual regression

Screenshot baselines with configurable pixel thresholds — full screen or cropped to a single widget. Catch unintended UI changes before they ship.

HTTP mocking

Mock API responses directly in test files. No external packages, no separate mock server.

AI assertions, local-first

see "checkout looks correct" with ai — against your own OpenAI/Anthropic key or a fully local model. Screenshot redaction built in; nothing routes through our servers.

Native UI automation

Drive pickers and share sheets outside the Flutter tree with tap native and see native. Android via uiautomator — no extra dependencies.

CI/CD ready

JUnit XML, JSON, HTML reports. A Dockerfile for self-hosted CI is in the repository. GitHub Actions workflow included. Test sharding with --shard N.

Side by side

Same login test. Different experience.

FlutterProbe7 lines
test "user can log in"
  open the app
  tap "Sign In"
  type "user@test.com" into "Email"
  type "secret" into "Password"
  tap "Continue"
  see "Dashboard"
flutter_test13 lines
testWidgets('user can log in', (t) async {
  app.main();
  await t.pumpAndSettle();
  await t.tap(find.text('Sign In'));
  await t.pumpAndSettle();
  await t.enterText(
    find.byType(TextField).first,
    'user@test.com');
  await t.enterText(
    find.byType(TextField).last,
    'secret');
  await t.tap(find.text('Continue'));
  await t.pumpAndSettle();
  expect(find.text('Dashboard'),
    findsOneWidget);
});

Architecture

Two components. One WebSocket.

probe CLI

Parses .probe files, manages devices, dispatches commands, generates reports

Go · one binary
⇌JSON-RPC 2.0WebSocket :48686

ProbeAgent

Runs inside your app. Walks widget tree, executes actions, captures screenshots

Dart · dev dependency

Migrate from anything

probe-convert handles 7 source formats.

MaestroGherkinRobot FrameworkDetoxAppium PythonAppium JavaAppium JS

Questions

Probe Cloud and plans

What happens to my data?

Nothing leaves your machine unless you upload. When you do, the payload is scrubbed first: paths are made relative, secret-looking values are redacted, and screenshots and device identifiers are left out. Uploads are capped at 1 MiB and 5,000 results, and --upload-dry-run prints exactly what would be sent. Details: Probe Cloud and the privacy policy (a draft for review).

Can I cancel, and what about refunds?

Nothing can be bought yet. The rules for paid plans are in the terms and the refund policy; both are drafts for review and not final.

My CI is not GitHub Actions. Can I still use FlutterProbe?

Yes. The CLI runs on any CI system and writes JUnit XML, JSON and HTML reports with no account. Uploads to Probe Cloud are built for GitHub Actions only for now.

What is a design partner?

One of the first 10 Flutter teams to try Probe Cloud and tell us what is missing. It is free during the beta, and when paid plans launch you get 50% off for life on any paid plan, in exchange for honest feedback and a short call or written notes after using it on a real project. The beta may change or end. Apply as a design partner with the GitHub form (public) or by email. The design-partner guide says what you install and what to expect.

Using FlutterProbe

Do I need to modify my Flutter app?

Add flutter_probe_agent as a dev dependency and call ProbeAgent.start() in your main entrypoint, guarded by --dart-define=PROBE_AGENT=true. No other changes needed. The agent is stripped from release builds.

Can I run tests in CI/CD?

Yes. FlutterProbe includes GitHub Actions workflows for Android (ubuntu + emulator) and iOS (macOS + simulator) with 3-way test sharding, automated app build, and HTML report generation. Use --format junit -o results.xml for CI integration.

Can I test on physical iOS devices?

Yes. FlutterProbe supports physical iOS devices via USB or WiFi. WiFi is recommended — it provides a stable, drop-free connection. USB-C cables can cause intermittent disconnects due to charge/data mode switching. Build with --dart-define=PROBE_WIFI=true and connect with --host <device-ip>.

Why does my USB connection keep dropping?

USB-C cables switch between charging and data transfer modes, which momentarily disconnects the device. FlutterProbe auto-reconnects (up to 2 retries), but this adds latency. Switch to WiFi testing to eliminate drops entirely, or use a USB-A cable which doesn’t have this issue.

Does restart the app work over WiFi?

Yes. The CLI pre-shares a token with the agent before restarting. After restart, the agent uses the pre-shared token — no USB log reading needed.

What operations are unsupported on physical devices?

clear app data, allow/deny permission, and set location are skipped with a warning on physical iOS devices (no simulator sandbox access). On Android physical devices, only set location is unsupported. All other operations work normally.

Try it now

$brew install AlphaWaveSystems/tap/probe

Free to use for any purpose, including production, except offering a competing hosted testing service (the CLI is under BSL 1.1; the Dart agent is MIT).