Flutter end-to-end tests in plain English
Write what a user does. FlutterProbe runs it on your emulator, simulator or phone, through direct access to the widget tree, and shows what broke and why.
Or apply by email: the form opens a public GitHub issue.
A real test next to its real report
Excerpts from real runs of the repository's own fixture app on an Android emulator.
test "type into a field, tap a button, wait for the result" wait until "Page loaded" appears type "hello" into css:"#message" in webview store text of css:"#message" in webview as typed type "<typed>" into #echo_input see "Echo: hello" tap "Contact support" in webview wait until "Message sent" appears in webview see css:"#result" in webview
✓ the page is shown and its title is found (2.808s) ✓ type into a field, tap a button, wait for the result (3.569s) ✓ the result text is read, not only found (3.864s) ✓ stored text has its runs of spaces collapsed, and can be typed into a Flutter field (4.709s) ✓ stored text drops the characters that show nothing (4.323s) ✓ a Flutter button reloads the page and the page starts again (4.232s) 6 passed (23.505s) ✗ a title the page does not have line 15: see "Getting Stopped" in webview: No element in the WebView page "Fixture page" matches "Getting Stopped"
The left listing is a whole test of the file, and the six passing lines are one run of that file: webview.probe (Flutter 3.44.8, emulator, Android 15). The failing block is not part of that run: it is a test written to fail, from a separate run of a variant file of the same app (webview.probe.expect-fail), to show the message you get when a step cannot find what it looks for; one repeated prefix is trimmed from that message. The WebView steps are Android only; every other step works on iOS too.
Three ways in
Only the first one needs nothing from us. Say which one fits, and here is what happens next.
Free
Install the CLI, add the agent to your app and run tests on your own devices in about ten minutes.
- No account, no sign-up.
- Unlimited local runs, unlimited seats.
Design partner
For the first 10 Flutter teams who want Probe Cloud and will tell us what is missing. Free during the beta, 50% off for life on any paid plan when paid plans launch. The beta may change or end.
- You send an application: a GitHub issue form (public) or an email.
- We answer by hand and tell you what access you get and when.
Team or Scale
Paid plans for teams that want the shared history. There is no checkout yet and nothing to buy today.
- Design partners get 50% off for life when paid plans launch.
- Prices and allowances are planned, not final.
Probe Cloud: your test history, shared with the team
A design-partner preview. The free CLI never needs it.
Trend
The pass rate for each day, so a slow slide shows before a release.
Failing tests
Tests that failed, newest failure first, with the file they live in.
Unstable tests
Tests that passed and also failed or needed a retry, with how often.
Pull requests
The newest commit of each pull request, compared with the base branch.
The hosted service is not live yet. Uploads are built for GitHub Actions; for any other CI, the JUnit and JSON reports work as before.
Plans
The CLI and agent are free. The paid plans are planned and by invitation: nothing can be bought yet.
Free
Local runs, unlimited seats. 1,000 results a month in Probe Cloud (preview).
Team
Unlimited seats, no overage. 100,000 results a month.
Scale
Unlimited seats, no overage. 500,000 results a month.
Planned prices in US dollars, not offers, and not final. A pay-what-you-use account is planned too. All details on the pricing page.
Three steps, with the exact commands
Install
One binary, no runtime dependencies. Then add flutter_probe_agent to your app as a dev dependency.
brew install AlphaWaveSystems/tap/probeWrite and run
Scaffold with probe init, write plain English in a .probe file, run it on a connected device.
probe test tests/ -vSee what a report holds
Works today and sends nothing: it prints exactly what Probe Cloud would receive. Design partners get access to send it.
probe test tests/ --upload-dry-runEverything you need
No plugins to install. No extra packages. Batteries included.
Natural language
ProbeScript reads like English. No locators, no async/await. QA, product, and engineering can all read the tests.
Direct widget-tree access
The Dart agent in your app reads the live widget tree. No XPath, no accessibility bridge, and widget keys work as selectors.
Cross-platform
Same tests on Android emulators, iOS simulators, and physical devices. WiFi mode for physical iOS devices. Permissions, lifecycle, and platform quirks handled automatically.
Test recording
Record taps, swipes, and text input on a real device. FlutterProbe writes the .probe file as you interact. probe test --record also keeps a step-by-step recording of each test: screenshots, widget trees and the app's Dart HTTP calls, viewable offline.
Visual regression
Screenshot baselines with configurable pixel thresholds — full screen or cropped to a single widget. Catch unintended UI changes before they ship.
HTTP mocking
Mock API responses directly in test files. No external packages, no separate mock server.
AI assertions, local-first
see "checkout looks correct" with ai — against your own OpenAI/Anthropic key or a fully local model. Screenshot redaction built in; nothing routes through our servers.
Native UI automation
Drive pickers and share sheets outside the Flutter tree with tap native and see native. Android via uiautomator — no extra dependencies.
CI/CD ready
JUnit XML, JSON, HTML reports. A Dockerfile for self-hosted CI is in the repository. GitHub Actions workflow included. Test sharding with --shard N.
Side by side
Same login test. Different experience.
test "user can log in" open the app tap "Sign In" type "user@test.com" into "Email" type "secret" into "Password" tap "Continue" see "Dashboard"
testWidgets('user can log in', (t) async { app.main(); await t.pumpAndSettle(); await t.tap(find.text('Sign In')); await t.pumpAndSettle(); await t.enterText( find.byType(TextField).first, 'user@test.com'); await t.enterText( find.byType(TextField).last, 'secret'); await t.tap(find.text('Continue')); await t.pumpAndSettle(); expect(find.text('Dashboard'), findsOneWidget); });
Architecture
Two components. One WebSocket.
probe CLI
Parses .probe files, manages devices, dispatches commands, generates reports
Go · one binaryProbeAgent
Runs inside your app. Walks widget tree, executes actions, captures screenshots
Dart · dev dependencyMigrate from anything
probe-convert handles 7 source formats.
Questions
Probe Cloud and plans
What happens to my data?
Nothing leaves your machine unless you upload. When you do, the payload is scrubbed first: paths are made relative, secret-looking values are redacted, and screenshots and device identifiers are left out. Uploads are capped at 1 MiB and 5,000 results, and --upload-dry-run prints exactly what would be sent. Details: Probe Cloud and the privacy policy (a draft for review).
Can I cancel, and what about refunds?
Nothing can be bought yet. The rules for paid plans are in the terms and the refund policy; both are drafts for review and not final.
My CI is not GitHub Actions. Can I still use FlutterProbe?
Yes. The CLI runs on any CI system and writes JUnit XML, JSON and HTML reports with no account. Uploads to Probe Cloud are built for GitHub Actions only for now.
What is a design partner?
One of the first 10 Flutter teams to try Probe Cloud and tell us what is missing. It is free during the beta, and when paid plans launch you get 50% off for life on any paid plan, in exchange for honest feedback and a short call or written notes after using it on a real project. The beta may change or end. Apply as a design partner with the GitHub form (public) or by email. The design-partner guide says what you install and what to expect.
Using FlutterProbe
Do I need to modify my Flutter app?
Add flutter_probe_agent as a dev dependency and call ProbeAgent.start() in your main entrypoint, guarded by --dart-define=PROBE_AGENT=true. No other changes needed. The agent is stripped from release builds.
Can I run tests in CI/CD?
Yes. FlutterProbe includes GitHub Actions workflows for Android (ubuntu + emulator) and iOS (macOS + simulator) with 3-way test sharding, automated app build, and HTML report generation. Use --format junit -o results.xml for CI integration.
Can I test on physical iOS devices?
Yes. FlutterProbe supports physical iOS devices via USB or WiFi. WiFi is recommended — it provides a stable, drop-free connection. USB-C cables can cause intermittent disconnects due to charge/data mode switching. Build with --dart-define=PROBE_WIFI=true and connect with --host <device-ip>.
Why does my USB connection keep dropping?
USB-C cables switch between charging and data transfer modes, which momentarily disconnects the device. FlutterProbe auto-reconnects (up to 2 retries), but this adds latency. Switch to WiFi testing to eliminate drops entirely, or use a USB-A cable which doesn’t have this issue.
Does restart the app work over WiFi?
Yes. The CLI pre-shares a token with the agent before restarting. After restart, the agent uses the pre-shared token — no USB log reading needed.
What operations are unsupported on physical devices?
clear app data, allow/deny permission, and set location are skipped with a warning on physical iOS devices (no simulator sandbox access). On Android physical devices, only set location is unsupported. All other operations work normally.
Try it now
Free to use for any purpose, including production, except offering a competing hosted testing service (the CLI is under BSL 1.1; the Dart agent is MIT).