Repository navigation
feat(fixtures): Navigation widget family: app bar and routes, measured on two named devices - #792
Merged
Merged
Conversation
…negative variants and matrix results
…tion finding names its devices
jay23606
added a commit
to jay23606/flutter-probe
that referenced
this pull request
Nov 17, 2026
…d on two named devices (AlphaWaveSystems#792) ## Summary **Widget family: Navigation (app bar and routes).** Item 2 of `docs/compat/NEXT.md`: a fixture screen, a ProbeScript suite, the variants that must fail, and the matrix results, set only from runs on named devices. - **Screen** (`PROBE_FIXTURE_WIDGETS=navigation`, `native-test-apps/flutter/lib/widget_families.dart`): - a `Scaffold` whose `AppBar` is titled "Inbox", with an action `IconButton` with the tooltip "Search" (`wf_search`), counted in the state text `Queries: N`; - a button "Open message" (`wf_open_message`) that pushes a `MaterialPageRoute` titled "Message", whose body is `From: Ada`. No label is part of a state text, since text selectors match substrings; that is why the counter is `Queries` and not `Searches`. Every control has a `Semantics(identifier:)` and a `ValueKey` of the same name, as in the other families. - **Suite** `probe-tests/widgets/navigation.probe` (4 tests): the bar's title, and its action by tooltip and by `#id`; a pushed route is seen; **the route under the top one is not seen**; `go back` returns to the first route. - **Negative variants** `probe-tests/negative/navigation.probe.expect-fail` (5 tests, each with its `# Expected:` line): the route under the top one seen; one tap counted twice; the second route before it is pushed; the second route after `go back`; a tooltip that is not on the screen. - **Matrix:** a Navigation section, and the family row now points to it. The cells are set only for `AppBar` and `Navigator`; `NavigationBar`, `TabBar` and `Drawer` stay not assessed. `NEXT.md` item 2 says it is assessed; the README lists the family. CHANGELOG under Added. ## What it settles - **The route rule holds on a device.** While the message route (opaque) is on top, the first route's title, texts and button are not matched: `don't see "Inbox"`, `"Queries: 0"` and `"Open message"` pass, and `see "Inbox"` fails. This is the agent's rule (`_isVisible` and `_routeOnScreen` in `probe_agent/lib/src/finder.dart`), which until now had only a unit test (PT-03). - **Finding: during a pop, the leaving route is still matched, on both devices.** Right after `go back`, `wait until "Inbox" appears` passes at once, while the message route is still on screen in Material's page transition. The first run of the suite failed at `don't see "From: Ada"`: it found 2 elements. Those steps alone, 5 runs on each device: the leaving route was found in 4 of 5 runs on `Compat-Pixel-API35` and in 5 of 5 on `Compat-iPhone`. A test that checks a pop now waits for the leaving route to go (`wait until "From: Ada" disappears`). The suite says this in a comment, and the negative N4 uses the same wait. - `go back` is the agent calling `Navigator.pop()`. It is not the Android back key or the iOS edge swipe; neither was run. ## Evidence (devices) Flutter **3.44.8** (installed; `pubspec.lock` unchanged, nothing added or upgraded). The probe CLI was built from this branch; the agent is the fixture's path dependency (0.27.0). - **Android:** AVD `Compat-Pixel-API35` (created for this work from the installed `android-35/google_apis/arm64-v8a` image, Pixel 6 profile), Android 15 (API 35), `emulator-5590`. The name was read back (`adb -s emulator-5590 emu avd name`) before each step. Host port 48690, device port 48686. - **iOS:** simulator `Compat-iPhone`, UDID `A57AEE67-3473-4C20-BE25-8847C2A9D6BE` (created for this work), iPhone 17 Pro, iOS 26.5, always addressed by UDID. The agent was built with `--dart-define=PROBE_PORT=48691`, its own port, since a simulator shares the host's network. | Run | `Compat-Pixel-API35` | `Compat-iPhone` | |---|---|---| | `navigation.probe`, 3 runs | 4 passed, 3 times | 4 passed, 3 times | | the negative file, run as its header says | 0 passed, 5 failed, each with its stated message | 0 passed, 5 failed, each with its stated message | Flutter 3.50.0 was not run for this family; the matrix says so. I've asked where a 3.50.0 SDK copy is; if one is available, the suite runs on it in a follow-up and the cells name both versions. ## Not covered `NavigationBar`, `TabBar`, `Drawer`; a route that is not opaque (a dialog, a popup menu, a sheet, where the page under it stays matched by design; item 8); named routes and `go_router`; the system back key and the iOS edge swipe; a nested `Navigator`; Cupertino routes (item 10); a physical device; other screen sizes, text scales and accessibility services. CI does not build the fixture. ## Test plan - [x] The suite and the negative file on both named devices (table above). - [x] `make ci-local`: 5 of 5 required checks at cc5bf40 (Build, Go unit tests, probe-convert tests, Lint, Disclosure guard). - [x] `go test ./scripts/`: the negative-suite guards accept the new file. - [x] `dart format` and `flutter analyze lib`: no issues. ## Review follow-up - **The transition finding names its devices**, in the matrix, the CHANGELOG and here: Android 4 of 5 runs, iOS 5 of 5 (`pop-repro`: tap, wait for the message, `go back`, `wait until "Inbox" appears`, `don't see "From: Ada"`). - **A positive control in test 3:** `see "Inbox"` and `see "Queries: 0"` run before the push, so the `don't see` steps can't pass on a text that never existed. N1 of the negative file sees `"Inbox"` before the push too, and fails only at the check after it (line 17). - **`NEXT.md` item 2** says "back: in-app pop only (not the system back key or iOS edge swipe)". - **Rerun after the change**, on each device: `navigation.probe` passed 4 of 4 three times, and the negative file gave 0 passed and 5 failed, each with its stated message. Main is merged in; there was no conflict.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Widget family: Navigation (app bar and routes). Item 2 of
docs/compat/NEXT.md: a fixture screen, a ProbeScript suite, the variants that must fail, and the matrix results, set only from runs on named devices.Screen (
PROBE_FIXTURE_WIDGETS=navigation,native-test-apps/flutter/lib/widget_families.dart):ScaffoldwhoseAppBaris titled "Inbox", with an actionIconButtonwith the tooltip "Search" (wf_search), counted in the state textQueries: N;wf_open_message) that pushes aMaterialPageRoutetitled "Message", whose body isFrom: Ada.No label is part of a state text, since text selectors match substrings; that is why the counter is
Queriesand notSearches. Every control has aSemantics(identifier:)and aValueKeyof the same name, as in the other families.Suite
probe-tests/widgets/navigation.probe(4 tests): the bar's title, and its action by tooltip and by#id; a pushed route is seen; the route under the top one is not seen;go backreturns to the first route.Negative variants
probe-tests/negative/navigation.probe.expect-fail(5 tests, each with its# Expected:line): the route under the top one seen; one tap counted twice; the second route before it is pushed; the second route aftergo back; a tooltip that is not on the screen.Matrix: a Navigation section, and the family row now points to it. The cells are set only for
AppBarandNavigator;NavigationBar,TabBarandDrawerstay not assessed.NEXT.mditem 2 says it is assessed; the README lists the family. CHANGELOG under Added.What it settles
don't see "Inbox","Queries: 0"and"Open message"pass, andsee "Inbox"fails. This is the agent's rule (_isVisibleand_routeOnScreeninprobe_agent/lib/src/finder.dart), which until now had only a unit test (PT-03).go back,wait until "Inbox" appearspasses at once, while the message route is still on screen in Material's page transition. The first run of the suite failed atdon't see "From: Ada": it found 2 elements. Those steps alone, 5 runs on each device: the leaving route was found in 4 of 5 runs onCompat-Pixel-API35and in 5 of 5 onCompat-iPhone. A test that checks a pop now waits for the leaving route to go (wait until "From: Ada" disappears). The suite says this in a comment, and the negative N4 uses the same wait.go backis the agent callingNavigator.pop(). It is not the Android back key or the iOS edge swipe; neither was run.Evidence (devices)
Flutter 3.44.8 (installed;
pubspec.lockunchanged, nothing added or upgraded). The probe CLI was built from this branch; the agent is the fixture's path dependency (0.27.0).Compat-Pixel-API35(created for this work from the installedandroid-35/google_apis/arm64-v8aimage, Pixel 6 profile), Android 15 (API 35),emulator-5590. The name was read back (adb -s emulator-5590 emu avd name) before each step. Host port 48690, device port 48686.Compat-iPhone, UDIDA57AEE67-3473-4C20-BE25-8847C2A9D6BE(created for this work), iPhone 17 Pro, iOS 26.5, always addressed by UDID. The agent was built with--dart-define=PROBE_PORT=48691, its own port, since a simulator shares the host's network.Compat-Pixel-API35Compat-iPhonenavigation.probe, 3 runsFlutter 3.50.0 was not run for this family; the matrix says so. I've asked where a 3.50.0 SDK copy is; if one is available, the suite runs on it in a follow-up and the cells name both versions.
Not covered
NavigationBar,TabBar,Drawer; a route that is not opaque (a dialog, a popup menu, a sheet, where the page under it stays matched by design; item 8); named routes andgo_router; the system back key and the iOS edge swipe; a nestedNavigator; Cupertino routes (item 10); a physical device; other screen sizes, text scales and accessibility services. CI does not build the fixture.Test plan
make ci-local: 5 of 5 required checks at cc5bf40 (Build, Go unit tests, probe-convert tests, Lint, Disclosure guard).go test ./scripts/: the negative-suite guards accept the new file.dart formatandflutter analyze lib: no issues.Review follow-up
pop-repro: tap, wait for the message,go back,wait until "Inbox" appears,don't see "From: Ada").see "Inbox"andsee "Queries: 0"run before the push, so thedon't seesteps can't pass on a text that never existed. N1 of the negative file sees"Inbox"before the push too, and fails only at the check after it (line 17).NEXT.mditem 2 says "back: in-app pop only (not the system back key or iOS edge swipe)".navigation.probepassed 4 of 4 three times, and the negative file gave 0 passed and 5 failed, each with its stated message. Main is merged in; there was no conflict.