Skip to content

fix(agent): answer only once a frame is laid out at the view's real size (issue #793) - #803

Merged
pbertsch merged 4 commits into
mainfrom
fix/agent-wait-for-laid-out-view
Nov 17, 2026
Merged

pbertsch merged 4 commits into
mainfrom
fix/agent-wait-for-laid-out-view

Conversation

@pbertsch

@pbertsch pbertsch commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes #793: a step right after restart the app no longer runs on a half-built screen.

The cause, measured

On an Android 15 emulator, after a relaunch, an app draws its first frames with a 0×0 view for about 1 to 1.2 s (logged per frame in a temporary build; see #793). A ListView laid out then builds only its first child.

My first version waited while the view was 0×0. It did not fix the bug: buttons.probe without waits still failed 2 of 10 runs. I instrumented the agent to log every request: none of 530 requests arrived while the size was 0×0. The real gap is one step later. physicalSize changes the moment the engine reports it, but the widgets are laid out at that size only in the next frame. A request that arrives in between passes a "has a real size" check and still sees the tree laid out at 0×0.

The change (agent; the CLI only reads a diagnostic)

  • probe_agent/lib/src/view_ready.dart (new):
    • ProbeAgent.start calls ViewReady.instance.attach(), which registers a persistent frame callback that records the size each frame was laid out at.
    • The view is ready when the size of the last laid-out frame equals the view's current, non-empty size. When it isn't, wait() waits until it is.
    • The hot path is a synchronous check (isReady) with no await.
    • Never slow when there's nothing to wait for: no wait when the app has no view (views empty) or is not resumed (paused, hidden, inactive). It stops at once if the app goes to the background during the wait.
    • The cap is 2.5 s, measured by a Timer, not a stopwatch. After a capped wait, the agent doesn't wait again until the view's metrics change, so there is at most one capped wait per "size unknown" episode.
    • A resize (a rotation) waits for at most one frame.
    • Without attach (an embedder that never calls start), the earlier rule applies: a real size is ready.
  • executor.dart: every request first does if (!ViewReady.instance.isReady) await ViewReady.instance.wait();, before the request's own work. A step's own waiting (a wait until, a scroll) isn't shortened, and the wait isn't counted against it twice: it happens before the step's own deadline starts.
  • probe.ping reports view_ready_ms, the wait in ms (0 when none). The capability waits_for_view_size is added; the existing names use underscores, like press_key. Both are for diagnostics.
  • Go: PingResult.ViewReadyMS is kept raw. HandshakeResult.ViewReadyMS accepts only a digits-only integer from 0 to 60000, and is −1 when absent or anything else (null, negative, a fraction, an exponent, a string). CapabilityWaitsForViewSize is added. Nothing depends on them.
  • Compatibility: an older CLI ignores the field and gets the fix. A new CLI with an older agent behaves as before. There's no protocol change.

Why 2.5 s

The cap stays well under every limit around the handshake:

  • the dial timeout: 30 s by default in the CLI, 5 s in Studio; the HTTP client's ping runs inside it and retries every second;
  • the reconnect loop (ReconnectWithToken, 2 s between dials, up to the token timeout);
  • the keep-alive ping (every 5 s, 2 s write deadline).

The 0×0 phase measured about 1.2 s.

Tests

probe_agent/test/view_ready_test.dart (11 tests):

  • a 0×0 view builds part of the list, and wait ends once the real size is laid out, with the whole list there;
  • the gap: the size is reported but no frame laid out yet, so the view isn't ready;
  • the wait lasts until a frame is laid out, using binding.delayed to advance time without a frame;
  • the cap ends the wait, and the same episode doesn't wait again;
  • a metrics change starts a new episode;
  • a paused app isn't waited for, and going to the background mid-wait ends the wait;
  • no view: no wait and no error;
  • a rotation waits for one frame;
  • without attach, the earlier rule applies.

probe_agent/test/executor_view_ready_test.dart (3 tests): a ping during 0×0 answers only after the real size, with view_ready_ms and the capability; a ping with a real size has view_ready_ms 0; a ping in the gap waits for the frame.

internal/probelink/view_ready_ms_test.go: the field's parsing (12 inputs), decoding with and without the field, and the capability name.

All 263 probe_agent tests pass; executor_ping_test's pinned field set now includes view_ready_ms.

Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. 9 of 10 were killed.

Mutant Result
no views still waits killed
a backgrounded app is waited for killed
the cap is not remembered killed
a metrics change does not start a new episode killed
a reported size counts as laid out (the gap) killed
the frame callback records nothing killed
without attach, a real size is not ready killed
the executor does not wait killed
ping without view_ready_ms killed
after the frame wait, return without re-checking the laid-out size survived (equivalent under the test binding)

The survivor: in a widget test, endOfFrame always completes a frame laid out at the current size, so the re-check can't fail there. On a device, a frame already under way when the size changed would complete laid out at the old size; the re-check keeps waiting then. It is kept as a guard.

Device evidence

AVD Compat-Pixel-API35 (Pixel 6 profile, Android 15, API 35, emulator-5590, name read back before each run) and simulator Compat-iPhone (iPhone 17 Pro, iOS 26.5, UDID A57AEE67-3473-4C20-BE25-8847C2A9D6BE). Flutter 3.44.8. Each run is the merged buttons.probe without the interim waits of #795, so restart the app before every test.

Agent Compat-Pixel-API35 Compat-iPhone
main 3 of 10 runs passed (measured twice: 3 of 10 each time) passed in every run
first version (waits while 0×0) 8 of 10 (2 half-built screens), plus one CLI hang, see below 5 of 5
this PR 29 of 30 (20 of 20, then 9 of 10 with the machine loaded) 5 of 5
this PR, cold boot (-no-snapshot-load) 10 of 10 with the machine idle; 7 of 10 while my Go and Studio tests loaded the machine, all 3 failures the reconnect timeout below

No run of this PR showed a half-built screen.

Connect time (a one-step test with restart the app, so relaunch, reconnect and handshake), 10 runs each, wall time:

main agent this PR
Compat-Pixel-API35 median 4.90 s (one run failed with the reconnect timeout: 35.5 s) median 4.90 s
Compat-iPhone median 7.20 s median 7.30 s

No regression beyond run-to-run noise.

Not fixed here: the reconnect timeout

restart the app: reconnect with pre-shared token: agent not reachable within 30s happened with both agents: once with main's agent in the connect-time runs, once in the 30 runs of this PR, three times on the cold-booted emulator while the machine was loaded, and twice in another session's runs on main. It is a failed dial (the WebSocket upgrade, which doesn't go through the request handler this PR changes), and it tracks machine load. It needs its own investigation.

The CLI hang seen once with the first version (the CLI waited 25 minutes on a closed socket) did not recur in about 60 later runs. A CLI call on a closed WebSocket that never fails is still worth a look, and goes with that investigation.

Release

This changes probe_agent, a pub.dev package. Its CHANGELOG entry is under Unreleased. The version bump happens at release time, per the release checklist; pubspec.yaml and probeAgentVersion are unchanged here.

Test plan

  • flutter test in probe_agent: 263 passed. flutter analyze: the 25 issues reported are all in test files this PR doesn't touch.
  • go test ./internal/probelink/ ./internal/runner/ ./internal/cli/ pass.
  • go test ./studio/...: studio/e2e ok. One Studio test, TestExportWritesTheEnginesJSONReport, failed once while the machine was loaded and passed 3 of 3 alone; this PR doesn't touch it.
  • staticcheck, GOOS=windows go vet ./internal/....
  • Device evidence (above).

Review follow-up

  • Blocker: ProbeAgent.start() threw without a binding. attach() touched WidgetsBinding.instance, so an app that called start() before runApp failed at launch. Now start() calls WidgetsFlutterBinding.ensureInitialized() (as runApp would) before attaching. attach() itself never throws: if the frame callback can't be added, the agent stays unattached and keeps the earlier rule.
    • test/view_ready_no_binding_test.dart (no binding in that file) checks attach() doesn't throw and stays unattached.
    • test/agent_start_no_binding_test.dart runs ProbeAgent.start() with no binding and checks it completes and attaches. The agent starts only with the define, so the test is skipped without it: flutter test --dart-define=PROBE_AGENT=true test/agent_start_no_binding_test.dart passes. A redundant outer try in start() was removed (it gave an equivalent mutant).
  • Defect: one capped wait per episode, on every cap exit. When the size was real but no frame came within the cap, the inner exit returned without marking the episode, so every later request waited 2.5 s again. Every cap exit now goes through _giveUp(). Test: a real size, no frame delivered, binding.delayed past the cap, then gaveUp is true and the view is ready.
  • Nit: only a new size starts a new episode. didChangeMetrics also fires for keyboard insets and padding. The episode is now reset only when physicalSize differs from the size at the cap. The test changes the insets and the pixel ratio (same episode), then the size (a new one).
  • "At most one capped wait per episode" now holds on every path.
  • Mutants: 14 of 14 killed:
    • 4 new: attach throwing without a binding; the inner cap exit not marking the episode; any metrics change resetting it; start() not making the binding;
    • the 10 earlier ones, rerun on the changed code: all killed. The earlier equivalent survivor (the post-frame re-check) is now killed too, since the restructured exit is testable.
  • flutter test in probe_agent: 265 passed, 1 skipped (the define-gated test, run separately).

Second review follow-up

  • Blocker: the agent never initialises the binding. The previous push called WidgetsFlutterBinding.ensureInitialized() in start(). An app that installs its own binding after start() (its own subclass, an error-reporting package's, IntegrationTestWidgetsFlutterBinding) then failed in debug with "Binding is already initialized". That call is removed, and the frame callback is added only when a binding exists:
    • at start(), if the app made its binding first. The fixture does this, and so does the documented setup.
    • otherwise, on the agent's first request under the app's binding (attachToAppBinding, a bool check once attached). When no frame is pending at that moment, the current size is taken as laid out, since a new size always schedules a frame, so that first request doesn't wait.
    • under a test binding (frames only on pump) it doesn't attach, and the earlier rule applies, as ProbeSync already does for settling. Without this, 9 executor widget tests hung.
    • attach() never throws.
  • Tests:
    • view_ready_lazy_attach_test.dart (no binding in the file): attach does nothing; an app-defined WidgetsFlutterBinding subclass is then created without an assertion; the first request attaches, with view_ready_ms 0; a second request doesn't try again (attachAttempts).
    • agent_start_no_binding_test.dart (define-gated): ProbeAgent.start() with no binding doesn't attach, then the app's own binding subclass is created without an assertion, then attaching works.
    • agent_start_with_binding_test.dart (define-gated): start() after the app's binding attaches at once.
    • Both define-gated tests pass with --dart-define=PROBE_AGENT=true. The full suite: 266 passed, 2 skipped (those two).
  • Mutants: 5 of 5 killed:
    • start() making the binding;
    • start() not attaching;
    • no lazy attach on a request;
    • lazy attach under a test binding (the hanging executor tests kill it);
    • a late attach not taking the settled size.
  • Device: the final build, buttons.probe without waits on Compat-Pixel-API35, passed 10 of 10 with the machine otherwise idle. (The previous build's rerun was 8 of 10, both failures the reconnect timeout while my Go tests loaded the machine.) The fixture makes its binding before start(), so on the device the agent attaches at start(); the lazy path is covered by the unit tests.

@pbertsch
pbertsch requested a review from a team as a code owner November 17, 2026 14:26
…throws; one capped wait per episode on every cap exit; only a new size starts a new episode
…ng exists (at start or on the first request); not under a test binding
@pbertsch
pbertsch merged commit edb0fce into main Nov 17, 2026
10 of 11 checks passed
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…ize (issue AlphaWaveSystems#793) (AlphaWaveSystems#803)

## Summary

Fixes AlphaWaveSystems#793: a step right after `restart the app` no longer runs on a half-built screen.

### The cause, measured

On an Android 15 emulator, after a relaunch, an app draws its first frames with a **0×0 view** for about 1 to 1.2 s (logged per frame in a temporary build; see AlphaWaveSystems#793). A `ListView` laid out then builds only its first child.

My first version waited while the view was 0×0. **It did not fix the bug:** `buttons.probe` without waits still failed 2 of 10 runs. I instrumented the agent to log every request: **none of 530 requests arrived while the size was 0×0**. The real gap is one step later. `physicalSize` changes the moment the engine reports it, but the widgets are laid out at that size only in the next frame. A request that arrives in between passes a "has a real size" check and still sees the tree laid out at 0×0.

### The change (agent; the CLI only reads a diagnostic)

- **`probe_agent/lib/src/view_ready.dart`** (new):
  - `ProbeAgent.start` calls `ViewReady.instance.attach()`, which registers a persistent frame callback that records the size each frame was laid out at.
  - The view is **ready** when the size of the last laid-out frame equals the view's current, non-empty size. When it isn't, `wait()` waits until it is.
  - **The hot path** is a synchronous check (`isReady`) with no `await`.
  - **Never slow when there's nothing to wait for:** no wait when the app has no view (`views` empty) or is not resumed (paused, hidden, inactive). It stops at once if the app goes to the background during the wait.
  - **The cap** is 2.5 s, measured by a `Timer`, not a stopwatch. After a capped wait, the agent doesn't wait again until the view's metrics change, so there is at most one capped wait per "size unknown" episode.
  - **A resize** (a rotation) waits for at most one frame.
  - **Without `attach`** (an embedder that never calls `start`), the earlier rule applies: a real size is ready.
- **`executor.dart`**: every request first does `if (!ViewReady.instance.isReady) await ViewReady.instance.wait();`, before the request's own work. A step's own waiting (a `wait until`, a scroll) isn't shortened, and the wait isn't counted against it twice: it happens before the step's own deadline starts.
- **`probe.ping`** reports `view_ready_ms`, the wait in ms (0 when none). The capability `waits_for_view_size` is added; the existing names use underscores, like `press_key`. Both are for diagnostics.
- **Go:** `PingResult.ViewReadyMS` is kept raw. `HandshakeResult.ViewReadyMS` accepts only a digits-only integer from 0 to 60000, and is −1 when absent or anything else (null, negative, a fraction, an exponent, a string). `CapabilityWaitsForViewSize` is added. Nothing depends on them.
- **Compatibility:** an older CLI ignores the field and gets the fix. A new CLI with an older agent behaves as before. There's no protocol change.

### Why 2.5 s

The cap stays well under every limit around the handshake:
- the dial timeout: 30 s by default in the CLI, **5 s in Studio**; the HTTP client's ping runs inside it and retries every second;
- the reconnect loop (`ReconnectWithToken`, 2 s between dials, up to the token timeout);
- the keep-alive ping (every 5 s, 2 s write deadline).

The 0×0 phase measured about 1.2 s.

## Tests

`probe_agent/test/view_ready_test.dart` (11 tests):
- a 0×0 view builds part of the list, and `wait` ends once the real size is laid out, with the whole list there;
- **the gap:** the size is reported but no frame laid out yet, so the view isn't ready;
- the wait lasts until a frame is laid out, using `binding.delayed` to advance time without a frame;
- the cap ends the wait, and the same episode doesn't wait again;
- a metrics change starts a new episode;
- a paused app isn't waited for, and going to the background mid-wait ends the wait;
- no view: no wait and no error;
- a rotation waits for one frame;
- without `attach`, the earlier rule applies.

`probe_agent/test/executor_view_ready_test.dart` (3 tests): a ping during 0×0 answers only after the real size, with `view_ready_ms` and the capability; a ping with a real size has `view_ready_ms` 0; **a ping in the gap waits for the frame.**

`internal/probelink/view_ready_ms_test.go`: the field's parsing (12 inputs), decoding with and without the field, and the capability name.

All 263 `probe_agent` tests pass; `executor_ping_test`'s pinned field set now includes `view_ready_ms`.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. 9 of 10 were killed.

| Mutant | Result |
|---|---|
| no views still waits | killed |
| a backgrounded app is waited for | killed |
| the cap is not remembered | killed |
| a metrics change does not start a new episode | killed |
| **a reported size counts as laid out (the gap)** | killed |
| the frame callback records nothing | killed |
| without attach, a real size is not ready | killed |
| the executor does not wait | killed |
| ping without `view_ready_ms` | killed |
| after the frame wait, return without re-checking the laid-out size | **survived (equivalent under the test binding)** |

The survivor: in a widget test, `endOfFrame` always completes a frame laid out at the current size, so the re-check can't fail there. On a device, a frame already under way when the size changed would complete laid out at the old size; the re-check keeps waiting then. It is kept as a guard.

## Device evidence

AVD `Compat-Pixel-API35` (Pixel 6 profile, Android 15, API 35, `emulator-5590`, name read back before each run) and simulator `Compat-iPhone` (iPhone 17 Pro, iOS 26.5, UDID `A57AEE67-3473-4C20-BE25-8847C2A9D6BE`). Flutter 3.44.8. Each run is the merged `buttons.probe` **without** the interim waits of AlphaWaveSystems#795, so `restart the app` before every test.

| Agent | `Compat-Pixel-API35` | `Compat-iPhone` |
|---|---|---|
| `main` | 3 of 10 runs passed (measured twice: 3 of 10 each time) | passed in every run |
| first version (waits while 0×0) | 8 of 10 (2 half-built screens), plus one CLI hang, see below | 5 of 5 |
| **this PR** | **29 of 30** (20 of 20, then 9 of 10 with the machine loaded) | **5 of 5** |
| **this PR, cold boot** (`-no-snapshot-load`) | **10 of 10** with the machine idle; 7 of 10 while my Go and Studio tests loaded the machine, all 3 failures the reconnect timeout below | |

No run of this PR showed a half-built screen.

**Connect time** (a one-step test with `restart the app`, so relaunch, reconnect and handshake), 10 runs each, wall time:

| | `main` agent | this PR |
|---|---|---|
| `Compat-Pixel-API35` | median 4.90 s (one run failed with the reconnect timeout: 35.5 s) | median 4.90 s |
| `Compat-iPhone` | median 7.20 s | median 7.30 s |

No regression beyond run-to-run noise.

### Not fixed here: the reconnect timeout

`restart the app: reconnect with pre-shared token: agent not reachable within 30s` happened with **both** agents: once with `main`'s agent in the connect-time runs, once in the 30 runs of this PR, three times on the cold-booted emulator while the machine was loaded, and twice in another session's runs on `main`. It is a failed *dial* (the WebSocket upgrade, which doesn't go through the request handler this PR changes), and it tracks machine load. It needs its own investigation.

The CLI hang seen once with the first version (the CLI waited 25 minutes on a closed socket) did not recur in about 60 later runs. A CLI call on a closed WebSocket that never fails is still worth a look, and goes with that investigation.

## Release

This changes `probe_agent`, a pub.dev package. Its CHANGELOG entry is under Unreleased. The version bump happens at release time, per the release checklist; `pubspec.yaml` and `probeAgentVersion` are unchanged here.

## Test plan

- [x] `flutter test` in `probe_agent`: 263 passed. `flutter analyze`: the 25 issues reported are all in test files this PR doesn't touch.
- [x] `go test ./internal/probelink/ ./internal/runner/ ./internal/cli/` pass.
- [x] `go test ./studio/...`: `studio/e2e` ok. One Studio test, `TestExportWritesTheEnginesJSONReport`, failed once while the machine was loaded and passed 3 of 3 alone; this PR doesn't touch it.
- [x] `staticcheck`, `GOOS=windows go vet ./internal/...`.
- [x] Device evidence (above).

## Review follow-up

- **Blocker: `ProbeAgent.start()` threw without a binding.** `attach()` touched `WidgetsBinding.instance`, so an app that called `start()` before `runApp` failed at launch. Now `start()` calls `WidgetsFlutterBinding.ensureInitialized()` (as `runApp` would) before attaching. `attach()` itself never throws: if the frame callback can't be added, the agent stays unattached and keeps the earlier rule.
  - `test/view_ready_no_binding_test.dart` (no binding in that file) checks `attach()` doesn't throw and stays unattached.
  - `test/agent_start_no_binding_test.dart` runs `ProbeAgent.start()` with no binding and checks it completes and attaches. The agent starts only with the define, so the test is skipped without it: `flutter test --dart-define=PROBE_AGENT=true test/agent_start_no_binding_test.dart` passes. A redundant outer `try` in `start()` was removed (it gave an equivalent mutant).
- **Defect: one capped wait per episode, on every cap exit.** When the size was real but no frame came within the cap, the inner exit returned without marking the episode, so every later request waited 2.5 s again. Every cap exit now goes through `_giveUp()`. Test: a real size, no frame delivered, `binding.delayed` past the cap, then `gaveUp` is true and the view is ready.
- **Nit: only a new size starts a new episode.** `didChangeMetrics` also fires for keyboard insets and padding. The episode is now reset only when `physicalSize` differs from the size at the cap. The test changes the insets and the pixel ratio (same episode), then the size (a new one).
- "At most one capped wait per episode" now holds on every path.
- **Mutants:** 14 of 14 killed:
  - 4 new: `attach` throwing without a binding; the inner cap exit not marking the episode; any metrics change resetting it; `start()` not making the binding;
  - the 10 earlier ones, rerun on the changed code: all killed. The earlier equivalent survivor (the post-frame re-check) is now killed too, since the restructured exit is testable.
- `flutter test` in `probe_agent`: 265 passed, 1 skipped (the define-gated test, run separately).

## Second review follow-up

- **Blocker: the agent never initialises the binding.** The previous push called `WidgetsFlutterBinding.ensureInitialized()` in `start()`. An app that installs its own binding after `start()` (its own subclass, an error-reporting package's, `IntegrationTestWidgetsFlutterBinding`) then failed in debug with "Binding is already initialized". That call is removed, and the frame callback is added only when a binding exists:
  - at `start()`, if the app made its binding first. The fixture does this, and so does the documented setup.
  - otherwise, on the agent's first request under the app's binding (`attachToAppBinding`, a bool check once attached). When no frame is pending at that moment, the current size is taken as laid out, since a new size always schedules a frame, so that first request doesn't wait.
  - under a test binding (frames only on `pump`) it doesn't attach, and the earlier rule applies, as `ProbeSync` already does for settling. Without this, 9 executor widget tests hung.
  - `attach()` never throws.
- **Tests:**
  - `view_ready_lazy_attach_test.dart` (no binding in the file): `attach` does nothing; an app-defined `WidgetsFlutterBinding` subclass is then created without an assertion; the first request attaches, with `view_ready_ms` 0; a second request doesn't try again (`attachAttempts`).
  - `agent_start_no_binding_test.dart` (define-gated): `ProbeAgent.start()` with no binding doesn't attach, then the app's own binding subclass is created without an assertion, then attaching works.
  - `agent_start_with_binding_test.dart` (define-gated): `start()` after the app's binding attaches at once.
  - Both define-gated tests pass with `--dart-define=PROBE_AGENT=true`. The full suite: 266 passed, 2 skipped (those two).
- **Mutants:** 5 of 5 killed:
  - `start()` making the binding;
  - `start()` not attaching;
  - no lazy attach on a request;
  - lazy attach under a test binding (the hanging executor tests kill it);
  - a late attach not taking the settled size.
- **Device:** the final build, `buttons.probe` without waits on `Compat-Pixel-API35`, passed **10 of 10** with the machine otherwise idle. (The previous build's rerun was 8 of 10, both failures the reconnect timeout while my Go tests loaded the machine.) The fixture makes its binding before `start()`, so on the device the agent attaches at `start()`; the lazy path is covered by the unit tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Steps right after restart the app can run before the first frame has settled on slow devices

1 participant