Skip to content

fix(runner): a parallel run names each lane's device as plain text on one line - #776

Merged
pbertsch merged 3 commits into
mainfrom
fix/clean-parallel-device-labels
Nov 17, 2026
Merged

pbertsch merged 3 commits into
mainfrom
fix/clean-parallel-device-labels

Conversation

@pbertsch

Copy link
Copy Markdown
Member

Summary

A parallel run (--devices) now names each lane's device as plain text on one line. This is part 2 of 3. Part 1 (#775) covered the names printed by probe device, and part 3 covers the run's device context.

On main, internal/runner/orchestrator.go printed a lane's device id and name as adb or simctl reported them:

  • deviceLabel (the "Device N: …" lane list);
  • shortID (the [device] label of the progress, warning and "Connected" lines and of the result rows; it cut the id to 12 bytes, which can split a multi-byte character);
  • the [id] prefix of the port-forward, hold, token, connect and handshake errors;
  • the run's error summary.

A name can hold a new line, an escape sequence, a bidi override or a zero-width character, and so could print a second line that looked like probe's own ("pass --yes to continue").

The change

  • shortID cleans the id with device.CleanOneLine and cuts it to 12 characters. A plain serial reads exactly as before (emulator-5554 → emulator-555).
  • label (new) is the whole id for error messages, cleaned.
  • deviceLabel returns the cleaned name, or the cleaned id when there is no name.
  • Display only: the lane's DeviceID and DeviceName are unchanged. That is the id given to adb and simctl, used for the port, and written to the JSON and JUnit reports, whose encoders escape it.

Tests (through the real run)

  • TestParallelRunPrintsHostileDeviceLabelsCleaned runs a one-lane parallel run with the orchestrator harness. The lane's name and id hold a new line with a fake "pass --yes to continue" line, an escape sequence, an OSC title sequence, a bidi override and a zero-width space. No printed line forges its own line or carries one of those characters; the color codes probe adds itself are allowed. The name's visible text is shown.
  • TestFailedLaneErrorNamesTheDeviceCleaned runs a lane whose connect fails, because no fake agent is on its port. Its error and the run's summary name the hostile id on one clean line, and a lane with no name is listed by its cleaned id.
  • TestLaneLabelsAreDisplayOnly: the lane keeps its raw id and name; shortID cuts by characters (Japanese text) and leaves a plain serial as before.

Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 5 were killed. "label prints the raw id" survived at first, because the happy-path run never reaches an error message. The failed-lane test was added for it, and it is now killed.

Mutant Result
shortID prints the raw id killed
shortID cuts bytes killed
deviceLabel prints the raw name killed
label prints the raw id killed
the Connected line prints the raw id killed

Test plan

  • go test ./internal/runner/ passes.
  • go vet ./internal/runner/ passes.
  • Mutation check (table above).

@pbertsch
pbertsch merged commit 7be9895 into main Nov 17, 2026
10 of 11 checks passed
pbertsch added a commit that referenced this pull request Nov 17, 2026
…line (#786)

## Summary

A follow-up from the review of #776: a parallel lane's **error** is now printed as one clean line, as its device name already was.

The error of a lane that failed can carry text from the device or the agent: a refused dial or handshake, adb's output. `internal/runner/orchestrator.go` printed it with `%v` in four places, so a new line, an escape sequence or a bidi override in it reached the terminal (or the run's error) as is:
- the retry line, `[<lane>] attempt N failed: …`;
- the failure line, `[<lane>] all attempts failed: …`;
- the run's error, `device errors: [<lane>] …`;
- the device summary, `<lane>  error: …`.

All four now go through `errShown(err)`, which is `device.CleanOneLine(err.Error(), 400)`. The error itself is unchanged: the report's not-run results keep it as it is, and so do the retry decision (`errors.As` on `connectError`) and the caller.

## Tests

`TestFailedLaneErrorTextIsPrintedCleaned`: the dial fails with `refused\n  pass --yes to continue<ESC>[2J<RLO>end`. Across the run's output, `PrintSummary` and the run's error, it checks that:
- no line starts with a forged `pass --yes`;
- no line holds an escape, carriage return, bidi or zero-width character (probe's own color codes aside);
- the visible text is shown;
- the retry, failure and summary lines are present.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 4 were killed.

| Mutant | Result |
|---|---|
| retry line prints the raw error | killed |
| failure line prints the raw error | killed |
| run error holds the raw error | killed |
| device summary prints the raw error | killed |

## Test plan

- [x] `go test ./internal/runner/` passes.
- [x] `go vet` and `staticcheck` pass on `./internal/runner/`.
- [x] Mutation check (table above).
pbertsch added a commit that referenced this pull request Nov 17, 2026
… the app id and serial as plain text (#779)

## Summary

This makes the lines a run prints about the app and the device show the app id and the serial as plain text on one line. It is part 3 of 3: #775 covered `probe device` and `internal/device`, and #776 the parallel run's lane labels.

**On main**, these printed `dc.AppID` and `dc.Serial` (or `deviceSerial`) as given:
- `internal/runner/device_context.go`: the single-device warning `[serial]`, force-stopping, relaunching (×5), the reinstall question, clearing data, the data-path error, the set-language error, iOS notification reinstall, granting permissions, killing, launching;
- `internal/runner/reconnect_hint.go`: the three "the app … is not running" hints;
- `internal/runner/runner.go`: the pull error;
- `internal/cli/test.go`: "Physical device detected";
- `internal/cli/record.go`: "Recording on".

A serial comes from `adb` or `simctl`. The app id is validated when it comes from `probe.yaml`, but these lines print whatever the device context holds. A value with a new line, an escape sequence, a bidi override or a zero-width character could forge a line or change the terminal.

## The change

- `DeviceContext.appShown()` and `serialShown()` return `device.CleanOneLine(value, 80)`. Each of the 15 print sites in `device_context.go` and the hint and pull sites use them. The two CLI sites call `device.CleanOneLine` directly.
- **Display only:** every value given to `adb`, `simctl` or `devicectl`, and every comparison, still uses `dc.AppID` and `dc.Serial` as they are.
- The reconnect hint is only reached for an app id that passes `ValidPackageName`, so a hostile id never got there. It is cleaned anyway, the same way.
- Script-authored names printed by the runner (test, recipe and measurement names, mostly `%q`) are not device output and are not part of this change.

## Tests (through the real print path)

`TestLifecycleLinesPrintAppAndSerialCleaned` runs kill, launch, restart and clear data on a `DeviceContext` with the recording fake `adb` (`recordingDevice`). Its app id holds a new line with a fake "pass --yes to continue" line, an escape sequence, a bidi override and a zero-width space, and its serial an OSC title sequence. For each step:
- no printed line forges its own line or carries one of those characters (probe's own color codes are allowed);
- every `adb -s` call gets the **real** serial.

## Mutation check

Each mutant was applied, the test was run, and the file was restored byte for byte. All 5 were killed.

| Mutant | Result |
|---|---|
| `appShown` returns the raw app id | killed |
| `serialShown` returns the raw serial | killed |
| the kill line prints the raw app id | killed |
| the clear line prints the raw serial | killed |
| adb gets the cleaned serial (display-only rule broken) | killed |

## Not covered

- The two CLI lines ("Physical device detected", "Recording on") are cleaned with the same call but have no test of their own: they sit deep in the `probe test` and `probe record` flows.

## Test plan

- [x] `go test ./internal/runner/ ./internal/cli/` passes.
- [x] `go vet ./internal/runner/ ./internal/cli/` passes.
- [x] Mutation check (table above).


## Review follow-up

- **Windows build:** `printed_device_names_test.go` uses `recordingDevice`, which is defined only for `!windows`, so it now has the same build tag. `GOOS=windows go vet ./internal/...` passes.
- **More prints cleaned in `probe test`:**
  - "Connected to ProbeAgent on <serial>" (an auto-picked id comes from the device list and is not validated);
  - the `-v` "ensuring adb + device <serial> reachable" line;
  - the cloud run's "Connected to ProbeAgent on <device>";
  - both "App uploaded: <id>" lines (the id is the provider's API response).
- **Data-path errors** quote the path `simctl` returned with `%q`.
- **Tests for the other sites**, through their real paths with hostile values:
  - the three data-path errors;
  - set language on a physical iPhone;
  - grant all permissions (the grants still get the real id);
  - the reconnect hint, which is skipped for an app id that is not a package name, so its cleaning is a second line behind that gate.

  A failed pull is not printed, so its cleaning can't be observed; the CHANGELOG now says which sites are tested and which are a second line. A source check covers the `probe test` prints.
- **Mutants:** 10 of 10 killed.
  - raw data path (×3), raw serial in the data-path error;
  - raw app id in set language and in grant;
  - the hint's package-name gate removed;
  - raw connected, trace and upload prints.

## Second follow-up

- **Cloud prints:** the cloud upload and session start, which were the same code in two paths, are now one function, `uploadAndStartCloudSession`. Its "Starting cloud session on <device>" line and the relay "Connected to ProbeAgent via relay on <device>" line print the device cleaned, like the other cloud lines.
- **The provider gets the real values:** `TestCloudSessionGetsTheRealValuesAndPrintsThemCleaned` checks, with a fake provider, that `UploadApp` gets the app path and `StartSession` gets the id `UploadApp` returned and the target device as they are, while both printed lines are clean.
- **Mutants:** 5 of 5 killed: the provider gets a cleaned device, the provider gets a cleaned app id, and the starting, upload and relay lines printed raw.
- **CHANGELOG:** says the data-path errors quote the path with `%q`, and names the cloud lines.
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
… one line (AlphaWaveSystems#776)

## Summary

A parallel run (`--devices`) now names each lane's device as plain text on one line. This is part 2 of 3. Part 1 (AlphaWaveSystems#775) covered the names printed by `probe device`, and part 3 covers the run's device context.

**On main**, `internal/runner/orchestrator.go` printed a lane's device id and name as `adb` or `simctl` reported them:
- `deviceLabel` (the "Device N: …" lane list);
- `shortID` (the `[device]` label of the progress, warning and "Connected" lines and of the result rows; it cut the id to 12 **bytes**, which can split a multi-byte character);
- the `[id]` prefix of the port-forward, hold, token, connect and handshake errors;
- the run's error summary.

A name can hold a new line, an escape sequence, a bidi override or a zero-width character, and so could print a second line that looked like probe's own ("pass --yes to continue").

## The change

- **`shortID`** cleans the id with `device.CleanOneLine` and cuts it to 12 **characters**. A plain serial reads exactly as before (`emulator-5554` → `emulator-555`).
- **`label`** (new) is the whole id for error messages, cleaned.
- **`deviceLabel`** returns the cleaned name, or the cleaned id when there is no name.
- **Display only:** the lane's `DeviceID` and `DeviceName` are unchanged. That is the id given to `adb` and `simctl`, used for the port, and written to the JSON and JUnit reports, whose encoders escape it.

## Tests (through the real run)

- `TestParallelRunPrintsHostileDeviceLabelsCleaned` runs a one-lane parallel run with the orchestrator harness. The lane's name and id hold a new line with a fake "pass --yes to continue" line, an escape sequence, an OSC title sequence, a bidi override and a zero-width space. No printed line forges its own line or carries one of those characters; the color codes probe adds itself are allowed. The name's visible text is shown.
- `TestFailedLaneErrorNamesTheDeviceCleaned` runs a lane whose connect fails, because no fake agent is on its port. Its error and the run's summary name the hostile id on one clean line, and a lane with no name is listed by its cleaned id.
- `TestLaneLabelsAreDisplayOnly`: the lane keeps its raw id and name; `shortID` cuts by characters (Japanese text) and leaves a plain serial as before.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 5 were killed. "label prints the raw id" survived at first, because the happy-path run never reaches an error message. The failed-lane test was added for it, and it is now killed.

| Mutant | Result |
|---|---|
| `shortID` prints the raw id | killed |
| `shortID` cuts bytes | killed |
| `deviceLabel` prints the raw name | killed |
| `label` prints the raw id | killed |
| the Connected line prints the raw id | killed |

## Test plan

- [x] `go test ./internal/runner/` passes.
- [x] `go vet ./internal/runner/` passes.
- [x] Mutation check (table above).
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…line (AlphaWaveSystems#786)

## Summary

A follow-up from the review of AlphaWaveSystems#776: a parallel lane's **error** is now printed as one clean line, as its device name already was.

The error of a lane that failed can carry text from the device or the agent: a refused dial or handshake, adb's output. `internal/runner/orchestrator.go` printed it with `%v` in four places, so a new line, an escape sequence or a bidi override in it reached the terminal (or the run's error) as is:
- the retry line, `[<lane>] attempt N failed: …`;
- the failure line, `[<lane>] all attempts failed: …`;
- the run's error, `device errors: [<lane>] …`;
- the device summary, `<lane>  error: …`.

All four now go through `errShown(err)`, which is `device.CleanOneLine(err.Error(), 400)`. The error itself is unchanged: the report's not-run results keep it as it is, and so do the retry decision (`errors.As` on `connectError`) and the caller.

## Tests

`TestFailedLaneErrorTextIsPrintedCleaned`: the dial fails with `refused\n  pass --yes to continue<ESC>[2J<RLO>end`. Across the run's output, `PrintSummary` and the run's error, it checks that:
- no line starts with a forged `pass --yes`;
- no line holds an escape, carriage return, bidi or zero-width character (probe's own color codes aside);
- the visible text is shown;
- the retry, failure and summary lines are present.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 4 were killed.

| Mutant | Result |
|---|---|
| retry line prints the raw error | killed |
| failure line prints the raw error | killed |
| run error holds the raw error | killed |
| device summary prints the raw error | killed |

## Test plan

- [x] `go test ./internal/runner/` passes.
- [x] `go vet` and `staticcheck` pass on `./internal/runner/`.
- [x] Mutation check (table above).
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
… the app id and serial as plain text (AlphaWaveSystems#779)

## Summary

This makes the lines a run prints about the app and the device show the app id and the serial as plain text on one line. It is part 3 of 3: AlphaWaveSystems#775 covered `probe device` and `internal/device`, and AlphaWaveSystems#776 the parallel run's lane labels.

**On main**, these printed `dc.AppID` and `dc.Serial` (or `deviceSerial`) as given:
- `internal/runner/device_context.go`: the single-device warning `[serial]`, force-stopping, relaunching (×5), the reinstall question, clearing data, the data-path error, the set-language error, iOS notification reinstall, granting permissions, killing, launching;
- `internal/runner/reconnect_hint.go`: the three "the app … is not running" hints;
- `internal/runner/runner.go`: the pull error;
- `internal/cli/test.go`: "Physical device detected";
- `internal/cli/record.go`: "Recording on".

A serial comes from `adb` or `simctl`. The app id is validated when it comes from `probe.yaml`, but these lines print whatever the device context holds. A value with a new line, an escape sequence, a bidi override or a zero-width character could forge a line or change the terminal.

## The change

- `DeviceContext.appShown()` and `serialShown()` return `device.CleanOneLine(value, 80)`. Each of the 15 print sites in `device_context.go` and the hint and pull sites use them. The two CLI sites call `device.CleanOneLine` directly.
- **Display only:** every value given to `adb`, `simctl` or `devicectl`, and every comparison, still uses `dc.AppID` and `dc.Serial` as they are.
- The reconnect hint is only reached for an app id that passes `ValidPackageName`, so a hostile id never got there. It is cleaned anyway, the same way.
- Script-authored names printed by the runner (test, recipe and measurement names, mostly `%q`) are not device output and are not part of this change.

## Tests (through the real print path)

`TestLifecycleLinesPrintAppAndSerialCleaned` runs kill, launch, restart and clear data on a `DeviceContext` with the recording fake `adb` (`recordingDevice`). Its app id holds a new line with a fake "pass --yes to continue" line, an escape sequence, a bidi override and a zero-width space, and its serial an OSC title sequence. For each step:
- no printed line forges its own line or carries one of those characters (probe's own color codes are allowed);
- every `adb -s` call gets the **real** serial.

## Mutation check

Each mutant was applied, the test was run, and the file was restored byte for byte. All 5 were killed.

| Mutant | Result |
|---|---|
| `appShown` returns the raw app id | killed |
| `serialShown` returns the raw serial | killed |
| the kill line prints the raw app id | killed |
| the clear line prints the raw serial | killed |
| adb gets the cleaned serial (display-only rule broken) | killed |

## Not covered

- The two CLI lines ("Physical device detected", "Recording on") are cleaned with the same call but have no test of their own: they sit deep in the `probe test` and `probe record` flows.

## Test plan

- [x] `go test ./internal/runner/ ./internal/cli/` passes.
- [x] `go vet ./internal/runner/ ./internal/cli/` passes.
- [x] Mutation check (table above).


## Review follow-up

- **Windows build:** `printed_device_names_test.go` uses `recordingDevice`, which is defined only for `!windows`, so it now has the same build tag. `GOOS=windows go vet ./internal/...` passes.
- **More prints cleaned in `probe test`:**
  - "Connected to ProbeAgent on <serial>" (an auto-picked id comes from the device list and is not validated);
  - the `-v` "ensuring adb + device <serial> reachable" line;
  - the cloud run's "Connected to ProbeAgent on <device>";
  - both "App uploaded: <id>" lines (the id is the provider's API response).
- **Data-path errors** quote the path `simctl` returned with `%q`.
- **Tests for the other sites**, through their real paths with hostile values:
  - the three data-path errors;
  - set language on a physical iPhone;
  - grant all permissions (the grants still get the real id);
  - the reconnect hint, which is skipped for an app id that is not a package name, so its cleaning is a second line behind that gate.

  A failed pull is not printed, so its cleaning can't be observed; the CHANGELOG now says which sites are tested and which are a second line. A source check covers the `probe test` prints.
- **Mutants:** 10 of 10 killed.
  - raw data path (×3), raw serial in the data-path error;
  - raw app id in set language and in grant;
  - the hint's package-name gate removed;
  - raw connected, trace and upload prints.

## Second follow-up

- **Cloud prints:** the cloud upload and session start, which were the same code in two paths, are now one function, `uploadAndStartCloudSession`. Its "Starting cloud session on <device>" line and the relay "Connected to ProbeAgent via relay on <device>" line print the device cleaned, like the other cloud lines.
- **The provider gets the real values:** `TestCloudSessionGetsTheRealValuesAndPrintsThemCleaned` checks, with a fake provider, that `UploadApp` gets the app path and `StartSession` gets the id `UploadApp` returned and the target device as they are, while both printed lines are clean.
- **Mutants:** 5 of 5 killed: the provider gets a cleaned device, the provider gets a cleaned app id, and the starting, upload and relay lines printed raw.
- **CHANGELOG:** says the data-path errors quote the path with `%q`, and names the cloud lines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant