Repository navigation
fix(runner): the reports keep only files the run wrote, or a simulator's screenshot - #783
Merged
Merged
Conversation
… a host path as it is
… and save a copy, so a swapped link is never followed
…st-paths # Conflicts: # CHANGELOG.md
Merged
4 tasks done
…nto fix/pull-artifacts-confined # Conflicts: # CHANGELOG.md
…swapped link is never followed
pbertsch
added a commit
that referenced
this pull request
Nov 17, 2026
… a host path as it is (#782) ## Summary A screenshot path from the agent is no longer used as a host path as it is. **On main:** - `internal/probelink/client.go` `Client.Screenshot` and `http_client.go` `HTTPClient.Screenshot` saved the image when the answer carried `data`. Otherwise, and also when the data did not decode, they **returned the agent's `path` as the host path**. - The runner then used that path as a host file: - `captureRedactedScreenshot` (`executor.go`, the `with ai` steps) read it with `os.ReadFile` and, with `ai.redact` rules, **wrote the redacted image back over it** with `os.WriteFile(path, …, 0644)`. Without redact rules the bytes went on as the screenshot, for example to the AI provider. - `take screenshot`, `compare screenshot` (which crops next to it) and the failure screenshot attach or read it. So an agent could make probe read, copy or overwrite another host image. The current agent always sends the image in `data` (`probe_agent/lib/src/executor.dart`, "The image always travels in `data`"). Only an older agent, or a hostile one, sends a path without it. ## The change `savedScreenshot` (`internal/probelink/screenshot_save.go`) is now the only way an answer becomes a host path, and both clients return it: - **With data:** the image is saved in `reports/screenshots` under the base name of the agent's path, in a **new file** (`O_CREATE|O_EXCL`). If that name is taken, `-2`, `-3`… is added. An existing file, or a planted link of that name, is never written through. The name always ends in `.png`. It becomes `screenshot.png` when it can't be used on every system: `.`, `..`, nothing, a trailing dot or space, or a name Windows reserves (`CON`, `PRN`, `AUX`, `NUL`, `COM0`–`COM9`, `LPT0`–`LPT9`, with or without an extension). Undecodable data is an error, no longer a fallback to the path. - **Without data** (what an older agent on an iOS simulator sends, since a simulator shares the host's file system), the file is read by `internal/confined.ReadFile`, which has **no check-then-use gap**: - The file is **opened once**, with `O_NOFOLLOW | O_NONBLOCK`. A link as the last part of the path is refused, and a FIFO can't block the run. - On the **open file**: `Fstat` must say it is a regular file with **exactly one name** (`nlink == 1`), so a hard link can't give a host file an allowed name. - Its **real path is asked of the open descriptor**, not of the path: `fcntl(F_GETPATH)` on macOS, `/proc/self/fd/N` on Linux, `GetFinalPathNameByHandleW` on Windows. It must be in a simulator's data folder, `<simulators>/<UUID>/data/…`. - The bytes are **read from that same descriptor** (at most 64 MiB). - `savedScreenshot` saves those bytes as a new file in `reports/screenshots`, like data, and returns that path. **No consumer ever opens the agent's path**: not redaction, not the AI step, not `ScreenshotBytes`' fallback, not `compare screenshot` or `PullArtifacts`. A link swapped in after the check is never followed. - Anything else is an error naming the path (`%q`). An older agent on Android or a physical iPhone, whose path isn't on the host anyway, gets that error. - **Redaction** (`captureRedactedScreenshot`) writes the redacted image to a **new file**, `reports/screenshots/<name>-redacted-*.png` (`os.CreateTemp`: exclusive, mode 0600), and returns that path. It never writes over the file the screenshot came from. - The five file helpers in `http_client.go`, which were only used by the old save, are removed. ## Tests - `TestScreenshotWithoutDataRefusesAHostFile`: with no data, these are refused: a host file outside a simulator folder, `/etc/passwd`, a relative path, a non-UUID folder, a non-`.png` file, a file outside `data`, a folder named like a png, a `..` traversal out of the simulator folder, nothing, and **a link from a simulator's data folder to a host file**. A real screenshot in a simulator's data folder is accepted: its bytes are **copied into `reports/screenshots`**, and that copy is returned. The host file is unchanged. - `TestScreenshotWithoutDataSurvivesASwap`: **the race**. A goroutine renames, as fast as it can, a regular screenshot and a link to a host secret over the same path in a fake `<UUID>/data`, while the test reads it for 2 s. Last run: 102,838 reads, 27,852 returned the screenshot, **0 returned the secret**. The test also fails if no read returned the screenshot, so it can't pass by refusing everything. (Its first version swapped two links, so every read was refused and the read-by-path mutant survived; that was fixed.) - `TestScreenshotWithoutDataRefusesAHardLink`, `…RefusesALinkInsideTheSimulator` (a link even to another file in `data`), and `…RefusesAFIFO` (Unix: neither accepted nor blocking; 3 s guard). - `TestScreenshotFileName`: `.png` is forced; Windows reserved names (`CON`, `nul.png`, `COM1.png`, `lpt9`) and trailing dot or space give `screenshot.png`; `COM10.png` and `console.png` are kept. - `TestScreenshotWithDataIsANewFileInTheScreenshotsFolder`: hostile agent paths (`../../../x.png`, `/etc/x.png`, `..`, empty, a Windows path) are all saved inside the folder. An existing `taken.png` and a planted link `linked.png` get new names; the old file and the link's target are unchanged. - `TestScreenshotWithBadDataIsAnError`: undecodable data is an error even with a path that would pass. - `TestScreenshotWithoutAUsableNameIsScreenshotPNG`. - `TestBothClientsUseSavedScreenshot`: source check of both clients, and the HTTP client against a fake agent server that answers `{"path":"/etc/passwd"}`. - `TestRedactionNeverWritesOverTheScreenshotItCameFrom`: the source PNG is byte-identical after a redacted capture. The new file is in `reports/screenshots`, holds the redacted image and has mode 0600. - The existing `TestScreenshotStillSavesAFileForTestRuns` passes unchanged: the base name is kept. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. **Round 2 (this push): 11 of 11 killed.** - First run: 3 survived, 1 did not compile. - "The regular-file check is gone" survived because a folder fails to read anyway. That led to the FIFO case, and to `O_NONBLOCK`: a FIFO used to block the open. - "O_NOFOLLOW is dropped" survived because the real-path check covered it. That led to the link-inside-the-simulator case. - "The bytes are read again by path" survived because the swap test was too weak; it was fixed (see Tests). - "The real path must end in .png" was **equivalent**, because with `O_NOFOLLOW` the base name cannot change, so that check was removed. - One mutant was rewritten to compile. | Mutant | Result | |---|---| | the opened file's real path is not checked | killed | | the link count is not checked | killed | | the regular-file check is gone | killed | | **the bytes are read again by path after the check (the race)** | killed | | `O_NOFOLLOW` is dropped | killed | | no data: the agent's path is returned | killed | | the UUID folder check is gone | killed | | the `data` folder check is gone | killed | | Windows reserved names allowed | killed | | trailing dot or space allowed | killed | | `.png` not forced | killed | **Round 1, still valid:** - The with-data mutants: `O_TRUNC`, bad data falling back to the path, the base name not checked, the HTTP client returning the agent's path. All killed. - Redaction writing over the source: rerun on this push, killed. ## Not covered - The WebSocket client (`Client.Screenshot`) is checked through its source and through the shared `savedScreenshot`, not against a fake WebSocket agent. - `PullArtifacts` (an agent-named host file copied into the reports folder) is a separate change (#783). It gets the same open-once read. - `internal/confined` has no test file of its own: every branch of it is exercised through the probelink tests above, and each check has a mutant. On Windows it is only built and vetted here (`GOOS=windows go vet`, `staticcheck`), not run. ## Test plan - [x] `go test ./internal/probelink/ ./internal/runner/ ./internal/cli/` passes. - [x] `go vet` and `staticcheck` pass on `./internal/probelink/`, `./internal/confined/` and `./internal/runner/`, and with `GOOS=linux` and `GOOS=windows` on `confined` and `probelink`. - [x] No raw format characters in the changed files. - [x] Mutation check (table above).
…nfined # Conflicts: # CHANGELOG.md # internal/confined/confined.go
…file's folder; too-large message; any-case video extensions; confined tests
…k, never as wide as /, the working or the home folder
…r opened once, and only when it is the folder that was checked
jay23606
added a commit
to jay23606/flutter-probe
that referenced
this pull request
Nov 17, 2026
… a host path as it is (AlphaWaveSystems#782) ## Summary A screenshot path from the agent is no longer used as a host path as it is. **On main:** - `internal/probelink/client.go` `Client.Screenshot` and `http_client.go` `HTTPClient.Screenshot` saved the image when the answer carried `data`. Otherwise, and also when the data did not decode, they **returned the agent's `path` as the host path**. - The runner then used that path as a host file: - `captureRedactedScreenshot` (`executor.go`, the `with ai` steps) read it with `os.ReadFile` and, with `ai.redact` rules, **wrote the redacted image back over it** with `os.WriteFile(path, …, 0644)`. Without redact rules the bytes went on as the screenshot, for example to the AI provider. - `take screenshot`, `compare screenshot` (which crops next to it) and the failure screenshot attach or read it. So an agent could make probe read, copy or overwrite another host image. The current agent always sends the image in `data` (`probe_agent/lib/src/executor.dart`, "The image always travels in `data`"). Only an older agent, or a hostile one, sends a path without it. ## The change `savedScreenshot` (`internal/probelink/screenshot_save.go`) is now the only way an answer becomes a host path, and both clients return it: - **With data:** the image is saved in `reports/screenshots` under the base name of the agent's path, in a **new file** (`O_CREATE|O_EXCL`). If that name is taken, `-2`, `-3`… is added. An existing file, or a planted link of that name, is never written through. The name always ends in `.png`. It becomes `screenshot.png` when it can't be used on every system: `.`, `..`, nothing, a trailing dot or space, or a name Windows reserves (`CON`, `PRN`, `AUX`, `NUL`, `COM0`–`COM9`, `LPT0`–`LPT9`, with or without an extension). Undecodable data is an error, no longer a fallback to the path. - **Without data** (what an older agent on an iOS simulator sends, since a simulator shares the host's file system), the file is read by `internal/confined.ReadFile`, which has **no check-then-use gap**: - The file is **opened once**, with `O_NOFOLLOW | O_NONBLOCK`. A link as the last part of the path is refused, and a FIFO can't block the run. - On the **open file**: `Fstat` must say it is a regular file with **exactly one name** (`nlink == 1`), so a hard link can't give a host file an allowed name. - Its **real path is asked of the open descriptor**, not of the path: `fcntl(F_GETPATH)` on macOS, `/proc/self/fd/N` on Linux, `GetFinalPathNameByHandleW` on Windows. It must be in a simulator's data folder, `<simulators>/<UUID>/data/…`. - The bytes are **read from that same descriptor** (at most 64 MiB). - `savedScreenshot` saves those bytes as a new file in `reports/screenshots`, like data, and returns that path. **No consumer ever opens the agent's path**: not redaction, not the AI step, not `ScreenshotBytes`' fallback, not `compare screenshot` or `PullArtifacts`. A link swapped in after the check is never followed. - Anything else is an error naming the path (`%q`). An older agent on Android or a physical iPhone, whose path isn't on the host anyway, gets that error. - **Redaction** (`captureRedactedScreenshot`) writes the redacted image to a **new file**, `reports/screenshots/<name>-redacted-*.png` (`os.CreateTemp`: exclusive, mode 0600), and returns that path. It never writes over the file the screenshot came from. - The five file helpers in `http_client.go`, which were only used by the old save, are removed. ## Tests - `TestScreenshotWithoutDataRefusesAHostFile`: with no data, these are refused: a host file outside a simulator folder, `/etc/passwd`, a relative path, a non-UUID folder, a non-`.png` file, a file outside `data`, a folder named like a png, a `..` traversal out of the simulator folder, nothing, and **a link from a simulator's data folder to a host file**. A real screenshot in a simulator's data folder is accepted: its bytes are **copied into `reports/screenshots`**, and that copy is returned. The host file is unchanged. - `TestScreenshotWithoutDataSurvivesASwap`: **the race**. A goroutine renames, as fast as it can, a regular screenshot and a link to a host secret over the same path in a fake `<UUID>/data`, while the test reads it for 2 s. Last run: 102,838 reads, 27,852 returned the screenshot, **0 returned the secret**. The test also fails if no read returned the screenshot, so it can't pass by refusing everything. (Its first version swapped two links, so every read was refused and the read-by-path mutant survived; that was fixed.) - `TestScreenshotWithoutDataRefusesAHardLink`, `…RefusesALinkInsideTheSimulator` (a link even to another file in `data`), and `…RefusesAFIFO` (Unix: neither accepted nor blocking; 3 s guard). - `TestScreenshotFileName`: `.png` is forced; Windows reserved names (`CON`, `nul.png`, `COM1.png`, `lpt9`) and trailing dot or space give `screenshot.png`; `COM10.png` and `console.png` are kept. - `TestScreenshotWithDataIsANewFileInTheScreenshotsFolder`: hostile agent paths (`../../../x.png`, `/etc/x.png`, `..`, empty, a Windows path) are all saved inside the folder. An existing `taken.png` and a planted link `linked.png` get new names; the old file and the link's target are unchanged. - `TestScreenshotWithBadDataIsAnError`: undecodable data is an error even with a path that would pass. - `TestScreenshotWithoutAUsableNameIsScreenshotPNG`. - `TestBothClientsUseSavedScreenshot`: source check of both clients, and the HTTP client against a fake agent server that answers `{"path":"/etc/passwd"}`. - `TestRedactionNeverWritesOverTheScreenshotItCameFrom`: the source PNG is byte-identical after a redacted capture. The new file is in `reports/screenshots`, holds the redacted image and has mode 0600. - The existing `TestScreenshotStillSavesAFileForTestRuns` passes unchanged: the base name is kept. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. **Round 2 (this push): 11 of 11 killed.** - First run: 3 survived, 1 did not compile. - "The regular-file check is gone" survived because a folder fails to read anyway. That led to the FIFO case, and to `O_NONBLOCK`: a FIFO used to block the open. - "O_NOFOLLOW is dropped" survived because the real-path check covered it. That led to the link-inside-the-simulator case. - "The bytes are read again by path" survived because the swap test was too weak; it was fixed (see Tests). - "The real path must end in .png" was **equivalent**, because with `O_NOFOLLOW` the base name cannot change, so that check was removed. - One mutant was rewritten to compile. | Mutant | Result | |---|---| | the opened file's real path is not checked | killed | | the link count is not checked | killed | | the regular-file check is gone | killed | | **the bytes are read again by path after the check (the race)** | killed | | `O_NOFOLLOW` is dropped | killed | | no data: the agent's path is returned | killed | | the UUID folder check is gone | killed | | the `data` folder check is gone | killed | | Windows reserved names allowed | killed | | trailing dot or space allowed | killed | | `.png` not forced | killed | **Round 1, still valid:** - The with-data mutants: `O_TRUNC`, bad data falling back to the path, the base name not checked, the HTTP client returning the agent's path. All killed. - Redaction writing over the source: rerun on this push, killed. ## Not covered - The WebSocket client (`Client.Screenshot`) is checked through its source and through the shared `savedScreenshot`, not against a fake WebSocket agent. - `PullArtifacts` (an agent-named host file copied into the reports folder) is a separate change (AlphaWaveSystems#783). It gets the same open-once read. - `internal/confined` has no test file of its own: every branch of it is exercised through the probelink tests above, and each check has a mutant. On Windows it is only built and vetted here (`GOOS=windows go vet`, `staticcheck`), not run. ## Test plan - [x] `go test ./internal/probelink/ ./internal/runner/ ./internal/cli/` passes. - [x] `go vet` and `staticcheck` pass on `./internal/probelink/`, `./internal/confined/` and `./internal/runner/`, and with `GOOS=linux` and `GOOS=windows` on `confined` and `probelink`. - [x] No raw format characters in the changed files. - [x] Mutation check (table above).
jay23606
added a commit
to jay23606/flutter-probe
that referenced
this pull request
Nov 17, 2026
…r's screenshot (AlphaWaveSystems#783) ## Summary > AlphaWaveSystems#782 (which added `internal/confined`) is merged, and `main` is merged in, so this diff is only this PR's change. The reports now keep only files the run wrote, or a simulator's screenshot. **On main**, `runner.PullArtifacts` (called by `probe test` after a run, `internal/cli/test.go`) took each test's artifact paths and: - on **iOS**, copied whatever host file the path named into the reports folder (`copyFileIfDifferent(remotePath, …)`); - on **Android**, copied any host file that existed at an absolute artifact path; - **kept any existing absolute `.mov`, `.mp4` or `.webm` path** in the report as it was; - wrote the copy with `os.WriteFile`, which follows a link already at the destination. An artifact path can come from the agent: a screenshot path, and so the crop beside it. AlphaWaveSystems#782 stops the screenshot path at the client. This change confines `PullArtifacts` itself, so the reports (which can be uploaded) never take another host file, whatever produced the path. ## The change - **Every host file is read once from the open file.** It goes through `internal/confined.ReadFile` (from AlphaWaveSystems#782), so there is no gap between checking and reading: - open once with `O_NOFOLLOW | O_NONBLOCK`; - on the open file, `Fstat` must say it is a regular file with exactly one name; - its real path is asked of the descriptor (`F_GETPATH`, `/proc/self/fd`, `GetFinalPathNameByHandleW`); - the bytes are read from that descriptor. `copyHostArtifact` then writes those bytes with `writeFileReplacing`. The real path must be in one of these: - `localDir`, the screenshots folder; - the folders the caller adds. `probe test` adds four: the video folder; `./reports/screenshots`, where the probelink client saves the agent's screenshots (a crop and a redacted copy go next to them); `./reports/visual-diff`, for the compare screenshot diffs; and nothing else; - **a simulator's data folder** (`~/Library/Developer/CoreSimulator/Devices/<UUID>/data/…`, checked with `ios.ValidSimulatorUDID`). Roots are resolved once (`resolvedRoots`). Anything else is left out of the artifacts. **No parent folder is a root.** The first version allowed `filepath.Dir(localDir)`. With `-o`, `localDir` is `Dir(outFile)/screenshots`, so that root was the folder of the `-o` file: the working folder for `-o result.json`, the home folder for `-o ~/x.json`, `/tmp` for `-o /tmp/r.json`. An agent could then have `probe.yaml` or `sub/id_rsa` copied into the reports. Found in review and reproduced; fixed here. - **A file over 64 MiB** is reported as too large (`confined.ErrTooLarge`), not as "not a file this run wrote". - **iOS**: `copyHostArtifact` replaces `copyFileIfDifferent`; that helper and `copyFile` are removed. If the file already is the destination, nothing is written. - **Android host branch**: the same `copyHostArtifact`, now computed before the `switch` instead of inside a `case` condition. A path it refuses goes on to the device branch, which only reads the app's private storage, as before. - **Videos** (`.mov`, `.mp4`, `.webm`, **in any case**) are not read (they can be large). One is kept in place only when `confined.Check`, the same open-once check without the read, finds its real path in the **run's own folders**. - A video in a simulator's data folder is no longer kept: an app controls that folder, and nothing records videos there. - The report keeps the path as given, as before. - **What reads a kept path later.** Nothing in probe opens it again: the console reporter, JUnit (`artifactLines`), the JSON report, the HTML report (a link relative to the report's folder), the cloud result JSON (`screenshot_url`) and `probe report` only store or print the path string. A person who later opens the HTML report loads the file at that path in their browser. That is the run's own video folder, which an agent can't write to (it writes only into a simulator's data folder, which is not a video root). - **Writes** into the reports go to a new temporary file and are renamed into place (`writeFileReplacing`), so a link already at the destination is replaced, never written through. ## Tests `internal/runner/pull_artifacts_confined_test.go`: - iOS: a host file outside the run is neither copied nor listed; - a screenshot in a simulator's data folder is copied; a file outside `data`, or in a folder that is not a UUID, is not; - a link inside the run's folders that points out of them is refused; - Android: a host file outside the run is not copied; - only the run's videos are kept, and a folder named like a video is not kept; - a link at the destination is replaced, and its target is unchanged. - **the `-o ~/x.json` case**: with the screenshots folder in a "home" folder, a hostile agent names `home/probe.yaml`, `home/sub/id_rsa` and `home/notes.MOV`. On iOS and on Android nothing is copied and nothing is listed; - an upper-case `CLIP.MOV` in the video folder is kept in place, not copied; - a file over 64 MiB gives the too-large error. The existing Android host-copy test passed only because Go's `t.TempDir()` folders share a parent, which the old parent root allowed. It now passes the source's folder as a root, standing for the probelink save folder. `internal/confined/confined_test.go` (new): a regular file is read and its real path returned. These are refused: a file outside, a folder, a missing file, a link out, a link even to an allowed file, a hard link, a file `allowed` rejects, and (on Unix) a FIFO, without blocking. **Size cap:** exactly 64 MiB is read, 64 MiB + 1 is `ErrTooLarge`, and `Check` has no limit (sparse files). Also `Under`. `internal/runner/pull_artifacts_swap_test.go` (darwin and linux): - **the race**: a goroutine renames a regular screenshot and a link to a host secret over the same path in a fake `<UUID>/data`, while `PullArtifacts` runs for 2 s. Last run: 9,443 runs, 6,549 copies, **0 copies holding the secret**. The test fails if nothing was copied, so it can't pass by refusing everything; - a hard link to a host file in a simulator's data folder is not copied; - a video in a simulator's data folder is not kept. `internal/cli/pull_artifacts_roots_test.go`: a source check that `probe test` passes `./reports/screenshots`. The existing `PullArtifacts` tests (FP-19, the Android host copy) pass unchanged. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. On this push, 20 of 20 were killed. - First run: 2 survived. "Video extension case-sensitive" was missing an upper-case video *inside* the video folder. "Too-large reported as not the run's" was missed because its test's name didn't match the mutant runner's `-run`. Both were fixed and rerun. | Mutant | Result | |---|---| | iOS checks, then reads by path (the race) | killed | | every real path is allowed | killed | | the Android host copy has no check | killed | | a video is kept without a check | killed | | a video in a simulator's data folder is kept | killed | | **the `-o` file's folder (`Dir(localDir)`) is a root again** | killed | | `localDir` is not a root | killed | | the video extension check is case-sensitive | killed | | too large is reported as "not the run's" | killed | | `confined`: no size cap | killed | | `confined`: cap off by one (`LimitReader(maxSize)`) | killed | | `confined`: `allowed` not asked | killed | | `confined`: link count ignored | killed | | `probe test` passes the `-o` folder | killed | | `probe test` passes no video folder | killed | | `probe test` passes no client folder | killed | | roots are not resolved | killed | | the simulator `data` check is gone | killed | | the simulator UUID check is gone | killed | | the copy writes through a link | killed | ## Test plan - [x] `go test ./internal/runner/ ./internal/cli/ ./internal/confined/ ./internal/probelink/` passes. - [x] `go vet` and `staticcheck` pass on `runner`, `confined` and `cli`, and with `GOOS=windows`: `go vet ./internal/...` and `staticcheck` on `runner` and `confined`. - [x] No raw format characters in the changed files. - [x] Mutation check (table above). ## Root hardening (review follow-up) A future caller's mistake must not become allow-all. `artifactRoot` checks every root: - **MkdirAll first, then resolve.** `localDir` is created, then resolved with links followed. - **Not a link.** A root whose last part is a link is refused (`Lstat`). A screenshots folder planted as a link to a host folder would otherwise make that folder allowed. When `localDir` itself is refused, nothing is copied (that would write through the link) and no path the agent named is kept. **Behavior change:** if someone made the screenshots folder itself a symlink on purpose, a run now copies no artifacts. A symlinked parent, such as `reports/` or macOS's `/var`, is fine. - **Not too wide.** A root that is a filesystem root, the working folder, the home folder, or a folder above either is refused. So are `.` and `/`. - **Absolute.** A folder the caller adds must be absolute. `localDir` may be relative to the working folder; that is how `probe test` passes it. Tests (`pull_artifacts_roots_test.go`, darwin and linux): - `.`, `/`, a relative `sub`, the home folder and its parent as extra roots: nothing copied or listed, on iOS and Android; - a linked screenshots folder: nothing listed, and nothing written into the link's target; - a screenshots folder that did not exist before the run is created and used; - the working folder as the screenshots folder is refused; - `/` is refused with the working/home check switched off. Mutants on this push: 26 of 26 killed. - **6 new:** the roots resolved before `MkdirAll`; relative extra roots allowed; the `Lstat` link check gone; the filesystem-root check gone; the wide-folder check gone; a refused `localDir` used anyway. - Two survived on the first run: the link check was also caught by `IsDir` on `Lstat`, and `/` was also caught by the wide check. The link mutant now removes the whole `Lstat` check, and a test with the wide check off covers `/`. - **17** from the earlier set, still valid. - **3** re-expressed for the new code: the parent of `localDir` as a root again, `localDir` not a root, roots not resolved. ## Write side (review follow-up) The review found that the copies were written to `filepath.Join(localDir, name)`, the unresolved path, while the check was made on the resolved folder. A process of the same user swapping the folder's last part for a link to a host folder landed 12,220 of 185,605 files in the host folder. Now: - **`openArtifactDir`** opens the screenshots folder once with `os.OpenRoot` (Go 1.24+). It keeps the folder only when the opened folder is the folder that was checked: `os.SameFile` against what `Lstat` said of the unresolved path in `checkedArtifactRoot`, before its real path was read. That closes two gaps: - a swap between the check and the open; - a swap between the `Lstat` and the `EvalSymlinks` of the check. My own flipper test found this one (1 file in about 107,000 runs) before the fix. - **`writeInDir`** writes every copy, from the iOS branch, the Android host branch and the Android device branch, **through that open folder**: a new temp file (`O_EXCL`), then `Root.Rename`. A link swapped into the folder's place after the open can't redirect a write, and `os.Root` refuses to follow a link out of the folder. - `writeFileReplacing` (path-based) is removed. A file is named by its base name only. The listed path is the same as before (the absolute path under `localDir`), so report paths don't change. - The `ModeSymlink` test next to `IsDir` on an `Lstat` was redundant: to `Lstat`, a link is not a directory. That is why the reviewer's mutant removing it survived. It is removed, with a comment saying why. Tests (darwin and linux): - `TestPullArtifactsWritesSurviveAFolderSwap`: a goroutine keeps parking the real folder, putting a link to a host folder in its place for about 100 µs and putting the folder back, moving aside any folder `MkdirAll` recreated so the flipping never stops. Three runs: 115,852 / 111,533 / 115,429 runs, about 4,200 copies each, **0 files in the host folder**. It fails if nothing was copied. - `TestPullArtifactsRefusesAFolderSwappedBeforeItIsResolved` and `TestOpenArtifactDirRefusesAFolderSwappedBeforeTheOpen`: deterministic swaps at each gap, through test hooks. - `TestOpenArtifactDirRefusesALink`. Mutants on this push: every one is killed, 30 in all. - **6 write-side:** - each write reopens the folder by path (the old bug); - the opened folder not compared with the checked one; - the opened folder compared with a fresh `Lstat` (the resolve gap); - the `Lstat` in `openArtifactDir` gone; - the `Lstat` in `artifactRoot` gone; - a write that follows a link at the name. - **22** from the earlier sets, still valid. - **2** re-expressed: resolved before `MkdirAll`; the Android host copy without a check. - At first, three survived because two tests' names didn't match the mutant runner's `-run` and the flipper got stuck; both were fixed. Then one survived because the resolve gap is rarely hit by chance; the hook test covers it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The reports now keep only files the run wrote, or a simulator's screenshot.
On main,
runner.PullArtifacts(called byprobe testafter a run,internal/cli/test.go) took each test's artifact paths and:copyFileIfDifferent(remotePath, …));.mov,.mp4or.webmpath in the report as it was;os.WriteFile, which follows a link already at the destination.An artifact path can come from the agent: a screenshot path, and so the crop beside it. #782 stops the screenshot path at the client. This change confines
PullArtifactsitself, so the reports (which can be uploaded) never take another host file, whatever produced the path.The change
Every host file is read once from the open file. It goes through
internal/confined.ReadFile(from fix(probelink): a screenshot path from the agent is no longer used as a host path as it is #782), so there is no gap between checking and reading:O_NOFOLLOW | O_NONBLOCK;Fstatmust say it is a regular file with exactly one name;F_GETPATH,/proc/self/fd,GetFinalPathNameByHandleW);copyHostArtifactthen writes those bytes withwriteFileReplacing. The real path must be in one of these:localDir, the screenshots folder;probe testadds four: the video folder;./reports/screenshots, where the probelink client saves the agent's screenshots (a crop and a redacted copy go next to them);./reports/visual-diff, for the compare screenshot diffs; and nothing else;~/Library/Developer/CoreSimulator/Devices/<UUID>/data/…, checked withios.ValidSimulatorUDID).Roots are resolved once (
resolvedRoots). Anything else is left out of the artifacts.No parent folder is a root. The first version allowed
filepath.Dir(localDir). With-o,localDirisDir(outFile)/screenshots, so that root was the folder of the-ofile: the working folder for-o result.json, the home folder for-o ~/x.json,/tmpfor-o /tmp/r.json. An agent could then haveprobe.yamlorsub/id_rsacopied into the reports. Found in review and reproduced; fixed here.A file over 64 MiB is reported as too large (
confined.ErrTooLarge), not as "not a file this run wrote".iOS:
copyHostArtifactreplacescopyFileIfDifferent; that helper andcopyFileare removed. If the file already is the destination, nothing is written.Android host branch: the same
copyHostArtifact, now computed before theswitchinstead of inside acasecondition. A path it refuses goes on to the device branch, which only reads the app's private storage, as before.Videos (
.mov,.mp4,.webm, in any case) are not read (they can be large). One is kept in place only whenconfined.Check, the same open-once check without the read, finds its real path in the run's own folders.artifactLines), the JSON report, the HTML report (a link relative to the report's folder), the cloud result JSON (screenshot_url) andprobe reportonly store or print the path string. A person who later opens the HTML report loads the file at that path in their browser. That is the run's own video folder, which an agent can't write to (it writes only into a simulator's data folder, which is not a video root).Writes into the reports go to a new temporary file and are renamed into place (
writeFileReplacing), so a link already at the destination is replaced, never written through.Tests
internal/runner/pull_artifacts_confined_test.go:iOS: a host file outside the run is neither copied nor listed;
a screenshot in a simulator's data folder is copied; a file outside
data, or in a folder that is not a UUID, is not;a link inside the run's folders that points out of them is refused;
Android: a host file outside the run is not copied;
only the run's videos are kept, and a folder named like a video is not kept;
a link at the destination is replaced, and its target is unchanged.
the
-o ~/x.jsoncase: with the screenshots folder in a "home" folder, a hostile agent nameshome/probe.yaml,home/sub/id_rsaandhome/notes.MOV. On iOS and on Android nothing is copied and nothing is listed;an upper-case
CLIP.MOVin the video folder is kept in place, not copied;a file over 64 MiB gives the too-large error.
The existing Android host-copy test passed only because Go's
t.TempDir()folders share a parent, which the old parent root allowed. It now passes the source's folder as a root, standing for the probelink save folder.internal/confined/confined_test.go(new): a regular file is read and its real path returned. These are refused: a file outside, a folder, a missing file, a link out, a link even to an allowed file, a hard link, a fileallowedrejects, and (on Unix) a FIFO, without blocking. Size cap: exactly 64 MiB is read, 64 MiB + 1 isErrTooLarge, andCheckhas no limit (sparse files). AlsoUnder.internal/runner/pull_artifacts_swap_test.go(darwin and linux):<UUID>/data, whilePullArtifactsruns for 2 s. Last run: 9,443 runs, 6,549 copies, 0 copies holding the secret. The test fails if nothing was copied, so it can't pass by refusing everything;internal/cli/pull_artifacts_roots_test.go: a source check thatprobe testpasses./reports/screenshots.The existing
PullArtifactstests (FP-19, the Android host copy) pass unchanged.Mutation check
Each mutant was applied, the tests were run, and the file was restored byte for byte. On this push, 20 of 20 were killed.
-run. Both were fixed and rerun.-ofile's folder (Dir(localDir)) is a root againlocalDiris not a rootconfined: no size capconfined: cap off by one (LimitReader(maxSize))confined:allowednot askedconfined: link count ignoredprobe testpasses the-ofolderprobe testpasses no video folderprobe testpasses no client folderdatacheck is goneTest plan
go test ./internal/runner/ ./internal/cli/ ./internal/confined/ ./internal/probelink/passes.go vetandstaticcheckpass onrunner,confinedandcli, and withGOOS=windows:go vet ./internal/...andstaticcheckonrunnerandconfined.Root hardening (review follow-up)
A future caller's mistake must not become allow-all.
artifactRootchecks every root:localDiris created, then resolved with links followed.Lstat). A screenshots folder planted as a link to a host folder would otherwise make that folder allowed. WhenlocalDiritself is refused, nothing is copied (that would write through the link) and no path the agent named is kept. Behavior change: if someone made the screenshots folder itself a symlink on purpose, a run now copies no artifacts. A symlinked parent, such asreports/or macOS's/var, is fine..and/.localDirmay be relative to the working folder; that is howprobe testpasses it.Tests (
pull_artifacts_roots_test.go, darwin and linux):.,/, a relativesub, the home folder and its parent as extra roots: nothing copied or listed, on iOS and Android;/is refused with the working/home check switched off.Mutants on this push: 26 of 26 killed.
MkdirAll; relative extra roots allowed; theLstatlink check gone; the filesystem-root check gone; the wide-folder check gone; a refusedlocalDirused anyway.IsDironLstat, and/was also caught by the wide check. The link mutant now removes the wholeLstatcheck, and a test with the wide check off covers/.localDiras a root again,localDirnot a root, roots not resolved.Write side (review follow-up)
The review found that the copies were written to
filepath.Join(localDir, name), the unresolved path, while the check was made on the resolved folder. A process of the same user swapping the folder's last part for a link to a host folder landed 12,220 of 185,605 files in the host folder.Now:
openArtifactDiropens the screenshots folder once withos.OpenRoot(Go 1.24+). It keeps the folder only when the opened folder is the folder that was checked:os.SameFileagainst whatLstatsaid of the unresolved path incheckedArtifactRoot, before its real path was read. That closes two gaps:Lstatand theEvalSymlinksof the check. My own flipper test found this one (1 file in about 107,000 runs) before the fix.writeInDirwrites every copy, from the iOS branch, the Android host branch and the Android device branch, through that open folder: a new temp file (O_EXCL), thenRoot.Rename. A link swapped into the folder's place after the open can't redirect a write, andos.Rootrefuses to follow a link out of the folder.writeFileReplacing(path-based) is removed. A file is named by its base name only. The listed path is the same as before (the absolute path underlocalDir), so report paths don't change.ModeSymlinktest next toIsDiron anLstatwas redundant: toLstat, a link is not a directory. That is why the reviewer's mutant removing it survived. It is removed, with a comment saying why.Tests (darwin and linux):
TestPullArtifactsWritesSurviveAFolderSwap: a goroutine keeps parking the real folder, putting a link to a host folder in its place for about 100 µs and putting the folder back, moving aside any folderMkdirAllrecreated so the flipping never stops. Three runs: 115,852 / 111,533 / 115,429 runs, about 4,200 copies each, 0 files in the host folder. It fails if nothing was copied.TestPullArtifactsRefusesAFolderSwappedBeforeItIsResolvedandTestOpenArtifactDirRefusesAFolderSwappedBeforeTheOpen: deterministic swaps at each gap, through test hooks.TestOpenArtifactDirRefusesALink.Mutants on this push: every one is killed, 30 in all.
Lstat(the resolve gap);LstatinopenArtifactDirgone;LstatinartifactRootgone;MkdirAll; the Android host copy without a check.-runand the flipper got stuck; both were fixed. Then one survived because the resolve gap is rarely hit by chance; the hook test covers it.