Skip to content

fix(cli): a tools path from probe.yaml runs only when it is an absolute path outside the repository - #772

Merged
pbertsch merged 3 commits into
mainfrom
fix/cli-tools-path-from-config
Nov 17, 2026
Merged

pbertsch merged 3 commits into
mainfrom
fix/cli-tools-path-from-config

Conversation

@pbertsch

@pbertsch pbertsch commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

A probe.yaml belongs to the repository it sits in, and tools.adb and tools.flutter in it name a program. Before this change, probe test and probe doctor used those values as given, so a repository could make probe test run a binary it carries. This change makes the CLI apply the rule Studio already used for tools.adb.

On main:

  • internal/cli/test.go: the parallel path (around line 553) and the single-device path (around line 923) put cfg.Tools.ADB and cfg.Tools.Flutter into device.ToolPaths unchecked.
  • internal/cli/doctor.go: the flutter and adb checks look those values up, and doctor's device list runs them.
  • Studio enforced the rule in studio/adb_tools.go workspaceADB, as documented in tools/studio.md.

The change

  • internal/toolpath (new): Studio's rule, moved out of Studio and generalized to flutter. A path passes when all of these hold:

    • it is an absolute path to a regular, executable file with the tool's name: adb, or adb.exe on Windows; flutter, or flutter.bat or flutter.exe on Windows;
    • it is outside the repository of the probe.yaml: the folder above it with a .git (a directory or a file, so worktrees and submodules count), or with none, outside the probe.yaml's folder;
    • symlinks are resolved first, and both the given name and the resolved name are checked.

    A relative path is never resolved.

  • CLI. configTool and runToolPaths (internal/cli/config_tools.go) are used by both probe test paths and by probe doctor.

    • A refused path stops probe test with a message that names the key and the file, gives the reason, and says the way out: tools.adb in /…/probe.yaml is not used: it is inside the repository of this probe.yaml. Set it to the full path of adb outside this repository, remove it to use the one on PATH, or pass --adb.
    • probe doctor reports the same message as a fail check, and its device list uses only a passing path.
    • --adb and --flutter are used as given, since the person typed them. A flag for one tool does not excuse a bad tools. value for the other.
  • config.Config.Source (yaml:"-", never read from the file) records the absolute path of the probe.yaml a config came from, so the rule knows which repository the file belongs to.

  • Studio calls toolpath.Check and keeps its own wording through a small replacer. Its messages are unchanged, and its existing tests pass unmodified.

Breaking: a probe.yaml with a relative tools: path, or one inside its own repository, now stops probe test. The escape, stated in the CHANGELOG and the docs: pass --adb or --flutter, or move the binary outside the repository and give its absolute path.

Docs:

  • advanced/configuration.md: the tools table and the rule.
  • platform/android.md: the tools.adb example.
  • The root README's config example.
  • The probe init template comments.
  • The CHANGELOG, under Changed, marked breaking.

Tests

  • internal/toolpath/toolpath_test.go:
    • Names per tool and system;
    • Check passes a tool outside the repository and an empty value;
    • Check refuses a relative path, a missing file, another program's name, a link to another program's name, a non-executable file, a directory, a tool elsewhere in the repository, one in the project folder, and a link from outside into the repository;
    • with no repository, the folder is the boundary;
    • a .git file marks a repository;
    • IsExecutableFile.
  • internal/cli/config_tools_test.go:
    • LoadFile records Source, and records none without a file;
    • the outside path is used, and the in-repository path is refused with the key, the file, the reason and --adb;
    • a relative tools.flutter is refused;
    • flags win as typed, and a flag for adb does not excuse tools.flutter;
    • no tools section means PATH;
    • doctor fails both refused checks;
    • a source check that test.go and doctor.go never read cfg.Tools. directly.

Mutation check

Each mutant was applied, the tests were run (Studio's against Studio's own suite), and the file was restored byte for byte. All 18 were killed. One first left a variable unused and did not compile; it was rewritten, and that run is the one counted.

Mutant Result
a relative path is accepted killed
symlinks are not resolved killed
the name is not checked killed
the resolved name is not checked killed
a non-executable file is accepted killed
the repository is not the boundary killed
no folder boundary without a repository killed
a .git file is not a repository killed
Windows flutter is not flutter.bat killed
a directory counts as executable on Windows killed
configTool does not check killed
LoadFile does not record the source killed
a flag does not win killed
probe test (parallel) uses cfg.Tools as it is killed
doctor does not fail a refused flutter killed
doctor's device list runs cfg.Tools killed
Studio: workspaceADB does not check killed
Studio: its wording is lost killed

Not covered

  • No run on Windows. The Windows names come from the Flutter SDK's bin/flutter.bat and adb.exe, and the tests that need execute bits and symlinks are skipped there.
  • gofmt -l flags internal/config/config.go on main already: field alignment, pre-existing. This change adds no new difference.

Test plan

  • go test ./internal/cli/ ./internal/config/ ./internal/toolpath/ ./internal/device/ passes.
  • go test . in studio/ passes (with a placeholder frontend/dist, not committed).
  • go vet passes on the changed packages and Studio.
  • npm run build in website/ passes (62 pages).
  • Mutation check (table above).

Review follow-up

  • Blocker: folders, not strings. Check compared paths with filepath.Rel after EvalSymlinks, which doesn't normalize letter case. On APFS, /x/evil/tools/adb was "not under" /x/Evil and the repository's binary ran. The new toolpath.Within(folder, p) walks each folder from the resolved file up to the filesystem root and compares it with the protected folder by os.SameFile, so spelling doesn't matter. TestAnotherSpellingOfTheRepositoryIsStillTheRepository creates Evil/Tools/adb and checks the lowercase path; it skips where the file system is case-sensitive, and it ran here (APFS). Studio's workspaceADB calls toolpath.Check, so it gets the fix too. go test ./studio/... passes.
  • (3) Nested repositories: RepoRoots returns every folder up to the filesystem root that holds a .git, and the tool must be outside all of them (TestTheOuterRepositoryIsProtectedToo).
  • (4) A symlinked probe.yaml: configTool checks the link's folder and the real file's folder (toolpath.CheckIn); tests at both levels.
  • (2) No .git at all (Docker COPY, git archive): only the probe.yaml's own folder is protected. This is now stated prominently in advanced/configuration.md, platform/android.md and the CHANGELOG, with the advice to keep the .git or pass the flags. I didn't add a refusal of the parent folder for an e2e/ or tests/ subfolder: guessing a project root from folder names would refuse legitimate layouts.
  • (5) Doctor's wording: configTool and runToolPaths take withFlag. probe test says "or pass --adb"; doctor's message doesn't offer a flag it doesn't have.
  • (6) Exit code: TestProbeTestExitsNonZeroOnARefusedToolsPath runs probe test through the root command and checks the error names tools.adb and ExitCode(err) != 0. It resets --config afterwards: without that, four later port_follow tests in the package inherited its probe.yaml and failed. That was found and fixed while doing this.
  • (1) A wider source guard: every non-test Go file under internal/, studio/ and cmd/ that reads .Tools.ADB or .Tools.Flutter must be in an allowlist of the two checked readers (internal/cli/config_tools.go, studio/app.go). The test also fails if it finds fewer readers than the allowlist.
  • Mutants: 6 of 6 killed:
    • Within comparing spelling;
    • only the innermost repository;
    • CheckIn checking only the first folder;
    • the real folder of a symlinked yaml ignored;
    • doctor offering --adb;
    • no-repository folder not protected.
  • Gate: go test ./internal/toolpath/ ./internal/cli/ and go test ./studio/... pass; staticcheck, and GOOS=windows go vet for ./internal/... and studio, are clean. Main is merged in.

…te path outside the repository, the rule Studio uses
@pbertsch
pbertsch requested a review from a team as a code owner November 17, 2026 06:03
… spelling), every enclosing repository and the real folder of a symlinked probe.yaml
@pbertsch
pbertsch merged commit 5b48342 into main Nov 17, 2026
10 of 12 checks passed
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…te path outside the repository (AlphaWaveSystems#772)

## Summary

A `probe.yaml` belongs to the repository it sits in, and `tools.adb` and `tools.flutter` in it name a program. Before this change, `probe test` and `probe doctor` used those values as given, so a repository could make `probe test` run a binary it carries. This change makes the CLI apply the rule Studio already used for `tools.adb`.

**On main:**
- `internal/cli/test.go`: the parallel path (around line 553) and the single-device path (around line 923) put `cfg.Tools.ADB` and `cfg.Tools.Flutter` into `device.ToolPaths` unchecked.
- `internal/cli/doctor.go`: the flutter and adb checks look those values up, and doctor's device list runs them.
- Studio enforced the rule in `studio/adb_tools.go` `workspaceADB`, as documented in `tools/studio.md`.

## The change

- **`internal/toolpath`** (new): Studio's rule, moved out of Studio and generalized to `flutter`. A path passes when all of these hold:
  - it is an absolute path to a regular, executable file with the tool's name: `adb`, or `adb.exe` on Windows; `flutter`, or `flutter.bat` or `flutter.exe` on Windows;
  - it is outside the repository of the `probe.yaml`: the folder above it with a `.git` (a directory or a file, so worktrees and submodules count), or with none, outside the `probe.yaml`'s folder;
  - symlinks are resolved first, and both the given name and the resolved name are checked.

  A relative path is never resolved.
- **CLI.** `configTool` and `runToolPaths` (`internal/cli/config_tools.go`) are used by both `probe test` paths and by `probe doctor`.
  - A refused path stops `probe test` with a message that names the key and the file, gives the reason, and says the way out: `tools.adb in /…/probe.yaml is not used: it is inside the repository of this probe.yaml. Set it to the full path of adb outside this repository, remove it to use the one on PATH, or pass --adb`.
  - `probe doctor` reports the same message as a `fail` check, and its device list uses only a passing path.
  - **`--adb` and `--flutter` are used as given**, since the person typed them. A flag for one tool does not excuse a bad `tools.` value for the other.
- **`config.Config.Source`** (`yaml:"-"`, never read from the file) records the absolute path of the `probe.yaml` a config came from, so the rule knows which repository the file belongs to.
- **Studio** calls `toolpath.Check` and keeps its own wording through a small replacer. Its messages are unchanged, and its existing tests pass unmodified.

**Breaking:** a `probe.yaml` with a relative `tools:` path, or one inside its own repository, now stops `probe test`. The escape, stated in the CHANGELOG and the docs: pass `--adb` or `--flutter`, or move the binary outside the repository and give its absolute path.

**Docs:**
- `advanced/configuration.md`: the `tools` table and the rule.
- `platform/android.md`: the `tools.adb` example.
- The root README's config example.
- The `probe init` template comments.
- The CHANGELOG, under Changed, marked breaking.

## Tests

- **`internal/toolpath/toolpath_test.go`:**
  - `Names` per tool and system;
  - `Check` passes a tool outside the repository and an empty value;
  - `Check` refuses a relative path, a missing file, another program's name, a link to another program's name, a non-executable file, a directory, a tool elsewhere in the repository, one in the project folder, and a link from outside into the repository;
  - with no repository, the folder is the boundary;
  - a `.git` file marks a repository;
  - `IsExecutableFile`.
- **`internal/cli/config_tools_test.go`:**
  - `LoadFile` records `Source`, and records none without a file;
  - the outside path is used, and the in-repository path is refused with the key, the file, the reason and `--adb`;
  - a relative `tools.flutter` is refused;
  - flags win as typed, and a flag for adb does not excuse `tools.flutter`;
  - no tools section means PATH;
  - doctor fails both refused checks;
  - a source check that `test.go` and `doctor.go` never read `cfg.Tools.` directly.

## Mutation check

Each mutant was applied, the tests were run (Studio's against Studio's own suite), and the file was restored byte for byte. All 18 were killed. One first left a variable unused and did not compile; it was rewritten, and that run is the one counted.

| Mutant | Result |
|---|---|
| a relative path is accepted | killed |
| symlinks are not resolved | killed |
| the name is not checked | killed |
| the resolved name is not checked | killed |
| a non-executable file is accepted | killed |
| the repository is not the boundary | killed |
| no folder boundary without a repository | killed |
| a `.git` file is not a repository | killed |
| Windows flutter is not `flutter.bat` | killed |
| a directory counts as executable on Windows | killed |
| `configTool` does not check | killed |
| `LoadFile` does not record the source | killed |
| a flag does not win | killed |
| `probe test` (parallel) uses `cfg.Tools` as it is | killed |
| doctor does not fail a refused flutter | killed |
| doctor's device list runs `cfg.Tools` | killed |
| Studio: `workspaceADB` does not check | killed |
| Studio: its wording is lost | killed |

## Not covered

- No run on Windows. The Windows names come from the Flutter SDK's `bin/flutter.bat` and `adb.exe`, and the tests that need execute bits and symlinks are skipped there.
- `gofmt -l` flags `internal/config/config.go` on main already: field alignment, pre-existing. This change adds no new difference.

## Test plan

- [x] `go test ./internal/cli/ ./internal/config/ ./internal/toolpath/ ./internal/device/` passes.
- [x] `go test .` in `studio/` passes (with a placeholder `frontend/dist`, not committed).
- [x] `go vet` passes on the changed packages and Studio.
- [x] `npm run build` in `website/` passes (62 pages).
- [x] Mutation check (table above).


## Review follow-up

- **Blocker: folders, not strings.** `Check` compared paths with `filepath.Rel` after `EvalSymlinks`, which doesn't normalize letter case. On APFS, `/x/evil/tools/adb` was "not under" `/x/Evil` and the repository's binary ran. The new `toolpath.Within(folder, p)` walks each folder from the resolved file up to the filesystem root and compares it with the protected folder by `os.SameFile`, so spelling doesn't matter. `TestAnotherSpellingOfTheRepositoryIsStillTheRepository` creates `Evil/Tools/adb` and checks the lowercase path; it skips where the file system is case-sensitive, and it ran here (APFS). Studio's `workspaceADB` calls `toolpath.Check`, so it gets the fix too. `go test ./studio/...` passes.
- **(3) Nested repositories:** `RepoRoots` returns every folder up to the filesystem root that holds a `.git`, and the tool must be outside all of them (`TestTheOuterRepositoryIsProtectedToo`).
- **(4) A symlinked `probe.yaml`:** `configTool` checks the link's folder and the real file's folder (`toolpath.CheckIn`); tests at both levels.
- **(2) No `.git` at all** (Docker `COPY`, `git archive`): only the `probe.yaml`'s own folder is protected. This is now stated prominently in `advanced/configuration.md`, `platform/android.md` and the CHANGELOG, with the advice to keep the `.git` or pass the flags. I didn't add a refusal of the parent folder for an `e2e/` or `tests/` subfolder: guessing a project root from folder names would refuse legitimate layouts.
- **(5) Doctor's wording:** `configTool` and `runToolPaths` take `withFlag`. `probe test` says "or pass --adb"; doctor's message doesn't offer a flag it doesn't have.
- **(6) Exit code:** `TestProbeTestExitsNonZeroOnARefusedToolsPath` runs `probe test` through the root command and checks the error names `tools.adb` and `ExitCode(err) != 0`. It resets `--config` afterwards: without that, four later `port_follow` tests in the package inherited its `probe.yaml` and failed. That was found and fixed while doing this.
- **(1) A wider source guard:** every non-test Go file under `internal/`, `studio/` and `cmd/` that reads `.Tools.ADB` or `.Tools.Flutter` must be in an allowlist of the two checked readers (`internal/cli/config_tools.go`, `studio/app.go`). The test also fails if it finds fewer readers than the allowlist.
- **Mutants:** 6 of 6 killed:
  - `Within` comparing spelling;
  - only the innermost repository;
  - `CheckIn` checking only the first folder;
  - the real folder of a symlinked yaml ignored;
  - doctor offering `--adb`;
  - no-repository folder not protected.
- **Gate:** `go test ./internal/toolpath/ ./internal/cli/` and `go test ./studio/...` pass; `staticcheck`, and `GOOS=windows go vet` for `./internal/...` and `studio`, are clean. Main is merged in.
pbertsch added a commit that referenced this pull request Nov 17, 2026
…nsensitive on Windows; AST-based guard (#808)

## Summary

Follow-ups from the review of #772, which keeps a `probe.yaml` tools path outside the repository.

1. **Fail closed without a file identity.** `toolpath.Within` compares folders with `os.SameFile`. Some Windows network shares and file systems report no file identity, and a folder that can't be read gives none either. In those cases, `Within` used to say "not within" and let the path run. Now it says **within**, so the path is refused.
   - `hasIdentity` is per OS: device and inode on macOS and Linux; on Windows, the volume serial number and file index from an open handle, where all zero or an error means none.
   - It is a variable, so tests can replace it.
2. **Tool names on Windows** are compared without regard to case, as Windows does: `ADB.EXE` is `adb.exe` there. macOS and Linux stay exact.
3. **The source guard is AST-based.** It parses every non-test Go file under `internal/`, `studio/` and `cmd/`, and flags any selector of a field called `Tools`, so `t := cfg.Tools; t.ADB` is caught as well as `cfg.Tools.ADB`. The allowlist is the two files that check the path: `internal/cli/config_tools.go` and `studio/app.go`. It also fails if either stops using the field, which means the walk or the allowlist is out of date.
4. **Docs** (`advanced/configuration.md`):
   - a `.git` in a parent folder (your home, for example) protects every folder below it: move the SDK outside it, or pass `--adb`/`--flutter`;
   - a hard link or bind mount outside the repository passes, and making one needs write access outside it;
   - the Windows name rule and the fail-closed rule.

## Tests

- `TestWithinFailsClosedWithoutAFileIdentity`: with `hasIdentity` replaced, no identity anywhere, and none for one folder on the way, both count as within. `Check` refuses an outside tool without identities and accepts it with them.
- `TestWithinFailsClosedForAFolderThatCannotBeRead`.
- `TestToolNameIsCaseInsensitiveOnlyOnWindows`.
- The guard test, rewritten as above.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 5 were killed. The guard mutant first didn't compile (no `config` import in `doctor.go`), so it was placed in `test.go`.

| Mutant | Result |
|---|---|
| an unreadable protected folder is not within | killed |
| no identity on the way is not within | killed |
| names exact on Windows too | killed |
| names case-insensitive everywhere | killed |
| a new reader through an alias (`t := c.Tools; t.ADB`) | killed |

## Test plan

- [x] `go test ./internal/toolpath/ ./internal/cli/` pass.
- [x] `go test ./studio/...` passes (Studio's `workspaceADB` uses `toolpath.Check`).
- [x] `staticcheck`; `GOOS=windows go vet ./internal/...` and windows `staticcheck` on `internal/toolpath` (which builds `identity_windows.go`).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant