Skip to content

fix(vscode): only test runs receive the secret variables, and Test Explorer runs get the settings' secrets - #770

Merged
pbertsch merged 2 commits into
mainfrom
fix/vscode-child-env-allowlist
Nov 17, 2026
Merged

pbertsch merged 2 commits into
mainfrom
fix/vscode-child-env-allowlist

Conversation

@pbertsch

@pbertsch pbertsch commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (PROBE_CLOUD_KEY, PROBE_CLOUD_SECRET, PROBE_RELAY_CLI_TOKEN) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment.

Why not an allowlist. An environment allowlist was considered and rejected:

  • A ${NAME} in a ProbeScript step reads any environment variable (internal/runner/executor.go, expandEnv, os.LookupEnv), as documented in probescript/syntax.md. An allowlist would make ${USER_EMAIL} type its literal text.
  • Most runs go through a terminal, which starts the person's own shell and its profile anyway.
  • A process of the same user can read the same files directly.

The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust).

The spawn sites, on main:

  • vscode/src/probe.ts runCommand (probe device list, probe lint): env: { ...process.env }.
  • vscode/src/commands/deviceView.ts (the device view's hub): env: { ...process.env }.
  • vscode/src/testing/testController.ts (Test Explorer runs of probe test): no env, so it inherited VS Code's environment and did not get the secrets from the settings.

The change

  • childEnv(base, { allow, add }) in vscode/src/secretArgs.ts (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts add on top. The base is not changed.
    • Names are compared without regard to case. Windows reads Probe_Cloud_Key as PROBE_CLOUD_KEY, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through.
    • A name in add replaces any casing of itself, so two casings of one variable never reach a child.
  • runCommand and the device view use childEnv(process.env), so they get no secrets.
  • Test Explorer uses childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() }): the secrets VS Code's environment has, with the settings' values on top.
  • The terminal runs are unchanged: they get their secrets from buildTestRun (fix(vscode): the cloud key, the cloud secret and the relay token never go on a command line #768).
  • Not changed, and stated in the docs: the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it.

Docs.

  • The extension README gets a "What else the programs inherit" paragraph.
  • The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with probePath described as a user setting.
  • The CHANGELOG entry is under Changed, not breaking.

#769 is merged and main is merged into this branch. The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux.

Tests

vscode/src/test/childEnv.test.ts, with node --test:

  • a non-test child loses exactly the three, and keeps PATH, HOME, a ${NAME} variable and PROBE_CLOUD_URL;
  • mixed-case names (Probe_Cloud_Key, probe_cloud_secret) are removed, and a settings value replaces another casing of its name;
  • the base is not changed;
  • allow keeps only the named secrets;
  • add overrides an inherited secret;
  • a source check that each of the three spawn sites uses childEnv with its role, and that none passes { ...process.env } any more. The spawn sites import vscode, so a unit test cannot call them.

Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it.

Mutant Result
childEnv strips nothing killed
childEnv ignores allow killed
childEnv drops add killed
the settings do not override the inherited secret killed
childEnv changes its base killed
runCommand passes the whole environment killed
the device view passes the whole environment killed
Test Explorer gets no settings secrets killed
childEnv compares names with case killed
add does not replace another casing killed

Test plan

  • npm test in vscode/ passes on the branch with main merged in: 73 tests.
  • npm run compile passes.
  • Mutation check (table above).

@pbertsch
pbertsch requested a review from a team as a code owner November 17, 2026 05:52
@pbertsch
pbertsch merged commit 3830a9e into main Nov 17, 2026
11 of 12 checks passed
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…plorer runs get the settings' secrets (AlphaWaveSystems#770)

## Summary

Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (`PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET`, `PROBE_RELAY_CLI_TOKEN`) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment.

**Why not an allowlist.** An environment allowlist was considered and rejected:
- A `${NAME}` in a ProbeScript step reads any environment variable (`internal/runner/executor.go`, `expandEnv`, `os.LookupEnv`), as documented in `probescript/syntax.md`. An allowlist would make `${USER_EMAIL}` type its literal text.
- Most runs go through a terminal, which starts the person's own shell and its profile anyway.
- A process of the same user can read the same files directly.

The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust).

**The spawn sites, on main:**
- `vscode/src/probe.ts` `runCommand` (`probe device list`, `probe lint`): `env: { ...process.env }`.
- `vscode/src/commands/deviceView.ts` (the device view's hub): `env: { ...process.env }`.
- `vscode/src/testing/testController.ts` (Test Explorer runs of `probe test`): no `env`, so it inherited VS Code's environment and did not get the secrets from the settings.

## The change

- `childEnv(base, { allow, add })` in `vscode/src/secretArgs.ts` (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts `add` on top. The base is not changed.
  - Names are compared without regard to case. Windows reads `Probe_Cloud_Key` as `PROBE_CLOUD_KEY`, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through.
  - A name in `add` replaces any casing of itself, so two casings of one variable never reach a child.
- `runCommand` and the device view use `childEnv(process.env)`, so they get no secrets.
- Test Explorer uses `childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() })`: the secrets VS Code's environment has, with the settings' values on top.
- The terminal runs are unchanged: they get their secrets from `buildTestRun` (AlphaWaveSystems#768).
- **Not changed, and stated in the docs:** the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it.

**Docs.**
- The extension README gets a "What else the programs inherit" paragraph.
- The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with `probePath` described as a user setting.
- The CHANGELOG entry is under Changed, not breaking.

**AlphaWaveSystems#769 is merged and main is merged into this branch.** The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux.

## Tests

`vscode/src/test/childEnv.test.ts`, with `node --test`:
- a non-test child loses exactly the three, and keeps `PATH`, `HOME`, a `${NAME}` variable and `PROBE_CLOUD_URL`;
- mixed-case names (`Probe_Cloud_Key`, `probe_cloud_secret`) are removed, and a settings value replaces another casing of its name;
- the base is not changed;
- `allow` keeps only the named secrets;
- `add` overrides an inherited secret;
- a source check that each of the three spawn sites uses `childEnv` with its role, and that none passes `{ ...process.env }` any more. The spawn sites import `vscode`, so a unit test cannot call them.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it.

| Mutant | Result |
|---|---|
| `childEnv` strips nothing | killed |
| `childEnv` ignores `allow` | killed |
| `childEnv` drops `add` | killed |
| the settings do not override the inherited secret | killed |
| `childEnv` changes its base | killed |
| `runCommand` passes the whole environment | killed |
| the device view passes the whole environment | killed |
| Test Explorer gets no settings secrets | killed |
| `childEnv` compares names with case | killed |
| `add` does not replace another casing | killed |

## Test plan

- [x] `npm test` in `vscode/` passes on the branch with main merged in: 73 tests.
- [x] `npm run compile` passes.
- [x] Mutation check (table above).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant