Skip to content

fix(vscode): the cloud key, the cloud secret and the relay token never go on a command line - #768

Merged
pbertsch merged 3 commits into
mainfrom
fix/vscode-secrets-not-in-argv
Nov 17, 2026
Merged

pbertsch merged 3 commits into
mainfrom
fix/vscode-secrets-not-in-argv

Conversation

@pbertsch

@pbertsch pbertsch commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

The VS Code extension typed the cloud key, the cloud secret and the relay token into its terminal as command-line flags. A command line is visible to other users in the process list and stays in shell history. With this change these values never go on a command line.

The finding, checked against the code on main:

  • vscode/src/probe.ts:174-184: buildTestArgs pushed --cloud-key, --cloud-secret and --relay-token with their values, taken from the options or the workspace settings.
  • vscode/src/terminal/probeTerminal.ts:34: runInTerminal types the whole command with terminal.sendText. Every buildTestArgs caller runs through it: extension.ts, commands/test.ts, providers/testExplorer.ts, providers/sessionManager.ts and views/runProfile.ts.
  • The CLI read these three only as flags (internal/cli/test.go:189-200).

The change

CLI

  • probe test reads --cloud-key, --cloud-secret and --relay-token from PROBE_CLOUD_KEY, PROBE_CLOUD_SECRET and PROBE_RELAY_CLI_TOKEN when the flag is not given (internal/cli/secret_flags.go).
  • A flag that is given wins, even when it is given empty, so existing scripts behave the same.
  • The relay token's variable is not PROBE_RELAY_TOKEN. That name is the agent's dart-define, which holds the relay session's agent token. cloud.RelaySession has both an agent_token and a cli_token, and they are different values. A CI job that exports the agent token for the app build would otherwise hand the CLI the wrong token.

Extension

  • buildTestArgs no longer writes the three flags, and it no longer takes them as options. buildTestRun strips them with withoutSecrets before calling it.
  • The new buildTestRun returns { args, env }, with the secrets from the options or the settings in env (vscode/src/secretArgs.ts).
  • All 13 test-run call sites pass run.env. runInTerminal now takes a required env, so a missed caller fails to compile. Other commands (init, lint, report, convert, record, devices, studio) pass {}.
  • runInTerminal throws if a command line still carries one of the three flags, in either the --flag value or the --flag=value form.
  • The terminal is created with env. A terminal's environment is fixed when it is created, so a terminal is replaced when a run's secrets differ from the ones it was made with. The check compares a fingerprint (name, length and a 32-bit hash), which stays in memory and is never printed.

Settings (breaking for a workspace that sets them)

  • flutterprobe.cloudKey, flutterprobe.cloudSecret and flutterprobe.relayToken are now "scope": "machine", and so are the two settings that decide where they are sent: relayUrl (it receives the relay CLI token) and cloudProvider (it receives the key and the secret). SECRET_SETTINGS in secretArgs.ts lists the five, and a test checks each in package.json. VS Code's contribution-points reference says such a setting can be written only in user or remote settings, cannot be overridden in workspace or folder settings, and is left out of Settings Sync.
  • So a value in .vscode/settings.json is now ignored, and a repository cannot set these settings. On activation the extension shows a one-time notice (per workspace, through workspaceState) naming any of the five that the workspace's .vscode/settings.json still sets, so the value is not dropped silently.
  • What this does not cover: probe test also reads the workspace's probe.yaml, whose cloud: section can name a provider and a relay service URL. And which binary the extension runs is a separate setting (tool paths, restricted in the follow-up PR); until then, only open workspaces you trust. The README, the CHANGELOG and the notice say so.
  • Where the secrets live: in the FlutterProbe terminal's environment for the terminal's whole life, inherited by every command typed there and by the programs probe starts (adb, flutter). Whether VS Code restores a terminal's environment after a restart was not verified. The README says this too.
  • The vscode README, the CHANGELOG (under Changed) and cloud-farms.md say how to move the value: to user settings, or to the environment variable.
  • VS Code still stores settings in plain text. Moving them to SecretStorage would be a separate change.

Docs. The vscode README covers the settings table, a note on the change and the setup examples (now "user settings"). The root README's cloud example uses export PROBE_CLOUD_KEY=… PROBE_CLOUD_SECRET=…. website/src/content/docs/tools/cloud-farms.md documents the variables and the two relay tokens. The CHANGELOG has the entry.

Tests

  • Go (internal/cli/secret_flags_test.go):
    • each flag falls back to its variable;
    • a flag that is given wins, and a flag given empty stays empty;
    • a non-secret flag reads no variable;
    • test.go has the three flags and reads them only through secretFlag, never GetString.
  • Extension (vscode/src/test/secretArgs.test.ts, node --test):
    • the variable names, and empty values left out;
    • a secret flag is found in both forms;
    • the fingerprint tells values apart and does not contain them;
    • a source check that runInTerminal refuses a secret flag and creates the terminal with env. It imports vscode, so a unit test cannot call it.
    • a source check that no file but secretArgs.ts names one of the three flags. It is a prefix match, so it also catches '--cloud-key=' + value.
    • the five secret and destination settings are machine-scoped (parsed from package.json);
    • terminalAction: a terminal is reused only when alive and made with the same secrets, otherwise replaced or created; plus a source check that getOrCreateTerminal follows it;
    • staleWorkspaceSettings finds the settings a workspace file still names, and only exact keys;
    • withoutSecrets takes the three out of the options;
    • a source check of the hand-off: all 13 runInTerminal calls with run.args pass run.env, no caller uses buildTestArgs, and the run profile hands its form's three secrets to buildTestRun;
    • a source check that activation shows the notice once.

Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 22 were killed. Five mutants first failed to compile or matched more than one place; they were rewritten, and those runs are the ones counted. One survived at first (getOrCreateTerminal ignoring terminalAction); the source test was extended and it is now killed.

Mutant Result
CLI: a secret flag ignores its variable killed
CLI: a flag given empty falls back to the variable killed
CLI: the relay token reads PROBE_RELAY_TOKEN killed
CLI: probe test reads --cloud-secret with GetString killed
CLI: a non-secret flag reads a variable killed
ext: buildTestArgs puts --cloud-secret back killed
ext: secretEnv keeps an empty value killed
ext: the cloud secret goes to the wrong variable killed
ext: secretFlagIn misses the --flag=value form killed
ext: the fingerprint ignores the value killed
ext: runInTerminal does not refuse a secret flag killed
ext: the terminal is created without env killed
ext: buildTestArgs pushes the --cloud-key=value form killed
ext: relayUrl is not machine-scoped killed
ext: cloudProvider is not machine-scoped killed
ext: a terminal with other secrets is reused killed
ext: getOrCreateTerminal ignores terminalAction killed
ext: a stale workspace setting is not found killed
ext: the notice is not shown on activation killed
ext: the run profile drops the cloud secret killed
ext: the test explorer passes no env killed
ext: withoutSecrets keeps the relay token killed

Not covered

  • An end-to-end run in a VS Code window. The terminal's environment was checked through the source and the types, not by running the extension.
  • A secret in the environment is still readable by processes of the same user, as every environment is. This change removes the exposure to other users and to shell history.

Test plan

  • go test ./internal/cli/ and go vet ./internal/cli/ pass.
  • npm test in vscode/ passes: 59 tests, including the 12 new ones.
  • Mutation check (table above).
  • gofmt clean on the changed Go files.

@pbertsch
pbertsch requested a review from a team as a code owner November 17, 2026 05:32
…oped too, a stale workspace value is noticed, and the review's tests
@pbertsch
pbertsch merged commit 3a83a8f into main Nov 17, 2026
11 of 12 checks passed
pbertsch added a commit that referenced this pull request Nov 17, 2026
…plorer runs get the settings' secrets (#770)

## Summary

Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (`PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET`, `PROBE_RELAY_CLI_TOKEN`) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment.

**Why not an allowlist.** An environment allowlist was considered and rejected:
- A `${NAME}` in a ProbeScript step reads any environment variable (`internal/runner/executor.go`, `expandEnv`, `os.LookupEnv`), as documented in `probescript/syntax.md`. An allowlist would make `${USER_EMAIL}` type its literal text.
- Most runs go through a terminal, which starts the person's own shell and its profile anyway.
- A process of the same user can read the same files directly.

The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust).

**The spawn sites, on main:**
- `vscode/src/probe.ts` `runCommand` (`probe device list`, `probe lint`): `env: { ...process.env }`.
- `vscode/src/commands/deviceView.ts` (the device view's hub): `env: { ...process.env }`.
- `vscode/src/testing/testController.ts` (Test Explorer runs of `probe test`): no `env`, so it inherited VS Code's environment and did not get the secrets from the settings.

## The change

- `childEnv(base, { allow, add })` in `vscode/src/secretArgs.ts` (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts `add` on top. The base is not changed.
  - Names are compared without regard to case. Windows reads `Probe_Cloud_Key` as `PROBE_CLOUD_KEY`, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through.
  - A name in `add` replaces any casing of itself, so two casings of one variable never reach a child.
- `runCommand` and the device view use `childEnv(process.env)`, so they get no secrets.
- Test Explorer uses `childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() })`: the secrets VS Code's environment has, with the settings' values on top.
- The terminal runs are unchanged: they get their secrets from `buildTestRun` (#768).
- **Not changed, and stated in the docs:** the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it.

**Docs.**
- The extension README gets a "What else the programs inherit" paragraph.
- The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with `probePath` described as a user setting.
- The CHANGELOG entry is under Changed, not breaking.

**#769 is merged and main is merged into this branch.** The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux.

## Tests

`vscode/src/test/childEnv.test.ts`, with `node --test`:
- a non-test child loses exactly the three, and keeps `PATH`, `HOME`, a `${NAME}` variable and `PROBE_CLOUD_URL`;
- mixed-case names (`Probe_Cloud_Key`, `probe_cloud_secret`) are removed, and a settings value replaces another casing of its name;
- the base is not changed;
- `allow` keeps only the named secrets;
- `add` overrides an inherited secret;
- a source check that each of the three spawn sites uses `childEnv` with its role, and that none passes `{ ...process.env }` any more. The spawn sites import `vscode`, so a unit test cannot call them.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it.

| Mutant | Result |
|---|---|
| `childEnv` strips nothing | killed |
| `childEnv` ignores `allow` | killed |
| `childEnv` drops `add` | killed |
| the settings do not override the inherited secret | killed |
| `childEnv` changes its base | killed |
| `runCommand` passes the whole environment | killed |
| the device view passes the whole environment | killed |
| Test Explorer gets no settings secrets | killed |
| `childEnv` compares names with case | killed |
| `add` does not replace another casing | killed |

## Test plan

- [x] `npm test` in `vscode/` passes on the branch with main merged in: 73 tests.
- [x] `npm run compile` passes.
- [x] Mutation check (table above).
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…r go on a command line (AlphaWaveSystems#768)

## Summary

The VS Code extension typed the cloud key, the cloud secret and the relay token into its terminal as command-line flags. A command line is visible to other users in the process list and stays in shell history. With this change these values never go on a command line.

**The finding, checked against the code on main:**
- `vscode/src/probe.ts:174-184`: `buildTestArgs` pushed `--cloud-key`, `--cloud-secret` and `--relay-token` with their values, taken from the options or the workspace settings.
- `vscode/src/terminal/probeTerminal.ts:34`: `runInTerminal` types the whole command with `terminal.sendText`. Every `buildTestArgs` caller runs through it: `extension.ts`, `commands/test.ts`, `providers/testExplorer.ts`, `providers/sessionManager.ts` and `views/runProfile.ts`.
- The CLI read these three only as flags (`internal/cli/test.go:189-200`).

## The change

**CLI**
- `probe test` reads `--cloud-key`, `--cloud-secret` and `--relay-token` from `PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET` and `PROBE_RELAY_CLI_TOKEN` when the flag is not given (`internal/cli/secret_flags.go`).
- A flag that is given wins, even when it is given empty, so existing scripts behave the same.
- The relay token's variable is not `PROBE_RELAY_TOKEN`. That name is the agent's dart-define, which holds the relay session's **agent** token. `cloud.RelaySession` has both an `agent_token` and a `cli_token`, and they are different values. A CI job that exports the agent token for the app build would otherwise hand the CLI the wrong token.

**Extension**
- `buildTestArgs` no longer writes the three flags, and it no longer takes them as options. `buildTestRun` strips them with `withoutSecrets` before calling it.
- The new `buildTestRun` returns `{ args, env }`, with the secrets from the options or the settings in `env` (`vscode/src/secretArgs.ts`).
- All 13 test-run call sites pass `run.env`. `runInTerminal` now takes a required `env`, so a missed caller fails to compile. Other commands (init, lint, report, convert, record, devices, studio) pass `{}`.
- `runInTerminal` throws if a command line still carries one of the three flags, in either the `--flag value` or the `--flag=value` form.
- The terminal is created with `env`. A terminal's environment is fixed when it is created, so a terminal is replaced when a run's secrets differ from the ones it was made with. The check compares a fingerprint (name, length and a 32-bit hash), which stays in memory and is never printed.

**Settings (breaking for a workspace that sets them)**
- `flutterprobe.cloudKey`, `flutterprobe.cloudSecret` and `flutterprobe.relayToken` are now `"scope": "machine"`, and so are the two settings that decide where they are sent: `relayUrl` (it receives the relay CLI token) and `cloudProvider` (it receives the key and the secret). `SECRET_SETTINGS` in `secretArgs.ts` lists the five, and a test checks each in `package.json`. VS Code's contribution-points reference says such a setting can be written only in user or remote settings, cannot be overridden in workspace or folder settings, and is left out of Settings Sync.
- So a value in `.vscode/settings.json` is now ignored, and a repository cannot set these settings. On activation the extension shows a one-time notice (per workspace, through `workspaceState`) naming any of the five that the workspace's `.vscode/settings.json` still sets, so the value is not dropped silently.
- What this does **not** cover: `probe test` also reads the workspace's `probe.yaml`, whose `cloud:` section can name a provider and a relay service URL. And which binary the extension runs is a separate setting (tool paths, restricted in the follow-up PR); until then, only open workspaces you trust. The README, the CHANGELOG and the notice say so.
- **Where the secrets live:** in the FlutterProbe terminal's environment for the terminal's whole life, inherited by every command typed there and by the programs `probe` starts (`adb`, `flutter`). Whether VS Code restores a terminal's environment after a restart was not verified. The README says this too.
- The vscode README, the CHANGELOG (under Changed) and `cloud-farms.md` say how to move the value: to user settings, or to the environment variable.
- VS Code still stores settings in plain text. Moving them to `SecretStorage` would be a separate change.

**Docs.** The vscode README covers the settings table, a note on the change and the setup examples (now "user settings"). The root README's cloud example uses `export PROBE_CLOUD_KEY=… PROBE_CLOUD_SECRET=…`. `website/src/content/docs/tools/cloud-farms.md` documents the variables and the two relay tokens. The CHANGELOG has the entry.

## Tests

- **Go** (`internal/cli/secret_flags_test.go`):
  - each flag falls back to its variable;
  - a flag that is given wins, and a flag given empty stays empty;
  - a non-secret flag reads no variable;
  - `test.go` has the three flags and reads them only through `secretFlag`, never `GetString`.
- **Extension** (`vscode/src/test/secretArgs.test.ts`, `node --test`):
  - the variable names, and empty values left out;
  - a secret flag is found in both forms;
  - the fingerprint tells values apart and does not contain them;
  - a source check that `runInTerminal` refuses a secret flag and creates the terminal with `env`. It imports `vscode`, so a unit test cannot call it.
  - a source check that no file but `secretArgs.ts` names one of the three flags. It is a prefix match, so it also catches `'--cloud-key=' + value`.
  - the five secret and destination settings are machine-scoped (parsed from `package.json`);
  - `terminalAction`: a terminal is reused only when alive and made with the same secrets, otherwise replaced or created; plus a source check that `getOrCreateTerminal` follows it;
  - `staleWorkspaceSettings` finds the settings a workspace file still names, and only exact keys;
  - `withoutSecrets` takes the three out of the options;
  - a source check of the hand-off: all 13 `runInTerminal` calls with `run.args` pass `run.env`, no caller uses `buildTestArgs`, and the run profile hands its form's three secrets to `buildTestRun`;
  - a source check that activation shows the notice once.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 22 were killed. Five mutants first failed to compile or matched more than one place; they were rewritten, and those runs are the ones counted. One survived at first (`getOrCreateTerminal` ignoring `terminalAction`); the source test was extended and it is now killed.

| Mutant | Result |
|---|---|
| CLI: a secret flag ignores its variable | killed |
| CLI: a flag given empty falls back to the variable | killed |
| CLI: the relay token reads `PROBE_RELAY_TOKEN` | killed |
| CLI: `probe test` reads `--cloud-secret` with `GetString` | killed |
| CLI: a non-secret flag reads a variable | killed |
| ext: `buildTestArgs` puts `--cloud-secret` back | killed |
| ext: `secretEnv` keeps an empty value | killed |
| ext: the cloud secret goes to the wrong variable | killed |
| ext: `secretFlagIn` misses the `--flag=value` form | killed |
| ext: the fingerprint ignores the value | killed |
| ext: `runInTerminal` does not refuse a secret flag | killed |
| ext: the terminal is created without `env` | killed |
| ext: `buildTestArgs` pushes the `--cloud-key=value` form | killed |
| ext: `relayUrl` is not machine-scoped | killed |
| ext: `cloudProvider` is not machine-scoped | killed |
| ext: a terminal with other secrets is reused | killed |
| ext: `getOrCreateTerminal` ignores `terminalAction` | killed |
| ext: a stale workspace setting is not found | killed |
| ext: the notice is not shown on activation | killed |
| ext: the run profile drops the cloud secret | killed |
| ext: the test explorer passes no env | killed |
| ext: `withoutSecrets` keeps the relay token | killed |

## Not covered

- An end-to-end run in a VS Code window. The terminal's environment was checked through the source and the types, not by running the extension.
- A secret in the environment is still readable by processes of the same user, as every environment is. This change removes the exposure to other users and to shell history.

## Test plan

- [x] `go test ./internal/cli/` and `go vet ./internal/cli/` pass.
- [x] `npm test` in `vscode/` passes: 59 tests, including the 12 new ones.
- [x] Mutation check (table above).
- [x] `gofmt` clean on the changed Go files.
jay23606 added a commit to jay23606/flutter-probe that referenced this pull request Nov 17, 2026
…plorer runs get the settings' secrets (AlphaWaveSystems#770)

## Summary

Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (`PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET`, `PROBE_RELAY_CLI_TOKEN`) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment.

**Why not an allowlist.** An environment allowlist was considered and rejected:
- A `${NAME}` in a ProbeScript step reads any environment variable (`internal/runner/executor.go`, `expandEnv`, `os.LookupEnv`), as documented in `probescript/syntax.md`. An allowlist would make `${USER_EMAIL}` type its literal text.
- Most runs go through a terminal, which starts the person's own shell and its profile anyway.
- A process of the same user can read the same files directly.

The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust).

**The spawn sites, on main:**
- `vscode/src/probe.ts` `runCommand` (`probe device list`, `probe lint`): `env: { ...process.env }`.
- `vscode/src/commands/deviceView.ts` (the device view's hub): `env: { ...process.env }`.
- `vscode/src/testing/testController.ts` (Test Explorer runs of `probe test`): no `env`, so it inherited VS Code's environment and did not get the secrets from the settings.

## The change

- `childEnv(base, { allow, add })` in `vscode/src/secretArgs.ts` (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts `add` on top. The base is not changed.
  - Names are compared without regard to case. Windows reads `Probe_Cloud_Key` as `PROBE_CLOUD_KEY`, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through.
  - A name in `add` replaces any casing of itself, so two casings of one variable never reach a child.
- `runCommand` and the device view use `childEnv(process.env)`, so they get no secrets.
- Test Explorer uses `childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() })`: the secrets VS Code's environment has, with the settings' values on top.
- The terminal runs are unchanged: they get their secrets from `buildTestRun` (AlphaWaveSystems#768).
- **Not changed, and stated in the docs:** the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it.

**Docs.**
- The extension README gets a "What else the programs inherit" paragraph.
- The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with `probePath` described as a user setting.
- The CHANGELOG entry is under Changed, not breaking.

**AlphaWaveSystems#769 is merged and main is merged into this branch.** The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux.

## Tests

`vscode/src/test/childEnv.test.ts`, with `node --test`:
- a non-test child loses exactly the three, and keeps `PATH`, `HOME`, a `${NAME}` variable and `PROBE_CLOUD_URL`;
- mixed-case names (`Probe_Cloud_Key`, `probe_cloud_secret`) are removed, and a settings value replaces another casing of its name;
- the base is not changed;
- `allow` keeps only the named secrets;
- `add` overrides an inherited secret;
- a source check that each of the three spawn sites uses `childEnv` with its role, and that none passes `{ ...process.env }` any more. The spawn sites import `vscode`, so a unit test cannot call them.

## Mutation check

Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it.

| Mutant | Result |
|---|---|
| `childEnv` strips nothing | killed |
| `childEnv` ignores `allow` | killed |
| `childEnv` drops `add` | killed |
| the settings do not override the inherited secret | killed |
| `childEnv` changes its base | killed |
| `runCommand` passes the whole environment | killed |
| the device view passes the whole environment | killed |
| Test Explorer gets no settings secrets | killed |
| `childEnv` compares names with case | killed |
| `add` does not replace another casing | killed |

## Test plan

- [x] `npm test` in `vscode/` passes on the branch with main merged in: 73 tests.
- [x] `npm run compile` passes.
- [x] Mutation check (table above).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant