Repository navigation
fix(vscode): the cloud key, the cloud secret and the relay token never go on a command line - #768
Merged
Merged
Conversation
…r go on a command line
…oped too, a stale workspace value is noticed, and the review's tests
…inary is a separate setting
3 tasks done
pbertsch
added a commit
that referenced
this pull request
Nov 17, 2026
…plorer runs get the settings' secrets (#770) ## Summary Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (`PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET`, `PROBE_RELAY_CLI_TOKEN`) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment. **Why not an allowlist.** An environment allowlist was considered and rejected: - A `${NAME}` in a ProbeScript step reads any environment variable (`internal/runner/executor.go`, `expandEnv`, `os.LookupEnv`), as documented in `probescript/syntax.md`. An allowlist would make `${USER_EMAIL}` type its literal text. - Most runs go through a terminal, which starts the person's own shell and its profile anyway. - A process of the same user can read the same files directly. The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust). **The spawn sites, on main:** - `vscode/src/probe.ts` `runCommand` (`probe device list`, `probe lint`): `env: { ...process.env }`. - `vscode/src/commands/deviceView.ts` (the device view's hub): `env: { ...process.env }`. - `vscode/src/testing/testController.ts` (Test Explorer runs of `probe test`): no `env`, so it inherited VS Code's environment and did not get the secrets from the settings. ## The change - `childEnv(base, { allow, add })` in `vscode/src/secretArgs.ts` (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts `add` on top. The base is not changed. - Names are compared without regard to case. Windows reads `Probe_Cloud_Key` as `PROBE_CLOUD_KEY`, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through. - A name in `add` replaces any casing of itself, so two casings of one variable never reach a child. - `runCommand` and the device view use `childEnv(process.env)`, so they get no secrets. - Test Explorer uses `childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() })`: the secrets VS Code's environment has, with the settings' values on top. - The terminal runs are unchanged: they get their secrets from `buildTestRun` (#768). - **Not changed, and stated in the docs:** the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it. **Docs.** - The extension README gets a "What else the programs inherit" paragraph. - The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with `probePath` described as a user setting. - The CHANGELOG entry is under Changed, not breaking. **#769 is merged and main is merged into this branch.** The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux. ## Tests `vscode/src/test/childEnv.test.ts`, with `node --test`: - a non-test child loses exactly the three, and keeps `PATH`, `HOME`, a `${NAME}` variable and `PROBE_CLOUD_URL`; - mixed-case names (`Probe_Cloud_Key`, `probe_cloud_secret`) are removed, and a settings value replaces another casing of its name; - the base is not changed; - `allow` keeps only the named secrets; - `add` overrides an inherited secret; - a source check that each of the three spawn sites uses `childEnv` with its role, and that none passes `{ ...process.env }` any more. The spawn sites import `vscode`, so a unit test cannot call them. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it. | Mutant | Result | |---|---| | `childEnv` strips nothing | killed | | `childEnv` ignores `allow` | killed | | `childEnv` drops `add` | killed | | the settings do not override the inherited secret | killed | | `childEnv` changes its base | killed | | `runCommand` passes the whole environment | killed | | the device view passes the whole environment | killed | | Test Explorer gets no settings secrets | killed | | `childEnv` compares names with case | killed | | `add` does not replace another casing | killed | ## Test plan - [x] `npm test` in `vscode/` passes on the branch with main merged in: 73 tests. - [x] `npm run compile` passes. - [x] Mutation check (table above).
jay23606
added a commit
to jay23606/flutter-probe
that referenced
this pull request
Nov 17, 2026
…r go on a command line (AlphaWaveSystems#768) ## Summary The VS Code extension typed the cloud key, the cloud secret and the relay token into its terminal as command-line flags. A command line is visible to other users in the process list and stays in shell history. With this change these values never go on a command line. **The finding, checked against the code on main:** - `vscode/src/probe.ts:174-184`: `buildTestArgs` pushed `--cloud-key`, `--cloud-secret` and `--relay-token` with their values, taken from the options or the workspace settings. - `vscode/src/terminal/probeTerminal.ts:34`: `runInTerminal` types the whole command with `terminal.sendText`. Every `buildTestArgs` caller runs through it: `extension.ts`, `commands/test.ts`, `providers/testExplorer.ts`, `providers/sessionManager.ts` and `views/runProfile.ts`. - The CLI read these three only as flags (`internal/cli/test.go:189-200`). ## The change **CLI** - `probe test` reads `--cloud-key`, `--cloud-secret` and `--relay-token` from `PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET` and `PROBE_RELAY_CLI_TOKEN` when the flag is not given (`internal/cli/secret_flags.go`). - A flag that is given wins, even when it is given empty, so existing scripts behave the same. - The relay token's variable is not `PROBE_RELAY_TOKEN`. That name is the agent's dart-define, which holds the relay session's **agent** token. `cloud.RelaySession` has both an `agent_token` and a `cli_token`, and they are different values. A CI job that exports the agent token for the app build would otherwise hand the CLI the wrong token. **Extension** - `buildTestArgs` no longer writes the three flags, and it no longer takes them as options. `buildTestRun` strips them with `withoutSecrets` before calling it. - The new `buildTestRun` returns `{ args, env }`, with the secrets from the options or the settings in `env` (`vscode/src/secretArgs.ts`). - All 13 test-run call sites pass `run.env`. `runInTerminal` now takes a required `env`, so a missed caller fails to compile. Other commands (init, lint, report, convert, record, devices, studio) pass `{}`. - `runInTerminal` throws if a command line still carries one of the three flags, in either the `--flag value` or the `--flag=value` form. - The terminal is created with `env`. A terminal's environment is fixed when it is created, so a terminal is replaced when a run's secrets differ from the ones it was made with. The check compares a fingerprint (name, length and a 32-bit hash), which stays in memory and is never printed. **Settings (breaking for a workspace that sets them)** - `flutterprobe.cloudKey`, `flutterprobe.cloudSecret` and `flutterprobe.relayToken` are now `"scope": "machine"`, and so are the two settings that decide where they are sent: `relayUrl` (it receives the relay CLI token) and `cloudProvider` (it receives the key and the secret). `SECRET_SETTINGS` in `secretArgs.ts` lists the five, and a test checks each in `package.json`. VS Code's contribution-points reference says such a setting can be written only in user or remote settings, cannot be overridden in workspace or folder settings, and is left out of Settings Sync. - So a value in `.vscode/settings.json` is now ignored, and a repository cannot set these settings. On activation the extension shows a one-time notice (per workspace, through `workspaceState`) naming any of the five that the workspace's `.vscode/settings.json` still sets, so the value is not dropped silently. - What this does **not** cover: `probe test` also reads the workspace's `probe.yaml`, whose `cloud:` section can name a provider and a relay service URL. And which binary the extension runs is a separate setting (tool paths, restricted in the follow-up PR); until then, only open workspaces you trust. The README, the CHANGELOG and the notice say so. - **Where the secrets live:** in the FlutterProbe terminal's environment for the terminal's whole life, inherited by every command typed there and by the programs `probe` starts (`adb`, `flutter`). Whether VS Code restores a terminal's environment after a restart was not verified. The README says this too. - The vscode README, the CHANGELOG (under Changed) and `cloud-farms.md` say how to move the value: to user settings, or to the environment variable. - VS Code still stores settings in plain text. Moving them to `SecretStorage` would be a separate change. **Docs.** The vscode README covers the settings table, a note on the change and the setup examples (now "user settings"). The root README's cloud example uses `export PROBE_CLOUD_KEY=… PROBE_CLOUD_SECRET=…`. `website/src/content/docs/tools/cloud-farms.md` documents the variables and the two relay tokens. The CHANGELOG has the entry. ## Tests - **Go** (`internal/cli/secret_flags_test.go`): - each flag falls back to its variable; - a flag that is given wins, and a flag given empty stays empty; - a non-secret flag reads no variable; - `test.go` has the three flags and reads them only through `secretFlag`, never `GetString`. - **Extension** (`vscode/src/test/secretArgs.test.ts`, `node --test`): - the variable names, and empty values left out; - a secret flag is found in both forms; - the fingerprint tells values apart and does not contain them; - a source check that `runInTerminal` refuses a secret flag and creates the terminal with `env`. It imports `vscode`, so a unit test cannot call it. - a source check that no file but `secretArgs.ts` names one of the three flags. It is a prefix match, so it also catches `'--cloud-key=' + value`. - the five secret and destination settings are machine-scoped (parsed from `package.json`); - `terminalAction`: a terminal is reused only when alive and made with the same secrets, otherwise replaced or created; plus a source check that `getOrCreateTerminal` follows it; - `staleWorkspaceSettings` finds the settings a workspace file still names, and only exact keys; - `withoutSecrets` takes the three out of the options; - a source check of the hand-off: all 13 `runInTerminal` calls with `run.args` pass `run.env`, no caller uses `buildTestArgs`, and the run profile hands its form's three secrets to `buildTestRun`; - a source check that activation shows the notice once. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. All 22 were killed. Five mutants first failed to compile or matched more than one place; they were rewritten, and those runs are the ones counted. One survived at first (`getOrCreateTerminal` ignoring `terminalAction`); the source test was extended and it is now killed. | Mutant | Result | |---|---| | CLI: a secret flag ignores its variable | killed | | CLI: a flag given empty falls back to the variable | killed | | CLI: the relay token reads `PROBE_RELAY_TOKEN` | killed | | CLI: `probe test` reads `--cloud-secret` with `GetString` | killed | | CLI: a non-secret flag reads a variable | killed | | ext: `buildTestArgs` puts `--cloud-secret` back | killed | | ext: `secretEnv` keeps an empty value | killed | | ext: the cloud secret goes to the wrong variable | killed | | ext: `secretFlagIn` misses the `--flag=value` form | killed | | ext: the fingerprint ignores the value | killed | | ext: `runInTerminal` does not refuse a secret flag | killed | | ext: the terminal is created without `env` | killed | | ext: `buildTestArgs` pushes the `--cloud-key=value` form | killed | | ext: `relayUrl` is not machine-scoped | killed | | ext: `cloudProvider` is not machine-scoped | killed | | ext: a terminal with other secrets is reused | killed | | ext: `getOrCreateTerminal` ignores `terminalAction` | killed | | ext: a stale workspace setting is not found | killed | | ext: the notice is not shown on activation | killed | | ext: the run profile drops the cloud secret | killed | | ext: the test explorer passes no env | killed | | ext: `withoutSecrets` keeps the relay token | killed | ## Not covered - An end-to-end run in a VS Code window. The terminal's environment was checked through the source and the types, not by running the extension. - A secret in the environment is still readable by processes of the same user, as every environment is. This change removes the exposure to other users and to shell history. ## Test plan - [x] `go test ./internal/cli/` and `go vet ./internal/cli/` pass. - [x] `npm test` in `vscode/` passes: 59 tests, including the 12 new ones. - [x] Mutation check (table above). - [x] `gofmt` clean on the changed Go files.
jay23606
added a commit
to jay23606/flutter-probe
that referenced
this pull request
Nov 17, 2026
…plorer runs get the settings' secrets (AlphaWaveSystems#770) ## Summary Programs the VS Code extension starts inherit VS Code's environment. This change removes the three secret variables (`PROBE_CLOUD_KEY`, `PROBE_CLOUD_SECRET`, `PROBE_RELAY_CLI_TOKEN`) from every child that is not a test run, and gives Test Explorer runs the secrets from the settings, as terminal runs already get them. Nothing else changes in the environment. **Why not an allowlist.** An environment allowlist was considered and rejected: - A `${NAME}` in a ProbeScript step reads any environment variable (`internal/runner/executor.go`, `expandEnv`, `os.LookupEnv`), as documented in `probescript/syntax.md`. An allowlist would make `${USER_EMAIL}` type its literal text. - Most runs go through a terminal, which starts the person's own shell and its profile anyway. - A process of the same user can read the same files directly. The boundary is which binaries the extension runs and in which workspaces (the tool path settings and Workspace Trust). **The spawn sites, on main:** - `vscode/src/probe.ts` `runCommand` (`probe device list`, `probe lint`): `env: { ...process.env }`. - `vscode/src/commands/deviceView.ts` (the device view's hub): `env: { ...process.env }`. - `vscode/src/testing/testController.ts` (Test Explorer runs of `probe test`): no `env`, so it inherited VS Code's environment and did not get the secrets from the settings. ## The change - `childEnv(base, { allow, add })` in `vscode/src/secretArgs.ts` (pure). It builds a new environment without each of the three secret variables that is not allowed, and puts `add` on top. The base is not changed. - Names are compared without regard to case. Windows reads `Probe_Cloud_Key` as `PROBE_CLOUD_KEY`, and Node keeps the casing an environment was given, so a differently cased secret would otherwise slip through. - A name in `add` replaces any casing of itself, so two casings of one variable never reach a child. - `runCommand` and the device view use `childEnv(process.env)`, so they get no secrets. - Test Explorer uses `childEnv(process.env, { allow: ALL_SECRETS, add: settingsSecretEnv() })`: the secrets VS Code's environment has, with the settings' values on top. - The terminal runs are unchanged: they get their secrets from `buildTestRun` (AlphaWaveSystems#768). - **Not changed, and stated in the docs:** the other terminals (Studio, record, convert, init, lint, report, devices) are created with no extra variables, but a VS Code terminal still inherits VS Code's own environment. If VS Code itself was started with one of the three variables set, those terminals have it. **Docs.** - The extension README gets a "What else the programs inherit" paragraph. - The VS Code docs page gets a Security section covering the secrets, the binaries and what children inherit, with `probePath` described as a user setting. - The CHANGELOG entry is under Changed, not breaking. **AlphaWaveSystems#769 is merged and main is merged into this branch.** The docs page's binaries bullet was re-read against the merged code. It now says the extension looks the binary up in the absolute folders of PATH and runs the path it found. When nothing is found, it says so on Windows and runs the bare name on macOS and Linux. ## Tests `vscode/src/test/childEnv.test.ts`, with `node --test`: - a non-test child loses exactly the three, and keeps `PATH`, `HOME`, a `${NAME}` variable and `PROBE_CLOUD_URL`; - mixed-case names (`Probe_Cloud_Key`, `probe_cloud_secret`) are removed, and a settings value replaces another casing of its name; - the base is not changed; - `allow` keeps only the named secrets; - `add` overrides an inherited secret; - a source check that each of the three spawn sites uses `childEnv` with its role, and that none passes `{ ...process.env }` any more. The spawn sites import `vscode`, so a unit test cannot call them. ## Mutation check Each mutant was applied, the tests were run, and the file was restored byte for byte. All 10 were killed: the first 8 on the first version, and the case-insensitive version re-checked with the 5 that apply to it. | Mutant | Result | |---|---| | `childEnv` strips nothing | killed | | `childEnv` ignores `allow` | killed | | `childEnv` drops `add` | killed | | the settings do not override the inherited secret | killed | | `childEnv` changes its base | killed | | `runCommand` passes the whole environment | killed | | the device view passes the whole environment | killed | | Test Explorer gets no settings secrets | killed | | `childEnv` compares names with case | killed | | `add` does not replace another casing | killed | ## Test plan - [x] `npm test` in `vscode/` passes on the branch with main merged in: 73 tests. - [x] `npm run compile` passes. - [x] Mutation check (table above).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The VS Code extension typed the cloud key, the cloud secret and the relay token into its terminal as command-line flags. A command line is visible to other users in the process list and stays in shell history. With this change these values never go on a command line.
The finding, checked against the code on main:
vscode/src/probe.ts:174-184:buildTestArgspushed--cloud-key,--cloud-secretand--relay-tokenwith their values, taken from the options or the workspace settings.vscode/src/terminal/probeTerminal.ts:34:runInTerminaltypes the whole command withterminal.sendText. EverybuildTestArgscaller runs through it:extension.ts,commands/test.ts,providers/testExplorer.ts,providers/sessionManager.tsandviews/runProfile.ts.internal/cli/test.go:189-200).The change
CLI
probe testreads--cloud-key,--cloud-secretand--relay-tokenfromPROBE_CLOUD_KEY,PROBE_CLOUD_SECRETandPROBE_RELAY_CLI_TOKENwhen the flag is not given (internal/cli/secret_flags.go).PROBE_RELAY_TOKEN. That name is the agent's dart-define, which holds the relay session's agent token.cloud.RelaySessionhas both anagent_tokenand acli_token, and they are different values. A CI job that exports the agent token for the app build would otherwise hand the CLI the wrong token.Extension
buildTestArgsno longer writes the three flags, and it no longer takes them as options.buildTestRunstrips them withwithoutSecretsbefore calling it.buildTestRunreturns{ args, env }, with the secrets from the options or the settings inenv(vscode/src/secretArgs.ts).run.env.runInTerminalnow takes a requiredenv, so a missed caller fails to compile. Other commands (init, lint, report, convert, record, devices, studio) pass{}.runInTerminalthrows if a command line still carries one of the three flags, in either the--flag valueor the--flag=valueform.env. A terminal's environment is fixed when it is created, so a terminal is replaced when a run's secrets differ from the ones it was made with. The check compares a fingerprint (name, length and a 32-bit hash), which stays in memory and is never printed.Settings (breaking for a workspace that sets them)
flutterprobe.cloudKey,flutterprobe.cloudSecretandflutterprobe.relayTokenare now"scope": "machine", and so are the two settings that decide where they are sent:relayUrl(it receives the relay CLI token) andcloudProvider(it receives the key and the secret).SECRET_SETTINGSinsecretArgs.tslists the five, and a test checks each inpackage.json. VS Code's contribution-points reference says such a setting can be written only in user or remote settings, cannot be overridden in workspace or folder settings, and is left out of Settings Sync..vscode/settings.jsonis now ignored, and a repository cannot set these settings. On activation the extension shows a one-time notice (per workspace, throughworkspaceState) naming any of the five that the workspace's.vscode/settings.jsonstill sets, so the value is not dropped silently.probe testalso reads the workspace'sprobe.yaml, whosecloud:section can name a provider and a relay service URL. And which binary the extension runs is a separate setting (tool paths, restricted in the follow-up PR); until then, only open workspaces you trust. The README, the CHANGELOG and the notice say so.probestarts (adb,flutter). Whether VS Code restores a terminal's environment after a restart was not verified. The README says this too.cloud-farms.mdsay how to move the value: to user settings, or to the environment variable.SecretStoragewould be a separate change.Docs. The vscode README covers the settings table, a note on the change and the setup examples (now "user settings"). The root README's cloud example uses
export PROBE_CLOUD_KEY=… PROBE_CLOUD_SECRET=….website/src/content/docs/tools/cloud-farms.mddocuments the variables and the two relay tokens. The CHANGELOG has the entry.Tests
internal/cli/secret_flags_test.go):test.gohas the three flags and reads them only throughsecretFlag, neverGetString.vscode/src/test/secretArgs.test.ts,node --test):runInTerminalrefuses a secret flag and creates the terminal withenv. It importsvscode, so a unit test cannot call it.secretArgs.tsnames one of the three flags. It is a prefix match, so it also catches'--cloud-key=' + value.package.json);terminalAction: a terminal is reused only when alive and made with the same secrets, otherwise replaced or created; plus a source check thatgetOrCreateTerminalfollows it;staleWorkspaceSettingsfinds the settings a workspace file still names, and only exact keys;withoutSecretstakes the three out of the options;runInTerminalcalls withrun.argspassrun.env, no caller usesbuildTestArgs, and the run profile hands its form's three secrets tobuildTestRun;Mutation check
Each mutant was applied, the tests were run, and the file was restored byte for byte. All 22 were killed. Five mutants first failed to compile or matched more than one place; they were rewritten, and those runs are the ones counted. One survived at first (
getOrCreateTerminalignoringterminalAction); the source test was extended and it is now killed.PROBE_RELAY_TOKENprobe testreads--cloud-secretwithGetStringbuildTestArgsputs--cloud-secretbacksecretEnvkeeps an empty valuesecretFlagInmisses the--flag=valueformrunInTerminaldoes not refuse a secret flagenvbuildTestArgspushes the--cloud-key=valueformrelayUrlis not machine-scopedcloudProvideris not machine-scopedgetOrCreateTerminalignoresterminalActionwithoutSecretskeeps the relay tokenNot covered
Test plan
go test ./internal/cli/andgo vet ./internal/cli/pass.npm testinvscode/passes: 59 tests, including the 12 new ones.gofmtclean on the changed Go files.