Repository navigation
feat(cloud): add TestingBot real device cloud provider - #283
jochen-testingbot wants to merge 1 commit into
Conversation
Adds TestingBot as a sixth cloud device farm alongside BrowserStack, Sauce Labs, LambdaTest, AWS Device Farm and Firebase Test Lab. It implements the existing CloudProvider interface, so nothing on the shared cloud path in internal/cli/test.go changes. - UploadApp: POST /v1/storage (multipart "file"), returns the tb:// URL used as the appium:app capability. - ListDevices: GET /v1/devices, mapping platform_name/version onto the common Device shape. - StartSession/StopSession: Appium W3C hub at hub.testingbot.com, with credentials passed in tb:options as TestingBot documents. - GetSessionArtifacts: GET /v1/tests/<id>/assets for video and screenshots, polled up to 30s since assets are processed asynchronously. Credentials resolve from "key"/"secret" (TestingBot's own naming), falling back to the generic "username"/"access_key" so --cloud-key and --cloud-secret keep working without touching the CLI credential map. Like Sauce Labs and LambdaTest, the provider runs in relay mode; direct port forwarding would require the TestingBot Tunnel binary and returns an error pointing at --relay. Endpoints and payloads were taken from the TestingBot OpenAPI spec (api.testingbot.com/v1/openapi.json) and the Appium support docs. Signed-off-by: Jochen Delabie <jochen@testingbot.com>
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
|
Hi @jochen-testingbot, thank you for the contribution and the clear write-up, and sorry for the long silence. Could you please rebase on the current On the CLA: your sign comment is there, but the check kept failing because of a problem in our CLA workflow (it could neither be triggered by the sign comment nor record the signature). That is fixed now. Pushing the rebase will run the check again and record your signature. We are happy to help with anything further in the future, and a lot is coming soon. |
|
Thank you for the contribution and for disclosing your affiliation with TestingBot. We reviewed the code: the fixed HTTPS base URLs, basic auth only to those hosts and the bounded client timeout are good. Before we can consider it: (1) rebase on main (it has conflicted since September, including the shared test.go and config.go); (2) escape the session id with url.PathEscape where it is placed into URL paths; (3) bound the response body reads and avoid echoing the raw body in errors; (4) complete the CLA check. Separately, we also have to decide whether we can maintain another cloud provider that we cannot run against the live service ourselves; we will come back to that after the technical items are done. |
What
Adds TestingBot as a sixth cloud device farm, alongside BrowserStack, Sauce Labs, LambdaTest, AWS Device Farm and Firebase Test Lab.
Disclosure: I work at TestingBot. This is a vendor integration for our own platform, written against the public API — happy to adjust anything that doesn't fit how you'd like third-party providers handled here.
Why
TestingBot is a real device cloud in the same category as the three farms already supported, and
CloudProvideris a clean extension point, so this slots in without disturbing existing providers.How
internal/cloud/testingbot.goimplements the existingCloudProviderinterface. The only change to shared code is onecaseinNewProvider— nothing on the cloud path ininternal/cli/test.gomoves.UploadAppPOST /v1/storage(multipartfile) →tb://<appkey>forappium:appListDevicesGET /v1/devices→name/platform_name/versionStartSession/StopSessionhub.testingbot.com, credentials intb:optionsGetSessionArtifactsGET /v1/tests/<id>/assets→ video + screenshotsEndpoints and payload shapes were taken from the TestingBot OpenAPI spec (
api.testingbot.com/v1/openapi.json) and the Appium support docs, not from memory.Two decisions worth your review:
key/secret(TestingBot's own naming, matchingTESTINGBOT_KEY/TESTINGBOT_SECRET) and falls back to the genericusername/access_key. That keeps--cloud-key/--cloud-secretandprobe.yamlworking without editing the shared credential map intest.go, which would have touched all five existing providers. Happy to flip to generic-only if you'd rather keep one convention.ForwardPortpasses through whenRelayURLis set and otherwise returns an error pointing at--relay, exactly as Sauce Labs and LambdaTest do. Direct forwarding would need the TestingBot Tunnel binary; that can be a follow-up if it's wanted.Artifact collection polls up to 30s because TestingBot processes assets asynchronously — same shape as the existing BrowserStack video poll.
Tests
internal/cloud/testingbot_test.gocovers credential resolution (both naming schemes and the failure cases), upload, device listing, capability construction for Android and iOS, the WebDriver error path, relay/directForwardPort, artifact polling, and session teardown.To test against
httptestI made the base and hub URLs struct fields defaulting to the package constants. The other providers hardcode consts and currently have no tests; I kept that change insidetestingbot.gorather than refactoring them.make test— all 17 packages passgo build ./...andgo vetclean;gofmtclean on the added files (firebase.goandlambdatest.goare gofmt-dirty onmainalready; left alone to keep this focused)Not exercised against a live TestingBot account in this branch: request/response shapes are spec-verified, not wire-verified.
Also updated
README provider table,
probe.yaml.example, the--cloud-providerflag help, thecloud.providerconfig comment, and the VS Code extension (setting enum, positionalenumDescriptions, run-profile dropdown, extension README setup section).No CHANGELOG entry — happy to add one under
[Unreleased]in whatever wording you prefer if that's part of your flow.