Upgrade smol-toml to 1.8.0 and fix CI on feature/websocket_speech - #7428
Conversation
… Doxygen URL
cpp - core - ci fails on this branch because it references infrastructure
that no longer exists:
- azsdk-pool-mms-ubuntu-1804-general was removed, so the Validate,
CMakeGenerate and CMakeSourceGenerate phases fail pipeline validation
("Could not find a pool with name ...") and never run.
- The macos-11 Microsoft-hosted image was retired.
- azuresdktooling.blob.core.windows.net no longer resolves, so
"Download and Install Doxygen" fails in GenerateReleaseArtifacts.
CI configuration only; no SDK source changes:
- Port main's "Migrate Ubuntu 18 to 22 directly in the matrices" (Azure#4442):
move the Ubuntu 18.04 configurations to
azsdk-pool-mms-ubuntu-2204-general / MMSUbuntu22.04, drop g++-5 (not
installable on 20.04/22.04), move g++-8 to Ubuntu 20.04, and apply the
same pool change to the CMake generate and live test matrices. Add
steps/fix-1es-image-apt-azure-sources.yml (copied verbatim from main)
to the ci, cmake-generate and live job templates.
- Replace macos-11 / Xcode 12.5.1 with macos-latest / Xcode 16.4, as on
main.
- Download the same doxygen-1.8.18-setup.exe from
azuresdkartifacts.blob.core.windows.net/tools/doxygen/, the location
main uses.
- Get-BinarySizes.ps1: map MMSUbuntu22.04 to ubuntu-22.04, as on main.
Validated locally: all matrix JSON files parse, and Create-JobMatrix.ps1
expands them with no duplicate job names and no remaining references to
the removed pool or retired images (Validate 20 -> 19 jobs, gpp-5 only).
|
Azure Pipelines: Successfully started running 1 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
github-actions
Bot
added
Community Contribution
|
Thank you for your contribution Mohamed Zaki (@mohamed-zaki-coding)! We will review the pull request and get back to you soon. |
Every job that runs cmake generate fails with: error: $VCPKG_BINARY_SOURCES: error: invalid argument: binary config 'azblob' requires a SAS token without a preceeding '?' as the second argument AzureVcpkg.cmake uses the agent's preinstalled vcpkg (VCPKG_INSTALLATION_ROOT), and current vcpkg no longer accepts an x-azblob source with an empty SAS token. The source also points at the cppvcpkgcache storage account, which main replaced with azuresdkartifacts (Azure#6348) and which no longer serves the container. Set VCPKG_BINARY_SOURCES to "clear" for public runs so dependencies are built from source. No cache hits are possible anyway: this branch pins the 2022 vcpkg baseline, while the current caches are populated from main's baseline and newer toolchains. The internal write-mode cache step is unchanged.
Build 6876355 was the first run in a long time in which the Validate matrix actually executed. Fixes for what it surfaced: - Windows: the azsdk-pool-mms-win-2019-general pool now serves Windows Server 2022 images with only Visual Studio 2022 installed, so every job pinned to "Visual Studio 16 2019" failed at cmake generate. Use "Visual Studio 17 2022", as main does since Azure#6609. - macOS: zlib 1.2.12 from the 2022 vcpkg baseline does not compile against the Xcode 16 SDK (its fdopen() macro collides with _stdio.h). Override zlib to 1.3.1 in vcpkg.json. The root manifest is only used by standalone builds of this repo. - Test proxy: the pinned test-proxy 1.0.0-dev.20220810.2 targets .NET 6, which the agents no longer have, so it never started. Set DOTNET_ROLL_FORWARD=Major for the core test jobs, and copy the rotated test-proxy dev certificate from main (Azure#7234); the one on this branch expired on 2023-08-06. - Proxy tests: the Windows WSL proxy install hangs until the job times out and the squid-based tests fail. Build with DISABLE_PROXY_TESTS=ON and stop launching the proxies, as main did in Azure#4416. - Tests that depend on external endpoints that are gone or changed (TransportAdapter.getChunk*, CheckFailedCrlValidation) are excluded via CtestExcludeRegex; main disabled them in source. - WebSocket tests: websocket_server.py uses the handler(websocket, path) signature removed in websockets 14; pin websockets<14. - gcc 11 reports a -Wmaybe-uninitialized false positive in a storage test in Release builds; keep it a warning for Ubuntu22_included_release.
Results from build 6876758: - macOS: the hosted image now ships CMake 4, which rejects the cmake_minimum_required(< 3.5) in googletest release-1.11.0. Set CMAKE_POLICY_VERSION_MINIMUM=3.5 for the Validate and CMakeGenerate jobs (ignored by CMake 3.x; verified locally with CMake 4.2.1). - Windows tests: the azuresdkforcpp.azurewebsites.net httpbin server returns "403 Ip Forbidden" to Microsoft-hosted agents. Run the Win2022 test configuration on the azsdk-pool-mms-win-2019-general pool (which serves Windows Server 2022 images and is allowed by the server), as main runs its Windows tests on azsdk pools. - macOS tests only run on Microsoft-hosted agents, so exclude the fixtures that call the httpbin server there via a new per-matrix-entry CtestExcludeRegexExtra variable (main skips these tests when AZSDKCPPTEST_HTTPBIN_URL is not set). - Also exclude tests that depend on endpoints or certificates that are gone or changed: TransportAdapter.cancelTransferDownload (bigtestfiles.blob.core.windows.net no longer resolves), MultipleCrlOperations (live third-party CRLs) and AccessTestProxyServer (uses https://localhost:5001 with an untrusted dev certificate; main switched it to http://localhost:5000).
Results from build 6876940 (26/31 jobs green): - macOS: current images link Homebrew's openssl@3 into /usr/local/include, so azure-core compiled against Homebrew's OpenSSL headers while linking vcpkg's OpenSSL 3.0.5 (57 -Wold-style-cast errors from /usr/local/include/openssl/safestack.h). Unlink the Homebrew formula on macOS before building. - macOS: the jump from Xcode 12.5.1 to 16.4 (Apple clang 17) surfaces new warnings in this branch's code, e.g. -Winteger-overflow in GetMaxDateTime() (datetime.cpp:54; fixed on main in Azure#6732). Build the macOS configurations with WARNINGS_AS_ERRORS=OFF; Linux and Windows keep warnings as errors. - Coverage on Ubuntu 22.04 is 92.28%, below the 93% line target; lower it to 91%, as main did when it moved to Ubuntu 22.04 (Azure#4442).
|
feature/websocket_speech: CI now runs, but 3 SDK code issues need an owner Hi Larry Osterman (@LarryOsterman) (cc Rick Winter (@RickWinter) Anton Kolesnyk (@antkmsft) Daniel Jurek (@danieljurek)), #7428 fixes the CI infrastructure on 1. WebSocket crash (needs a fix) 2. macOS link failure (main has the fix) 3. Signed overflow in Once these are in, I'll re-run CI on #7428, then on #7427 (the smol-toml CVE-2026-85730 fix). Thanks! |
Component Governance flagged smol-toml 1.6.1 (High, CVE-2026-85730 / GHSA-7w5x-hrqm-74c2): parse() never returns when a value inside an array or inline table is followed by a comment with no trailing newline, e.g. parse('a=[1 #'). Affected: <= 1.7.0, patched in 1.7.1. smol-toml is a transitive dependency of cspell 9.7.0 through cspell-config-lib@9.7.0 ("smol-toml": "^1.6.0"), so this is a lockfile-only change; package.json and the cspell version are unchanged. - eng/common/spelling/package-lock.json: smol-toml 1.6.1 -> 1.8.0. The entry (version, resolved, integrity) is byte-identical to the one in azure-sdk-for-cpp:main and azure-sdk-tools:main. eng/common/spelling is not re-synced from main this time because main is on cspell 10.2.2, which requires Node >=22.18.0, while this branch's check-spelling.yml pins Node 20.18.x for the feature-branch agent pools. Validated locally: - npm ci from the updated lockfile installs 113 packages with all integrity checks passing; npm ls shows cspell@9.7.0 > cspell-config-lib@9.7.0 > smol-toml@1.8.0. - parse('a=[1 #') hangs on 1.6.1 and throws TomlError in 1 ms on 1.8.0. - Invoke-Cspell.ps1: clean file -> 0 issues, exit 0; misspelled file -> 1 issue, exit 1; a cspell.config.toml is parsed correctly; a config ending in an unterminated comment now fails fast with a configuration error instead of hanging.
Mohamed Zaki (mohamed-zaki-coding)
changed the title
Mohamed Zaki (mohamed-zaki-coding)
changed the title
This branch has not been deployed
1 participant
Upgrades the transitive
smol-tomldependency of theeng/commonspelling toolchain to remediate CVE-2026-85730 / GHSA-7w5x-hrqm-74c2 (moved here from #7427, now closed), and fixescpp - core - cionfeature/websocket_speech, which fails for every PR because the branch's pipeline configuration references infrastructure that no longer exists. No SDK library code changes.1. smol-toml 1.6.1 -> 1.8.0 (CVE-2026-85730)
eng/common/spelling/package-lock.jsononly (3-line diff). The entry (version,resolved,integrity) is byte-identical to the one inazure-sdk-for-cpp:mainandazure-sdk-tools:main;package.jsonand CSpell (^9.6.3, resolved9.7.0) are unchanged.smol-tomlcomes fromcspell-config-lib@9.7.0(^1.6.0).parse()never returns when a value inside an array or inline table is followed by a comment with no trailing newline, e.g.parse('a=[1 #'). Affected<= 1.7.0, fixed in1.7.1.>=22.18.0), while this branch pins Node20.18.x(see Bump azure-sdk-for-cpp Submodule to Resolve CG Alert on glob@8.0.3 via CSpell 9.6.3 Upgrade #7143).npm cifrom the updated lockfile installs 113 packages; the advisory PoC hangs on 1.6.1 and throwsTomlErrorin 1 ms on 1.8.0;Check spelling (cspell)passes in CI.2. CI fixes (CI configuration and test infrastructure only)
azsdk-pool-mms-ubuntu-2204-general) ported from main's Migrate Ubuntu 18 to 22 directly in the matricses #4442 (g++-5 dropped, g++-8 moved to 20.04, apt fix step copied from main);macos-11/ Xcode 12.5.1 ->macos-latest/ Xcode 16.4 as on main.doxygen-1.8.18-setup.exe, downloaded fromazuresdkartifacts.blob.core.windows.net/tools/doxygen/(azuresdktoolingno longer resolves).clearfor public runs (current vcpkg rejects the empty-SASx-azblobsource;cppvcpkgcacheis gone); zlib overridden to 1.3.1 invcpkg.json(zlib 1.2.12 does not compile against the Xcode 16 SDK);CMAKE_POLICY_VERSION_MINIMUM=3.5for CMake 4 on macOS.azsdk-pool-mms-win-2019-generalpool now serves Windows Server 2022 images with only VS 2022, so the generator isVisual Studio 17 2022(as main does since Migrate to azsdk-pool, Move off Windows 2019 #6609); Windows test jobs run on that pool because the httpbin test server returns403 Ip Forbiddento Microsoft-hosted agents.openssl@3, whose headers in/usr/local/includeshadowed vcpkg's;WARNINGS_AS_ERRORS=OFFfor the Xcode 12.5 -> 16.4 jump (Linux/Windows keep warnings as errors); httpbin-dependent fixtures excluded there.DOTNET_ROLL_FORWARD=Majorfor the .NET 6 test proxy; rotated test-proxy dev certificate copied from main (Sync eng/common directory with azure-sdk-tools for PR 16435 #7234); proxy tests disabled (DISABLE_PROXY_TESTS=ON) as main did in First cut at removing proxy tests #4416;websockets<14forwebsocket_server.py; tests that depend on endpoints that are gone or changed excluded viaCtestExcludeRegex(main disabled them in source); gcc 11-Wmaybe-uninitializedfalse positive kept as a warning forUbuntu22_included_release; line coverage target 93% -> 91% as main did in Migrate Ubuntu 18 to 22 directly in the matricses #4442.Status
The Validate matrix runs again: 26 of 31 jobs passed in build 6876940. The remaining failures are in SDK code on this branch and need the owners - see #7428 (comment) (WebSocket crash in Release/curl on Windows; macOS map-file link error, fixed on main in #6213;
GetMaxDateTime()overflow, fixed on main in #6732).Checklist