Skip to content

ci(prof): run additional allocation test on all PHP versions#3592

Merged
realFlowControl merged 3 commits intomasterfrom
florian/foobar
Mar 10, 2026
Merged

ci(prof): run additional allocation test on all PHP versions#3592
realFlowControl merged 3 commits intomasterfrom
florian/foobar

Conversation

@realFlowControl
Copy link
Copy Markdown
Member

@realFlowControl realFlowControl commented Jan 21, 2026

Description

This PR is based on #3559 with the intention to run that new correctness test on all PHP versions and provide a value and error range that matches all PHP versions. Additionally this runs the allocation_time_combined test without an adjusted sampling distance for allocations 🎉

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@realFlowControl realFlowControl changed the base branch from master to levi/perf-time-alloc-piggyback January 21, 2026 09:04
@datadog-official
Copy link
Copy Markdown

datadog-official bot commented Jan 21, 2026

⚠️ Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1028 Tests failed

testSearchPhpBinaries from integration.DDTrace\Tests\Integration\PHPInstallerTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integration\PHPInstallerTest::testSearchPhpBinaries
Test code or tested code printed unexpected output: Searching for available php binaries, this operation might take a while.
testSimplePushAndProcess from laravel-58-test.DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V5_8\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69b0443c00000000b7d5c92645b2ba67
tid: 69b0443c00000000
hexProcessTraceId: b7d5c92645b2ba67
hexProcessSpanId: a551c06431b1354f
processTraceId: 13246715045195463271
processSpanId: 11912514025934632271

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
testSimplePushAndProcess from laravel-8x-test.DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest (Datadog) (Fix with Cursor)
DDTrace\Tests\Integrations\Laravel\V8_x\QueueTest::testSimplePushAndProcess
Test code or tested code printed unexpected output: spanLinksTraceId: 69b044bd00000000e4865f5a0527e1b3
tid: 69b044bd00000000
hexProcessTraceId: e4865f5a0527e1b3
hexProcessSpanId: 3dd959933f6e6093
processTraceId: 16466953927764992435
processSpanId: 4456691795219079315

phpvfscomposer://tests/vendor/phpunit/phpunit/phpunit:106
View all

ℹ️ Info

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e9c2a9b | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.32%. Comparing base (41b371e) to head (e9c2a9b).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3592      +/-   ##
==========================================
- Coverage   62.39%   62.32%   -0.08%     
==========================================
  Files         142      142              
  Lines       13586    13586              
  Branches     1775     1775              
==========================================
- Hits         8477     8467      -10     
- Misses       4304     4311       +7     
- Partials      805      808       +3     

see 3 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41b371e...e9c2a9b. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Jan 21, 2026

Benchmarks [ profiler ]

Benchmark execution time: 2026-03-10 16:24:35

Comparing candidate commit e9c2a9b in PR branch florian/foobar with baseline commit 41b371e in branch master.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 26 metrics, 7 unstable metrics.

scenario:php-profiler-timeline-memory-control

  • 🟥 cpu_user_time [+33.879ms; +46.399ms] or [+5.625%; +7.704%]
  • 🟥 execution_time [+35.388ms; +41.999ms] or [+5.604%; +6.651%]

scenario:php-profiler-timeline-memory-with-profiler

  • 🟥 execution_time [+26.875ms; +45.485ms] or [+2.706%; +4.580%]

@realFlowControl realFlowControl changed the title Florian/foobar DO NOT MERGE ci(prof): run additional allocation test on all PHP versions Jan 21, 2026
@realFlowControl realFlowControl marked this pull request as ready for review January 21, 2026 13:19
@realFlowControl realFlowControl requested review from a team as code owners January 21, 2026 13:19
Base automatically changed from levi/perf-time-alloc-piggyback to master January 21, 2026 18:01
On PHP >= 8.4 (with frameless functions), typical CPU utilization of
that `str_replace()` function is at around 51%, without frameless
functions (PHP <= 8.3) the CPU utilisation for `str_replace()` is at
about 99%, this runs that test always and makes sure we match both
cases.
Comment on lines +40 to +41
"percent": 70,
"error_margin": 30
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stack is for wall-time. The idea here is that str_replace will use enough CPU that the allocation time spent is negligible, and the replace operation will dominate it.

The variability comes from whether it's piggy-backing on the allocation interrupt (for frameless versions, where we don't get an interrupt on the frameless call) or on older versions without it, we get an interrupt, and it's more accurate.

@realFlowControl realFlowControl merged commit 42d0acc into master Mar 10, 2026
2057 of 2073 checks passed
@realFlowControl realFlowControl deleted the florian/foobar branch March 10, 2026 16:48
@github-actions github-actions bot added this to the 1.17.0 milestone Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants