You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Test runner's --test-rerun-failures could swallow actual failures on retries #63424
The following example test file will reproduce the bug:
// Demonstrates that `node --test --test-rerun-failures` can mark a failing test as passing on retry without executing its body.// Root cause: the runtime disambiguator at lib/internal/test_runner/test.js:793-801 keys tests purely by `file:line:column`.// A `t.test()` registered inside a factory function gets the same source location regardless of which parent invoked the factory.// The disambiguator counter is assigned in execution order across the whole run, so on retry - when the set of running tests changes,// the surviving failing test inherits a counter slot that, in attempt 0, belonged to a different (passing) sibling.// Node matches by that slot, replaces `this.fn` with a synthetic noop replay, and reports the failure as a pass.import{describe,it}from'node:test';import{strictasassert}from'node:assert';functionmakeSuite(shouldPass,label){returnasync(t)=>{awaitt.test('inner',async()=>{if(!shouldPass)assert.fail(`${label} should fail`);});};}describe('parents',{concurrency: false},()=>{it('A passes',makeSuite(true,'A'));it('B fails',makeSuite(false,'B'));// the only real failureit('C passes',makeSuite(true,'C'));});
Executed with:
node --test --test-rerun-failures=rerun.json repro.mjs # attempt 0# B fails as expected; rerun.json now contains A and C plus their# inner subtests keyed as "repro.mjs:13:13" and "repro.mjs:13:13:(1)"
node --test --test-rerun-failures=rerun.json repro.mjs # retry# all 6 tests reported as passing, exit code 0# B's assert.fail was never executed
How often does it reproduce? Is there a required condition?
Happens consistently
What is the expected behavior? Why is that the expected behavior?
The test inner should not be replaced with a noop for B fails, allowing it to actually execute
What do you see instead?
The inner test of B fails is skipped completely, due to a erroneous disambiguation
Activity
test_runner: fix --test-rerun-failures swallowing failures on retry
test_runner: fix --test-rerun-failures swallowing failures on retry
test_runner: fix --test-rerun-failures swallowing failures on retry
test_runner: fix --test-rerun-failures swallowing failures on retry
test_runner: fix --test-rerun-failures swallowing failures on retry