Remove race condition in 022_crash_temp_files.pl test.

It's possible for the query that "waits for restart" to complete a
successful iteration before the postmaster has noticed its SIGKILL'd
child and begun the restart cycle.  (This is a bit hard to believe
perhaps, but it's been seen at least twice in the buildfarm, mainly
on ancient platforms that likely have quirky schedulers.)

To provide a more secure interlock, wait for the other session
we're using to report that it's been forcibly shut down.

Patch by me, based on a suggestion from Andres Freund.
Back-patch to v14 where this test case came in.

Discussion: https://postgr.es/m/1801850.1649047827@sss.pgh.pa.us

diff --git a/src/test/recovery/t/022_crash_temp_files.pl b/src/test/recovery/t/022_crash_temp_files.pl
index 36906b4aca6c8d536a5fd5b13627e29f438d355c..24fb141785da5ad3db4d83a09273afaad3378e0b 100644 (file)
--- a/src/test/recovery/t/022_crash_temp_files.pl
+++ b/src/test/recovery/t/022_crash_temp_files.pl
@@ -125,11 +125,24 @@ $killme_stderr2 = '';
 my $ret = PostgreSQL::Test::Utils::system_log('pg_ctl', 'kill', 'KILL', $pid);
 is($ret, 0, 'killed process with KILL');
 
-# Close psql session
+# Close that psql session
 $killme->finish;
+
+# Wait till the other session reports failure, ensuring that the postmaster
+# has noticed its dead child and begun a restart cycle.
+$killme_stdin2 .= qq[
+SELECT pg_sleep($PostgreSQL::Test::Utils::timeout_default);
+];
+ok( pump_until(
+               $killme2,
+               $psql_timeout,
+               \$killme_stderr2,
+               qr/WARNING:  terminating connection because of crash of another server process|server closed the connection unexpectedly|connection to server was lost|could not send data to server/m
+       ),
+       "second psql session died successfully after SIGKILL");
 $killme2->finish;
 
-# Wait till server restarts
+# Wait till server finishes restarting
 $node->poll_query_until('postgres', undef, '');
 
 # Check for temporary files
@@ -214,11 +227,24 @@ $killme_stderr2 = '';
 $ret = PostgreSQL::Test::Utils::system_log('pg_ctl', 'kill', 'KILL', $pid);
 is($ret, 0, 'killed process with KILL');
 
-# Close psql session
+# Close that psql session
 $killme->finish;
+
+# Wait till the other session reports failure, ensuring that the postmaster
+# has noticed its dead child and begun a restart cycle.
+$killme_stdin2 .= qq[
+SELECT pg_sleep($PostgreSQL::Test::Utils::timeout_default);
+];
+ok( pump_until(
+               $killme2,
+               $psql_timeout,
+               \$killme_stderr2,
+               qr/WARNING:  terminating connection because of crash of another server process|server closed the connection unexpectedly|connection to server was lost|could not send data to server/m
+       ),
+       "second psql session died successfully after SIGKILL");
 $killme2->finish;
 
-# Wait till server restarts
+# Wait till server finishes restarting
 $node->poll_query_until('postgres', undef, '');
 
 # Check for temporary files -- should be there