Convert pg_restore's ready_list to a priority queue.
authorNathan Bossart <nathan@postgresql.org>
Tue, 19 Sep 2023 21:31:29 +0000 (14:31 -0700)
committerNathan Bossart <nathan@postgresql.org>
Tue, 19 Sep 2023 21:31:29 +0000 (14:31 -0700)
commit9bfd44bbde4261181bf94738f3b041c629c65a7e
tree03dd7b5b3b71a9af58ab4b51161bcf27c5356ef2
parent1f998863b0bc6fc8ef3d971d9c6d2c29b52d8ba2
Convert pg_restore's ready_list to a priority queue.

Presently, parallel restores spend a lot of time sorting this list
so that we pick the largest items first.  With many tables, this
sorting can become a significant bottleneck.  There are a couple of
reports from the field about this, and it is easy to reproduce.

This commit improves the performance of parallel pg_restore with
many tables by converting its ready_list to a priority queue, i.e.,
a binary heap.  We will first try to run the highest priority item,
but if it cannot be chosen due to the lock heuristic, we'll do a
sequential scan through the heap nodes until we find one that is
runnable.  This means that we might end up picking an item with a
much lower priority.  However, we expect that we will typically be
able to pick one of the first few items, which should usually have
a relatively high priority.

Suggested-by: Tom Lane
Tested-by: Pierre Ducroquet
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us
src/bin/pg_dump/pg_backup_archiver.c