pgsql: Fix run-time partition pruning for appends with multiple source

Fix run-time partition pruning for appends with multiple source rels.

The previous coding here supposed that if run-time partitioning applied to
a particular Append/MergeAppend plan, then all child plans of that node
must be members of a single partitioning hierarchy. This is totally wrong,
since an Append could be formed from a UNION ALL: we could have multiple
hierarchies sharing the same Append, or child plans that aren't part of any
hierarchy.

To fix, restructure the related plan-time and execution-time data
structures so that we can have a separate list or array for each
partitioning hierarchy. Also track subplans that are not part of any
hierarchy, and make sure they don't get pruned.

Per reports from Phil Florent and others. Back-patch to v11, since
the bug originated there.

David Rowley, with a lot of cosmetic adjustments by me; thanks also
to Amit Langote for review.

Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/1b54e91faabf3764b6786915881e514e42dccf89

Modified Files
--------------
src/backend/executor/execPartition.c | 399 +++++++++++++++-----------
src/backend/executor/nodeAppend.c | 4 +-
src/backend/nodes/copyfuncs.c | 16 +-
src/backend/nodes/outfuncs.c | 16 +-
src/backend/nodes/readfuncs.c | 15 +-
src/backend/optimizer/path/allpaths.c | 27 +-
src/backend/optimizer/plan/createplan.c | 52 +++-
src/backend/optimizer/plan/planner.c | 1 +
src/backend/partitioning/partprune.c | 211 +++++++++++---
src/include/executor/execPartition.h | 68 +++--
src/include/nodes/nodes.h | 1 +
src/include/nodes/plannodes.h | 49 +++-
src/include/partitioning/partprune.h | 6 +-
src/test/regress/expected/partition_prune.out | 140 +++++++++
src/test/regress/sql/partition_prune.sql | 45 +++
15 files changed, 780 insertions(+), 270 deletions(-)

Re: Tom Lane 2018-08-02 <E1fl0m0-0005T3-DO(at)gemulon(dot)postgresql(dot)org>
> Fix run-time partition pruning for appends with multiple source rels.
>
> The previous coding here supposed that if run-time partitioning applied to
> a particular Append/MergeAppend plan, then all child plans of that node
> must be members of a single partitioning hierarchy. This is totally wrong,
> since an Append could be formed from a UNION ALL: we could have multiple
> hierarchies sharing the same Append, or child plans that aren't part of any
> hierarchy.
>
> To fix, restructure the related plan-time and execution-time data
> structures so that we can have a separate list or array for each
> partitioning hierarchy. Also track subplans that are not part of any
> hierarchy, and make sure they don't get pruned.
>
> Per reports from Phil Florent and others. Back-patch to v11, since
> the bug originated there.

Since this commit added a new node type, all following node types got
renumbered. This means all extension modules using the information
compiled against 11beta2 need to be recompiled against 11beta3.

For apt.postgresql.org, the list of affected modules (so far, haven't
yet checked all) is pgsql-ogr-fdw, plr, wal2json.

I believe this should be mentioned in the beta3 release notes.

Christoph

(Thanks to Andrew Gierth for helping me pinpoint the issue.)

Christoph Berg <myon(at)debian(dot)org> writes:
> Since this commit added a new node type, all following node types got
> renumbered. This means all extension modules using the information
> compiled against 11beta2 need to be recompiled against 11beta3.

That's standard procedure for beta releases. We don't normally start
to worry about keeping binary ABI compatibility until the .0 release.

> I believe this should be mentioned in the beta3 release notes.

Too late ...

regards, tom lane