Add support for prepared transactions to built-in logical replication.
authorAmit Kapila <akapila@postgresql.org>
Wed, 14 Jul 2021 02:03:50 +0000 (07:33 +0530)
committerAmit Kapila <akapila@postgresql.org>
Wed, 14 Jul 2021 02:03:50 +0000 (07:33 +0530)
commita8fd13cab0ba815e9925dc9676e6309f699b5f72
treebfebac6bfc2d32a9212e33f9090bd700b0316fae
parent6c9c2831668345122fd0f92280b30f3bbe2dd4e6
Add support for prepared transactions to built-in logical replication.

To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:

* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.

* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.

* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.

We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.

The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.

We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.

Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
43 files changed:
contrib/test_decoding/test_decoding.c
doc/src/sgml/catalogs.sgml
doc/src/sgml/protocol.sgml
doc/src/sgml/ref/alter_subscription.sgml
doc/src/sgml/ref/create_subscription.sgml
doc/src/sgml/ref/pg_dump.sgml
src/backend/access/transam/twophase.c
src/backend/catalog/pg_subscription.c
src/backend/catalog/system_views.sql
src/backend/commands/subscriptioncmds.c
src/backend/replication/libpqwalreceiver/libpqwalreceiver.c
src/backend/replication/logical/decode.c
src/backend/replication/logical/logical.c
src/backend/replication/logical/origin.c
src/backend/replication/logical/proto.c
src/backend/replication/logical/reorderbuffer.c
src/backend/replication/logical/snapbuild.c
src/backend/replication/logical/tablesync.c
src/backend/replication/logical/worker.c
src/backend/replication/pgoutput/pgoutput.c
src/backend/replication/slot.c
src/backend/replication/walreceiver.c
src/bin/pg_dump/pg_dump.c
src/bin/pg_dump/pg_dump.h
src/bin/psql/describe.c
src/bin/psql/tab-complete.c
src/include/access/twophase.h
src/include/catalog/catversion.h
src/include/catalog/pg_subscription.h
src/include/catalog/pg_subscription_rel.h
src/include/replication/logical.h
src/include/replication/logicalproto.h
src/include/replication/pgoutput.h
src/include/replication/reorderbuffer.h
src/include/replication/slot.h
src/include/replication/snapbuild.h
src/include/replication/walreceiver.h
src/include/replication/worker_internal.h
src/test/regress/expected/subscription.out
src/test/regress/sql/subscription.sql
src/test/subscription/t/021_twophase.pl [new file with mode: 0644]
src/test/subscription/t/022_twophase_cascade.pl [new file with mode: 0644]
src/tools/pgindent/typedefs.list