Add circular WAL decoding buffer.
authorThomas Munro <tmunro@postgresql.org>
Thu, 8 Apr 2021 11:03:34 +0000 (23:03 +1200)
committerThomas Munro <tmunro@postgresql.org>
Thu, 8 Apr 2021 11:20:42 +0000 (23:20 +1200)
commitf003d9f8721b3249e4aec8a1946034579d40d42c
treef53cd4a664fc1212156d32cd6f7dd6da507e0602
parent323cbe7c7ddcf18aaf24b7f6d682a45a61d4e31b
Add circular WAL decoding buffer.

Teach xlogreader.c to decode its output into a circular buffer, to
support optimizations based on looking ahead.

 * XLogReadRecord() works as before, consuming records one by one, and
   allowing them to be examined via the traditional XLogRecGetXXX()
   macros.

 * An alternative new interface XLogNextRecord() is added that returns
   pointers to DecodedXLogRecord structs that can be examined directly.

 * XLogReadAhead() provides a second cursor that lets you see
   further ahead, as long as data is available and there is enough space
   in the decoding buffer.  This returns DecodedXLogRecord pointers to the
   caller, but also adds them to a queue of records that will later be
   consumed by XLogNextRecord()/XLogReadRecord().

The buffer's size is controlled with wal_decode_buffer_size.  The buffer
could potentially be placed into shared memory, for future projects.
Large records that don't fit in the circular buffer are called
"oversized" and allocated separately with palloc().

Discussion: https://postgr.es/m/CA+hUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq=AovOddfHpA@mail.gmail.com
src/backend/access/transam/generic_xlog.c
src/backend/access/transam/xlog.c
src/backend/access/transam/xlogreader.c
src/backend/access/transam/xlogutils.c
src/backend/replication/logical/decode.c
src/bin/pg_rewind/parsexlog.c
src/bin/pg_waldump/pg_waldump.c
src/include/access/xlogreader.h