summaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)Author
2022-03-08Stabilize test_decoding touching with sequencesTomas Vondra
Some of the test_decoding regression tests are unstable due to modifying a sequence. The first increment of a sequence after a checkpoint is always logged (and thus decoded), which makes the output unpredictable. The runs are usually much shorter than a checkpoint internal, so these failures are rare, but we've seen a couple of them on animals that are either slow or are running with valgrind/clobber cache/... Fixed by skipping sequence decoding in most tests, with the exception of the test aimed at testing decoding of sequences. Reported-by: Amita Kapila Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
2022-03-08plpython: Code cleanup related to removal of Python 2 support.Andres Freund
Since 19252e8ec93 we reject Python 2 during build configuration. Now that the dust on the buildfarm has settled, remove Python 2 specific code, including the "Python 2/3 porting layer". The code to detect conflicts between plpython using Python 2 and 3 is not removed, in case somebody creates an out-of-tree version adding back support for Python 2. Reviewed-By: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-08plpython: Remove regression test infrastructure for Python 2.Andres Freund
Since 19252e8ec93 we reject Python 2 during build configuration. Now that the dust on the buildfarm has settled, remove regression testing infrastructure dealing with differing output between Python 2 / 3. Reviewed-By: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-08plpython: Remove plpythonu, plpython2u and associated transform extensions.Andres Freund
Since 19252e8ec93 we reject Python 2 during build configuration. Now that the dust on the buildfarm has settled, remove extension variants specific to Python 2. Reviewed-By: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20211031184548.g4sxfe47n2kyi55r@alap3.anarazel.de
2022-03-08Simplify SRFs using materialize mode in contrib/ modulesMichael Paquier
9e98583 introduced a helper to centralize building their needed state (tuplestore, tuple descriptors, etc.), checking for any errors. This commit updates all places of contrib/ that can be switched to use SetSingleFuncCall() as a drop-in replacement, resulting in the removal of a lot of boilerplate code in all the modules updated by this commit. Per analysis, some places remain as they are: - pg_logdir_ls() in adminpack/ uses historically TYPEFUNC_RECORD as return type, and I suspect that changing it may cause issues at run-time with some of its past versions, down to 1.0. - dblink/ uses a wrapper function doing exactly the work of SetSingleFuncCall(). Here the switch should be possible, but rather invasive so it does not seem the extra backpatch maintenance cost. - tablefunc/, similarly, uses multiple helper functions with portions of SetSingleFuncCall() spread across the code paths of this module. Author: Melanie Plageman Discussion: https://postgr.es/m/CAAKRu_bvDPJoL9mH6eYwvBpPtTGQwbDzfJbCM-OjkSZDu5yTPg@mail.gmail.com
2022-03-05Use PG_TEST_TIMEOUT_DEFAULT for pg_regress suite non-elapsing timeouts.Noah Misch
Currently, only contrib/test_decoding has this property. Use \getenv to load the timeout value. Discussion: https://postgr.es/m/20220218052842.GA3627003@rfd.leadboat.com
2022-03-05Introduce PG_TEST_TIMEOUT_DEFAULT for TAP suite non-elapsing timeouts.Noah Misch
Slow hosts may avoid load-induced, spurious failures by setting environment variable PG_TEST_TIMEOUT_DEFAULT to some number of seconds greater than 180. Developers may see faster failures by setting that environment variable to some lesser number of seconds. In tests, write $PostgreSQL::Test::Utils::timeout_default wherever the convention has been to write 180. This change raises the default for some briefer timeouts. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220218052842.GA3627003@rfd.leadboat.com
2022-03-03Clean up assorted failures under clang's -fsanitize=undefined checks.Tom Lane
Most of these are cases where we could call memcpy() or other libc functions with a NULL pointer and a zero count, which is forbidden by POSIX even though every production version of libc allows it. We've fixed such things before in a piecemeal way, but apparently never made an effort to try to get them all. I don't claim that this patch does so either, but it gets every failure I observe in check-world, using clang 12.0.1 on current RHEL8. numeric.c has a different issue that the sanitizer doesn't like: "ln(-1.0)" will compute log10(0) and then try to assign the resulting -Inf to an integer variable. We don't actually use the result in such a case, so there's no live bug. Back-patch to all supported branches, with the idea that we might start running a buildfarm member that tests this case. This includes back-patching c1132aae3 (Check the size in COPY_POINTER_FIELD), which previously silenced some of these issues in copyfuncs.c. Discussion: https://postgr.es/m/CALNJ-vT9r0DSsAOw9OXVJFxLENoVS_68kJ5x0p44atoYH+H4dg@mail.gmail.com
2022-02-28pg_stat_statements: Remove unnecessary call to GetUserId()Michael Paquier
The same is done a couple of lines above, so there is no need for the same, extra, call. Author: Dong Wook Lee Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/CAAcBya+szDd1Y6dJU4_dbH_Ye3=G=8O1oQGG01kv3Tpie7wELQ@mail.gmail.com
2022-02-24Simplify more checks related to set-returning functionsMichael Paquier
This makes more consistent the SRF-related checks in the area of PL/pgSQL, PL/Perl, PL/Tcl, pageinspect and some of the JSON worker functions, making it easier to grep for the same error patterns through the code, reducing a bit the translation work. It is worth noting that each_worker_jsonb()/each_worker() in jsonfuncs.c and pageinspect's brin_page_items() were doing a check on expectedDesc that is not required as they fetch their tuple descriptor directly from get_call_result_type(). This looks like a set of copy-paste errors that have spread over the years. This commit is a continuation of the changes begun in 07daca5, for any remaining code paths on sight. Like fcc2817, this makes the code more consistent, easing the integration of a larger patch that will refactor the way tuplestores are created and checked in a good portion of the set-returning functions present in core. I have worked my way through the changes of this patch by myself, and Ranier has proposed the same changes in a different thread in parallel, though there were some inconsistencies related in expectedDesc in what was proposed by him. Author: Michael Paquier, Ranier Vilela Discussion: https://postgr.es/m/CAAKRu_azyd1Z3W_r7Ou4sorTjRCs+PxeHw1CWJeXKofkE6TuZg@mail.gmail.com Discussion: https://postgr.es/m/CAEudQApm=AFuJjEHLBjBcJbxcw4pBMwg2sHwXyCXYcbBOj3hpg@mail.gmail.com
2022-02-24postgres_fdw: Add support for parallel commit.Etsuro Fujita
postgres_fdw commits remote (sub)transactions opened on remote server(s) in a local (sub)transaction one by one when the local (sub)transaction commits. This patch allows it to commit the remote (sub)transactions in parallel to improve performance. This is enabled by the server option "parallel_commit". The default is false. Etsuro Fujita, reviewed by Fujii Masao and David Zhang. Discussion: http://postgr.es/m/CAPmGK17dAZCXvwnfpr1eTfknTGdt%3DhYTV9405Gt5SqPOX8K84w%40mail.gmail.com
2022-02-21Disallow setting bogus GUCs within an extension's reserved namespace.Tom Lane
Commit 75d22069e tried to throw a warning for setting a custom GUC whose prefix belongs to a previously-loaded extension, if there is no such GUC defined by the extension. But that caused unstable behavior with parallel workers, because workers don't necessarily load extensions and GUCs in the same order their leader did. To make that work safely, we have to completely disallow the case. We now actually remove any such GUCs at the time of initial extension load, and then throw an error not just a warning if you try to add one later. While this might create a compatibility issue for a few people, the improvement in error-detection capability seems worth it; it's hard to believe that there's any good use-case for choosing such GUC names. This also un-reverts 5609cc01c (Rename EmitWarningsOnPlaceholders() to MarkGUCPrefixReserved()), since that function's old name is now even more of a misnomer. Florin Irion and Tom Lane Discussion: https://postgr.es/m/1902182.1640711215@sss.pgh.pa.us
2022-02-21pgcrypto: Remove unused error codePeter Eisentraut
PXE_DEV_READ_ERROR hasn't been used since random device support was removed from pgcrypto (fe0a0b5993dfe24e4b3bcf52fa64ff41a444b8f1).
2022-02-21pgcrypto: Remove unused error codePeter Eisentraut
PXE_MCRYPT_INTERNAL was apparently never used even when it was added.
2022-02-18Rearrange libpq's error reporting to avoid duplicated error text.Tom Lane
Since commit ffa2e4670, libpq accumulates text in conn->errorMessage across a whole query cycle. In some situations, we may report more than one error event within a cycle: the easiest case to reach is where we report a FATAL error message from the server, and then a bit later we detect loss of connection. Since, historically, each error PGresult bears the entire content of conn->errorMessage, this results in duplication of the FATAL message in any output that concatenates the contents of the PGresults. Accumulation in errorMessage still seems like a good idea, especially in view of the number of places that did ad-hoc error concatenation before ffa2e4670. So to fix this, let's track how much of conn->errorMessage has been read out into error PGresults, and only include new text in later PGresults. The tricky part of that is to be sure that we never discard an error PGresult once made (else we'd risk dropping some text, a problem much worse than duplication). While libpq formerly did that in some code paths, a little bit of rearrangement lets us postpone making an error PGresult at all until we are about to return it. A side benefit of that postponement is that it now becomes practical to return a dummy static PGresult in cases where we hit out-of-memory while trying to manufacture an error PGresult. This eliminates the admittedly-very-rare case where we'd return NULL from PQgetResult, indicating successful query completion, even though what actually happened was an OOM failure. Discussion: https://postgr.es/m/ab4288f8-be5c-57fb-2400-e3e857f53e46@enterprisedb.com
2022-02-18postgres_fdw: Make postgres_fdw.application_name support more escape sequences.Fujii Masao
Commit 6e0cb3dec1 allowed postgres_fdw.application_name to include escape sequences %a (application name), %d (database name), %u (user name) and %p (pid). In addition to them, this commit makes it support the escape sequences for session ID (%c) and cluster name (%C). These are helpful to investigate where each remote transactions came from. Author: Fujii Masao Reviewed-by: Ryohei Takahashi, Kyotaro Horiguchi Discussion: https://postgr.es/m/1041dc9a-c976-049f-9f14-e7d94c29c4b2@oss.nttdata.com
2022-02-17Remove all traces of tuplestore_donestoring() in the C codeMichael Paquier
This routine is a no-op since dd04e95 from 2003, with a macro kept around for compatibility purposes. This has led to the same code patterns being copy-pasted around for no effect, sometimes in confusing ways like in pg_logical_slot_get_changes_guts() from logical.c where the code was actually incorrect. This issue has been discussed on two different threads recently, so rather than living with this legacy, remove any uses of this routine in the C code to simplify things. The compatibility macro is kept to avoid breaking any out-of-core modules that depend on it. Reported-by: Tatsuhito Kasahara, Justin Pryzby Author: Tatsuhito Kasahara Discussion: https://postgr.es/m/20211217200419.GQ17618@telsasoft.com Discussion: https://postgr.es/m/CAP0=ZVJeeYfAeRfmzqAF2Lumdiv4S4FewyBnZd4DPTrsSQKJKw@mail.gmail.com
2022-02-16Split xlog.c into xlog.c and xlogrecovery.c.Heikki Linnakangas
This moves the functions related to performing WAL recovery into the new xlogrecovery.c source file, leaving xlog.c responsible for maintaining the WAL buffers, coordinating the startup and switch from recovery to normal operations, and other miscellaneous stuff that have always been in xlog.c. Reviewed-by: Andres Freund, Kyotaro Horiguchi, Robert Haas Discussion: https://www.postgresql.org/message-id/a31f27b4-a31d-f976-6217-2b03be646ffa%40iki.fi
2022-02-15Add isolation test for errors during logical slot creation.Andres Freund
I didn't include this test in 2f6501fa3c5, because I was not sure the error messages for the terminated connection is stable across platforms. But it sounds like it is, and if not, we'd want to do something about the instability anyway... Discussion: https://postgr.es/m/CAD21AoDAeEpAbZEyYJsPZJUmSPaRicVSBObaL7sPaofnKz+9zg@mail.gmail.com
2022-02-14Delete contrib/xml2's legacy implementation of xml_is_well_formed().Tom Lane
This function is unreferenced in modern usage; it was superseded in 9.1 by a core function of the same name. It has been left in place in the C code only so that pre-9.1 SQL definitions of the contrib/xml2 functions would continue to work. Eleven years seems like enough time for people to have updated to the extension-style version of the xml2 module, so let's drop this. We did this once before, in 20540710e, and then reverted it because the intended change of PGDLLEXPORT markings didn't happen. This time the reason is to suppress link-time duplicate-symbol warnings on AIX. That's not worth a lot perhaps, but the value of keeping this function has surely dropped to about zero by now. Discussion: https://postgr.es/m/2717731.1644778752@sss.pgh.pa.us
2022-02-14WAL log unchanged toasted replica identity key attributes.Amit Kapila
Currently, during UPDATE, the unchanged replica identity key attributes are not logged separately because they are getting logged as part of the new tuple. But if they are stored externally then the untoasted values are not getting logged as part of the new tuple and logical replication won't be able to replicate such UPDATEs. So we need to log such attributes as part of the old_key_tuple during UPDATE. Reported-by: Haiying Tang Author: Dilip Kumar and Amit Kapila Reviewed-by: Alvaro Herrera, Haiying Tang, Andres Freund Backpatch-through: 10 Discussion: https://postgr.es/m/OS0PR01MB611342D0A92D4F4BF26C0F47FB229@OS0PR01MB6113.jpnprd01.prod.outlook.com
2022-02-12Fix skip-empty-xacts with sequences in test_decodingTomas Vondra
Regression tests need to use skip-empty-xacts = false, because there might be accidental concurrent activity (like autovacuum), particularly on slow machines. The tests added by 80901b3291 failed to do that in a couple places, triggering occasional failures on buildfarm. Fixing the tests however uncovered a bug in the code, because sequence callbacks did not handle skip-empty-xacts properly. For trasactional increments we need to check/update the xact_wrote_changes flag, and emit the BEGIN if it's the first change in the transaction. Reported-by: Andres Freund Discussion: https://postgr.es/m/20220212220413.b25amklo7t4xb7ni%40alap3.anarazel.de
2022-02-11Add decoding of sequences to test_decodingTomas Vondra
Commit 0da92dc530 improved the logical decoding infrastructure to handle sequences, and did various changes to related parts (WAL logging etc.). But it did not include any implementation of the new callbacks added to OutputPluginCallbacks. This extends test_decoding with two callbacks to decode sequences. The decoding of sequences may be disabled using 'include-sequences', a new option of the output plugin. Author: Tomas Vondra, Cary Huang Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca
2022-02-11Replace Test::More plans with done_testingDaniel Gustafsson
Rather than doing manual book keeping to plan the number of tests to run in each TAP suite, conclude each run with done_testing() summing up the the number of tests that ran. This removes the need for maintaning and updating the plan count at the expense of an accurate count of remaining during the test suite runtime. This patch has been discussed a number of times, often in the context of other patches which updates tests, so a larger number of discussions can be found in the archives. Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/DD399313-3D56-4666-8079-88949DAC870F@yesql.se
2022-02-10Use Test::Builder::todo_start(), replacing $::TODO.Noah Misch
Some pre-2017 Test::More versions need perfect $Test::Builder::Level maintenance to find the variable. Buildfarm member snapper reported an overall failure that the file intended to hide via the TODO construct. That trouble was reachable in v11 and v10. For later branches, this serves as defense in depth. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220202055556.GB2745933@rfd.leadboat.com
2022-02-07Reduce non-leaf keys overlap in GiST indexes produced by a sorted buildAlexander Korotkov
The GiST sorted build currently chooses split points according to the only page space utilization. That may lead to higher non-leaf keys overlap and, in turn, slower search query answers. This commit makes the sorted build use the opclass's picksplit method. Once four pages at the level are accumulated, the picksplit method is applied until each split partition fits the page. Some of our split algorithms could show significant performance degradation while processing 4-times more data at once. But those opclasses haven't received the sorted build support and shouldn't receive it before their split algorithms are improved. Discussion: https://postgr.es/m/CAHqSB9jqtS94e9%3D0vxqQX5dxQA89N95UKyz-%3DA7Y%2B_YJt%2BVW5A%40mail.gmail.com Author: Aliaksandr Kalenik, Sergei Shoulbakov, Andrey Borodin Reviewed-by: Björn Harrtell, Darafei Praliaskouski, Andres Freund Reviewed-by: Alexander Korotkov
2022-02-03In basic_archive tests, insist on wal_level='replica'.Robert Haas
That's normally the default, but buildfarm member thorntail has other ideas. Nathan Bossart Discussion: http://postgr.es/m/20220203212539.GA1082940@nathanxps13
2022-02-03Allow archiving via loadable modules.Robert Haas
Running a shell command for each file to be archived has a lot of overhead and may not offer as much error checking as you want, or the exact semantics that you want. So, offer the option to call a loadable module for each file to be archived, rather than running a shell command. Also, add a 'basic_archive' contrib module as an example implementation that archives to a local directory. Nathan Bossart, with a little bit of kibitzing by me. Discussion: http://postgr.es/m/20220202224433.GA1036711@nathanxps13
2022-01-31Simplify coding around path_contains_parent_reference().Tom Lane
Given the existing stipulation that path_contains_parent_reference() must only be invoked on canonicalized paths, we can simplify things in the wake of commit c10f830c5. It is now only possible to see ".." at the start of a relative path. That means we can simplify path_contains_parent_reference() itself quite a bit, and it makes the two existing outside call sites dead code, since they'd already checked that the path is absolute. We could now fold path_contains_parent_reference() into its only remaining caller path_is_relative_and_below_cwd(). But it seems better to leave it as a separately callable function, in case any extensions are using it. Also document the pre-existing requirement for path_is_relative_and_below_cwd's input to be likewise canonicalized. Shenhao Wang and Tom Lane Discussion: https://postgr.es/m/OSBPR01MB4214FA221FFE046F11F2AD74F2D49@OSBPR01MB4214.jpnprd01.prod.outlook.com
2022-01-31Make canonicalize_path() more canonical.Tom Lane
Teach canonicalize_path() how to strip all unnecessary uses of "." and "..", replacing the previous ad-hoc code that got rid of only some such cases. In particular, we can always remove all such uses from absolute paths. The proximate reason to do this is that Windows rejects paths involving ".." in some cases (in particular, you can't put one in a symlink), so we ought to be sure we don't use ".." unnecessarily. Moreover, it seems like good cleanup on general principles. There is other path-munging code that could be simplified now, but we'll leave that for followup work. It is tempting to call this a bug fix and back-patch it. On the other hand, the misbehavior can only be reached if a highly privileged user does something dubious, so it's not unreasonable to say "so don't do that". And this patch could result in unexpected behavioral changes, in case anybody was expecting uses of ".." to stay put. So at least for now, just put it in HEAD. Shenhao Wang, editorialized a bit by me Discussion: https://postgr.es/m/OSBPR01MB4214FA221FFE046F11F2AD74F2D49@OSBPR01MB4214.jpnprd01.prod.outlook.com
2022-01-30Remove xloginsert.h from xlog.hAlvaro Herrera
xlog.h is directly and indirectly #included in a lot of places. With this change, xloginsert.h is no longer unnecessarily included in the large number of them that don't need it. Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Discussion: https://postgr.es/m/CALj2ACVe-W+WM5P44N7eG9C2_FmaeM8Dq5aCnD3fHt0Ba=WR6w@mail.gmail.com
2022-01-28Add HEADER support to COPY text formatPeter Eisentraut
The COPY CSV format supports the HEADER option to output a header line. This patch adds the same option to the default text format. On input, the HEADER option causes the first line to be skipped, same as with CSV. Author: Rémi Lapeyre <remi.lapeyre@lenstra.fr> Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com
2022-01-27postgres_fdw: Fix handling of a pending asynchronous request in ↵Etsuro Fujita
postgresReScanForeignScan(). Commit 27e1f1456 failed to process a pending asynchronous request made for a given ForeignScan node in postgresReScanForeignScan() (if any) in cases where we would only reset the next_tuple counter in that function, contradicting the assumption that there should be no pending asynchronous requests that have been made for async-capable subplans for the parent Append node after ReScan. This led to an assert failure in an assert-enabled build. I think this would also lead to mis-rewinding the cursor in that function in the case where we have already fetched one batch for the ForeignScan node and the asynchronous request has been made for the second batch, because even in that case we would just reset the counter when called from that function, so we would fail to execute MOVE BACKWARD ALL. To fix, modify that function to process the asynchronous request before restarting the scan. While at it, add a comment to a function to match other places. Per bug #17344 from Alexander Lakhin. Back-patch to v14 where the aforesaid commit came in. Patch by me. Test case by Alexander Lakhin, adjusted by me. Reviewed and tested by Alexander Lakhin and Dmitry Dolgov. Discussion: https://postgr.es/m/17344-226b78b00de73a7e@postgresql.org
2022-01-27On sparc64+ext4, suppress test failures from known WAL read failure.Noah Misch
Buildfarm members kittiwake, tadarida and snapper began to fail frequently when commits 3cd9c3b921977272e6650a5efbeade4203c4bca2 and f47ed79cc8a0cfa154dc7f01faaf59822552363f added tests of concurrency, but the problem was reachable before those commits. Back-patch to v10 (all supported versions). Discussion: https://postgr.es/m/20220116210241.GC756210@rfd.leadboat.com
2022-01-21postgres_fdw: Fix subabort cleanup of connections used in asynchronous ↵Etsuro Fujita
execution. Commit 27e1f1456 resets the per-connection states of connections used to scan foreign tables asynchronously during abort cleanup at main transaction end, but it failed to do so during subabort cleanup at subtransaction end, leading to a segmentation fault when re-executing an asynchronous-foreign-table-scan query in a transaction that was cancelled in a subtransaction of it. Fix by modifying pgfdw_abort_cleanup() to reset the per-connection state of a given connection also when called for subabort cleanup. Also, modify that function to do the reset in both the abort-cleanup and subabort-cleanup cases if necessary, to save cycles, and improve a comment on it a little bit. Back-patch to v14 where the aforesaid commit came in. Reviewed by Alexander Pyhalov Discussion: https://postgr.es/m/CAPmGK14cCV-JA7kNsyt2EUTKvZ4xkr2LNRthi1U1C3cqfGppAw@mail.gmail.com
2022-01-17Add Boolean nodePeter Eisentraut
Before, SQL-level boolean constants were represented by a string with a cast, and internal Boolean values in DDL commands were usually represented by Integer nodes. This takes the place of both of these uses, making the intent clearer and having some amount of type safety. Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/8c1a2e37-c68d-703c-5a83-7a6077f4f997@enterprisedb.com
2022-01-12Include permissive/enforcing state in sepgsql log messages.Tom Lane
SELinux itself does this (at least in modern releases), and it seems like a good idea to reduce confusion. Dave Page Discussion: https://postgr.es/m/CA+OCxowsQoLEYc=jN7OtNvOdX0Jg5L7nMYt++=k0X78HGq-sXg@mail.gmail.com
2022-01-11Improve error handling of cryptohash computationsMichael Paquier
The existing cryptohash facility was causing problems in some code paths related to MD5 (frontend and backend) that relied on the fact that the only type of error that could happen would be an OOM, as the MD5 implementation used in PostgreSQL ~13 (the in-core implementation is used when compiling with or without OpenSSL in those older versions), could fail only under this circumstance. The new cryptohash facilities can fail for reasons other than OOMs, like attempting MD5 when FIPS is enabled (upstream OpenSSL allows that up to 1.0.2, Fedora and Photon patch OpenSSL 1.1.1 to allow that), so this would cause incorrect reports to show up. This commit extends the cryptohash APIs so as callers of those routines can fetch more context when an error happens, by using a new routine called pg_cryptohash_error(). The error states are stored within each implementation's internal context data, so as it is possible to extend the logic depending on what's suited for an implementation. The default implementation requires few error states, but OpenSSL could report various issues depending on its internal state so more is needed in cryptohash_openssl.c, and the code is shaped so as we are always able to grab the necessary information. The core code is changed to adapt to the new error routine, painting more "const" across the call stack where the static errors are stored, particularly in authentication code paths on variables that provide log details. This way, any future changes would warn if attempting to free these strings. The MD5 authentication code was also a bit blurry about the handling of "logdetail" (LOG sent to the postmaster), so improve the comments related that, while on it. The origin of the problem is 87ae969, that introduced the centralized cryptohash facility. Extra changes are done for pgcrypto in v14 for the non-OpenSSL code path to cope with the improvements done by this commit. Reported-by: Michael Mühlbeyer Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/89B7F072-5BBE-4C92-903E-D83E865D9367@trivadis.com Backpatch-through: 14
2022-01-08Fix results of index-only scans on btree_gist char(N) indexes.Tom Lane
If contrib/btree_gist is used to make a GIST index on a char(N) (bpchar) column, and that column is retrieved via an index-only scan, what came out had all trailing spaces removed. Since that doesn't happen in any other kind of table scan, this is clearly a bug. The cause is that gbt_bpchar_compress() strips trailing spaces (using rtrim1) before a new index entry is made. That was probably a good idea when this code was first written, but since we invented index-only scans, it's not so good. One answer could be to mark this opclass as incapable of index-only scans. But to do so, we'd need an extension module version bump, followed by manual action by DBAs to install the updated version of btree_gist. And it's not really a desirable place to end up, anyway. Instead, let's fix the code by removing the unwanted space-stripping action and adjusting the opclass's comparison logic to ignore trailing spaces as bpchar normally does. This will not hinder cases that work today, since index searches with this logic will act the same whether trailing spaces are stored or not. It will not by itself fix the problem of getting space-stripped results from index-only scans, of course. Users who care about that can REINDEX affected indexes after installing this update, to immediately replace all improperly-truncated index entries. Otherwise, it can be expected that the index's behavior will change incrementally as old entries are replaced by new ones. Per report from Alexander Lakhin. Back-patch to all supported branches. Discussion: https://postgr.es/m/696c995b-b37f-5526-f45d-04abe713179f@gmail.com
2022-01-08Update copyright for 2022Bruce Momjian
Backpatch-through: 10
2022-01-07postgres_fdw: Add regression test for postgres_fdw.application_name GUC.Fujii Masao
Commit 449ab63505 added postgres_fdw.application_name GUC that specifies a value for application_name configuration parameter used when postgres_fdw establishes a connection to a foreign server. Also commit 6e0cb3dec1 allowed it to include escape sequences. Both commits added the regression tests for the GUC, but those tests were reverted by commits 98dbef90eb and 5e64ad3697 because they were unstable and caused some buildfarm members to report the failure. This is the third try to add the regression test for postgres_fdw.application_name GUC. One of issues to make the test unstable was to have used postgres_fdw_disconnect_all() to close the existing remote connections. The test expected that the remote connection and its corresponding backend at the remote server disappeared just after postgres_fdw_disconnect_all() was executed, but it could take a bit time for them to disappear. To make sure that they exit, this commit makes the test use pg_terminate_backend() with the timeout at the remote server, instead. If the timeout is set to greater than zero, this function waits until they are actually terminated (or until the given time has passed). Another issue was that the test didn't take into consideration the case where postgres_fdw.application_name containing some escape sequences was converted to the string larger than NAMEDATALEN. In this case it was truncated to less than NAMEDATALEN when it's passed to the remote server, but the test expected wrongly that full string of application_name was always viewable. This commit changes the test so that it can handle that case. Author: Fujii Masao Reviewed-by: Masahiko Sawada, Hayato Kuroda, Kyotaro Horiguchi Discussion: https://postgr.es/m/3220909.1631054766@sss.pgh.pa.us Discussion: https://postgr.es/m/20211224.180006.2247635208768233073.horikyota.ntt@gmail.com Discussion: https://postgr.es/m/e7b61420-a97b-8246-77c4-a0d48fba5a45@oss.nttdata.com
2022-01-05Enable routine running of citext's UTF8-specific test cases.Tom Lane
These test cases have been commented out since citext was invented, because at the time we had no nice way to deal with tests that have restrictions such as requiring UTF8 encoding. But now we do have a convention for that, ie put them into a separate test file with an early-exit path. So let's enable these tests to run when their prerequisites are satisfied. (We may have to tighten the prerequisites beyond the "encoding = UTF8 and locale != C" checks made here. But let's put it on the buildfarm and see what blows up.) Dag Lem Discussion: https://postgr.es/m/ygezgoacs4e.fsf_-_@sid.nimrod.no
2022-01-03Handle mixed returnable and non-returnable columns better in IOS.Tom Lane
We can revert the code changes of commit b5febc1d1 now, because commit 9a3ddeb51 installed a real solution for the difficulty that b5febc1d1 just dodged, namely that the planner might pick the wrong one of several index columns nominally containing the same value. It only matters which one we pick if we pick one that's not returnable, and that mistake is now foreclosed. Although both of the aforementioned commits were back-patched, I don't feel a need to take any risk by back-patching this one. The cases that it improves are very corner-ish. Discussion: https://postgr.es/m/3179992.1641150853@sss.pgh.pa.us
2022-01-03pg_stat_statements: Remove obsolete commentMichael Paquier
Since 4f0b096, pgss_store() does nothing if compute_query_id is disabled or if no other module computed a query identifier, but the top comment of this function did not reflect that. This behavior is already documented in its own code path, and this just removes the inconsistent comment. Author: Kyotaro Horiguchi Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/20211122.153823.1325120762360533122.horikyota.ntt@gmail.com Backpatch-through: 14
2021-12-27Revert changes about warnings/errors for placeholders.Tom Lane
Revert commits 5609cc01c, 2ed8a8cc5, and 75d22069e until we have a less broken idea of how this should work in parallel workers. Per buildfarm. Discussion: https://postgr.es/m/1640909.1640638123@sss.pgh.pa.us
2021-12-27Rename EmitWarningsOnPlaceholders() to MarkGUCPrefixReserved().Tom Lane
This seems like a clearer name for what it does now. Provide a compatibility macro so that extensions don't have to convert to the new name right away. Discussion: https://postgr.es/m/116024.1640111629@sss.pgh.pa.us
2021-12-24postgres_fdw: Revert unstable tests for postgres_fdw.application_name.Fujii Masao
Commit 6e0cb3dec1 added the tests that check that escape sequences in postgres_fdw.application_name setting are replaced with status information expectedly. But they were unstable and caused some buildfarm members to report the failure. This commit reverts those unstable tests.
2021-12-24postgres_fdw: Allow postgres_fdw.application_name to include escape sequences.Fujii Masao
application_name that used when postgres_fdw establishes a connection to a foreign server can be specified in either or both a connection parameter of a server object and GUC postgres_fdw.application_name. This commit allows those parameters to include escape sequences that begins with % character. Then postgres_fdw replaces those escape sequences with status information. For example, %d and %u are replaced with user name and database name in local server, respectively. This feature enables us to add information more easily to track remote transactions or queries, into application_name of a remote connection. Author: Hayato Kuroda Reviewed-by: Kyotaro Horiguchi, Masahiro Ikeda, Hou Zhijie, Fujii Masao Discussion: https://postgr.es/m/TYAPR01MB5866FAE71C66547C64616584F5EB9@TYAPR01MB5866.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/TYCPR01MB5870D1E8B949DAF6D3B84E02F5F29@TYCPR01MB5870.jpnprd01.prod.outlook.com
2021-12-22Remove assertion for ALTER TABLE .. DETACH PARTITION CONCURRENTLYMichael Paquier
One code path related to this flavor of ALTER TABLE was checking that the relation to detach has to be a normal table or a partitioned table, which would fail if using the command with a different relation kind. Views, sequences and materialized views cannot be part of a partition tree, so these would cause the command to fail anyway, but the assertion was triggered. Foreign tables can be part of a partition tree, and again the assertion would have failed. The simplest solution is just to remove this assertion, so as we get the same failure as the non-concurrent code path. While on it, add a regression test in postgres_fdw for the concurrent partition detach of a foreign table, as per a suggestion from Alexander Lakhin. Issue introduced in 71f4c8c. Reported-by: Alexander Lakhin Author: Michael Paquier, Alexander Lakhin Reviewed-by: Peter Eisentraut, Kyotaro Horiguchi Discussion: https://postgr.es/m/17339-a9e09aaf38a3457a@postgresql.org Backpatch-through: 14
2021-12-21Add missing EmitWarningsOnPlaceholders() calls.Tom Lane
Extensions that define any custom GUCs should call EmitWarningsOnPlaceholders after doing so, to help catch misspellings. Many of our contrib modules hadn't gotten the memo on that, though. Also add such calls to src/test/modules extensions that have GUCs. While these aren't really user-facing, they should illustrate good practice not faulty practice. Shinya Kato Discussion: https://postgr.es/m/524fa2c0a34f34b68fbfa90d0760d515@oss.nttdata.com