summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)Author
2023-01-26DROP ROLE regress_role_limited_admin at end of testRobert Haas
This is required by project policy, and I overlooked the need for it (again) by accident. Reported by Álvaro Herrara. Discussion: http://postgr.es/m/20230126114659.x3yuypot7p6zj73c@alvherre.pgsql
2023-01-26Don't install postmaster symlink anymorePeter Eisentraut
This has long been deprecated. Some of the build systems didn't even install it. Also remove man page. Reviewed-by: Karl O. Pinc <kop@karlpinc.com> Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-26Remove gratuitous references to postmaster programPeter Eisentraut
"postgres" has long been officially preferred over "postmaster" as the name of the program to invoke to run the server. Some example scripts and code comments still used the latter. Change those. Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-26Revert "Add eager and lazy freezing strategies to VACUUM."Peter Geoghegan
This reverts commit 4d417992613949af35530b4e8e83670c4e67e1b2. Broad concerns about regressions caused by eager freezing strategy have been raised. Whether or not these concerns can be worked through in any time frame is far from certain. Discussion: https://postgr.es/m/20230126004347.gepcmyenk2csxrri@awork3.anarazel.de
2023-01-26Make auto_explain print the query identifier in verbose modeMichael Paquier
When auto_explain.log_verbose is on, auto_explain should print in the logs plans equivalent to the EXPLAIN (VERBOSE). However, when compute_query_id is on, query identifiers were not showing up, being only handled by EXPLAIN (VERBOSE). This brings auto_explain on par with EXPLAIN regarding that. Note that like EXPLAIN, auto_explain does not show the query identifier when compute_query_id=regress. The change is done so as the choice of printing the query identifier is done in ExplainPrintPlan() rather than in ExplainOnePlan(), to avoid a duplication of the logic dealing with the query ID. auto_explain is the only in-core caller of ExplainPrintPlan(). While looking at the area, I have noticed that more consolidation between EXPLAIN and auto_explain would be in order for the logging of the plan duration and the buffer usage. This refactoring is left as a future change. Author: Atsushi Torikoshi Reviewed-by: Justin Pryzby, Julien Rouhaud Discussion: https://postgr.es/m/1ea21936981f161bccfce05765c03bee@oss.nttdata.com
2023-01-26Fix rare sharedtuplestore.c corruption.Thomas Munro
If the final chunk of an oversized tuple being written out to disk was exactly 32760 bytes, it would be corrupted due to a fencepost bug. Bug #17619. Back-patch to 11 where the code arrived. While testing that (see test module in archives), I (tmunro) noticed that the per-participant page counter was not initialized to zero as it should have been; that wasn't a live bug when it was written since DSM memory was originally always zeroed, but since 14 min_dynamic_shared_memory might be configured and it supplies non-zeroed memory, so that is also fixed here. Author: Dmitry Astapov <dastapov@gmail.com> Discussion: https://postgr.es/m/17619-0de62ceda812b8b5%40postgresql.org
2023-01-25Add eager and lazy freezing strategies to VACUUM.Peter Geoghegan
Eager freezing strategy avoids large build-ups of all-visible pages. It makes VACUUM trigger page-level freezing whenever doing so will enable the page to become all-frozen in the visibility map. This is useful for tables that experience continual growth, particularly strict append-only tables such as pgbench's history table. Eager freezing significantly improves performance stability by spreading out the cost of freezing over time, rather than doing most freezing during aggressive VACUUMs. It complements the insert autovacuum mechanism added by commit b07642db. VACUUM determines its freezing strategy based on the value of the new vacuum_freeze_strategy_threshold GUC (or reloption) with logged tables. Tables that exceed the size threshold use the eager freezing strategy. Unlogged tables and temp tables always use eager freezing strategy, since the added cost is negligible there. Non-permanent relations won't incur any extra overhead in WAL written (for the obvious reason), nor in pages dirtied (since any extra freezing will only take place on pages whose PD_ALL_VISIBLE bit needed to be set either way). VACUUM uses lazy freezing strategy for logged tables that fall under the GUC size threshold. Page-level freezing triggers based on the criteria established in commit 1de58df4, which added basic page-level freezing. Eager freezing is strictly more aggressive than lazy freezing. Settings like vacuum_freeze_min_age still get applied in just the same way in every VACUUM, independent of the strategy in use. The only mechanical difference between eager and lazy freezing strategies is that only the former applies its own additional criteria to trigger freezing pages. Note that even lazy freezing strategy will trigger freezing whenever a page happens to have required that an FPI be written during pruning, provided that the page will thereby become all-frozen in the visibility map afterwards (due to the FPI optimization from commit 1de58df4). The vacuum_freeze_strategy_threshold default setting is 4GB. This is a relatively low setting that prioritizes performance stability. It will be reviewed at the end of the Postgres 16 beta period. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Jeff Davis <pgsql@j-davis.com> Reviewed-By: Andres Freund <andres@anarazel.de> Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com> Discussion: https://postgr.es/m/CAH2-WzkFok_6EAHuK39GaW4FjEFQsY=3J0AAd6FXk93u-Xq3Fg@mail.gmail.com
2023-01-25plpython: Stop undefining _POSIX_C_SOURCE, _XOPEN_SOURCEAndres Freund
We undefined them to avoid warnings about macro redefinitions. But we haven't fully followed the necessary include order, since at least 147c2482542, in 2011. Recently the combination of the include order rules not being followed and undefining _POSIX_C_SOURCE started to cause a compile failure, starting with 03023a2664f. Undefining _POSIX_C_SOURCE hides clock_gettime(), which is referenced in an inline function as of 03023a2664f, whereas it was a macro before. After seeing some evidence that undefining _POSIX_C_SOURCE et al isn't required, I tried to build postgres with plpython on most of our supported platforms (except DragonFlyBSD and Illumos, but similar systems were tested), with/without the #undefines. No compiler warning / behavioral difference. The oldest supported python version, 3.2, defines _POSIX_C_SOURCE to 200112L ad _XOPEN_SOURCE to 600, whereas newer versions of python use 200809L/700 respectively. As _POSIX_C_SOURCE/_XOPEN_SOURCE will default to the newer operating system on most platforms, it's possible that when using python 3.2 new warnings would be emitted - but that seems acceptable. It's possible that this approach won't work on some older platforms. But getting rid of most of the include-order complexity seems promising, and it's an easily revertible patch if we end up having to go another way. Discussion: https://postgr.es/m/20230124165814.2njc7gnvubn2amh6@awork3.anarazel.de
2023-01-25plpython: Avoid the need to redefine *printf macrosAndres Freund
Until now we undefined and then redefined a lot of *printf macros due to worries about conflicts with Python.h macro definitions. Current Python.h doesn't define any *printf macros, and older versions just defined snprintf, vsnprintf, guarded by #if defined(MS_WIN32) && !defined(HAVE_SNPRINTF). Thus we can replace the undefine/define section with a single #define HAVE_SNPRINTF 1 Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20230124165814.2njc7gnvubn2amh6@awork3.anarazel.de
2023-01-25Avoid type cheats for invalid dsa_handles and dshash_table_handles.Tom Lane
Invent separate macros for "invalid" values of these types, so that we needn't embed knowledge of their representations into calling code. These are all zeroes anyway ATM, so this is not fixing any live bug, but it makes the code cleaner and more future-proof. I (tgl) also chose to move DSM_HANDLE_INVALID into dsm_impl.h, since it seems like it should live beside the typedef for dsm_handle. Hou Zhijie, Nathan Bossart, Kyotaro Horiguchi, Tom Lane Discussion: https://postgr.es/m/OS0PR01MB5716860B1454C34E5B179B6694C99@OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-01-25Process pending postmaster work before connections.Thomas Munro
Modify the new event loop code from commit 7389aad6 so that it checks for work requested by signal handlers even if it doesn't see a latch event yet. This gives priority to shutdown and reload requests where the latch will be reported later in the event array, or in a later call to WaitEventSetWait(), due to scheduling details. In particular, this guarantees that a SIGHUP-then-connect sequence (as seen in authentication tests) causes the postmaster to process the reload before accepting the connection. If the WaitEventSetWait() call saw the socket as ready, and the reload signal was generated before the connection, then the latest time the signal handler should be able to run is after poll/epoll_wait/kevent returns but before we check the pending_pm_reload_request flag. While here, also shift the handling of child exit below reload requests, per Tom Lane's observation that that might start new processes, so we should make sure we pick up new settings first. This probably explains the one-off failure of build farm animal malleefowl. Reported-by: Hou Zhijie <houzj.fnst@fujitsu.com> Reported-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/OS0PR01MB57163D3BF2AB42ECAA94E5C394C29%40OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-01-24Update more obsolete multixact.c comments.Peter Geoghegan
Update some remaining comments in multixact.c that still described SLRU truncation as happening in the checkpointer, rather than during VACUUM. Follow-up to commit 5212d447. Shi yu, with tweaks by me. Author: Shi yu <shiy.fnst@fujitsu.com> Discussion: https://postgr.es/m/OSZPR01MB631066BF246F8F74E83222FCFDC69@OSZPR01MB6310.jpnprd01.prod.outlook.com
2023-01-24Improve exclude pattern file processing in pgindentAndrew Dunstan
This makes two small changes that will improve pgindent's usefulness in a git hook. First, it looks for the exclude file relative to the current directory. And second, it applies the filters to filenames given on the command line as well as those found in a directory sweep. It might prove necessary to make further efforts to find the exclude file, and even to allow multiple exclude files, but for now this should be enough for most purposes. Reviewed by Jelte Fennema
2023-01-24Adjust interaction of CREATEROLE with role properties.Robert Haas
Previously, a CREATEROLE user without SUPERUSER could not alter REPLICATION users in any way, and could not set the BYPASSRLS attribute. However, they could manipulate the CREATEDB property even if they themselves did not possess it. With this change, a CREATEROLE user without SUPERUSER can set or clear the REPLICATION, BYPASSRLS, or CREATEDB property on a new role or a role that they have rights to manage if and only if that property is set for their own role. This implements the standard idea that you can't give permissions you don't have (but you can give the ones you do have). We might in the future want to provide more powerful ways to constrain what a CREATEROLE user can do - for example, to limit whether CONNECTION LIMIT can be set or the values to which it can be set - but that is left as future work. Patch by me, reviewed by Nathan Bossart, Tushar Ahuja, and Neha Sharma. Discussion: http://postgr.es/m/CA+TgmobX=LHg_J5aT=0pi9gJy=JdtrUVGAu0zhr-i5v5nNbJDg@mail.gmail.com
2023-01-24Fix the Drop Database hang.Amit Kapila
The drop database command waits for the logical replication sync worker to accept ProcSignalBarrier and the worker's slot creation waits for the drop database to finish which leads to a deadlock. This happens because the tablesync worker holds interrupts while creating a slot. We prevent cancel/die interrupts while creating a slot in the table sync worker because it is possible that before the server finishes this command, a concurrent drop subscription happens which would complete without removing this slot and that leads to the slot existing until the end of walsender. However, the slot will eventually get dropped at the walsender exit time, so there is no danger of the dangling slot. This patch reallows cancel/die interrupts while creating a slot and modifies the test to wait for slots to become zero to prevent finding an ephemeral slot. The reported hang doesn't happen in PG14 as the drop database starts to wait for ProcSignalBarrier with PG15 (commits 4eb2176318 and e2f65f4255) but it is good to backpatch this till PG14 as it is not a good idea to prevent interrupts during a network call that could block indefinitely. Reported-by: Lakshmi Narayanan Sreethar Diagnosed-by: Andres Freund Author: Hou Zhijie Reviewed-by: Vignesh C, Amit Kapila Backpatch-through: 14, where it was introduced in commit 6b67d72b60 Discussion: https://postgr.es/m/CA+kvmZELXQ4ZD3U=XCXuG3KvFgkuPoN1QrEj8c-rMRodrLOnsg@mail.gmail.com
2023-01-24libpqwalreceiver: Convert to libpq-be-fe-helpers.hAndres Freund
In contrast to the changes to dblink and postgres_fdw, this does not fix a bug, as libpqwalreceiver did already process interrupts. Besides reducing code duplication, the conversion leads to libpqwalreceiver now using reserving file descriptors for libpq connections. While not strictly required for the use in walreceiver, we are also using libpqwalreceiver for logical replication, where it does seem more important. Even if we eventually decide to backpatch the prior commits, there'd be no need to backpatch this commit, due to not fixing an active bug. Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2@awork3.anarazel.de
2023-01-24Add helper library for use of libpq inside the server environmentAndres Freund
Currently dblink and postgres_fdw don't process interrupts during connection establishment. Besides preventing query cancellations etc, this can lead to undetected deadlocks, as global barriers are not processed. Libpqwalreceiver in contrast, processes interrupts during connection establishment. The required code is not trivial, so duplicating it into additional places does not seem like a good option. These aforementioned undetected deadlocks are the reason for the spate of CI test failures in the FreeBSD 'test_running' step. For now the helper library is just a header, as it needs to be linked into each extension using libpq, and it seems too small to be worth adding a dedicated static library for. The conversion to the helper are done in subsequent commits. Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2@awork3.anarazel.de
2023-01-24Fix error handling in libpqrcv_connect()Andres Freund
When libpqrcv_connect (also known as walrcv_connect()) failed, it leaked the libpq connection. In most paths that's fairly harmless, as the calling process will exit soon after. But e.g. CREATE SUBSCRIPTION could lead to a somewhat longer lived leak. Fix by releasing resources, including the libpq connection, on error. Add a test exercising the error code path. To make it reliable and safe, the test tries to connect to port=-1, which happens to fail during connection establishment, rather than during connection string parsing. Reviewed-by: Noah Misch <noah@leadboat.com> Discussion: https://postgr.es/m/20230121011237.q52apbvlarfv6jm6@awork3.anarazel.de Backpatch: 11-
2023-01-24Use OFFSET 0 instead of ORDER BY to stop subquery pullupDavid Rowley
b762fed64 recently changed this test to prevent subquery pullup to allow us to test Memoize with lateral_vars. As pointed out by Tom Lane, OFFSET 0 is our standard way of preventing subquery pullups, so do it that way instead. Discussion: https://postgr.es/m/2144818.1674517061@sss.pgh.pa.us Backpatch-through: 14, same as b762fed64
2023-01-23Fix LATERAL join test in test memoize.sqlDavid Rowley
The test in question was meant to be testing Memoize to ensure it worked correctly when the inner side of the join contained lateral vars, however, nothing in the lateral subquery stopped it from being pulled up into the main query, so the planner did that, and that meant no more lateral vars. Here we add a simple ORDER BY to stop the planner from being able to pullup the lateral subquery. Author: Richard Guo Discussion: https://postgr.es/m/CAMbWs4_LHJaN4L-tXpKMiPFnsCJWU1P8Xh59o0W7AA6UN99=cQ@mail.gmail.com Backpatch-through: 14, where Memoize was added.
2023-01-23Fix XLogPageRead() commentPeter Eisentraut
7fcbf6a and 2ff6555 changed the function signature of XLogPageRead() but did not update the comment. XLogReaderRoutine contains up to date information about the API, so no need to repeat all that at XLogPageRead(), but fix the mentions of the no longer existing function arguments.
2023-01-23Add non-decimal integer support to type numeric.Dean Rasheed
This enhances the numeric type input function, adding support for hexadecimal, octal, and binary integers of any size, up to the limits of the numeric type. Since 6fcda9aba8, such non-decimal integers have been accepted by the parser as integer literals and passed through to numeric_in(). This commit gives numeric_in() the ability to handle them. While at it, simplify the handling of NaN and infinities, reducing the number of calls to pg_strncasecmp(), and arrange for pg_strncasecmp() to not be called at all for regular numbers. This gives a significant performance improvement for decimal inputs, more than offsetting the small performance hit of checking for non-decimal input. Discussion: https://postgr.es/m/CAEZATCV8XShnmT9HZy25C%2Bo78CVOFmUN5EM9FRAZ5xvYTggPMg%40mail.gmail.com
2023-01-23Fix pgindent --show-diff option.Tom Lane
At least on my machine, the initial coding of this didn't actually work, because interpolation of "$post_fh->filename" doesn't act as intended. I threw in some double quotes too, just in case anybody tries to run this in a path containing spaces.
2023-01-23Remove special outfuncs/readfuncs handling of RangeVar.catalogname.Tom Lane
Historically we skipped writing/reading this field, but that no longer works under WRITE_READ_PARSE_PLAN_TREES since we expanded the coverage of that option to include utility commands (787102b56). Remove the special case and just treat this field normally. Bump catversion out of an abundance of caution --- I do not think we currently ever store RangeVar nodes in the catalogs, but perhaps I'm wrong. Per report from Pavel Stehule. Discussion: https://postgr.es/m/CAFj8pRAYvYu-qU7-NieqRRyaQZk-yr3UjtHQ2LR62PS9M1dZMA@mail.gmail.com
2023-01-23Add a test using ldapbindpasswd in pg_hba.confAndrew Dunstan
This feature has not been covered in tests up to now. John Naylor and Andrew Dunstan Discussion: https://postgr.es/m/06005bfb-0fd7-9d08-e0e5-440f277b73b4@dunslane.net
2023-01-23Restructure Ldap TAP testAndrew Dunstan
The code for detecting the Ldap installation and setting up a test server is broken out into a reusable module that can be used for additional tests to be added in later patches. Discussion: https://postgr.es/m/06005bfb-0fd7-9d08-e0e5-440f277b73b4@dunslane.net
2023-01-23Add non-destructive modes to pgindentAndrew Dunstan
This adds two modes of running pgindent, neither of which results in any changes being made to the source code. The --show-diff option shows what changes would have been made, and the --silent-diff option just exits with a status of 2 if any changes would be made. The second of these is intended for scripting use in places such as git hooks. Along the way some code cleanup is done, and a --help option is also added. Reviewed by Tom Lane Discussion: https://postgr.es/m/c9c9fa6d-6de6-48c2-4f8b-0fbeef026439@dunslane.net
2023-01-23Optimise numeric division for 3 and 4 base-NBASE digit divisors.Dean Rasheed
On platforms with 128-bit integer support, introduce a new function div_var_int64(), along the same lines as div_var_int() added in d1b307eef2 for divisors with 1 or 2 base-NBASE digits, and use it to speed up div_var() and div_var_fast() in a similar way when the divisor has 3 or 4 base-NBASE digits. This gives significant performance gains for divisors with 9-16 decimal digits. Joel Jacobson. Discussion: https://postgr.es/m/b7a5893d-af18-4c0b-8918-96de5f1bbf39%40app.fastmail.com https://postgr.es/m/CAEZATCXGm%3DDyTq%3DFrcOqC0gPMVveKUYTaD5KRRoajrUTiWxVMw%40mail.gmail.com
2023-01-23Run pgindent on heapam.cDavid Rowley
An upcoming patch by Melanie Plageman does some refactoring work in this area. Run pgindent on that file now before making any changes so that it's easier to maintain/evolve each of the individual patches doing the refactor work. Additionally, add a few new required typedefs to the list to make it easier to do future pgindent runs on this file during the refactor work. Discussion: https://postgr.es/m/CAAKRu_YSOnhKsDyFcqJsKtBSrd32DP-jjXmv7hL0BPD-z0TGXQ@mail.gmail.com
2023-01-23Fix and clarify function comment on LogicalTapeSetCreate.Heikki Linnakangas
Commit c4649cce39 removed the "shared" and "ntapes" arguments, but the comment still talked about "shared". It also talked about "a shared file handle", which was technically correct because even before commit c4649cce39, the "shared file handle" referred to the "fileset" argument, not "shared". But it was very confusing. Improve the comment. Also add a comment on what the "preallocate" argument does. Backpatch to v15, just to make backpatching other patches easier in the future. Discussion: https://www.postgresql.org/message-id/af989685-91d5-aad4-8f60-1d066b5ec309@enterprisedb.com Reviewed-by: Peter Eisentraut
2023-01-23Harden new parallel string_agg/array_agg regression testDavid Rowley
Per buildfarm member mandrill, it seems that max_parallel_workers_per_gather may not always be set to the default value of 2 when the new test added in 16fd03e95 is executed. Here let's just explicitly set that to 2 so that the planner never opts to use more than that many parallel workers.
2023-01-23Allow parallel aggregate on string_agg and array_aggDavid Rowley
This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-22Track logrep apply workers' last start times to avoid useless waits.Tom Lane
Enforce wal_retrieve_retry_interval on a per-subscription basis, rather than globally, and arrange to skip that delay in case of an intentional worker exit. This probably makes little difference in the field, where apply workers wouldn't be restarted often; but it has a significant impact on the runtime of our logical replication regression tests (even though those tests use artificially-small wal_retrieve_retry_interval settings already). Nathan Bossart, with mostly-cosmetic editorialization by me Discussion: https://postgr.es/m/20221122004119.GA132961@nathanxps13
2023-01-21Allow REPLICA IDENTITY to be set on an index that's not (yet) valid.Tom Lane
The motivation for this change is that when pg_dump dumps a partitioned index that's marked REPLICA IDENTITY, it generates a command sequence that applies REPLICA IDENTITY before the partitioned index has been marked valid, causing restore to fail. We could perhaps change pg_dump to not do it like that, but that would be difficult and would not fix existing dump files with the problem. There seems to be very little reason for the backend to disallow this anyway --- the code ignores indisreplident when the index isn't valid --- so instead let's fix it by allowing the case. Commit 9511fb37a previously expressed a concern that allowing indisreplident to be set on invalid indexes might allow us to wind up in a situation where a table could have indisreplident set on multiple indexes. I'm not sure I follow that concern exactly, but in any case the only way that could happen is because relation_mark_replica_identity is too trusting about the existing set of markings being valid. Let's just rip out its early-exit code path (which sure looks like premature optimization anyway; what are we doing expending code to make redundant ALTER TABLE ... REPLICA IDENTITY commands marginally faster and not-redundant ones marginally slower?) and fix it to positively guarantee that no more than one index is marked indisreplident. The pg_dump failure can be demonstrated in all supported branches, so back-patch all the way. I chose to back-patch 9511fb37a as well, just to keep indisreplident handling the same in all branches. Per bug #17756 from Sergey Belyashov. Discussion: https://postgr.es/m/17756-dd50e8e0c8dd4a40@postgresql.org
2023-01-21Reject CancelRequestPacket having unexpected length.Noah Misch
When the length was too short, the server read outside the allocation. That yielded the same log noise as sending the correct length with (backendPID,cancelAuthCode) matching nothing. Change to a message about the unexpected length. Given the attacker's lack of control over the memory layout and the general lack of diversity in memory layouts at the code in question, we doubt a would-be attacker could cause a segfault. Hence, while the report arrived via security@postgresql.org, this is not a vulnerability. Back-patch to v11 (all supported versions). Andrey Borodin, reviewed by Tom Lane. Reported by Andrey Borodin.
2023-01-21instr_time: Represent time as an int64 on all platformsAndres Freund
Until now we used struct timespec for instr_time on all platforms but windows. Using struct timespec causes a fair bit of memory (struct timeval is 16 bytes) and runtime overhead (much more complicated additions). Instead we can convert the time to nanoseconds in INSTR_TIME_SET_CURRENT(), making the remaining operations cheaper. Representing time as int64 nanoseconds provides sufficient range, ~292 years relative to a starting point (depending on clock source, relative to the unix epoch or the system's boot time). That'd not be sufficient for calendar time stored on disk, but is plenty for runtime interval time measurement. On windows instr_time already is represented as cycles. It might make sense to represent time as cycles on other platforms as well, as using cycle acquisition instructions like rdtsc directly can reduce the overhead of time acquisition substantially. This could be done in a fairly localized manner as the code stands after this commit. Because the windows and non-windows paths are now more similar, use a common set of macros. To make that possible, most of the use of LARGE_INTEGER had to be removed, which looks nicer anyway. To avoid users of the API relying on the integer representation, we wrap the 64bit integer inside struct struct instr_time. Author: Andres Freund <andres@anarazel.de> Author: Lukas Fittl <lukas@fittl.com> Author: David Geier <geidav.pg@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20230113195547.k4nlrmawpijqwlsa@awork3.anarazel.de
2023-01-21Zero initialize uses of instr_time about to trigger compiler warningsAndres Freund
These are all not necessary from a correctness POV. However, in the near future instr_time will be simplified to an int64, at which point gcc would otherwise start to warn about the changed places. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20230116023639.rn36vf6ajqmfciua@awork3.anarazel.de
2023-01-21Rework format of comments in headers for nodesMichael Paquier
This is similar to 835d476, except that this one is to add node attributes related to query jumbling and avoid long lines in the headers and in the node structures changed by this commit. Reviewed-by: Peter Eisentraut Discussion: https://postgr.es/m/Y5BHOUhX3zTH/ig6@paquier.xyz
2023-01-21Move queryjumble.c code to src/backend/nodes/Michael Paquier
This will ease a follow-up move that will generate automatically this code. The C file is renamed, for consistency with the node-related files whose code are generated by gen_node_support.pl: - queryjumble.c -> queryjumblefuncs.c - utils/queryjumble.h -> nodes/queryjumble.h Per a suggestion from Peter Eisentraut. Reviewed-by: Peter Eisentraut Discussion: https://postgr.es/m/Y5BHOUhX3zTH/ig6@paquier.xyz
2023-01-20Bump catversion for 6e2775e4d4e47775f0d933e4a93c148024a3bc63.Robert Haas
It creates a new predefined role.
2023-01-20Add new GUC reserved_connections.Robert Haas
This provides a way to reserve connection slots for non-superusers. The slots reserved via the new GUC are available only to users who have the new predefined role pg_use_reserved_connections. superuser_reserved_connections remains as a final reserve in case reserved_connections has been exhausted. Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me. Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20Rename ReservedBackends variable to SuperuserReservedConnections.Robert Haas
This is in preparation for adding a new reserved_connections GUC, but aligning the GUC name with the variable name is also a good idea on general principle. Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me. Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20Update docs and error message for superuser_reserved_connections.Robert Haas
Commit ea92368cd1da1e290f9ab8efb7f60cb7598fc310 made max_wal_senders a separate pool of backends from max_connections, but the documentation and error message for superuser_reserved_connections weren't updated at the time, and as a result are somewhat misleading. Update. This is arguably a back-patchable bug fix, but because it seems quite minor, no back-patch. Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me. Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20Remove SHM_QUEUEAndres Freund
Prior patches got rid of all the uses of SHM_QUEUE. ilist.h style lists are more widely used and have an easier to use interface. As there are no users left, remove SHM_QUEUE. Reviewed-by: Thomas Munro <thomas.munro@gmail.com> (in an older version) Discussion: https://postgr.es/m/20221120055930.t6kl3tyivzhlrzu2@awork3.anarazel.de Discussion: https://postgr.es/m/20200211042229.msv23badgqljrdg2@alap3.anarazel.de
2023-01-20Use dlists instead of SHM_QUEUE for predicate lockingAndres Freund
Part of a series to remove SHM_QUEUE. ilist.h style lists are more widely used and have an easier to use interface. Reviewed-by: Thomas Munro <thomas.munro@gmail.com> (in an older version) Discussion: https://postgr.es/m/20221120055930.t6kl3tyivzhlrzu2@awork3.anarazel.de Discussion: https://postgr.es/m/20200211042229.msv23badgqljrdg2@alap3.anarazel.de
2023-01-20Support the same patterns for pg-user in pg_ident.conf as in pg_hba.confMichael Paquier
While pg_hba.conf has support for non-literal username matches, and this commit extends the capabilities that are supported for the PostgreSQL user listed in an ident entry part of pg_ident.conf, with support for: 1. The "all" keyword, where all the requested users are allowed. 2. Membership checks using the + prefix. 3. Using a regex to match against multiple roles. 1. is a feature that has been requested by Jelte Fennema, 2. something that has been mentioned independently by Andrew Dunstan, and 3. is something I came up with while discussing how to extend the first one, whose implementation is facilitated by 8fea868. This allows matching certain system users against many different postgres users with a single line in pg_ident.conf. Without this, one would need one line for each of the postgres users that a system user can log in as, which can be cumbersome to maintain. Tests are added to the TAP test of peer authentication to provide coverage for all that. Note that this introduces a set of backward-incompatible changes to be able to detect the new patterns, for the following cases: - A role named "all". - A role prefixed with '+' characters, which is something that would not have worked in HBA entries anyway. - A role prefixed by a slash character, similarly to 8fea868. Any of these can be still be handled by using quotes in the Postgres role defined in an ident entry. A huge advantage of this change is that the code applies the same checks for the Postgres roles in HBA and ident entries, via the common routine check_role(). **This compatibility change should be mentioned in the release notes.** Author: Jelte Fennema Discussion: https://postgr.es/m/DBBPR83MB0507FEC2E8965012990A80D0F7FC9@DBBPR83MB0507.EURPRD83.prod.outlook.com
2023-01-20Avoid harmless warning from pg_dump --if-exists mode.Tom Lane
If the public schema has a non-default owner (perhaps due to dropping and recreating it) then use of pg_dump's "--if-exists" option results in a warning message: warning: could not find where to insert IF EXISTS in statement "-- *not* dropping schema, since initdb creates it" This is harmless since the dump output is the same either way, but nonetheless it's undesirable. It's the fault of commit a7a7be1f2, which created situations where a TOC entry's "defn" or "dropStmt" fields could be just comments. Although that commit fixed up the kluges in pg_backup_archiver.c that munge defn strings, it missed doing so for the one that munges dropStmts. Per bug# 17753 from Justin Zhang. Discussion: https://postgr.es/m/17753-9c8773631747ee1c@postgresql.org
2023-01-20Use appendStringInfoSpaces in more placesDavid Rowley
This adjusts a few places which were appending a string constant containing spaces onto a StringInfo. We have appendStringInfoSpaces for that job, so let's use that instead. For the change to jsonb.c's add_indent() function, appendStringInfoString was being called inside a loop to append 4 spaces on each loop. This meant that enlargeStringInfo would get called once per loop. Here it should be much more efficient to get rid of the loop and just calculate the number of spaces with "level * 4" and just append all the spaces in one go. Here we additionally adjust the appendStringInfoSpaces function so it makes use of memset rather than a while loop to apply the required spaces to the StringInfo. One of the problems with the while loop was that it was incrementing one variable and decrementing another variable once per loop. That's more work than what's required to get the job done. We may as well use memset for this rather than trying to optimize the existing loop. Some testing has shown memset is faster even for very small sizes. Discussion: https://postgr.es/m/CAApHDvp_rKkvwudBKgBHniNRg67bzXVjyvVKfX0G2zS967K43A@mail.gmail.com
2023-01-19Improve comment about GetWALAvailability's WALAVAIL_REMOVED code.Tom Lane
Sirisha Chamarthi and Kyotaro Horiguchi Discussion: https://postgr.es/m/CAKrAKeXt-=bgm=d+EDmcC9kWoikp8kbVb3LH0K3K+AGGsykpHQ@mail.gmail.com
2023-01-19Fix ts_headline() to handle ORs and phrase queries more honestly.Tom Lane
This patch largely reverts what I did in commits c9b0c678d and 78e73e875. The maximum cover length limit that I added in 78e73e875 (to band-aid over c9b0c678d's performance issues) creates too many user-visible behavior discrepancies, as complained of for example in bug #17691. The real problem with hlCover() is not what I thought at the time, but more that it seems to have been designed with only AND tsquery semantics in mind. It doesn't work quite right for OR, and even less so for NOT or phrase queries. However, we can improve that situation by building a variant of TS_execute() that returns a list of match locations. We already get an ExecPhraseData struct representing match locations for the primitive case of a simple match, as well as one for a phrase match; we just need to add some logic to combine these for AND and OR operators. The result is a list of ExecPhraseDatas, which hlCover can regard as having simple AND semantics, so that its old algorithm works correctly. There's still a lot not to like about ts_headline's behavior, but I think the remaining issues have to do with the heuristics used in mark_hl_words and mark_hl_fragments (which, likewise, were not revisited when phrase search was added). Improving those is a task for another day. Patch by me; thanks to Alvaro Herrera for review. Discussion: https://postgr.es/m/840.1669405935@sss.pgh.pa.us