| Age | Commit message (Collapse) | Author |
|---|
|
In pool_write_flush() and pool_flus_it() write(2) is used. write(2)
returns ssize_t and in the code there are some places where the return
value was assigned to an int variable. As a result it was possible
that the return value was evaluated to negative value, which led to an
error.
Backpatch-through: v4.3
|
|
Also update typedefs.list and README.pgpool.
|
|
get_config() uses ctime() to generate printable form of
memqcache_stats_start_time. But it did not take into account that
ctime() adds newline at the end of result. As a result, not only the
output of memqcache_stats_start_time was with unnecessary newline but
next row printed empty items.
Author: Tatsuo Ishii <ishii@postgresql.org>
Discussion: https://www.postgresql.org/message-id/20251130.102712.131456481338876013.ishii%40postgresql.org
Backpatch-through: v4.3
|
|
Previously, the directory for storing pgpool_status and lock files was specified by the logdir parameter.
However, since the name logdir was confusing, the parameter has been renamed to work_dir for clarity.
|
|
Previously pgpool assumed that time_t to be a simple long. This causes
a lot of compile time warnings on certain systems, for example
OpenBSD because on the system time_t is __int64, which results in long
long. This commit upcasts such calculations to long long to avoid the
issue.
This way times can't get truncated and for the places where time_t is
actually used as a time and not a time diff this would allow pgpool to
keep working correctly post Y2038 on 64 bit clean time_t systems
(e.g. i386 OpenBSD).
Moreover, json_get_long_value_for_key is changed to
json_get_llong_value_for_key and changed the parameter to long long.
This makes it more in line _json_value's integer, which is defined as
int64. This should also give 32 bit platforms proper retrieval of the
max value of an integer and may or may not solve some weird integer
overflow issues.
Backpatch for 4.6 was rebased by Gyorgy Sarvari.
Author: Martijn van Duren <pgpool@list.imperialat.at>
Reviewed-by: Tatsuo Ishii <ishii@postgresql.org>
Reviewed-by: Gyorgy Sarvari <skandigraun@gmail.com>
Backpatch-through: v4.6
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004584.html
Discussion: https://www.postgresql.org/message-id/20251003.211957.2067537305399895611.ishii%40postgresql.org
Discussion: https://github.com/pgpool/pgpool2/pull/128
|
|
This commit includes multiple fixes to compile Pgpool-II in Fedora 42,
which uses gcc 15 (C23).
- Modify pool_type.h. "bool" is now standard in C99 and
above. PostgreSQL decided to require C99 to compile it. So we follow
the way, which is just including <stdbool.h>. Also we define
TRUE/FALSE to (bool) 1 and (bool) 0 respectively. They are used only
in Windows build in PostgreSQL but we still use them in some
places. Eventually we should replace it with true/false since we do
not support Windows.
- It is now required that function pointer arguments matches the
function prototype to be called. For example:
static pid_t worker_fork_a_child(ProcessType type, void (*func) (), void *params);
should be:
static pid_t worker_fork_a_child(ProcessType type, void (*func) (void *), void *params);
Also the prototype of pool_create_relcache() is changed,
- raw_expression_tree_walker() calls walker() in many places. Now
callers of walker() should cast the first argument of it using (Node
*). We replace the call:
return walker(((RangeVar *) node)->alias, context);
with:
return WALK(((RangeVar *) node)->alias, context);
where WALK is defined as:
#define WALK(n,c) walker((Node *) (n), c)
- Note: we have lots of warnings regarding OpenSSL while compiling
Pgpool-II in Fedora42. The version used in Fedora42:
$ openssl -version
OpenSSL 3.2.4 11 Feb 2025 (Library: OpenSSL 3.2.4 11 Feb 2025)
The fix is not included in this commit. We need
to look into it in the future.
Discussion: https://github.com/pgpool/pgpool2/issues/124
Backpatch-through: v4.6
|
|
|
|
Prior to version 4.6, the online recovery database was hardcoded to "template1".
This commit introduces a new configuration parameter, "recovery_database",
which allows users to specify the database used for online recovery.
The default value is "postgres".
|
|
This commit adds new pgpool_adm extension function:
pcp_proc_info. Also add new fields: client_host, client_port and SQL
statement to pcp_proc_info and "show pool_pools". With these additions
now it is possible to track the relationship among clients of pgpool,
pgpool itself and PostgreSQL.
Moreover the commit allows to know what commands (statements) are last
executed by using pcp_proc_info. Previously it was not possible unless
looking into the pgpool log.
lipcp.so version is bumped from 2.0 to 2.1.
|
|
Previously we used random() for choosing load balancing node. However
PostgreSQL has better random number generator: pg_prng.c. This commit
imports the file and use pg_prng_double() to generate random number in
range [0.0, 1.0). The seed is generated using pg_strong_random().
Other notes regarding the port:
- Some of functions in the file were not ported because they require
additional library: pg_bitutils.c. In the future we may revisit and
import pg_bitutils.c.
- All conditional compiling regarding "sun" or "_sun" are removed. It
seems the platform is not used for running pgpool anymore.
- Since srandom() is not necessary any more, related code are removed
from pgpool_main.c, child.c and pcp_worker.c.
Author: Martijn van Duren <pgpool@list.imperialat.at>, Tatsuo Ishii <ishii@postgresql.org>
Discussion: [pgpool-hackers: 4588] Shuffle random functions and use better random numbers
https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004589.html
|
|
This reverts commit 66fcd561d74c8f00326bad94300053bd7ea13566.
It was accidentally committed.
|
|
Previously we used random() for choosing load balancing node. However
PostgreSQL has better random number generator: pg_prng.c. This commit
imports the file and use pg_prng_double() to generate random number in
range [0.0, 1.0).
Other notes regarding the port:
- pg_prng needs to be initialized using pg_prng_strong_seed() per
process. Currently the only caller is child.c (per session
process). If other process needs to use pg_prng, it needs the same
initialization as child.c.
- Some of functions in the file were not ported because they require
additional library: pg_bitutils.c. In the future we may revisit and
import pg_bitutils.c.
- likely/unlikely are ignored. In the future we may revisit import
them.
- All conditional compiling regarding "sun" or "_sun" are removed. It
seems the platform is not used for running pgpool anymore.
- Since srandom() is not necessary any more, related code are removed
from pgpool_main.c, child.c and pcp_worker.c.
Discussion: [pgpool-hackers: 4588] Shuffle random functions and use better random numbers
https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004589.html
|
|
In the commit I forgot to test without SSL case, which requires to
include <errno.h>.
Author: Bo Peng <pengbo@sraoss.co.jp>
|
|
Our PostmasterRandmon() was imported from PostgreSQL long time ago (in
2016). In the same year PostgreSQL replaced PostmasterRandmon() with
pg_strong_random()(src/port/pg_strong_random.c). This commit follows
it.
pg_strong_random() looks better than PostmasterRandmon(), since it's
more secure and portable. Moreover no initialization is necessary.
Reviewed-by: Martijn van Duren <pgpool@list.imperialat.at>
Discussion: [pgpool-hackers: 4588] Shuffle random functions and use better random numbers
https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004589.html
|
|
- va_list is defined stdarg.h[0]
- pthread_t is defined in pthread.h / sys/types.h[1]
On OpenBSD sys/types.h doesn't suffice, so include pthread.h.
- LibreSSL has removed HMAC_CTX_init(), and has support for HMAC_CTX_new
since 2018. I've talked to Theo Buehler of LibreSSL and he said that he'd
prefer to simply remove the LIBRESSL_VERSION_NUMBER, but if desired by
upstream the LIBRESSL_VERSION_NUMBER should be 0x2070100fL.
- WIFEXITED is defined in sys/wait.h[2]
Author: Martijn van Duren (pgpool@list.imperialat.at)
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004583.html
Backpatch-through: v4.2
|
|
Pgpool would crash when the watchdog was enabled if wd_authkey contained special characters (e.g., a backslash).
The patch was originally created by Martijn van Duren and revised by Bo Peng.
|
|
Previously pool_signal did not set SA_RESTART flag. Thus any system
calls interrupted by a signal does not restart. Some of our code are
prepared so that they restart if a system call is interrupted by a
signal. But not sure all places are prepared too. So add the
flag. Note, PostgreSQL always uses the flag.
|
|
When enabled, log protocol messages from each backend. Possible
options are "none", "terse" and "verbose". "none" disables the feature
and is the default. "verbose" prints the log each time pgpool receives
a message from backend. "terse" is similar to verbose except it does
not print logs for repeated message to save log lines. If different
kind of message received, pgpool prints a log message including the
number of the message. One downside of "terse" is, the repeated
message will not be printed if the pgpool child process is killed
before different kind of message arrives.
For testing, 039.log_backend_messages is added.
Discussion: [pgpool-hackers: 4535] New feature: log_backend_messages
https://www.pgpool.net/pipermail/pgpool-hackers/2024-November/004536.html
|
|
In the client side implementation of SSL negotiation
(pool_ssl_negotiate_clientserver()), it was possible for a
man-in-the-middle attacker to send a long error message to confuse
Pgpool-II or client while in the SSL negotiation phase. This commit
rejects the negotiation immediately (issue a FATAL error) and exits
the session to prevent such an attack.
This resembles PostgreSQL's CVE-2024-10977.
Backpatch-through: v4.1
|
|
Major changes of PostgreSQL 17 parser include:
- Allow MERGE to use NOT MATCHED BY SOURCE and RETURNING clause:
MERGE INTO ... WHEN NOT MATCHED BY SOURCE ...
MERGE INTO ... RETURNING ...
- Add new COPY option ON_ERROR ignore and LOG_VERBOSITY:
COPY ... WITH (ON_ERROR ignore);
COPY ... WITH (LOG_VERBOSITY verbose);
- Allow to use '*' to specify the COPY FROM options FORCE_NOT_NULL and FORCE_NULL for all columns.
COPY ... WITH (FORCE_NOT_NULL *);
COPY ... WITH (FORCE_NULL *);
- Add EXPLAIN option SERIALIZE and MEMORY
EXPLAIN (MEMORY) ...
EXPLAIN (ANALYZE, SERIALIZE ...) ...
- Allow ALTER TABLE to use SET STATISTICS DEFAULT to set a column to the default statistics target
ALTER TABLE ... ALTER COLUMN ... SET STATISTICS DEFAULT;
- Allow ALTER TABLE to change a column's generation expression
ALTER TABLE ... ALTER COLUMN ... SET EXPRESSION;
- Add DEFAULT setting for ALTER TABLE .. SET ACCESS METHOD
ALTER TABLE ... SET STATISTICS DEFAULT;
- Allow event triggers to use login event:
CREATE EVENT TRIGGER ... ON login ...
- Add event trigger support for REINDEX.
|
|
When the query cache feature is enabled, it was possible that a user
can read rows from tables that should not be visible for the user
through query cache.
- If query cache is created for a row security enabled table for user
A, and then other user B accesses the table via SET ROLE or SET
SESSION_AUTHORIZATION in the same session, it was possible for the
user B to retrieve rows which should not be visible from the user B.
- If query cache is created for a table for user A, and then other
user B accesses the table via SET ROLE or SET SESSION_AUTHORIZATION
in the same session, it was possible for the user B to retrieve rows
which should not be visible from the user B.
- If query cache is created for a table for a user, and then the
access right of the table is revoked from the user by REVOKE
command, still it was possible for the user to to retrieve the rows
through the query cache.
Besides the vulnerabilities, there were multiple bugs with the query
cache feature.
- If query cache is created for a row security enabled table for a
user, and then ALTER DATABASE BYPASSRLS or ALTER ROLE BYPASSRLS
disable the row security of the table, subsequent SELECT still
returns the same rows as before through the query cache.
- If query cache is created for a table for a user, and then ALTER
TABLE SET SCHEMA changes the search path to not allow to access the
table, subsequent SELECT still returns the rows as before through
the query cache.
To fix above, following changes are made:
- Do not allow to create query cache/use query cache for row security
enabled tables (even if the table is included in
cache_safe_memqcache_table_list).
- Do not allow to create query cache/use query cache if SET ROLE/SET
AUTHORIZATION is executed in the session (query cache invalidation
is performed when a table is modified as usual).
- Remove entire query cache if REVOKE/ALTER DATABASE/ALTER TABLE/ALTER
ROLE is executed. If the command is executed in an explicit
transaction, do not create query cache/use query cache until the
transaction gets committed (query cache invalidation is performed
when a table is modified as usual). If the transaction is aborted,
do not remove query cache.
Patch is created by Tatsuo Ishii.
Backpatch-through: v4.1
|
|
processes_reporting() accidentaly called both send_row_description()
and send_row_description_and_data_rows().
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2024-June/004472.html
[pgpool-hackers: 4471] [PATCH] printing empty row first in query "show pool_process"
Author: Kwangwon Seo
Back patch to V4.2 where the problem started.
|
|
Author: Umar Hayat
|
|
(4bfca73c6788cee498d74e938fa38c38b9abb6a2).
|
|
The following ERROR messages are downgraded to DEBUG messages.
(1) ERROR:unable to flush data to frontend
(2) ERROR: unable to read data from frontend
DETAIL: EOF encountered with frontend
(3) ERROR: unable to read data
DETAIL: child connection forced to terminate due to client_idle_limit:30 is reached
(1) and (2)
These messages are cuased when the client did not send a terminate message
before disconnecting to pgpool.
For example, when the client process was forcefully terminated, the error occurs.
Although they are harmless, it can sometimes confuse users.
(3)
If we set "client_idle_limit" to a non-zero value, the connection
will be disconnected if it remains idle since the last query.
The disconnection is caused by Pgpool-II settings,
but Pgpool-II handles the log message as an "ERROR".
Because the ERROR messages above are normal messages, I decide to downgrade them.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-June/004351.html
|
|
The definition for NAMEDATALEN is now in
src/include/pg_config_manual.h only. Also replace POOL_NAMEDATALEN
with NAMEDATALEN in src/utils/pool_select_walker.c.
I tried to eliminate copying pool_config_manual.h from PostgreSQL but
it seems it is a little bit difficult to do that at this moment.
There are some definitions like pg_attribute_format_arg were added to
it.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-August/004366.html
|
|
|
|
Major changes of PostgreSQL 16 parser include:
- Add new option DEFAULT to COPY FROM
COPY ... FROM stdin WITH (default 'xx');
- Allow the STORAGE type to be specified by CREATE TABLE
CREATE TABLE t1 (
c1 VARCHAR(10) STORAGE PLAIN,
c2 TEXT STORAGE EXTENDED
);
- Add EXPLAIN option GENERIC_PLAN to display the generic plan for a parameterized query
EXPLAIN (GENERIC_PLAN) SELECT ...;
- Allow subqueries in the FROM clause to omit aliases
SELECT COUNT(*) FROM (SELECT ... FROM ...);
- Add SQL/JSON constructors
- Add VACUUM options
SKIP_DATABASE_STATS, ONLY_DATABASE_STATS to skip or update all frozen statistics
PROCESS_MAIN to only process TOAST tables
VACUUM (SKIP_DATABASE_STATS);
VACUUM (PROCESS_MAIN FALSE) t1 ;
|
|
The new field is "1" if pgpool process is connected by a client and
the session uses the backend id as a load balance node. Users can
execute the commands to find out if there's any session that uses the
backend as the load balance node. If so, shutting down the backend may
cause session disconnection.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-July/004353.html
Note: I accidentally pushed only doc part of the commit in 4658f84870e8edfd39920f273bab1a12d71d8986.
This is a follow-up commit for actual codes.
|
|
user connection.
This commit adds a new parameter "user_redirect_preference_list" to
allow to send SELECT queries to a specific backend node for a
specific user connection.
|
|
There were a few places where schema qualification were not used while
issuing system catalog inquiry.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-June/004346.html
|
|
|
|
Previously PREPARE/EXECUTE/DEALLOCATE are not load balanced. In
streaming replication/logical replication mode, they were sent to
always primary node. In native replication/snapshot isolation
mode,they were always sent to all nodes.
Now they can be load balanced if the SQL statement prepared by PREPARE
command is a read only SELECT.
For this purpose following changes were made:
- is_select_query() looks into "query" member if node is
PrepareStmt. Also second argument "sql" (query string) is not now
mandatory. If sql is NULL, warning is emitted and this function
returns false. If allow_sql_comments is off and node is PrepareStmt
and query is SelectStmt, is_select_query() does not return false
anymore.
- pool_has_function_call() looks into "query" member if node is
PrepareStmt.
- Add PREPARE/EXECUTE/DEALLOCATE test cases to 001.load_balance test.
- Add send_prepare() function which is similar to parse_before_bind in
extended query protocol case to keep up
disable_load_balance_on_write rule. send_prepare() is called by
SimpleQuery() when EXECUTE message is sent by frontend in SL mode so
that it sends PREPARE message to primary node if it has not sent to
primary because of load balance. Note that send_prepare() does
nothing if the clustering mode is other than SL mode. In native
replication mode or snapshot isolation mode, all backend has the
same data, and there's no point to keep up
disable_load_balance_on_write rule.
- Remove descriptions of restrictions regarding load balance for
PREPARE/EXECUTE/DEALLOCATE in "Restrictions" section in the docs.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-May/004334.html
|
|
They are generated files and we should not keep in the git repository.
|
|
|
|
This allows to disable logging about normal PCP Process fork and exit
status. When pcp command is executed, pgpool logs its fork/exit event
even if there's no error. This could fill up the pgpool log.
Abnormal fork/exit event will be logged even if the parameter is
disabled.
Author: Maximilien Cuony
Review and Japanese document by: Tatsuo Ishii
|
|
This is a follow up to commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=caeb3a8681d7aa0f5b24539de12d29dd5a8d9997
Also add configure and src/utils/psqlscan.c to .gitignore.
|
|
Psqlscan is a module in the PostgreSQL source tree. It is essentially
subset of PostgreSQL SQL scanner but it is specialized for detecting
the end of each SQL statement. Therefore we can count the number of
SQL statements in a query string by using it.
Because psqlscan is not designed as an external library, it is hard to
call it from outside of PostgreSQL source tree. So I decided to import
psqlscan source code. This module consists of multiple files. Program
sources are deployed in src/utils directory and header files are
deployed in src/include/utils directory.
psqlscan module was originally designed for frontend programs and uses
malloc directly. So I changed them so that it calls palloc and
friends. Additionally pgstrcasecmp.c and pqexpbuffer.c are also
imported. They are used by psqlscan.
The example usage of psqlscan can be found in
multi_statement_query():src/protocol/pool_proto_modules.c.
Discussion:
https://www.pgpool.net/pipermail/pgpool-hackers/2023-February/004291.html
|
|
Some of objects such as function and cast did not use "pg_catalog."
schema qualification. This does not lead to immediate security
concern but using the schema qualification is always good practice.
Not that for this I had to increase some buffer length:
- POOL_RELCACHE.query was changed from 1024 to 1500.
- query buffer size in pool_search_relcache was changed from 1024 to 1500.
Back patch to all supported branches: 4.4 to 4.0
|
|
This was missed in commit: 85ce852329c0e9775076234cd4a82c20fa173659
|
|
test=# show pool_version;
pool_version
--------------
(0 rows)
pool_version
-----------------------
4.3.5 (tamahomeboshi)
(1 row)
|
|
Previously pool_search_relcache() obtained only shared lock, which is
not safe enough to call pool_catalog_commit_cache() because it
registers new cache entry. Unfortunately our locking system does not
allow to escalate a shared lock to an exclusive lock. So we release
the shared lock then acquire the exclusive lock before calling
pool_catalog_commit_cache(). There's a window between them and we may
end up with an effort to register duplicate cache entry. But
underlying infrastructure of the query system will reject it and
should be safe.
Back-patch through 4.4 where the shared locking of the query cache
system was introduced.
|
|
|
|
process_management_mode and process_management_strategy.
|
|
This feature allows selecting between static and dynamic process management modes.
Static process management is the same as the existing behavior of Pgpool-II, where it
spawns all child processes at startup. The new Dynamic mode keeps track of idle
processes and forks or kills processes to keep this number within
the specified boundaries.
Four new settings, process_management_mode, process_management_strategy,
min_spare_children, and max_spare_children are added to configure the process
management behavior, while process_management_strategy allows selecting
between three possible scaling-down strategies.
The first version of the patch was shared by "zhoujianshen at highgo.com" and reworked by me
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-September/004189.html
Reviewed by: Bo Peng and Tatsuo Ishii
|
|
This patch be able to specify a command that is used by trusted_servers
for checking up stream connection. Previously, ping command was only used,
and it was hard coded.
Default is 'ping -q -c3 %h' which means same as before.
|
|
The segfaults were in get_pools() and get_processes(). They first
extracted pid of particular process info slot on shared memory then
searched the slot again by using pid as the key. Because these steps
were not protected by any locking, it was possible that the search
using the pid failed and returned NULL if the process id is
overwritten by pgpool parent which is responsible for forking new
child process after the process exiting. As a result any subsequent
reference to the NULL pointer generated segfaults.
Solution is, first get the pointer to the process info slot then
extract the process id member from the pointer. This way, still
concurrent updating to the shared memory info by the parent process is
possible (which may lead to strange results in the output) but at
least we can avoid segfaults.
|
|
Declaration of variables must be before any execute statements.
|
|
Declaration of variables must be before any execute statements.
|
|
Major changes of PostgreSQL 15 parser include:
- Add new SQL MERGE command
MERGE INTO ... USING ...
- Add new option HEADER MATCH to COPY FROM
COPY ... FROM stdin WITH (HEADER MATCH);
- Allow foreign key ON DELETE SET actions
CREATE TABLE t1 (
...
FOREIGN KEY (c1, c2) REFERENCES t2 ON DELETE SET NULL (c2)
);
- Allow SET ACCESS METHOD in ALTER TABLE
ALTER TABLE ... SET ACCESS METHOD ...;
|
