| Age | Commit message (Collapse) | Author |
|---|
|
Noticed by Andrew Wong
|
|
This includes latest GXID and sequence information. This configuration
allows an external operator to modify sequence information with a simple
text editor before restarting a GTM.
Some functionalities have been added to include sequence name encoding and
decoding when interacting with control file.
Patch by Andrei Martsinchyk
|
|
gtm_report_failure was a function used to allow report of an error
between XCM and GTM-Standby. XCM is not supported anymore, in consequence
this function becomes useless. It was deactivated by the way.
|
|
Some functionalities related to relation locator determination
depending on the COPY operation done and remote query generation
are removed from copy.c to remotecopy.c and added in an XC-only
specific file for two reasons:
1) Externalize the process of COPY to allow other modules to use it.
2) Reduce the footprint of XC code in postgres core code.
The information of CopyStateData is necessary to generate a complete
remote query but this data is private to copy.c, so an intermediate
structure is used here to complete the externalization of those
functionalities. If in a (distant?) future, CopyStateData is removed
outside copy.c, we should use the same data for remotecopy.c, however
this is still not the case in Postgres 9.2.
|
|
With this commit, reconnection of a GTM-Proxy to a GTM instance
is controlled only with the interval time between 2 reconnections.
While being in reconnection waiting process, it is possible to
specify new GTM connection information with gtm_ctl reconnect.
In consequence, 5 setting parameters of GTM-Proxy are removed here:
- err_wait_count
- err_wait_interval
- err_wait_idle
- gtm_connect_retry_count
- gtm_connect_retry_idle
Related information and documentation also is removed.
|
|
This includes locator, XC planner, pool, barrier and node manager.
This clean-up should have been done long ago...
|
|
|
|
This commit externalizes all the features used for the interpretation
of SQL clauses TO NODE, TO GROUP and DISTRIBUTE BY. This reduces the footprint
of XC code in heap.c. It is important to note here that this code cannot be
completely removed from heap.c as there is a necessary dependency regarding
system columns that cannot be used as distribution columns for HASH and MODULO
tables.
|
|
If a node accidentally connects to the Standby and reconnects to the
Master afterwards, it will get an inconsistent GXID value since GTM
Master is not aware of GXIDs generated by GTM Standby.
Patch by Andrei Martsinchyk
|
|
This avoid to have another thread waiting for it, as it is possible
it waits for it.
Patch by Andrei Martsinchyk
|
|
In the case of a GTM standby crash during the initial backup, the
master GTM remained locked. The origin of this problem was during
GTM backup process, where GTM locks all its threads one by one,
performs a backup and then releases everything.
This commit adds an additional control layer able to detect the
thread status during backup. The new status GTM_THREAD_BACKUP is
introduced for this purpose. Once backup is performed, the thread
status is changed to GTM_THREAD_RUNNING.
Hence, if the handler (GTM) is invoked upon sudden disconnection for
backend (here GTM-Standby) disconnect of its connection and it sees
that the current thread has status GTM_THREAD_BACKUP it goes ahead
and releases the locks.
This commit also includes some fixes for whitespaces.
Patch by Andrei Martsinchyk
|
|
Wrong variable was referenced when writing terminating null at the
end of a string. It caused seg faults, because the referenced variable
was not initialized yet.
Patch by Andrei Martsinchyk.
|
|
It looks like it was accidentally copy/pasted. This has as consequence
to write the coordinator name twice when serializing, so the result
could not be deserialized properly.
Patch by Andrei Martsinchyk
|
|
This had consequences on the data types of the values sent by
GTM which was not matching those GTM Standby was trying to read.
Patch by Andrei Martsinchyk
|
|
This refactoring allows the usage of clauses TO NODE, TO GROUP and
DISTRIBUTE BY in a more flexible way by decoupling empty extensions.
This is particularly useful to plug-in those clauses to other existing
SQL of Postgres.
A slight grammar modification is being introduced in this commit, which
is the addition of brackets when defining a node list. This has consequences
on the following queries:
- EXECUTE DIRECT
- CLEAN CONNECTION
- CREATE TABLE (subset of nodes)
- CREATE TABLE AS
- CREATE NODE GROUP
This simple modification allows the use of TO NODE/TO GROUP extensions
when they are listed in a list of commands separated by commas.
By the way, this modification really lowers the possibility of
shift-reduce conflicts in bison when implementing new features in XC
using existing clauses.
|
|
Those functions were created in regression tests xc_for_update and
xc_prepared_xacts. Now their creation is done in xc_create_function.
|
|
When a table was distributed on a portion of nodes. COPY process
was not correctly choosing the output node during a COPY TO. This
problem has been put into light by a case where table was replicated
without using the first datanode of the cluster. It happened that
there was the same problem for hash/modulo tables as well.
A test case in xc_copy has been added to cover this problem, and
a small portion managing COPY TO in execRemote.c has been cleaned.
|
|
|
|
Those tests are using a similar PL/PGSQL function to create tables with
a node list, being transparent on the cluster structure. This function
was created 3 times. Now this function is created in another test
called xc_create_function launched before all the others.
A table name conflict has been resolved in xc_misc to allow this test
to run in parallel with the other XC-related tests.
|
|
gtm_ctl can control the status of a GTM server like pg_ctl does
with PostgreSQL instances (in the case of XC Coordinator and
Datanode).
Noticed by Koichi Suzuki, formatting of status output to get something
consistent with pg_ctl and clean up by me.
|
|
Some headers were missing, some did not follow PostgreSQL format.
|
|
different walkers to checking if an expression or a query respectively, is
shippable. The logic to decide whether an expression or query is shippable or
not needs to be same except for few differences between shippability of
expression and query, arising from the information in both of them. If we want
to code a rule for shippability, we need to make it to both of these functions,
becoming a maintenance burden. The commit unifies the walkers into one and makes
appropriate changes in naming.
|
|
|
|
This is to realign XC regressions in consequence of merge with
commit c1d9579 on PostgreSQL master branch.
|
|
This is to realign XC regressions with merge c1d9579.
|
|
This is to realign XC regressions with merge c1d9579.
|
|
This is to realign XC regressions with merge c1d9579.
|
|
This merge with PostgreSQL master has been requested by Ashutosh.
Commit c1d9579 contains a modification of pull_var_clause allowing
to filter aggregate expressions inside Var clauses which is useful
for Postgres-XC planner optimizations regarding remote query path
determination.
At this point, Postgres-XC master is half-way between the intersection
of postgres/master&postgres/REL9_1_STABLE and the intersection of
postgres/master&postgres/REL9_2_STABLE. The rest of Postgres-XC merge
up to the merge-base of postgres/master&postgres/REL9_2_STABLE is
planned to be done in a couple of weeks. Until this time it is not planned
to release an XC version which is based on a stable branch of Postgres,
so it is OK to let XC master code at this point of postgres master.
Conflicts:
configure
configure.in
src/Makefile.shlib
src/backend/access/heap/heapam.c
src/backend/access/index/indexam.c
src/backend/access/transam/transam.c
src/backend/access/transam/xact.c
src/backend/access/transam/xlog.c
src/backend/commands/view.c
src/backend/executor/execAmi.c
src/backend/storage/lmgr/proc.c
src/include/catalog/catversion.h
src/include/catalog/pg_proc.h
src/include/pg_config.h.win32
src/pl/plpgsql/src/plpgsql--1.0.sql
src/test/regress/expected/prepared_xacts.out
src/test/regress/expected/subselect.out
src/test/regress/pg_regress.c
src/test/regress/sql/prepared_xacts.sql
|
|
This function was referencing to a boolean flag as a COPY TO/FROM switch
but is not used by code.
|
|
This fix adds an additional boolean flag in GTM-Proxy thread info
to make it similar to the one used in GTM. The problem which was
happening was that GTM-Proxy tried to use the memory structure of
the thread info of GTM.
Crash problems are solved but we should find in the future a smarter
way to manage GTM and GTM-Proxy memory allocation APIs as they are
now strongly dependant on each other and we may find similar issues
if some new developments are done on GTM and/or GTM-Proxy.
Report from Plexo Rama and patch from Koichi Suzuki.
|
|
This solves 2 problems:
- Avoid to have an empty output file when gtm_ctl has a log file defined
- Change log file name of GTM according to what is given by gtm_ctl
Patch by Michael Meskes
|
|
In RemoteQuery node we provide base_tlist as the target list representing the
data expected from the datanodes. This provides the scan tuple descriptor.
The actual targetlist expected from the RemoteQuery node is saved in
scan.plan.targetlist like other scan plans. This serves as the projection list.
Rest of the logic to link the scan targetlist and project targetlist, to apply
quals is built in ExecScan().
Since RemoteQuery nodes have projection capabilities, we don't need Result nodes
above RemoteQuery nodes just to perform projection. ExecRemoteQuery() now uses
a template similar to ExecSeqScan().
Since ExecRemoteQuery() takes care of the materialization, we don't need
Materialize node above RemoteQuery node.
|
|
Remote DML planning was not taking into account the node list
to be used on a relation when modifying data on it.
This has been enlighten by a test case with INSERT SELECT on
tables replicated defined on a sub-cluster. For example, in the
case of a cluster with 2 Datanodes dn1 and dn2:
CREATE TABLE aa1 (a int) DISTRIBUTE BY REPLICATION TO NODE dn1;
CREATE TABLE aa2 (a int) DISTRIBUTE BY HASH(a);
INSERT INTO aa2 VALUES (1),(2),(3),(4);
INSERT INTO aa1 SELECT * FROM aa2;
The last INSERT SELECT was inserting data on all the nodes instead
of doing it only on Datanode dn1.
|
|
This is an additional fix consequence of commit 9a7d611.
It makes the output plan of join.out consistent with join_1.out.
|
|
There was an error on GTM side on when comparing sequence key strings
related to key size when sequence drop is based on a sequence key type
GTM_SEQ_DB_NAME.
Here is an example that was failing:
create database test;
\c test
create sequence toto;
\c postgres
-- Sequence not dropped on GTM when database is dropped
drop database test;
create database test;
\c test
-- error cannot re-create sequence, GTM complains
-- because sequence is seen as already created on GTM
create sequence toto;
Report and test case by Dimitrije Radojevic
|
|
is duplicated in two places. Move this code into a function
GetRelationNodesByQuals() to avoid duplication.
|
|
Some brackets were used on mandatory arguments of some clauses.
|
|
Regression test join failed on rare occasions depending on the cluster
configuration.
|
|
The PostgreSQL mailing list for bugs was incorrectly used.
|
|
Documentation was referring to both, as well as initgtm.
The option to be used should be -Z to be consistent with pg_ctl
when choosing Coordinator or Datanode.
|
|
Performing node registration on GTM just after postmaster start is dangerous
because the master/slave status of the node is unknown. So node
registration is done once node has reached a stable state, meaning that
Xlog process has confirmed to postmaster that recovery has been completed.
Registration is also blocked for slave nodes. A standby node is created
from a hot backup of its master so master and its slaves will share the
same node name. Two nodes are not authorized to register to GTM with the
same node name to avoid identification confusion in the cluster, so it
looks normal to let the master and the master only identify itself.
A a direct consequence, unregistration too is done only for master node.
Synchronous and asynchronous node replication works correctly with this fix,
as well as node promotion (Ex: "pg_ctl promote -D $DATA").
|
|
Both features are not directly related to the core features,
so it is better to move them there.
The installation of pgxc_clean can be done with the same way as
a normal contrib module. pgxc_ddl is not installed by default,
but is kept for future developments.
Documentation is moved the same way.
|
|
This unifies the convention name for Coordinator and Datanode in Postgres-XC
with documentation.
|
|
When an INSERT query was planned using create_remoteinsert_plan,
the tag returned to client was "INSERT 1 1", which is incorrect
as the first digit should be the newly-insert Oid in case table
being inserted uses WITH OIDS. Oids are not consistent among nodes
in XC cluster, so set the first digit value to InvalidOid, which
is the same way of doing with FQS-ed INSERT queries or INSERT having
multiple values.
|
|
Successive ALTER SEQUENCE with START/RESTART requests was messing
the start value of sequences involved.
Handling of sequence key string is reinforced to take into account
the last "\0" when this data is sent from client. This was causing GTM
to deal with sequence names with incorrect last characters, resulting
in strange logs.
Per report from Hitoshi Hemmi
|
|
Two problems related to EXTENSION creation are fixed here.
1) Queries CREATE/ALTER EXTENSION were not sent to remote nodes.
2) A raw query string directly obtained from the extension script
was sent to remote nodes, failing extension creation if more than
2 queries were planned. This process is now controlled at node level
and only CREATE/ALTER extension queries are sent to remote nodes.
Extension installation works correctly but no checks have been made with
modules like dblink, they should however work like in vanilla Postgres...
|
|
Specifying a database or all databases is mandatory.
Per report from Hitoshi Hemmi and patch from Koichi Suzuki
|
|
The origin of the problem was that BEGIN messages were being sent to remote nodes
for all the queries launched in do_query of execRemote.c. However, read-only
queries were registered as read connections to remote nodes, meaning that a
COMMIT message was not sent in this case. This had as direct consequence to keep
transactions open on remote nodes.
For example, in such a test case:
create table tab(a int, b int);
insert into tab values(1,2),(3,4);
begin;
select * from tab;
prepare transaction 'pt2';
select * from tab;
The transaction is begun on local Coordinator and remote nodes, but it is not
committed on remote nodes. The origin of the problem is that it doesn't make
sense to send a BEGIN message to remote nodes when SELECT is launched inside
this transaction block, but it needs only to be sent when the first write operation
is done.
Then, when the second SELECT was launched, the remote Datanodes were complaining
that a transaction was already open.
This fix has been checked with regressions and DBT-1.
It improves the performance output of DBT-1 by 15% for 5 nodes thanks
to all the BEGIN messages not sent to remote nodes.
|
|
The same kind of query was used twice, for the same results.
Moreover, this query was not always giving consistent results
depending on cluster configuration used.
|
|
This commit fixes issue 3528952, which made a Coordinator sending
COMMIT messages unnecessarily to remote nodes for read connections.
This caused Datanodes to log a lot of warning messages of the type
"WARNING: there is no transaction in progress". This issue was
reproducible when cluster was under a huge load.
Things are simplified in pgxc_node_remote_prepare and this function
does not manage anymore with COMMIT messages, what it was not meant
to do by the way.
The modifications of this commit revert partially things implemented
in commit 077b37369a4b26b5aa0db7b23667a1803e2af4de but does not
change the process of SELECT FOR UPDATE transactions as connections
using such queries are registed as write connections to remote nodes.
All the test cases introduced by commits 077b373 and 9e66733 are kept.
They pass correctly with this patch, and measurements with DBT-1 are
passing correctly.
|
