This was written by Lamar Owen:
+ +2001-06-22
+ + What open source development process is used by the PostgreSQL + team? + +Read HACKERS for six months (or a full release cycle, whichever + is longer). Really. HACKERS _is_the process. The process is not + well documented (AFAIK -- it may be somewhere that I am not aware + of) -- and it changes continually.
+ + What development environment (OS, system, compilers, etc) is + required to develop code? + +Developers Corner on the + website has links to this information. The distribution tarball + itself includes all the extra tools and documents that go beyond a + good Unix-like development environment. In general, a modern unix + with a modern gcc, GNU make or equivalent, autoconf (of a + particular version), and good working knowledge of those tools are + required.
+ + What areas need support? + +The TODO list.
+ +You've made the first step, by finding and subscribing to + HACKERS. Once you find an area to look at in the TODO, and have + read the documentation on the internals, etc, then you check out a + current CVS,write what you are going to write (keeping your CVS + checkout up to date in the process), and make up a patch (as a + context diff only) and send to the PATCHES list, prefereably.
+ +Discussion on the patch typically happens here. If the patch + adds a major feature, it would be a good idea to talk about it + first on the HACKERS list, in order to increase the chances of it + being accepted, as well as toavoid duplication of effort. Note that + experienced developers with a proven track record usually get the + big jobs -- for more than one reason. Also note that PostgreSQL is + highly portable -- nonportable code will likely be dismissed out of + hand.
+ +Once your contributions get accepted, things move from there. + Typically, you would be added as a developer on the list on the + website when one of the other developers recommends it. Membership + on the steering committee is by invitation only, by the other + steering committee members, from what I have gathered watching + froma distance.
+ +I make these statements from having watched the process for over + two years.
+ +To see a good example of how one goes about this, search the + archives for the name 'Tom Lane' and see what his first post + consisted of, and where he took things. In particular, note that + this hasn't been _that_ long ago -- and his bugfixing and general + deep knowledge with this codebase is legendary. Take a few days to + read after him. And pay special attention to both the sheer + quantity as well as the painstaking quality of his work. Both are + in high demand.
+ +The source code is over 250,000 lines. Many problems/features + are isolated to one specific area of the code. Others require + knowledge of much of the source. If you are confused about where to + start, ask the hackers list, and they will be glad to assess the + complexity and give pointers on where to start.
+ +Another thing to keep in mind is that many fixes and features + can be added with surprisingly little code. I often start by adding + code, then looking at other areas in the code where similar things + are done, and by the time I am finished, the patch is quite small + and compact.
+ +When adding code, keep in mind that it should use the existing + facilities in the source, for performance reasons and for + simplicity. Often a review of existing code doing similar things is + helpful.
+ +There are several ways to obtain the source tree. Occasional + developers can just get the most recent source tree snapshot from + ftp.postgresql.org. For regular developers, you can use CVS. CVS + allows you to download the source tree, then occasionally update + your copy of the source tree with any new changes. Using CVS, you + don't have to download the entire source each time, only the + changed files. Anonymous CVS does not allows developers to update + the remote source tree, though privileged developers can do this. + There is a CVS FAQ on our web site that describes how to use remote + CVS. You can also use CVSup, which has similarly functionality, and + is available from ftp.postgresql.org.
+ +To update the source tree, there are two ways. You can generate + a patch against your current source tree, perhaps using the + make_diff tools mentioned above, and send them to the patches list. + They will be reviewed, and applied in a timely manner. If the patch + is major, and we are in beta testing, the developers may wait for + the final release before applying your patches.
+ +For hard-core developers, Marc(scrappy@postgresql.org) will give + you a Unix shell account on postgresql.org, so you can use CVS to + update the main source tree, or you can ftp your files into your + account, patch, and cvs install the changes directly into the + source tree.
+ +First, use psql to make sure it is working as you expect. + Then run src/test/regress and get the output of + src/test/regress/checkresults with and without your changes, + to see that your patch does not change the regression test in + unexpected ways. This practice has saved me many times. The + regression tests test the code in ways I would never do, and has + caught many bugs in my patches. By finding the problems now, you + save yourself a lot of debugging later when things are broken, and + you can't figure out when it happened.
+ +Aside from the User documentation mentioned in the regular FAQ, @@ -179,7 +309,7 @@ There is also a script called unused_oids in pgsql/src/include/catalog that shows the unused oids.
-I have four good books, An Introduction to Database Systems, by C.J. Date, Addison, Wesley, A Guide to the SQL @@ -192,207 +322,245 @@ on-line written by Jim Gray at http://www.benchmarkresources.com.
-palloc() and pfree() are used in place of malloc() - and free() because we automatically free all memory allocated when - a transaction completes. This makes it easier to make sure we free - memory that gets allocated in one place, but only freed much later. - There are several contexts that memory can be allocated in, and - this controls when the allocated memory is automatically freed by - the backend.
+The files configure and configure.in are part of + the GNU autoconf package. Configure allows us to test for + various capabilities of the OS, and to set variables that can then + be tested in C programs and Makefiles. Autoconf is installed on the + PostgreSQL main server. To add options to configure, edit + configure.in, and then run autoconf to generate + configure.
-When configure is run by the user, it tests various OS + capabilities, stores those in config.status and + config.cache, and modifies a list of *.in files. For + example, if there exists a Makefile.in, configure generates + a Makefile that contains substitutions for all @var@ + parameters found by configure.
-We do this because this allows a consistent way to pass data - inside the backend in a flexible way. Every node has a - NodeTag which specifies what type of data is inside the - Node. Lists are groups of Nodes chained together as a - forward-linked list.
+When you need to edit files, make sure you don't waste time + modifying files generated by configure. Edit the *.in + file, and re-run configure to recreate the needed file. If + you run make distclean from the top-level source directory, + all files derived by configure are removed, so you see only the + file contained in the source distribution.
-Here are some of the List manipulation commands:
+-- You can print nodes easily inside gdb. First, to disable - output truncation when you use the gdb print command: --
-- lfirst(i)
+There are a variety of places that need to be modified to add a + new port. First, start in the src/template directory. Add an + appropriate entry for your OS. Also, use src/config.guess to + add your OS to src/template/.similar. You shouldn't match + the OS version exactly. The configure test will look for an + exact OS version number, and if not found, find a match without + version number. Edit src/configure.in to add your new OS. + (See configure item above.) You will need to run autoconf, or patch + src/configure too.
-- return the data at list element i.
+Then, check src/include/port and add your new OS file, + with appropriate values. Hopefully, there is already locking code + in src/include/storage/s_lock.h for your CPU. There is also + a src/makefiles directory for port-specific Makefile + handling. There is a backend/port directory if you need + special files for your OS.
-- lnext(i)
+1.9) Why don't we use threads in the + backend?
-- return the next list element after i.
+There are several reasons threads are not used:
-- foreach(i, list)
++
-- Historically, threads were unsupported and buggy.
-- - loop through list, assigning each list element to - i. It is important to note that i is a List *, - not the data in the List element. You need to use - lfirst(i) to get at the data. Here is a typical code - snipped that loops through a List containing Var *'s - and processes each one: -
+--List *i, *list; - - foreach(i, list) - { - Var *var = lfirst(i); +-- An error in one backend can corrupt other backends.
- /* process var here */ - } -- Speed improvements using threads are small compared to the + remaining backend startup time.
-- lcons(node, list)
+- The backend code would be more complex.
+- add node to the front of list, or create a - new list with node if list is NIL.
+1.10) How are RPM's packaged?
-- lappend(list, node)
+This was written by Lamar Owen:
-- add node to the end of list. This is more - expensive that lcons.
+2001-05-03
-- nconc(list1, list2)
+As to how the RPMs are built -- to answer that question sanely + requires me to know how much experience you have with the whole RPM + paradigm. 'How is the RPM built?' is a multifaceted question. The + obvious simple answer is that I maintain:
-- Concat list2 on to the end of list1.
++
-- A set of patches to make certain portions of the source tree + 'behave' in the different environment of the RPMset;
-- length(list)
+- The initscript;
-- return the length of the list.
+- Any other ancilliary scripts and files;
-- nth(i, list)
+- A README.rpm-dist document that tries to adequately document + both the differences between the RPM build and the WHY of the + differences, as well as useful RPM environment operations (like, + using syslog, upgrading, getting postmaster to start at OS boot, + etc);
-- return the i'th element in list.
+- The spec file that throws it all together. This is not a + trivial undertaking in a package of this size.
+- lconsi, ...
+I then download and build on as many different canonical + distributions as I can -- currently I am able to build on Red Hat + 6.2, 7.0, and 7.1 on my personal hardware. Occasionally I receive + opportunity from certain commercial enterprises such as Great + Bridge and PostgreSQL, Inc. to build on other distributions.
-- There are integer versions of these: lconsi, lappendi, - nthi. List's containing integers instead of Node - pointers are used to hold list of relation object id's and - other integer quantities.
-
-(gdb) set print elements 0
-
-
- Instead of printing values in gdb format, you can use the next two
- commands to print out List, Node, and structure contents in a
- verbose format that is easier to understand. List's are unrolled
- into nodes, and nodes are printed in detail. The first prints in a
- short format, and the second in a long format:
-
-(gdb) call print(any_pointer)
- (gdb) call pprint(any_pointer)
-
-
- The output appears in the postmaster log file, or on your screen if
- you are running a backend directly without a postmaster.
+ I test the build by installing the resulting packages and + running the regression tests. Once the build passes these tests, I + upload to the postgresql.org ftp server and make a release + announcement. I am also responsible for maintaining the RPM + download area on the ftp site.
-You'll notice I said 'canonical' distributions above. That + simply means that the machine is as stock 'out of the box' as + practical -- that is, everything (except select few programs) on + these boxen are installed by RPM; only official Red Hat released + RPMs are used (except in unusual circumstances involving software + that will not alter the build -- for example, installing a newer + non-RedHat version of the Dia diagramming package is OK -- + installing Python 2.1 on the box that has Python 1.5.2 installed is + not, as that alters the PostgreSQL build). The RPM as uploaded is + built to as close to out-of-the-box pristine as is possible. Only + the standard released 'official to that release' compiler is used + -- and only the standard official kernel is used as well.
-The source code is over 250,000 lines. Many problems/features - are isolated to one specific area of the code. Others require - knowledge of much of the source. If you are confused about where to - start, ask the hackers list, and they will be glad to assess the - complexity and give pointers on where to start.
+For a time I built on Mandrake for RedHat consumption -- no + more. Nonstandard RPM building systems are worse than useless. + Which is not to say that Mandrake is useless! By no means is + Mandrake useless -- unless you are building Red Hat RPMs -- and Red + Hat is useless if you're trying to build Mandrake or SuSE RPMs, for + that matter. But I would be foolish to use 'Lamar Owen's Super + Special RPM Blend Distro 0.1.2' to build for public consumption! + :-)
-Another thing to keep in mind is that many fixes and features - can be added with surprisingly little code. I often start by adding - code, then looking at other areas in the code where similar things - are done, and by the time I am finished, the patch is quite small - and compact.
+I _do_ attempt to make the _source_ RPM compatible with as many + distributions as possible -- however, since I have limited + resources (as a volunteer RPM maintainer) I am limited as to the + amount of testing said build will get on other distributions, + architectures, or systems.
-When adding code, keep in mind that it should use the existing - facilities in the source, for performance reasons and for - simplicity. Often a review of existing code doing similar things is - helpful.
+And, while I understand people's desire to immediately upgrade + to the newest version, realize that I do this as a side interest -- + I have a regular, full-time job as a broadcast + engineer/webmaster/sysadmin/Technical Director which occasionally + prevents me from making timely RPM releases. This happened during + the early part of the 7.1 beta cycle -- but I believe I was pretty + much on the ball for the Release Candidates and the final + release.
-I am working towards a more open RPM distribution -- I would + dearly love to more fully document the process and put everything + into CVS -- once I figure out how I want to represent things such + as the spec file in a CVS form. It makes no sense to maintain a + changelog, for instance, in the spec file in CVS when CVS does a + better job of changelogs -- I will need to write a tool to generate + a real spec file from a CVS spec-source file that would add version + numbers, changelog entries, etc to the result before building the + RPM. IOW, I need to rethink the process -- and then go through the + motions of putting my long RPM history into CVS one version at a + time so that version history information isn't lost.
-There are several ways to obtain the source tree. Occasional - developers can just get the most recent source tree snapshot from - ftp.postgresql.org. For regular developers, you can use CVS. CVS - allows you to download the source tree, then occasionally update - your copy of the source tree with any new changes. Using CVS, you - don't have to download the entire source each time, only the - changed files. Anonymous CVS does not allows developers to update - the remote source tree, though privileged developers can do this. - There is a CVS FAQ on our web site that describes how to use remote - CVS. You can also use CVSup, which has similarly functionality, and - is available from ftp.postgresql.org.
+As to why all these files aren't part of the source tree, well, + unless there was a large cry for it to happen, I don't believe it + should. PostgreSQL is very platform-agnostic -- and I like that. + Including the RPM stuff as part of the Official Tarball (TM) would, + IMHO, slant that agnostic stance in a negative way. But maybe I'm + too sensitive to that. I'm not opposed to doing that if that is the + consensus of the core group -- and that would be a sneaky way to + get the stuff into CVS :-). But if the core group isn't thrilled + with the idea (and my instinct says they're not likely to be), I am + opposed to the idea -- not to keep the stuff to myself, but to not + hinder the platform-neutral stance. IMHO, of course.
-To update the source tree, there are two ways. You can generate - a patch against your current source tree, perhaps using the - make_diff tools mentioned above, and send them to the patches list. - They will be reviewed, and applied in a timely manner. If the patch - is major, and we are in beta testing, the developers may wait for - the final release before applying your patches.
+Of course, there are many projects that DO include all the files + necessary to build RPMs from their Official Tarball (TM).
-For hard-core developers, Marc(scrappy@postgresql.org) will give - you a Unix shell account on postgresql.org, so you can use CVS to - update the main source tree, or you can ftp your files into your - account, patch, and cvs install the changes directly into the - source tree.
+This was written by Tom Lane:
-First, use psql to make sure it is working as you expect. - Then run src/test/regress and get the output of - src/test/regress/checkresults with and without your changes, - to see that your patch does not change the regression test in - unexpected ways. This practice has saved me many times. The - regression tests test the code in ways I would never do, and has - caught many bugs in my patches. By finding the problems now, you - save yourself a lot of debugging later when things are broken, and - you can't figure out when it happened.
+2001-05-07
-If you just do basic "cvs checkout", "cvs update", "cvs commit", + then you'll always be dealing with the HEAD version of the files in + CVS. That's what you want for development, but if you need to patch + past stable releases then you have to be able to access and update + the "branch" portions of our CVS repository. We normally fork off a + branch for a stable release just before starting the development + cycle for the next release.
-The structures passing around from the parser, rewrite, - optimizer, and executor require quite a bit of support. Most - structures have support routines in src/backend/nodes used - to create, copy, read, and output those structures. Make sure you - add support for your new field to these files. Find any other - places the structure may need code for your new field. mkid - is helpful with this (see above).
+The first thing you have to know is the branch name for the + branch you are interested in getting at. To do this, look at some + long-lived file, say the top-level HISTORY file, with "cvs status + -v" to see what the branch names are. (Thanks to Ian Lance Taylor + for pointing out that this is the easiest way to do it.) Typical + branch names are:
++ REL7_1_STABLE + REL7_0_PATCHES + REL6_5_PATCHES +-
OK, so how do you do work on a branch? By far the best way is to + create a separate checkout tree for the branch and do your work in + that. Not only is that the easiest way to deal with CVS, but you + really need to have the whole past tree available anyway to test + your work. (And you *better* test your work. Never forget that + dot-releases tend to go out with very little beta testing --- so + whenever you commit an update to a stable branch, you'd better be + doubly sure that it's correct.)
-Table, column, type, function, and view names are stored in - system tables in columns of type Name. Name is a - fixed-length, null-terminated type of NAMEDATALEN bytes. - (The default value for NAMEDATALEN is 32 bytes.)
+Normally, to checkout the head branch, you just cd to the place + you want to contain the toplevel "pgsql" directory and say
-typedef struct nameData
- {
- char data[NAMEDATALEN];
- } NameData;
- typedef NameData *Name;
-
+ cvs ... checkout pgsql
- Table, column, type, function, and view names that come into the
- backend via user queries are stored as variable-length,
- null-terminated character strings.
- Many functions are called with both types of names, ie. - heap_open(). Because the Name type is null-terminated, it is - safe to pass it to a function expecting a char *. Because there are - many cases where on-disk names(Name) are compared to user-supplied - names(char *), there are many cases where Name and char * are used - interchangeably.
+To get a past branch, you cd to whereever you want it and + say
++ cvs ... checkout -r BRANCHNAME pgsql ++ +
For example, just a couple days ago I did
++ mkdir ~postgres/REL7_1 + cd ~postgres/REL7_1 + cvs ... checkout -r REL7_1_STABLE pgsql ++ +
and now I have a maintenance copy of 7.1.*.
+ +When you've done a checkout in this way, the branch name is + "sticky": CVS automatically knows that this directory tree is for + the branch, and whenever you do "cvs update" or "cvs commit" in + this tree, you'll fetch or store the latest version in the branch, + not the head version. Easy as can be.
+ +So, if you have a patch that needs to apply to both the head and + a recent stable branch, you have to make the edits and do the + commit twice, once in your development tree and once in your stable + branch tree. This is kind of a pain, which is why we don't normally + fork the tree right away after a major release --- we wait for a + dot-release or two, so that we won't have to double-patch the first + wave of fixes.
+ +You first need to find the tuples(rows) you are interested in. @@ -460,330 +628,172 @@ ReleaseBuffer(), in the heap_fetch() case. Or it may be a palloc'ed tuple, that you must pfree() when finished. -
elog() is used to send messages to the front-end, and - optionally terminate the current query being processed. The first - parameter is an elog level of NOTICE, DEBUG, - ERROR, or FATAL. NOTICE prints on the user's - terminal and the postmaster logs. DEBUG prints only in the - postmaster logs. ERROR prints in both places, and terminates - the current query, never returning from the call. FATAL - terminates the backend process. The remaining parameters of - elog are a printf-style set of parameters to - print.
+Table, column, type, function, and view names are stored in + system tables in columns of type Name. Name is a + fixed-length, null-terminated type of NAMEDATALEN bytes. + (The default value for NAMEDATALEN is 32 bytes.)
+
+typedef struct nameData
+ {
+ char data[NAMEDATALEN];
+ } NameData;
+ typedef NameData *Name;
+
+
+ Table, column, type, function, and view names that come into the
+ backend via user queries are stored as variable-length,
+ null-terminated character strings.
- Many functions are called with both types of names, ie. + heap_open(). Because the Name type is null-terminated, it is + safe to pass it to a function expecting a char *. Because there are + many cases where on-disk names(Name) are compared to user-supplied + names(char *), there are many cases where Name and char * are used + interchangeably.
-The files configure and configure.in are part of - the GNU autoconf package. Configure allows us to test for - various capabilities of the OS, and to set variables that can then - be tested in C programs and Makefiles. Autoconf is installed on the - PostgreSQL main server. To add options to configure, edit - configure.in, and then run autoconf to generate - configure.
- -When configure is run by the user, it tests various OS - capabilities, stores those in config.status and - config.cache, and modifies a list of *.in files. For - example, if there exists a Makefile.in, configure generates - a Makefile that contains substitutions for all @var@ - parameters found by configure.
- -When you need to edit files, make sure you don't waste time - modifying files generated by configure. Edit the *.in - file, and re-run configure to recreate the needed file. If - you run make distclean from the top-level source directory, - all files derived by configure are removed, so you see only the - file contained in the source distribution.
- -There are a variety of places that need to be modified to add a - new port. First, start in the src/template directory. Add an - appropriate entry for your OS. Also, use src/config.guess to - add your OS to src/template/.similar. You shouldn't match - the OS version exactly. The configure test will look for an - exact OS version number, and if not found, find a match without - version number. Edit src/configure.in to add your new OS. - (See configure item above.) You will need to run autoconf, or patch - src/configure too.
- -Then, check src/include/port and add your new OS file, - with appropriate values. Hopefully, there is already locking code - in src/include/storage/s_lock.h for your CPU. There is also - a src/makefiles directory for port-specific Makefile - handling. There is a backend/port directory if you need - special files for your OS.
- -Normally, transactions can not see the rows they modify. This
- allows UPDATE foo SET x = x + 1 to work correctly.
However, there are cases where a transactions needs to see rows - affected in previous parts of the transaction. This is accomplished - using a Command Counter. Incrementing the counter allows - transactions to be broken into pieces so each piece can see rows - modified by previous pieces. CommandCounterIncrement() - increments the Command Counter, creating a new part of the - transaction.
- -There are several reasons threads are not used:
- -This was written by Lamar Owen:
- -2001-05-03
- -As to how the RPMs are built -- to answer that question sanely - requires me to know how much experience you have with the whole RPM - paradigm. 'How is the RPM built?' is a multifaceted question. The - obvious simple answer is that I maintain:
- -We do this because this allows a consistent way to pass data + inside the backend in a flexible way. Every node has a + NodeTag which specifies what type of data is inside the + Node. Lists are groups of Nodes chained together as a + forward-linked list.
-Here are some of the List manipulation commands:
-++
- lfirst(i)
-- The spec file that throws it all together. This is not a - trivial undertaking in a package of this size.
-
I then download and build on as many different canonical - distributions as I can -- currently I am able to build on Red Hat - 6.2, 7.0, and 7.1 on my personal hardware. Occasionally I receive - opportunity from certain commercial enterprises such as Great - Bridge and PostgreSQL, Inc. to build on other distributions.
+I test the build by installing the resulting packages and - running the regression tests. Once the build passes these tests, I - upload to the postgresql.org ftp server and make a release - announcement. I am also responsible for maintaining the RPM - download area on the ftp site.
+You'll notice I said 'canonical' distributions above. That - simply means that the machine is as stock 'out of the box' as - practical -- that is, everything (except select few programs) on - these boxen are installed by RPM; only official Red Hat released - RPMs are used (except in unusual circumstances involving software - that will not alter the build -- for example, installing a newer - non-RedHat version of the Dia diagramming package is OK -- - installing Python 2.1 on the box that has Python 1.5.2 installed is - not, as that alters the PostgreSQL build). The RPM as uploaded is - built to as close to out-of-the-box pristine as is possible. Only - the standard released 'official to that release' compiler is used - -- and only the standard official kernel is used as well.
+For a time I built on Mandrake for RedHat consumption -- no - more. Nonstandard RPM building systems are worse than useless. - Which is not to say that Mandrake is useless! By no means is - Mandrake useless -- unless you are building Red Hat RPMs -- and Red - Hat is useless if you're trying to build Mandrake or SuSE RPMs, for - that matter. But I would be foolish to use 'Lamar Owen's Super - Special RPM Blend Distro 0.1.2' to build for public consumption! - :-)
+
+List *i, *list;
+
+ foreach(i, list)
+ {
+ Var *var = lfirst(i);
- I _do_ attempt to make the _source_ RPM compatible with as many
- distributions as possible -- however, since I have limited
- resources (as a volunteer RPM maintainer) I am limited as to the
- amount of testing said build will get on other distributions,
- architectures, or systems.
+ /* process var here */
+ }
+
+
+ And, while I understand people's desire to immediately upgrade - to the newest version, realize that I do this as a side interest -- - I have a regular, full-time job as a broadcast - engineer/webmaster/sysadmin/Technical Director which occasionally - prevents me from making timely RPM releases. This happened during - the early part of the 7.1 beta cycle -- but I believe I was pretty - much on the ball for the Release Candidates and the final - release.
+I am working towards a more open RPM distribution -- I would - dearly love to more fully document the process and put everything - into CVS -- once I figure out how I want to represent things such - as the spec file in a CVS form. It makes no sense to maintain a - changelog, for instance, in the spec file in CVS when CVS does a - better job of changelogs -- I will need to write a tool to generate - a real spec file from a CVS spec-source file that would add version - numbers, changelog entries, etc to the result before building the - RPM. IOW, I need to rethink the process -- and then go through the - motions of putting my long RPM history into CVS one version at a - time so that version history information isn't lost.
+As to why all these files aren't part of the source tree, well, - unless there was a large cry for it to happen, I don't believe it - should. PostgreSQL is very platform-agnostic -- and I like that. - Including the RPM stuff as part of the Official Tarball (TM) would, - IMHO, slant that agnostic stance in a negative way. But maybe I'm - too sensitive to that. I'm not opposed to doing that if that is the - consensus of the core group -- and that would be a sneaky way to - get the stuff into CVS :-). But if the core group isn't thrilled - with the idea (and my instinct says they're not likely to be), I am - opposed to the idea -- not to keep the stuff to myself, but to not - hinder the platform-neutral stance. IMHO, of course.
+Of course, there are many projects that DO include all the files - necessary to build RPMs from their Official Tarball (TM).
+This was written by Tom Lane:
+2001-05-07
+If you just do basic "cvs checkout", "cvs update", "cvs commit", - then you'll always be dealing with the HEAD version of the files in - CVS. That's what you want for development, but if you need to patch - past stable releases then you have to be able to access and update - the "branch" portions of our CVS repository. We normally fork off a - branch for a stable release just before starting the development - cycle for the next release.
+The first thing you have to know is the branch name for the - branch you are interested in getting at. To do this, look at some - long-lived file, say the top-level HISTORY file, with "cvs status - -v" to see what the branch names are. (Thanks to Ian Lance Taylor - for pointing out that this is the easiest way to do it.) Typical - branch names are:
-- REL7_1_STABLE - REL7_0_PATCHES - REL6_5_PATCHES -+
OK, so how do you do work on a branch? By far the best way is to - create a separate checkout tree for the branch and do your work in - that. Not only is that the easiest way to deal with CVS, but you - really need to have the whole past tree available anyway to test - your work. (And you *better* test your work. Never forget that - dot-releases tend to go out with very little beta testing --- so - whenever you commit an update to a stable branch, you'd better be - doubly sure that it's correct.)
+Normally, to checkout the head branch, you just cd to the place - you want to contain the toplevel "pgsql" directory and say
-- cvs ... checkout pgsql -+
To get a past branch, you cd to whereever you want it and - say
+
- cvs ... checkout -r BRANCHNAME pgsql
+(gdb) set print elements 0
+
-
- For example, just a couple days ago I did
+ Instead of printing values in gdb format, you can use the next two + commands to print out List, Node, and structure contents in a + verbose format that is easier to understand. List's are unrolled + into nodes, and nodes are printed in detail. The first prints in a + short format, and the second in a long format:
- mkdir ~postgres/REL7_1
- cd ~postgres/REL7_1
- cvs ... checkout -r REL7_1_STABLE pgsql
+(gdb) call print(any_pointer)
+ (gdb) call pprint(any_pointer)
+
+ The output appears in the postmaster log file, or on your screen if
+ you are running a backend directly without a postmaster.
- and now I have a maintenance copy of 7.1.*.
- -When you've done a checkout in this way, the branch name is - "sticky": CVS automatically knows that this directory tree is for - the branch, and whenever you do "cvs update" or "cvs commit" in - this tree, you'll fetch or store the latest version in the branch, - not the head version. Easy as can be.
- -So, if you have a patch that needs to apply to both the head and - a recent stable branch, you have to make the edits and do the - commit twice, once in your development tree and once in your stable - branch tree. This is kind of a pain, which is why we don't normally - fork the tree right away after a major release --- we wait for a - dot-release or two, so that we won't have to double-patch the first - wave of fixes.
- -This was written by Lamar Owen:
- -2001-06-22
- - What open source development process is used by the PostgreSQL - team? - -Read HACKERS for six months (or a full release cycle, whichever - is longer). Really. HACKERS _is_the process. The process is not - well documented (AFAIK -- it may be somewhere that I am not aware - of) -- and it changes continually.
+The structures passing around from the parser, rewrite, + optimizer, and executor require quite a bit of support. Most + structures have support routines in src/backend/nodes used + to create, copy, read, and output those structures. Make sure you + add support for your new field to these files. Find any other + places the structure may need code for your new field. mkid + is helpful with this (see above).
-Developers Corner on the - website has links to this information. The distribution tarball - itself includes all the extra tools and documents that go beyond a - good Unix-like development environment. In general, a modern unix - with a modern gcc, GNU make or equivalent, autoconf (of a - particular version), and good working knowledge of those tools are - required.
+palloc() and pfree() are used in place of malloc() + and free() because we automatically free all memory allocated when + a transaction completes. This makes it easier to make sure we free + memory that gets allocated in one place, but only freed much later. + There are several contexts that memory can be allocated in, and + this controls when the allocated memory is automatically freed by + the backend.
-The TODO list.
+You've made the first step, by finding and subscribing to - HACKERS. Once you find an area to look at in the TODO, and have - read the documentation on the internals, etc, then you check out a - current CVS,write what you are going to write (keeping your CVS - checkout up to date in the process), and make up a patch (as a - context diff only) and send to the PATCHES list, prefereably.
+elog() is used to send messages to the front-end, and + optionally terminate the current query being processed. The first + parameter is an elog level of NOTICE, DEBUG, + ERROR, or FATAL. NOTICE prints on the user's + terminal and the postmaster logs. DEBUG prints only in the + postmaster logs. ERROR prints in both places, and terminates + the current query, never returning from the call. FATAL + terminates the backend process. The remaining parameters of + elog are a printf-style set of parameters to + print.
-Discussion on the patch typically happens here. If the patch - adds a major feature, it would be a good idea to talk about it - first on the HACKERS list, in order to increase the chances of it - being accepted, as well as toavoid duplication of effort. Note that - experienced developers with a proven track record usually get the - big jobs -- for more than one reason. Also note that PostgreSQL is - highly portable -- nonportable code will likely be dismissed out of - hand.
+Once your contributions get accepted, things move from there. - Typically, you would be added as a developer on the list on the - website when one of the other developers recommends it. Membership - on the steering committee is by invitation only, by the other - steering committee members, from what I have gathered watching - froma distance.
+Normally, transactions can not see the rows they modify. This
+ allows UPDATE foo SET x = x + 1 to work correctly.
I make these statements from having watched the process for over - two years.
+However, there are cases where a transactions needs to see rows + affected in previous parts of the transaction. This is accomplished + using a Command Counter. Incrementing the counter allows + transactions to be broken into pieces so each piece can see rows + modified by previous pieces. CommandCounterIncrement() + increments the Command Counter, creating a new part of the + transaction.
-To see a good example of how one goes about this, search the - archives for the name 'Tom Lane' and see what his first post - consisted of, and where he took things. In particular, note that - this hasn't been _that_ long ago -- and his bugfixing and general - deep knowledge with this codebase is legendary. Take a few days to - read after him. And pay special attention to both the sheer - quantity as well as the painstaking quality of his work. Both are - in high demand.