Hi!

INTERSECT and EXCEPT is available for postgresql-v6.4!

The patch against v6.4 is included at the end of the current text
(in uuencoded form!)

I also included the text of my Master's Thesis. (a postscript
version). I hope that you find something of it useful and would be
happy if parts of it find their way into the PostgreSQL documentation
project (If so, tell me, then I send the sources of the document!)

The contents of the document are:
  -) The first chapter might be of less interest as it gives only an
     overview on SQL.

  -) The second chapter gives a description on much of PostgreSQL's
     features (like user defined types etc. and how to use these features)

  -) The third chapter starts with an overview of PostgreSQL's internal
     structure with focus on the stages a query has to pass (i.e. parser,
     planner/optimizer, executor). Then a detailed description of the
     implementation of the Having clause and the Intersect/Except logic is
     given.

Originally I worked on v6.3.2 but never found time enough to prepare
and post a patch. Now I applied the changes to v6.4 to get Intersect
and Except working with the new version. Chapter 3 of my documentation
deals with the changes against v6.3.2, so keep that in mind when
comparing the parts of the code printed there with the patched sources
of v6.4.

Here are some remarks on the patch. There are some things that have
still to be done but at the moment I don't have time to do them
myself. (I'm doing my military service at the moment) Sorry for that
:-(

-) I used a rewrite technique for the implementation of the Except/Intersect
   logic which rewrites the query to a semantically equivalent query before
   it is handed to the rewrite system (for views, rules etc.), planner,
   executor etc.

-) In v6.3.2 the types of the attributes of two select statements
   connected by the UNION keyword had to match 100%. In v6.4 the types
   only need to be familiar (i.e. int and float can be mixed). Since this
   feature did not exist when I worked on Intersect/Except it
   does not work correctly for Except/Intersect queries WHEN USED IN
   COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the
   resulting table. This is because until now the types of the attributes of
   the first select statement have been used for the resulting table.
   When Intersects and/or Excepts are used in combination with Unions it
   might happen, that the first select statement of the original query
   appears at another position in the query which will be executed. The reason
   for this is the technique used for the implementation of
   Except/Intersect which does a query rewrite!)
   NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT
         queries!!!

-) I had to add the field intersect_clause to some data structures
   but did not find time to implement printfuncs for the new field.
   This does NOT break the debug modes but when an Except/Intersect
   is used the query debug output will be the already rewritten query.

-) Massive changes to the grammar rules for SELECT and INSERT statements
   have been necessary (see comments in gram.y and documentation for
   deatails) in order to be able to use mixed queries like
   (SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...;

-) When using UNION/EXCEPT/INTERSECT you will get:
   NOTICE: equal: "Don't know if nodes of type xxx are equal".
   I did not have  time to add comparsion support for all the needed nodes,
   but the default behaviour of the function equal met my requirements.
   I did not dare to supress this message!

   That's the reason why the regression test for union will fail: These
   messages are also included in the union.out file!

-) Somebody of you changed the union_planner() function for v6.4
   (I copied the targetlist to new_tlist and that was removed and
   replaced by a cleanup of the original targetlist). These chnages
   violated some having queries executed against views so I changed
   it back again. I did not have time to examine the differences between the
   two versions but now it works :-)
   If you want to find out, try the file queries/view_having.sql on
   both versions and compare the results . Two queries won't produce a
   correct result with your version.

regards

    Stefan

6 files changed, 35 insertions, 9 deletions

diff --git a/src/include/executor/nodeGroup.h b/src/include/executor/nodeGroup.h
index 0c23aa02d33..e56995fecdf 100644
--- a/src/include/executor/nodeGroup.h
+++ b/src/include/executor/nodeGroup.h
@@ -6,7 +6,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: nodeGroup.h,v 1.7 1998/09/01 04:35:56 momjian Exp $
+ * $Id: nodeGroup.h,v 1.8 1999/01/18 00:10:02 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -22,5 +22,8 @@ extern bool ExecInitGroup(Group *node, EState *estate, Plan *parent);
 extern int	ExecCountSlotsGroup(Group *node);
 extern void ExecEndGroup(Group *node);
 extern void ExecReScanGroup(Group *node, ExprContext *exprCtxt, Plan *parent);
+/***S*I***/
+extern void ExecReScanGroup(Group *node, ExprContext *exprCtxt, Plan *parent);
+
 
 #endif	 /* NODEGROUP_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 41730bf3463..d71d8c6b1e5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -6,7 +6,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: parsenodes.h,v 1.65 1999/01/05 15:45:49 vadim Exp $
+ * $Id: parsenodes.h,v 1.66 1999/01/18 00:10:06 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -58,6 +58,9 @@ typedef struct Query
 								 * BY */
 	Node	   *havingQual;		/* qualification of each group */
 
+        /***S*I***/
+	List	   *intersectClause;	
+
 	List	   *unionClause;	/* unions are linked under the previous
 								 * query */
 	Node	   *limitOffset;	/* # of result tuples to skip */
@@ -605,7 +608,9 @@ typedef struct InsertStmt
 	List	   *groupClause;	/* group by clause */
 	Node	   *havingClause;	/* having conditional-expression */
 	List	   *unionClause;	/* union subselect parameters */
-	bool		unionall;		/* union without unique sort */
+	bool	   unionall;		/* union without unique sort */
+        /***S*I***/
+        List       *intersectClause;  
 } InsertStmt;
 
 /* ----------------------
@@ -646,6 +651,10 @@ typedef struct SelectStmt
 	Node	   *whereClause;	/* qualifications */
 	List	   *groupClause;	/* group by clause */
 	Node	   *havingClause;	/* having conditional-expression */
+        /***S*I***/
+        List       *intersectClause;
+        List       *exceptClause;
+  
 	List	   *unionClause;	/* union subselect parameters */
 	List	   *sortClause;		/* sort clause (a list of SortGroupBy's) */
 	char	   *portalname;		/* the portal (cursor) to create */
diff --git a/src/include/parser/analyze.h b/src/include/parser/analyze.h
index 0c4f838d135..4be7637d744 100644
--- a/src/include/parser/analyze.h
+++ b/src/include/parser/analyze.h
@@ -5,7 +5,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: analyze.h,v 1.4 1998/09/01 04:37:25 momjian Exp $
+ * $Id: analyze.h,v 1.5 1999/01/18 00:10:11 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -15,5 +15,8 @@
 #include <parser/parse_node.h>
 
 extern QueryTreeList *parse_analyze(List *pl, ParseState *parentParseState);
+/***S*I***/
+extern void create_select_list(Node *ptr, List **select_list, bool *unionall_present);
+extern Node *A_Expr_to_Expr(Node *ptr, bool *intersect_present);
 
 #endif	 /* ANALYZE_H */
diff --git a/src/include/rewrite/rewriteHandler.h b/src/include/rewrite/rewriteHandler.h
index ecec766aece..0adf71baabc 100644
--- a/src/include/rewrite/rewriteHandler.h
+++ b/src/include/rewrite/rewriteHandler.h
@@ -6,7 +6,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: rewriteHandler.h,v 1.6 1998/09/01 04:38:01 momjian Exp $
+ * $Id: rewriteHandler.h,v 1.7 1999/01/18 00:10:12 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -34,5 +34,9 @@ typedef struct _rewrite_meta_knowledge RewriteInfo;
 
 
 extern List *QueryRewrite(Query *parsetree);
-
+/***S*I***/
+extern Query *Except_Intersect_Rewrite(Query *parsetree);
+extern void create_list(Node *ptr, List **intersect_list);
+extern Node *intersect_tree_analyze(Node *tree, Node *first_select, Node *parsetree);
+extern void check_targetlists_are_compatible(List *prev_target, List *current_target);
 #endif	 /* REWRITEHANDLER_H */
diff --git a/src/include/rewrite/rewriteManip.h b/src/include/rewrite/rewriteManip.h
index e1b54829bbc..dc3724515b5 100644
--- a/src/include/rewrite/rewriteManip.h
+++ b/src/include/rewrite/rewriteManip.h
@@ -6,7 +6,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: rewriteManip.h,v 1.11 1998/10/21 16:21:29 momjian Exp $
+ * $Id: rewriteManip.h,v 1.12 1999/01/18 00:10:16 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -25,6 +25,8 @@ void		AddQual(Query *parsetree, Node *qual);
 void		AddHavingQual(Query *parsetree, Node *havingQual);
 
 void		AddNotQual(Query *parsetree, Node *qual);
+void            AddNotHavingQual(Query *parsetree, Node *havingQual);
+
 void		FixNew(RewriteInfo *info, Query *parsetree);
 
 void HandleRIRAttributeRule(Query *parsetree, List *rtable, List *targetlist,
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 7d74644c620..258ef01aa48 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -6,7 +6,7 @@
  *
  * Copyright (c) 1994, Regents of the University of California
  *
- * $Id: elog.h,v 1.8 1998/09/01 04:39:03 momjian Exp $
+ * $Id: elog.h,v 1.9 1999/01/18 00:10:17 momjian Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -29,7 +29,12 @@
 #define ABORTX	0x4000			/* abort process after logging */
 #endif
 
-#define ELOG_MAXLEN 4096
+/***S*I***/
+/* Increase this to be able to use postmaster -d 3 with complex
+ * view definitions (which are transformed to very, very large INSERT statements
+ * and if -d 3 is used the query string of these statements is printed using
+ * vsprintf which expects enough memory reserved! */
+#define ELOG_MAXLEN 12288
 
 
 /* uncomment the following if you want your elog's to be timestamped */