From: Dmitry Kovalev Date: Thu, 2 Aug 2001 17:28:59 +0000 (+0000) Subject: A big bunch of improvements, contributed by Sam Drake and Raj Damani. X-Git-Tag: LDBM_PRE_GIANT_RWLOCK~1169 X-Git-Url: https://git.sur5r.net/?a=commitdiff_plain;h=2f4d324f60f31b3d09c338f31a99107a07b57186;p=openldap A big bunch of improvements, contributed by Sam Drake and Raj Damani. Summary of changes is cited below. The patch still needs some cosmetic changes to be made, but is ready for testing. -----Original Message----- From: Sam Drake [mailto:drake@timesten.com] Sent: Saturday, April 07, 2001 10:40 PM To: 'mitya@seismic.ru' Cc: openldap-devel@OpenLDAP.org Subject: RE: Slapd frontend performance issues FYI, here is a short description of the changes I made. I'll package up the changes asap, but it may take a couple of days. The performance numbers quoted in this report were seen at my location with a 100,000 object database ... the slower numbers I mentioned earlier were reported by a customer with a 1,000,000 object database. I also can't explain the very poor performance I saw with OpenLDAP and LDBM with a 100,000 object database. ...Sam Drake / TimesTen Performance Software ---------- Work Performed OpenLDAP 2.0.9, including back-sql, was built successfully on Solaris 8 using gcc. The LDAP server itself, slapd, passed all tests bundled with OpenLDAP. OpenLDAP was built using Sleepycat LDBM release 3.1.17 as the "native" storage manager. The experimental back-sql facility in slapd was also built successfully. It was built using Oracle release 8.1.7 and the Oracle ODBC driver and ODBC Driver Manager from Merant. Rudimentary testing was performed with the data and examples provided with back-sql, and back-sql was found to be functional. Slapd and back-sql were then tested with TimesTen, using TimesTen 4.1.1. Back-sql was not immediately functional with TimesTen due to a number of SQL limitations in the TimesTen product. Functional issues encountered were: 1. Back-sql issued SELECT statements including the construct, "UPPER(?)". While TimesTen supports UPPER, it does not support the use of parameters as input to builtin functions. Back-sql was modified to convert the parameter to upper case prior to giving it to the underlying database ... a change that is appropriate for all databases. 2. Back-sql issued SELECT statements using the SQL CONCAT function. TimesTen does not support this function. Back-sql was modified to concatentate the necessary strings itself (in "C" code) prior to passing the parameters to SQL. This change is also appropriate for all databases, not just TimesTen. Once these two issues were resolved, back-sql could successfully process LDAP searches using the sample data and examples provided with back-sql. While performance was not measured at this point, numerous serious performance problems were observed with the back-sql code and the generated SQL. In particular: 1. In the process of implementing an LDAP search, back-sql will generate and execute a SQL query for all object classes stored in back-sql. During the source of generating each SQL query, it is common for back-sql to determine that a particular object class can not possibly have any members satisfying the search. For example, this can occur if the query searches an attribute of the LDAP object that does not exist in the SQL schema. In this case, back-sql would generate and issue the SQL query anyway, including a clause such as "WHERE 1=0" in the generated SELECT. The overhead of parsing, optimizing and executing the query is non-trivial, and the answer (the empty set) is known in advance. Solution: Back-sql was modified to stop executing a SQL query when it can be predetermined that the query will return no rows. 2. Searches in LDAP are fundamentally case-insensitive ("abc" is equal to "aBc"). However, in SQL this is not normally the case. Back-sql thus generated SQL SELECT statements including clauses of the form, "WHERE UPPER(attribute) = 'JOE'". Even if an index is defined on the attribute in the relational database, the index can not be used to satisfy the query, as the index is case sensitive. The relational database then is forced to scan all rows in the table in order to satisfy the query ... an expensive and non-scalable proposition. Solution: Back-sql was modified to allow the schema designer to add additional "upper cased" columns to the SQL schema. These columns, if present, contain an upper cased version of the "standard" field, and will be used preferentially for searching. Such columns can be provided for all searchable columns, some columns, or no columns. An application using database "triggers" or similar mechanisms can automatically maintain these upper cased columns when the standard column is changed. 3. In order to implement the hierarchical nature of LDAP object hierarchies, OpenLDAP uses suffix searches in SQL. For example, to find all objects in the subtree "o=TimesTen,c=us", a SQL SELECT statement of the form, "WHERE UPPER(dn) LIKE '%O=TIMESTEN,C=US'" would be employed. Aside from the UPPER issue discussed above, a second performance problem in this query is the use of suffix search. In TimesTen (and most relational databases), indexes can be used to optimize exact-match searches and prefix searches. However, suffix searches must be performed by scanning every row in the table ... an expensive and non-scalable proposition. Solution: Back-sql was modified to optionally add a new "dn_ru" column to the ldap_entries table. This additional column, if present, contains a byte-reversed and upper cased version of the DN. This allows back-sql to generate indexable prefix searches. This column is also easily maintained automatically through the use of triggers. Results A simple database schema was generated holding the LDAP objects and attributes specified by our customer. An application was written to generate test databases. Both TimesTen and Oracle 8.1.7 were populated with 100,000 entry databases. Load Times Using "slapadd" followed by "slapindex", loading and indexing 100,000 entries in an LDBM database ran for 19 minutes 10 seconds. Using a C++ application that used ODBC, loading 100,000 entries into a disk based RDBMS took 17 minutes 53 seconds. Using a C++ application that used ODBC, loading 100,000 entries into TimesTen took 1 minute 40 seconds. Search Times The command, "timex timesearch.sh '(cn=fname210100*)'" was used to test search times. This command issues the same LDAP search 4000 times over a single LDAP connection. Both the client and server (slapd) were run on the same machine. With TimesTen as the database, 4000 queries took 14.93 seconds, for a rate of 267.9 per second. With a disk based RDBMS as the database, 4000 queries took 77.79 seconds, for a rate of 51.42 per second. With LDBM as the database, 1 query takes 76 seconds, or 0.076 per second. Something is clearly broken. --- diff --git a/servers/slapd/back-sql/back-sql.h b/servers/slapd/back-sql/back-sql.h index a63ca8eb9e..8038494ada 100644 --- a/servers/slapd/back-sql/back-sql.h +++ b/servers/slapd/back-sql/back-sql.h @@ -36,6 +36,8 @@ typedef struct ldap_pvt_thread_mutex_t dbconn_mutex; ldap_pvt_thread_mutex_t schema_mutex; SQLHENV db_env; + int isTimesTen; // TimesTen + int has_ldapinfo_dn_ru; /* Does ldapinfo.dn_ru exist in schema? */ }backsql_info; -#endif \ No newline at end of file +#endif diff --git a/servers/slapd/back-sql/entry-id.c b/servers/slapd/back-sql/entry-id.c index e75ded38ff..b1b2eaafad 100644 --- a/servers/slapd/back-sql/entry-id.c +++ b/servers/slapd/back-sql/entry-id.c @@ -37,15 +37,56 @@ backsql_entryID* backsql_dn2id(backsql_info *bi,backsql_entryID *id,SQLHDBC dbh, //SQLINTEGER nrows=0; RETCODE rc; + // TimesTen + char upperdn[BACKSQL_MAX_DN_LEN+1]; + char* toBind; + int i, j, k; + Debug(LDAP_DEBUG_TRACE,"==>backsql_dn2id(): dn='%s'\n",dn,0,0); - backsql_Prepare(dbh,&sth,bi->id_query,0); - if ((rc=backsql_BindParamStr(sth,1,dn,BACKSQL_MAX_DN_LEN)) != SQL_SUCCESS) - { + // begin TimesTen + Debug(LDAP_DEBUG_TRACE, "id_query '%s'\n", bi->id_query, 0, 0); + rc = backsql_Prepare(dbh,&sth,bi->id_query,0); + if (rc != SQL_SUCCESS) { + Debug(LDAP_DEBUG_TRACE, "backsql_dn2id(): error preparing SQL:\n", 0, 0, 0); + Debug(LDAP_DEBUG_TRACE, "%s\n", bi->id_query, 0, 0); + backsql_PrintErrors(SQL_NULL_HENV, dbh, sth, rc); + SQLFreeStmt(sth, SQL_DROP); + return NULL; + } + + if (bi->has_ldapinfo_dn_ru) { + /* Prepare an upper cased, byte reversed version that can be + searched using indexes */ + + for ((i=0, j=strlen(dn)-1); *(dn+i); (i++, j--)) { + *(upperdn+i) = toupper(*(dn+j)); + } + *(upperdn+i) = '\0'; + Debug(LDAP_DEBUG_TRACE,"==>backsql_dn2id(): upperdn='%s'\n",upperdn,0,0); + toBind = upperdn; + } + else { + if (bi->isTimesTen) { + for (i = 0; *(dn+i); i++) { + *(upperdn+i) = toupper(*(dn+i)); /* Copy while upper casing */ + } + *(upperdn+i) = '\0'; + Debug(LDAP_DEBUG_TRACE,"==>backsql_dn2id(): upperdn='%s'\n",upperdn,0,0); + toBind = upperdn; + } + else + toBind = dn; + } + + if ((rc=backsql_BindParamStr(sth,1,toBind, + BACKSQL_MAX_DN_LEN)) != SQL_SUCCESS) + // end TimesTen + { Debug(LDAP_DEBUG_TRACE,"backsql_dn2id(): error binding dn parameter:\n",0,0,0); backsql_PrintErrors(SQL_NULL_HENV,dbh,sth,rc); SQLFreeStmt(sth,SQL_DROP); return NULL; - } + } if ((rc=SQLExecute(sth)) != SQL_SUCCESS) { diff --git a/servers/slapd/back-sql/init.c b/servers/slapd/back-sql/init.c index dca5ee5fb0..ec21f5bb86 100644 --- a/servers/slapd/back-sql/init.c +++ b/servers/slapd/back-sql/init.c @@ -178,6 +178,15 @@ int backsql_db_open (BackendDB *bd) Debug(LDAP_DEBUG_TRACE,"backsql_db_open(): setting '%s' by default\n",backsql_def_delentry_query,0,0); si->delentry_query=ch_strdup(backsql_def_delentry_query); } + tmp.c_connid=-1; + dbh=backsql_get_db_conn(bd,&tmp); + if (!dbh) + { + Debug(LDAP_DEBUG_TRACE,"backsql_db_open(): connection failed, exiting\n",0,0,0 +); + return 1; + } + si->id_query=NULL; idq_len=0; if (si->upper_func==NULL) @@ -185,17 +194,22 @@ int backsql_db_open (BackendDB *bd) si->id_query=backsql_strcat(si->id_query,&idq_len,backsql_id_query,"dn=?",NULL); } else - { - si->id_query=backsql_strcat(si->id_query,&idq_len,backsql_id_query,si->upper_func,"(dn)=",si->upper_func,"(?)",NULL); - } - tmp.c_connid=-1; - dbh=backsql_get_db_conn(bd,&tmp); - if (!dbh) { - Debug(LDAP_DEBUG_TRACE,"backsql_db_open(): connection failed, exiting\n",0,0,0); - return 1; + if (si->has_ldapinfo_dn_ru) { + si->id_query=backsql_strcat(si->id_query,&idq_len,backsql_id_query,"dn_ru= +?",NULL); + } + else { + if (si->isTimesTen) { + si->id_query=backsql_strcat(si->id_query,&idq_len,backsql_id_query,si->upper_func,"(dn)=?",NULL); + } + else { + si->id_query=backsql_strcat(si->id_query,&idq_len,backsql_id_query,si->upper_func,"(dn)=",si->upper_func,"(?)",NULL); + } + } } - backsql_free_db_conn(bd,&tmp); + +backsql_free_db_conn(bd,&tmp); if (!si->schema_loaded) { Debug(LDAP_DEBUG_TRACE,"backsql_db_open(): test failed, schema map not loaded - exiting\n",0,0,0); diff --git a/servers/slapd/back-sql/rdbms_depend/oracle/backsql_create.sql b/servers/slapd/back-sql/rdbms_depend/oracle/backsql_create.sql index e88284bba8..2e4e6ecb41 100644 --- a/servers/slapd/back-sql/rdbms_depend/oracle/backsql_create.sql +++ b/servers/slapd/back-sql/rdbms_depend/oracle/backsql_create.sql @@ -25,6 +25,7 @@ create table ldap_attr_mappings ( oc_map_id number not null references ldap_oc_mappings(id), name varchar2(255) not null, sel_expr varchar2(255) not null, + sel_expr_u varchar2(255), from_tbls varchar2(255) not null, join_where varchar2(255), add_proc varchar2(255), @@ -43,6 +44,7 @@ alter table ldap_attr_mappings add create table ldap_entries ( id number not null , dn varchar2(255) not null , + dn_ru varchar2(255), oc_map_id number not null references ldap_oc_mappings(id), parent number not null , keyval number not null @@ -84,3 +86,5 @@ create table ldap_entry_objclasses entry_id number not null references ldap_entries(id), oc_name varchar(64) ); + +quit diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/backsql_create.sql b/servers/slapd/back-sql/rdbms_depend/timesten/backsql_create.sql new file mode 100644 index 0000000000..055e9df6b2 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/backsql_create.sql @@ -0,0 +1,66 @@ + +create table ldap_oc_mappings + ( + id integer not null primary key, + name varchar(64) not null, + keytbl varchar(64) not null, + keycol varchar(64) not null, + create_proc varchar(255), + delete_proc varchar(255), + expect_return tinyint not null +); + +create table ldap_attr_mappings + ( + id integer not null primary key, + oc_map_id integer not null, + name varchar(255) not null, + sel_expr varchar(255) not null, + sel_expr_u varchar(255), + from_tbls varchar(255) not null, + join_where varchar(255), + add_proc varchar(255), + delete_proc varchar(255), + param_order tinyint not null, + expect_return tinyint not null, + foreign key (oc_map_id) references ldap_oc_mappings(id) +); + +create table ldap_entries + ( + id integer not null primary key, + dn varchar(255) not null, + dn_ru varchar(255), + oc_map_id integer not null, + parent int NOT NULL , + keyval int NOT NULL, + foreign key (oc_map_id) references ldap_oc_mappings(id) +); + +create index ldap_entriesx1 on ldap_entries(dn_ru); + +create unique index unq1_ldap_entries on ldap_entries + ( + oc_map_id, + keyval + ); + +create unique index unq2_ldap_entries on ldap_entries + ( + dn + ); + +create table ldap_referrals + ( + entry_id integer not null, + url varchar(4096) not null, + foreign key (entry_id) references ldap_entries(id) +); + +create table ldap_entry_objclasses + ( + entry_id integer not null, + oc_name varchar(64), + foreign key (entry_id) references ldap_entries(id) + ); + diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/backsql_drop.sql b/servers/slapd/back-sql/rdbms_depend/timesten/backsql_drop.sql new file mode 100644 index 0000000000..7aa0b83023 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/backsql_drop.sql @@ -0,0 +1,9 @@ +DROP TABLE ldap_referrals; + +DROP TABLE ldap_entry_objclasses; + +DROP TABLE ldap_attr_mappings; + +DROP TABLE ldap_entries; + +DROP TABLE ldap_oc_mappings; diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/create_schema.sh b/servers/slapd/back-sql/rdbms_depend/timesten/create_schema.sh new file mode 100755 index 0000000000..947db21fda --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/create_schema.sh @@ -0,0 +1,4 @@ +ttIsql -connStr "DSN=ldap_tt;Overwrite=1" -f backsql_create.sql +ttIsql -connStr "DSN=ldap_tt" -f testdb_create.sql +ttIsql -connStr "DSN=ldap_tt" -f testdb_data.sql +ttIsql -connStr "DSN=ldap_tt" -f testdb_metadata.sql diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/Makefile b/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/Makefile new file mode 100644 index 0000000000..f9accd8876 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/Makefile @@ -0,0 +1,45 @@ +# +# Build TimesTen ODBC Sample Programs for Solaris 2.5.1. +# (c) Copyright 1996-1998, TimesTen Performance Software. +# All rights reserved. +# +# $Revision: 1.1 $ +# + +CPLUSPLUS = CC +TTCLASSES = ../../../../../../../../../cs/classes +ODBC = /opt/TimesTen4.1/32 +CFLAGS = -g -I$(ODBC)/include -I. -I$(TTCLASSES) -DUNIX +LDFLAGS = -g +DIRLIBS = $(TTCLASSES)/ttclasses.a -L $(ODBC)/lib -R $(ODBC)/lib -ltten -lpthread -lm -lrt +XLALIB = -L $(ODBC)/lib -lxla + +DIRPROGS= dnreverse + +DNREVERSE= dnreverse.o + +# +# Top-level targets +# + +all: $(DIRPROGS) + +direct: $(DIRPROGS) + +clean: + rm -rf $(DIRPROGS) *.o + + +# +# Direct-linked programs +# + +dnreverse: $(DNREVERSE) + $(CPLUSPLUS) -o dnreverse $(LDFLAGS) $(DNREVERSE) $(DIRLIBS) $(XLALIB) + +# +# .o files +# + +dnreverse.o: dnreverse.cpp + $(CPLUSPLUS) $(CFLAGS) -c dnreverse.cpp diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/dnreverse.cpp b/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/dnreverse.cpp new file mode 100644 index 0000000000..d35838cb46 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/dnreverse/dnreverse.cpp @@ -0,0 +1,380 @@ +// (c) Copyright 1999-2001 TimesTen Performance Software. All rights reserved. + +#include + +#include +#include +#include +#include + +#include + +TTConnectionPool pool; +TTXlaConnection conn; +TTConnection conn2; +TTCmd assignDn_ru; +TTCmd getNullDNs; + +//---------------------------------------------------------------------- +// This class contains all the logic to be implemented whenever +// the SCOTT.MYDATA table is changed. This is the table that is +// created by "sample.cpp", one of the other TTClasses demos. +// That application should be executed before this one in order to +// create and populate the table. +//---------------------------------------------------------------------- + +class LDAPEntriesHandler: public TTXlaTableHandler { +private: + // Definition of the columns in the table + int Id; + int Dn; + int Oc_map_id; + int Parent; + int Keyval; + int Dn_ru; + +protected: + +public: + LDAPEntriesHandler(TTXlaConnection& conn, const char* ownerP, const char* nameP); + ~LDAPEntriesHandler(); + + virtual void HandleDelete(ttXlaUpdateDesc_t*); + virtual void HandleInsert(ttXlaUpdateDesc_t*); + virtual void HandleUpdate(ttXlaUpdateDesc_t*); + + static void ReverseAndUpper(char* dnP, int id, bool commit=true); + +}; + +LDAPEntriesHandler::LDAPEntriesHandler(TTXlaConnection& conn, + const char* ownerP, const char* nameP) : + TTXlaTableHandler(conn, ownerP, nameP) +{ + Id = Dn = Oc_map_id = Parent = Keyval = Dn_ru = -1; + + // We are looking for several particular named columns. We need to get + // the ordinal position of the columns by name for later use. + + Id = tbl.getColNumber("ID"); + if (Id < 0) { + cerr << "target table has no 'ID' column" << endl; + exit(1); + } + Dn = tbl.getColNumber("DN"); + if (Dn < 0) { + cerr << "target table has no 'DN' column" << endl; + exit(1); + } + Oc_map_id = tbl.getColNumber("OC_MAP_ID"); + if (Oc_map_id < 0) { + cerr << "target table has no 'OC_MAP_ID' column" << endl; + exit(1); + } + Parent = tbl.getColNumber("PARENT"); + if (Parent < 0) { + cerr << "target table has no 'PARENT' column" << endl; + exit(1); + } + Keyval = tbl.getColNumber("KEYVAL"); + if (Keyval < 0) { + cerr << "target table has no 'KEYVAL' column" << endl; + exit(1); + } + Dn_ru = tbl.getColNumber("DN_RU"); + if (Dn_ru < 0) { + cerr << "target table has no 'DN_RU' column" << endl; + exit(1); + } + +} + +LDAPEntriesHandler::~LDAPEntriesHandler() +{ + +} + +void LDAPEntriesHandler::ReverseAndUpper(char* dnP, int id, bool commit) +{ + TTStatus stat; + char dn_rn[512]; + int i; + int j; + + // Reverse and upper case the given DN + + for ((j=0, i = strlen(dnP)-1); i > -1; (j++, i--)) { + dn_rn[j] = toupper(*(dnP+i)); + } + dn_rn[j] = '\0'; + + + // Update the database + + try { + assignDn_ru.setParam(1, (char*) &dn_rn[0]); + assignDn_ru.setParam(2, id); + assignDn_ru.Execute(stat); + } + catch (TTStatus stat) { + cerr << "Error updating id " << id << " ('" << dnP << "' to '" + << dn_rn << "'): " << stat; + exit(1); + } + + // Commit the transaction + + if (commit) { + try { + conn2.Commit(stat); + } + catch (TTStatus stat) { + cerr << "Error committing update: " << stat; + exit(1); + } + } + +} + + + +void LDAPEntriesHandler::HandleInsert(ttXlaUpdateDesc_t* p) +{ + char* dnP; + int id; + + row.Get(Dn, &dnP); + cerr << "DN '" << dnP << "': Inserted "; + row.Get(Id, &id); + + ReverseAndUpper(dnP, id); + +} + +void LDAPEntriesHandler::HandleUpdate(ttXlaUpdateDesc_t* p) +{ + char* newDnP; + char* oldDnP; + char oDn[512]; + int id; + + // row is 'old'; row2 is 'new' + row.Get(Dn, &oldDnP); + strcpy(oDn, oldDnP); + row.Get(Id, &id); + row2.Get(Dn, &newDnP); + + cerr << "old DN '" << oDn << "' / new DN '" << newDnP << "' : Updated "; + + if (strcmp(oDn, newDnP) != 0) { + // The DN field changed, update it + cerr << "(new DN: '" << newDnP << "')"; + ReverseAndUpper(newDnP, id); + } + else { + // The DN field did NOT change, leave it alone + } + + cerr << endl; + +} + +void LDAPEntriesHandler::HandleDelete(ttXlaUpdateDesc_t* p) +{ + char* dnP; + + row.Get(Dn, &dnP); + cerr << "DN '" << dnP << "': Deleted "; +} + + + + +//---------------------------------------------------------------------- + +int pleaseStop = 0; + +extern "C" { + void + onintr(int sig) + { + pleaseStop = 1; + cerr << "Stopping...\n"; + } +}; + +//---------------------------------------------------------------------- + +int +main(int argc, char* argv[]) +{ + + char* ownerP; + + TTXlaTableList list(&conn); // List of tables to monitor + + // Handlers, one for each table we want to monitor + + LDAPEntriesHandler* sampP = NULL; + + // Misc stuff + + TTStatus stat; + + ttXlaUpdateDesc_t ** arry; + + int records; + + SQLUBIGINT oldsize; + int j; + + if (argc < 2) { + cerr << "syntax: " << argv[0] << " " << endl; + exit(3); + } + + ownerP = argv[1]; + + signal(SIGINT, onintr); /* signal for CTRL-C */ +#ifdef _WIN32 + signal(SIGBREAK, onintr); /* signal for CTRL-BREAK */ +#endif + + // Before we do anything related to XLA, first we connect + // to the database. This is the connection we will use + // to perform non-XLA operations on the tables. + + try { + cerr << "Connecting..." << endl; + + conn2.Connect("DSN=ldap_tt", stat); + } + catch (TTStatus stat) { + cerr << "Error connecting to TimesTen: " << stat; + exit(1); + } + + try { + assignDn_ru.Prepare(&conn2, + "update ldap_entries set dn_ru=? where id=?", + "", stat); + getNullDNs.Prepare(&conn2, + "select dn, id from ldap_entries " + "where dn_ru is null " + "for update", + "", stat); + conn2.Commit(stat); + } + catch (TTStatus stat) { + cerr << "Error preparing update: " << stat; + exit(1); + } + + // If there are any entries with a NULL reversed/upper cased DN, + // fix them now. + + try { + cerr << "Fixing NULL reversed DNs" << endl; + getNullDNs.Execute(stat); + for (int k = 0;; k++) { + getNullDNs.FetchNext(stat); + if (stat.rc == SQL_NO_DATA_FOUND) break; + char* dnP; + int id; + getNullDNs.getColumn(1, &dnP); + getNullDNs.getColumn(2, &id); + // cerr << "Id " << id << ", Dn '" << dnP << "'" << endl; + LDAPEntriesHandler::ReverseAndUpper(dnP, id, false); + if (k % 1000 == 0) + cerr << "."; + } + getNullDNs.Close(stat); + conn2.Commit(stat); + } + catch (TTStatus stat) { + cerr << "Error updating NULL rows: " << stat; + exit(1); + } + + + // Go ahead and start up the change monitoring application + + cerr << "Starting change monitoring..." << endl; + try { + conn.Connect("DSN=ldap_tt", stat); + } + catch (TTStatus stat) { + cerr << "Error connecting to TimesTen: " << stat; + exit(1); + } + + /* set and configure size of buffer */ + conn.setXlaBufferSize((SQLUBIGINT) 1000000, &oldsize, stat); + if (stat.rc) { + cerr << "Error setting buffer size " << stat << endl; + exit(1); + } + + // Make a handler to process changes to the MYDATA table and + // add the handler to the list of all handlers + + sampP = new LDAPEntriesHandler(conn, ownerP, "ldap_entries"); + if (!sampP) { + cerr << "Could not create LDAPEntriesHandler" << endl; + exit(3); + } + list.add(sampP); + + // Enable transaction logging for the table we're interested in + + sampP->EnableTracking(stat); + + // Get updates. Dispatch them to the appropriate handler. + // This loop will handle updates to all the tables. + + while (pleaseStop == 0) { + conn.fetchUpdates(&arry, 1000, &records, stat); + if (stat.rc) { + cerr << "Error fetching updates" << stat << endl; + exit(1); + } + + // Interpret the updates + + for(j=0;j < records;j++){ + ttXlaUpdateDesc_t *p; + + p = arry[j]; + + list.HandleChange(p, stat); + + } // end for each record fetched + + if (records) { + cerr << "Processed " << records << " records\n"; + } + + if (records == 0) { +#ifdef _WIN32 + Sleep(250); +#else + struct timeval t; + t.tv_sec = 0; + t.tv_usec = 250000; // .25 seconds + select(0, NULL, NULL, NULL, &t); +#endif + } + } // end while pleasestop == 0 + + + // When we get to here, the program is exiting. + + list.del(sampP); // Take the table out of the list + delete sampP; + + conn.setXlaBufferSize(oldsize, NULL, stat); + + return 0; + +} + diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/slapd.conf b/servers/slapd/back-sql/rdbms_depend/timesten/slapd.conf new file mode 100644 index 0000000000..b0b15a386f --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/slapd.conf @@ -0,0 +1,31 @@ +# $OpenLDAP$ +# +# See slapd.conf(5) for details on configuration options. +# This file should NOT be world readable. +# +include /usr/local/etc/openldap/schema/core.schema +include /usr/local/etc/openldap/schema/cosine.schema +include /usr/local/etc/openldap/schema/inetorgperson.schema + +# Define global ACLs to disable default read access. + +# Do not enable referrals until AFTER you have a working directory +# service AND an understanding of referrals. +#referral ldap://root.openldap.org + +pidfile /usr/local/var/slapd.pid +argsfile /usr/local/var/slapd.args + +####################################################################### +# sql database definitions +####################################################################### + +database sql +suffix "o=sql,c=RU" +rootdn "cn=root,o=sql,c=RU" +rootpw secret +dbname ldap_tt +dbuser root +dbpasswd +subtree_cond "ldap_entries.dn LIKE ?" +insentry_query "INSERT INTO ldap_entries (dn,oc_map_id,parent,keyval) VALUES (?,?,?,?)" diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/testdb_create.sql b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_create.sql new file mode 100644 index 0000000000..768aec81c1 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_create.sql @@ -0,0 +1,36 @@ +CREATE TABLE persons ( + id int NOT NULL primary key, + name varchar(255) NOT NULL +) +unique hash on (id) pages=100; + +CREATE TABLE institutes ( + id int NOT NULL primary key, + name varchar(255) +) +unique hash on (id) pages=100; + +CREATE TABLE documents ( + id int NOT NULL primary key, + title varchar(255) NOT NULL, + abstract varchar(255) +) +unique hash on (id) pages=100; + +CREATE TABLE authors_docs ( + pers_id int NOT NULL, + doc_id int NOT NULL, + PRIMARY KEY + ( + pers_id, + doc_id + ) +) unique hash on (pers_id, doc_id) pages=100; + +CREATE TABLE phones ( + id int NOT NULL primary key, + phone varchar(255) NOT NULL , + pers_id int NOT NULL +) +unique hash on (id) pages=100; + diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/testdb_data.sql b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_data.sql new file mode 100644 index 0000000000..f141f414eb --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_data.sql @@ -0,0 +1,16 @@ +insert into institutes (id,name) values (1,'sql'); + +insert into persons (id,name) values (1,'Mitya Kovalev'); +insert into persons (id,name) values (2,'Torvlobnor Puzdoy'); +insert into persons (id,name) values (3,'Akakiy Zinberstein'); + +insert into phones (id,phone,pers_id) values (1,'332-2334',1); +insert into phones (id,phone,pers_id) values (2,'222-3234',1); +insert into phones (id,phone,pers_id) values (3,'545-4563',2); + +insert into documents (id,abstract,title) values (1,'abstract1','book1'); +insert into documents (id,abstract,title) values (2,'abstract2','book2'); + +insert into authors_docs (pers_id,doc_id) values (1,1); +insert into authors_docs (pers_id,doc_id) values (1,2); +insert into authors_docs (pers_id,doc_id) values (2,1); diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/testdb_drop.sql b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_drop.sql new file mode 100644 index 0000000000..17b12afd42 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_drop.sql @@ -0,0 +1,5 @@ +DROP TABLE persons; +DROP TABLE institutes; +DROP TABLE documents; +DROP TABLE authors_docs; +DROP TABLE phones; diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/testdb_metadata.sql b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_metadata.sql new file mode 100644 index 0000000000..098eea23b0 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/testdb_metadata.sql @@ -0,0 +1,108 @@ + +insert into ldap_oc_mappings +(id,name, keytbl, keycol, create_proc, +delete_proc,expect_return) +values +(1,'inetOrgPerson','persons','id', 'insert into persons (name) values ('');\n select last_insert_id();', +NULL,0); + +insert into ldap_oc_mappings +(id, name, keytbl, keycol,create_proc,delete_proc,expect_return) +values +(2, 'document','documents','id', NULL, NULL, 0); + +insert into ldap_oc_mappings +(id,name, keytbl, keycol,create_proc,delete_proc,expect_return) +values +(3,'organization','institutes','id', NULL, NULL, 0); + + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls,join_where,add_proc, +delete_proc,param_order,expect_return) +values +(1, 1, 'cn', 'persons.name', 'persons',NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id,name, sel_expr, from_tbls, +join_where, add_proc,delete_proc,param_order,expect_return) +values +(2, 1, 'telephoneNumber','phones.phone','persons,phones', +'phones.pers_id=persons.id', NULL, NULL, 3, 0); + +insert into ldap_attr_mappings +(id,oc_map_id, name, sel_expr, from_tbls, join_where,add_proc, +delete_proc,param_order,expect_return) +values +(3, 1, 'sn', 'persons.name','persons', NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where, +add_proc,delete_proc,param_order,expect_return) +values +(4, 2, 'description', 'documents.abstract','documents', NULL, +NULL, NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where, +add_proc,delete_proc,param_order,expect_return) +values +(5, 2, 'documentTitle','documents.title','documents',NULL, +NULL, NULL, 3, 0); + +-- insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +-- values (6,2,'documentAuthor','persons.name','persons,documents,authors_docs', +-- 'persons.id=authors_docs.pers_id AND documents.id=authors_docs.doc_id', +-- NULL,NULL,3,0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where,add_proc, +delete_proc,param_order,expect_return) +values +(7, 3, 'o', 'institutes.name', 'institutes', NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +values (8,1,'documentDN','ldap_entries.dn','ldap_entries,documents,authors_docs,persons', + 'ldap_entries.keyval=documents.id AND ldap_entries.oc_map_id=2 AND authors_docs.doc_id=documents.id AND authors_docs.pers_id=persons.id', + NULL,NULL,3,0); + +insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +values (9,2,'documentAuthor','ldap_entries.dn','ldap_entries,documents,authors_docs,persons', + 'ldap_entries.keyval=persons.id AND ldap_entries.oc_map_id=1 AND authors_docs.doc_id=documents.id AND authors_docs.pers_id=persons.id', + NULL,NULL,3,0); + +-- entries + +insert into ldap_entries +(id, dn, oc_map_id, parent, keyval) +values +(1, 'o=sql,c=RU', 3, 0, 1); + +insert into ldap_entries +(id, dn, oc_map_id, parent, keyval) +values +(2, 'cn=Mitya Kovalev,o=sql,c=RU', 1, 1, 1); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (3,'cn=Torvlobnor Puzdoy,o=sql,c=RU',1,1,2); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (4,'cn=Akakiy Zinberstein,o=sql,c=RU',1,1,3); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (5,'documentTitle=book1,o=sql,c=RU',2,1,1); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (6,'documentTitle=book2,o=sql,c=RU',2,1,2); + + +-- referrals + +insert into ldap_entry_objclasses (entry_id,oc_name) +values (4,'referral'); + +insert into ldap_referrals (entry_id,url) +values (4,'http://localhost'); diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/ttcreate_schema.sh b/servers/slapd/back-sql/rdbms_depend/timesten/ttcreate_schema.sh new file mode 100755 index 0000000000..c4c5df2e7a --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/ttcreate_schema.sh @@ -0,0 +1,4 @@ +ttIsql -connStr "DSN=ldap_tt;Overwrite=1" -f backsql_create.sql +ttIsql -connStr "DSN=ldap_tt" -f tttestdb_create.sql +ttIsql -connStr "DSN=ldap_tt" -f tttestdb_data.sql +ttIsql -connStr "DSN=ldap_tt" -f tttestdb_metadata.sql diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_create.sql b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_create.sql new file mode 100644 index 0000000000..f5955d2763 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_create.sql @@ -0,0 +1,42 @@ +CREATE TABLE persons ( + id int NOT NULL primary key, + name varchar(255) NOT NULL, + name_u varchar(255), + title varchar(255), + title_U varchar(255), + organization varchar(255) +) +unique hash on (id) pages=100; +create index personsx1 on persons(title_U); +create index personsx2 on persons(name_u); + +CREATE TABLE institutes ( + id int NOT NULL primary key, + name varchar(255) +) +unique hash on (id) pages=100; + +CREATE TABLE documents ( + id int NOT NULL primary key, + title varchar(255) NOT NULL, + abstract varchar(255) +) +unique hash on (id) pages=100; + +CREATE TABLE authors_docs ( + pers_id int NOT NULL, + doc_id int NOT NULL, + PRIMARY KEY + ( + pers_id, + doc_id + ) +) unique hash on (pers_id, doc_id) pages=100; + +CREATE TABLE phones ( + id int NOT NULL primary key, + phone varchar(255) NOT NULL , + pers_id int NOT NULL +) +unique hash on (id) pages=100; + diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_data.sql b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_data.sql new file mode 100644 index 0000000000..ca753398d4 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_data.sql @@ -0,0 +1,20 @@ +insert into institutes (id,name) values (1,'sql'); + +insert into persons (id,name, title, title_U, organization) values +(1,'Mitya Kovalev', 'Engineer', 'ENGINEER', 'Development'); +insert into persons (id,name, title, title_U, organization) values +(2,'Torvlobnor Puzdoy', 'Engineer', 'ENGINEER', 'Sales'); +insert into persons (id,name, title, title_U, organization) values +(3,'Akakiy Zinberstein', 'Engineer', 'ENGINEER', 'Marketing'); +update persons set name_u = upper(name); + +insert into phones (id,phone,pers_id) values (1,'332-2334',1); +insert into phones (id,phone,pers_id) values (2,'222-3234',1); +insert into phones (id,phone,pers_id) values (3,'545-4563',2); + +insert into documents (id,abstract,title) values (1,'abstract1','book1'); +insert into documents (id,abstract,title) values (2,'abstract2','book2'); + +insert into authors_docs (pers_id,doc_id) values (1,1); +insert into authors_docs (pers_id,doc_id) values (1,2); +insert into authors_docs (pers_id,doc_id) values (2,1); diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_drop.sql b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_drop.sql new file mode 100644 index 0000000000..17b12afd42 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_drop.sql @@ -0,0 +1,5 @@ +DROP TABLE persons; +DROP TABLE institutes; +DROP TABLE documents; +DROP TABLE authors_docs; +DROP TABLE phones; diff --git a/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_metadata.sql b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_metadata.sql new file mode 100644 index 0000000000..69bda8abf0 --- /dev/null +++ b/servers/slapd/back-sql/rdbms_depend/timesten/tttestdb_metadata.sql @@ -0,0 +1,122 @@ + +insert into ldap_oc_mappings +(id,name, keytbl, keycol, create_proc, +delete_proc,expect_return) +values +(1,'inetOrgPerson','persons','id', 'insert into persons (name) values ('');\n select last_insert_id();', +NULL,0); + +insert into ldap_oc_mappings +(id, name, keytbl, keycol,create_proc,delete_proc,expect_return) +values +(2, 'document','documents','id', NULL, NULL, 0); + +insert into ldap_oc_mappings +(id,name, keytbl, keycol,create_proc,delete_proc,expect_return) +values +(3,'organization','institutes','id', NULL, NULL, 0); + + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, sel_expr_u, from_tbls, +join_where,add_proc, delete_proc,param_order,expect_return) +values +(1, 1, 'cn', 'persons.name', 'persons.name_u','persons', +NULL, NULL, NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, sel_expr_u, from_tbls,join_where, +add_proc, delete_proc,param_order,expect_return) +values +(10, 1, 'title', 'persons.title', 'persons.title_u', 'persons',NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id,name, sel_expr, from_tbls, +join_where, add_proc,delete_proc,param_order,expect_return) +values +(2, 1, 'telephoneNumber','phones.phone','persons,phones', +'phones.pers_id=persons.id', NULL, NULL, 3, 0); + +insert into ldap_attr_mappings +(id,oc_map_id, name, sel_expr, from_tbls, join_where,add_proc, +delete_proc,param_order,expect_return) +values +(3, 1, 'sn', 'persons.name','persons', NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where,add_proc, +delete_proc,param_order,expect_return) +values +(30, 1, 'ou', 'persons.organization','persons', NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where, +add_proc,delete_proc,param_order,expect_return) +values +(4, 2, 'description', 'documents.abstract','documents', NULL, +NULL, NULL, 3, 0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where, +add_proc,delete_proc,param_order,expect_return) +values +(5, 2, 'documentTitle','documents.title','documents',NULL, +NULL, NULL, 3, 0); + +-- insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +-- values (6,2,'documentAuthor','persons.name','persons,documents,authors_docs', +-- 'persons.id=authors_docs.pers_id AND documents.id=authors_docs.doc_id', +-- NULL,NULL,3,0); + +insert into ldap_attr_mappings +(id, oc_map_id, name, sel_expr, from_tbls, join_where,add_proc, +delete_proc,param_order,expect_return) +values +(7, 3, 'o', 'institutes.name', 'institutes', NULL, NULL, +NULL, 3, 0); + +insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +values (8,1,'documentDN','ldap_entries.dn','ldap_entries,documents,authors_docs,persons', + 'ldap_entries.keyval=documents.id AND ldap_entries.oc_map_id=2 AND authors_docs.doc_id=documents.id AND authors_docs.pers_id=persons.id', + NULL,NULL,3,0); + +insert into ldap_attr_mappings (id,oc_map_id,name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return) +values (9,2,'documentAuthor','ldap_entries.dn','ldap_entries,documents,authors_docs,persons', + 'ldap_entries.keyval=persons.id AND ldap_entries.oc_map_id=1 AND authors_docs.doc_id=documents.id AND authors_docs.pers_id=persons.id', + NULL,NULL,3,0); + +-- entries + +insert into ldap_entries +(id, dn, oc_map_id, parent, keyval) +values +(1, 'o=sql,c=RU', 3, 0, 1); + +insert into ldap_entries +(id, dn, oc_map_id, parent, keyval) +values +(2, 'cn=Mitya Kovalev,o=sql,c=RU', 1, 1, 1); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (3,'cn=Torvlobnor Puzdoy,o=sql,c=RU',1,1,2); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (4,'cn=Akakiy Zinberstein,o=sql,c=RU',1,1,3); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (5,'documentTitle=book1,o=sql,c=RU',2,1,1); + +insert into ldap_entries (id,dn,oc_map_id,parent,keyval) +values (6,'documentTitle=book2,o=sql,c=RU',2,1,2); + + +-- referrals + +insert into ldap_entry_objclasses (entry_id,oc_name) +values (4,'referral'); + +insert into ldap_referrals (entry_id,url) +values (4,'http://localhost'); diff --git a/servers/slapd/back-sql/schema-map.c b/servers/slapd/back-sql/schema-map.c index ca76a30dbc..2055b8f322 100644 --- a/servers/slapd/back-sql/schema-map.c +++ b/servers/slapd/back-sql/schema-map.c @@ -114,6 +114,21 @@ int backsql_load_schema_map(backsql_info *si,SQLHDBC dbh) int tmpslen; Debug(LDAP_DEBUG_TRACE,"==>load_schema_map()\n",0,0,0); + + /* TimesTen : See if the ldap_entries.dn_ru field exists in the schema. */ + + rc = backsql_Prepare(dbh, &oc_sth, backsql_check_dn_ru_query, 0); + if (rc == SQL_SUCCESS) { + si->has_ldapinfo_dn_ru = 1; // Yes, the field exists + Debug(LDAP_DEBUG_TRACE, "ldapinfo.dn_ru field exists in the schema\n", 0, 0, +0); + } + else { + si->has_ldapinfo_dn_ru = 0; // No such field exists + } + + SQLFreeStmt(oc_sth, SQL_DROP); + rc=backsql_Prepare(dbh,&oc_sth,si->oc_query,0); if (rc != SQL_SUCCESS) { @@ -121,6 +136,8 @@ int backsql_load_schema_map(backsql_info *si,SQLHDBC dbh) backsql_PrintErrors(si->db_env,dbh,oc_sth,rc); return -1; } + Debug(LDAP_DEBUG_TRACE, "load_schema_map(): at_query '%s'\n", si->at_query,0,0); + rc=backsql_Prepare(dbh,&at_sth,si->at_query,0); if (rc != SQL_SUCCESS) { @@ -158,8 +175,15 @@ int backsql_load_schema_map(backsql_info *si,SQLHDBC dbh) oc_id=oc_map->id; Debug(LDAP_DEBUG_TRACE,"load_schema_map(): objectclass '%s': keytbl='%s' keycol='%s' ", oc_map->name,oc_map->keytbl,oc_map->keycol); - Debug(LDAP_DEBUG_TRACE,"create_proc='%s' delete_proc='%s' expect_return=%d; attributes:\n", - oc_map->create_proc,oc_map->delete_proc,oc_map->expect_return); + if (oc_map->delete_proc) { + Debug(LDAP_DEBUG_TRACE,"delete_proc='%s'\n", oc_map->delete_proc, 0, 0); + } + if (oc_map->create_proc) { + Debug(LDAP_DEBUG_TRACE,"create_proc='%s'\n", oc_map->create_proc, 0, 0); + } + Debug(LDAP_DEBUG_TRACE,"expect_return=%d; attributes:\n", + oc_map->expect_return, 0, 0); + Debug(LDAP_DEBUG_TRACE,"load_schema_map(): autoadding 'objectClass' and 'ref' mappings\n",0,0,0); backsql_add_sysmaps(oc_map); if ((rc=SQLExecute(at_sth)) != SQL_SUCCESS) @@ -177,9 +201,12 @@ int backsql_load_schema_map(backsql_info *si,SQLHDBC dbh) Debug(LDAP_DEBUG_TRACE,"join_where='%s',add_proc='%s' ",at_row.cols[3], at_row.cols[4],0); Debug(LDAP_DEBUG_TRACE,"delete_proc='%s'\n",at_row.cols[5],0,0); + Debug(LDAP_DEBUG_TRACE,"sel_expr_u='%s'\n", at_row.cols[8],0,0); // TimesTen at_map=(backsql_at_map_rec*)ch_calloc(1,sizeof(backsql_at_map_rec)); at_map->name=ch_strdup(at_row.cols[0]); at_map->sel_expr=ch_strdup(at_row.cols[1]); + at_map->sel_expr_u = (at_row.is_null[8]<0)?NULL:ch_strdup(at_row.cols[8 +]); tmps=NULL;tmpslen=0; backsql_merge_from_clause(&tmps,&tmpslen,at_row.cols[2]); at_map->from_tbls=ch_strdup(tmps); @@ -264,6 +291,8 @@ int backsql_free_attr(backsql_at_map_rec *at) if (at->query) ch_free(at->query); ch_free(at); + if (at->sel_expr_u) + ch_free(at->sel_expr_u); // TimesTen Debug(LDAP_DEBUG_TRACE,"<==free_attr()\n",0,0,0); return 1; } diff --git a/servers/slapd/back-sql/schema-map.h b/servers/slapd/back-sql/schema-map.h index 6e48c36fbb..3db6331dde 100644 --- a/servers/slapd/back-sql/schema-map.h +++ b/servers/slapd/back-sql/schema-map.h @@ -35,6 +35,7 @@ typedef struct //following flags are bitmasks (first bit used for add_proc, second - for modify, third - for delete_proc) int param_order; //order of parameters for procedures above; 1 means "data then keyval", 0 means "keyval then data" int expect_return; //flags whether one or more of procedures is a function (whether back-sql should bind first parameter as output for return code) + char *sel_expr_u; // TimesTen }backsql_at_map_rec; //defines to support bitmasks above @@ -47,4 +48,4 @@ backsql_oc_map_rec* backsql_oc_with_id(backsql_info *si,unsigned long id); backsql_at_map_rec* backsql_at_with_name(backsql_oc_map_rec* objclass,char* attr); int backsql_destroy_schema_map(backsql_info *si); -#endif \ No newline at end of file +#endif diff --git a/servers/slapd/back-sql/search.c b/servers/slapd/back-sql/search.c index 9cd5332589..5104e2c480 100644 --- a/servers/slapd/back-sql/search.c +++ b/servers/slapd/back-sql/search.c @@ -86,7 +86,10 @@ int backsql_process_filter_list(backsql_srch_info *bsi,Filter *f,int op) while(1) { res=backsql_process_filter(bsi,f); - + if (res < 0) + return -1; /* TimesTen : If the query has no answers, + don't bother to run the query. */ + f=f->f_next; if (f==NULL) break; @@ -116,12 +119,21 @@ int backsql_process_sub_filter(backsql_srch_info *bsi,Filter *f) return 0; bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"(",NULL); - + // TimesTen + Debug(LDAP_DEBUG_TRACE,"expr: '%s' '%s'\n",at->sel_expr, + at->sel_expr_u?at->sel_expr_u:"",0); if (bsi->bi->upper_func) { - bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len, - bsi->bi->upper_func,"(",at->sel_expr,")", - " LIKE '",NULL); + /* If a pre-upper-cased version of the column exists, use it. */ + if (at->sel_expr_u) { + bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len, + at->sel_expr_u," LIKE '",NULL); + } + else { + bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len, + bsi->bi->upper_func,"(",at->sel_expr,")", + " LIKE '",NULL); + } } else { @@ -171,6 +183,7 @@ int backsql_process_filter(backsql_srch_info *bsi,Filter *f) backsql_at_map_rec oc_attr={"objectClass","","",NULL,NULL,NULL,NULL}; char *at_name=NULL; int done=0,len=0; + int rc=0; // TimesTen Debug(LDAP_DEBUG_TRACE,"==>backsql_process_filter()\n",0,0,0); if (f==NULL || f->f_choice==SLAPD_FILTER_COMPUTED) @@ -181,16 +194,16 @@ int backsql_process_filter(backsql_srch_info *bsi,Filter *f) switch(f->f_choice) { case LDAP_FILTER_OR: - backsql_process_filter_list(bsi,f->f_or,LDAP_FILTER_OR); + rc = backsql_process_filter_list(bsi,f->f_or,LDAP_FILTER_OR); done=1; break; case LDAP_FILTER_AND: - backsql_process_filter_list(bsi,f->f_and,LDAP_FILTER_AND); + rc = backsql_process_filter_list(bsi,f->f_and,LDAP_FILTER_AND); done=1; break; case LDAP_FILTER_NOT: bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"NOT (",NULL); - backsql_process_filter(bsi,f->f_not); + rc = backsql_process_filter(bsi,f->f_not); bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,")",NULL); done=1; break; @@ -201,6 +214,9 @@ int backsql_process_filter(backsql_srch_info *bsi,Filter *f) at_name=f->f_av_desc->ad_cname->bv_val; break; } + + if (rc == -1) + goto impossible; /* TimesTen : Don't run the query */ if (done) goto done; @@ -217,7 +233,7 @@ int backsql_process_filter(backsql_srch_info *bsi,Filter *f) Debug(LDAP_DEBUG_TRACE,"backsql_process_filter(): attribute '%s' is not defined for objectclass '%s'\n", at_name,bsi->oc->name,0); bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len," 1=0 ",NULL); - return -1; + goto impossible; } backsql_merge_from_clause(&bsi->from,&bsi->from_len,at->from_tbls); @@ -237,10 +253,17 @@ int backsql_process_filter(backsql_srch_info *bsi,Filter *f) //to know whether upper_func is applicable, but for now //upper_func stuff is made for Oracle, where UPPER is //safely applicable to NUMBER etc. - if (bsi->bi->upper_func) - bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"(", + if (bsi->bi->upper_func) { + if (at->sel_expr_u) + bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"(", + at->sel_expr_u,"='", + ldap_pvt_str2upper(f->f_av_value->bv_val),"')", + NULL); + else + bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"(", bsi->bi->upper_func,"(",at->sel_expr,")='", - ldap_pvt_str2upper(f->f_av_value->bv_val),"')",NULL); + ldap_pvt_str2upper(f->f_av_value->bv_val),"')",NULL); + } else bsi->flt_where=backsql_strcat(bsi->flt_where,&bsi->fwhere_len,"(",at->sel_expr,"='", f->f_av_value->bv_val,"')",NULL); @@ -269,12 +292,20 @@ done: free(oc_attr.sel_expr); Debug(LDAP_DEBUG_TRACE,"<==backsql_process_filter()\n",0,0,0); return 1; + +impossible: + if (oc_attr.sel_expr!=NULL) + free(oc_attr.sel_expr); + Debug(LDAP_DEBUG_TRACE,"<==backsql_process_filter() returns -1\n",0,0,0); + return -1; + } char* backsql_srch_query(backsql_srch_info *bsi) { char *query=NULL; int q_len=0; + int rc; Debug(LDAP_DEBUG_TRACE,"==>backsql_srch_query()\n",0,0,0); bsi->sel=NULL; @@ -316,9 +347,19 @@ char* backsql_srch_query(backsql_srch_info *bsi) bsi->bi->subtree_cond,NULL); break; } - if (backsql_process_filter(bsi,bsi->filter)) + + rc = backsql_process_filter(bsi, bsi->filter); + if (rc>0) { query=backsql_strcat(query,&q_len,bsi->sel,bsi->from,bsi->join_where," AND ",bsi->flt_where,NULL); + } + else if (rc < 0) { + /* Indicates that there's no possible way the filter matches + anything. No need to issue the query. */ + Debug(LDAP_DEBUG_TRACE,"<==backsql_srch_query() returns NULL\n",0,0,0); + free(query); + query = NULL; + } free(bsi->sel); free(bsi->from); @@ -337,7 +378,9 @@ int backsql_oc_get_candidates(backsql_oc_map_rec *oc,backsql_srch_info *bsi) backsql_entryID base_id,*res,*c_id; //Entry *e; BACKSQL_ROW_NTS row; - //int i; + int i; + int j; + char temp_base_dn[BACKSQL_MAX_DN_LEN+1]; // TimesTen Debug(LDAP_DEBUG_TRACE,"==>backsql_oc_get_candidates(): oc='%s'\n",oc->name,0,0); bsi->oc=oc; @@ -349,6 +392,7 @@ int backsql_oc_get_candidates(backsql_oc_map_rec *oc,backsql_srch_info *bsi) } Debug(LDAP_DEBUG_TRACE,"Constructed query: %s\n",query,0,0); + if ((rc=backsql_Prepare(bsi->dbh,&sth,query,0)) != SQL_SUCCESS) { Debug(LDAP_DEBUG_TRACE,"backsql_oc_get_candidates(): error preparing query\n",0,0,0); @@ -366,7 +410,6 @@ int backsql_oc_get_candidates(backsql_oc_map_rec *oc,backsql_srch_info *bsi) switch(bsi->scope) { case LDAP_SCOPE_BASE: - case LDAP_SCOPE_SUBTREE: if ((rc=backsql_BindParamStr(sth,2,bsi->base_dn,BACKSQL_MAX_DN_LEN)) != SQL_SUCCESS) { Debug(LDAP_DEBUG_TRACE,"backsql_oc_get_candidates(): error binding base_dn parameter\n",0,0,0); @@ -374,6 +417,44 @@ int backsql_oc_get_candidates(backsql_oc_map_rec *oc,backsql_srch_info *bsi) return 1; } break; + + case LDAP_SCOPE_SUBTREE: + /* Sets the parameters for the SQL built earlier */ + /* NOTE that all the databases could actually use the TimesTen version, + which would be cleaner and would also eliminate the need for the + subtree_cond line in the configuration file. For now, I'm leaving + it the way it is, so non-TimesTen databases use the original code. + But at some point this should get cleaned up. */ + /* If "dn" is being used, do a suffix search. + If "dn_ru" is being used, do a prefix search. */ + + if (bsi->bi->has_ldapinfo_dn_ru) { + temp_base_dn[0] = '\0'; + for ((i=0, j=strlen(bsi->base_dn)-1); j >= 0; (i++, j--)) { + *(temp_base_dn+i) = toupper(*(bsi->base_dn+j)); + } + *(temp_base_dn+i) = '%'; + *(temp_base_dn+i+1) = '\0'; + } + else { + strcpy(temp_base_dn, "%"); + for (i = 0; *(bsi->base_dn+i); i++) { + *(temp_base_dn+i+1) = toupper(*(bsi->base_dn+i)); + } + *(temp_base_dn+i+1) = '\0'; + } + Debug(LDAP_DEBUG_TRACE, "dn '%s'\n", temp_base_dn, 0, 0); + + if ((rc=backsql_BindParamStr(sth,2,temp_base_dn,BACKSQL_MAX_DN_LEN)) != +SQL_SUCCESS) + { + Debug(LDAP_DEBUG_TRACE,"backsql_oc_get_candidates(): error binding base +_dn parameter (2)\n",0,0,0); + backsql_PrintErrors(bsi->bi->db_env,bsi->dbh,sth,rc); + return 1; + } + break; + case LDAP_SCOPE_ONELEVEL: res=backsql_dn2id(bsi->bi,&base_id,bsi->dbh,bsi->base_dn); if (res==NULL) @@ -429,6 +510,7 @@ int backsql_oc_get_candidates(backsql_oc_map_rec *oc,backsql_srch_info *bsi) } backsql_FreeRow(&row); SQLFreeStmt(sth,SQL_DROP); + Debug(LDAP_DEBUG_TRACE,"<==backsql_oc_get_candidates()\n",0,0,0); return 1; } @@ -460,6 +542,9 @@ int backsql_search(BackendDB *be,Connection *conn,Operation *op, send_ldap_result(conn,op,LDAP_OTHER,"","SQL-backend error",NULL,NULL); return 1; } + + /* TimesTen : Pass it along to the lower level routines */ + srch_info.isTimesTen = bi->isTimesTen; if (tlimit == 0 && be_isroot(be,op->o_dn)) { diff --git a/servers/slapd/back-sql/sql-wrap.c b/servers/slapd/back-sql/sql-wrap.c index 8a969f214c..50ab84daf2 100644 --- a/servers/slapd/back-sql/sql-wrap.c +++ b/servers/slapd/back-sql/sql-wrap.c @@ -195,6 +195,7 @@ int backsql_cmp_connid(backsql_db_conn *c1,backsql_db_conn *c2) int backsql_close_db_conn(backsql_db_conn *conn) { Debug(LDAP_DEBUG_TRACE,"==>backsql_close_db_conn()\n",0,0,0); + SQLTransact(NULL, conn->dbh, SQL_COMMIT); // TimesTen SQLDisconnect(conn->dbh); SQLFreeConnect(conn->dbh); Debug(LDAP_DEBUG_TRACE,"<==backsql_close_db_conn()\n",0,0,0); @@ -227,6 +228,8 @@ int backsql_free_db_env(backsql_info *si) backsql_db_conn* backsql_open_db_conn(backsql_info *si,int ldap_cid) { + char DBMSName[32]; // TimesTen + backsql_db_conn *dbc=(backsql_db_conn*)ch_calloc(1,sizeof(backsql_db_conn)); int rc; @@ -249,7 +252,32 @@ backsql_db_conn* backsql_open_db_conn(backsql_info *si,int ldap_cid) if (rc != SQL_SUCCESS_WITH_INFO) return NULL; } - + + /* TimesTen : Turn off autocommit. We must explicitly commit any transactions. */ + + SQLSetConnectOption(dbc->dbh, SQL_AUTOCOMMIT, SQL_AUTOCOMMIT_OFF); + + /* See if this connection is to TimesTen. If it is, + remember that fact for later use. */ + + si->isTimesTen = 0; /* Assume until proven otherwise */ + + DBMSName[0] = '\0'; + rc = SQLGetInfo(dbc->dbh, SQL_DBMS_NAME, (PTR) &DBMSName, + sizeof(DBMSName), NULL); + if (rc == SQL_SUCCESS) { + if (strcmp(DBMSName, "TimesTen") == 0 || + strcmp(DBMSName, "Front-Tier") == 0) { + Debug(LDAP_DEBUG_TRACE,"backsql_open_db_conn: TimesTen database!\n",0,0,0); + si->isTimesTen = 1; + } + } + else { + Debug(LDAP_DEBUG_TRACE,"backsql_open_db_conn: SQLGetInfo() failed:\n",0,0,0); + backsql_PrintErrors(si->db_env,dbc->dbh,SQL_NULL_HENV,rc); + } + // end TimesTen + Debug(LDAP_DEBUG_TRACE,"backsql_open_db_conn(): connected, adding to tree\n",0,0,0); ldap_pvt_thread_mutex_lock(&si->dbconn_mutex); avl_insert(&si->db_conns,dbc,(AVL_CMP)backsql_cmp_connid,backsql_dummy); @@ -293,7 +321,7 @@ SQLHDBC backsql_get_db_conn(Backend *be,Connection *ldapc) dbc=(backsql_db_conn*)avl_find(si->db_conns,&tmp,(AVL_CMP)backsql_cmp_connid); if (!dbc) dbc=backsql_open_db_conn(si,ldapc->c_connid); - + if (!dbc) { Debug(LDAP_DEBUG_TRACE,"backsql_get_db_conn(): could not get connection handle -- returning NULL\n",0,0,0); diff --git a/servers/slapd/back-sql/util.c b/servers/slapd/back-sql/util.c index 5d89c71f5c..a015674bdd 100644 --- a/servers/slapd/back-sql/util.c +++ b/servers/slapd/back-sql/util.c @@ -22,12 +22,16 @@ char backsql_def_oc_query[]="SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings"; -char backsql_def_at_query[]="SELECT name,sel_expr,from_tbls,join_where,add_proc,delete_proc,param_order,expect_return FROM ldap_attr_mappings WHERE oc_map_id=?"; +char backsql_def_at_query[]="SELECT name,sel_expr,from_tbls,join_where,add_proc, +delete_proc,param_order,expect_return,sel_expr_u FROM ldap_attr_mappings WHERE oc_map_id=?"; char backsql_def_delentry_query[]="DELETE FROM ldap_entries WHERE id=?"; char backsql_def_insentry_query[]="INSERT INTO ldap_entries (dn,oc_map_id,parent,keyval) VALUES (?,?,?,?)"; char backsql_def_subtree_cond[]="ldap_entries.dn LIKE CONCAT('%',?)"; char backsql_id_query[]="SELECT id,keyval,oc_map_id FROM ldap_entries WHERE "; +// TimesTen +char backsql_check_dn_ru_query[] = "SELECT dn_ru from ldap_entries"; + char* backsql_strcat(char* dest,int *buflen, ...) { va_list strs; diff --git a/servers/slapd/back-sql/util.h b/servers/slapd/back-sql/util.h index 110c9b6f2c..949ccaa221 100644 --- a/servers/slapd/back-sql/util.h +++ b/servers/slapd/back-sql/util.h @@ -43,6 +43,7 @@ typedef struct __backsql_srch_info Operation *op; char **attrs; Entry *e; + int isTimesTen; /* 1 if the db is TimesTen; 0 if it's not */ }backsql_srch_info; int backsql_process_filter(backsql_srch_info *bsi,Filter *f); @@ -54,8 +55,9 @@ Entry* backsql_id2entry(backsql_srch_info *bsi,Entry* e,backsql_entryID* id); extern char backsql_def_oc_query[],backsql_def_at_query[], backsql_def_delentry_query[],backsql_def_insentry_query[], backsql_def_subtree_cond[],backsql_id_query[]; +extern char backsql_check_dn_ru_query[]; int backsql_merge_from_clause(char **dest_from,int *dest_len,char *src_from); -#endif \ No newline at end of file +#endif