From: Will Ballantyne Date: Fri, 23 Apr 1999 23:00:44 +0000 (+0000) Subject: openldap guide sdf files and gifs X-Git-Tag: OPENLDAP_SLAPD_BACK_LDAP~145 X-Git-Url: https://git.sur5r.net/?a=commitdiff_plain;h=2c95a6fec528359e9db998f0ecdec7413bd7220b;p=openldap openldap guide sdf files and gifs --- diff --git a/doc/guide/LDAPwww.gif b/doc/guide/LDAPwww.gif new file mode 100644 index 0000000000..b593bc983e Binary files /dev/null and b/doc/guide/LDAPwww.gif differ diff --git a/doc/guide/config.sdf b/doc/guide/config.sdf new file mode 100644 index 0000000000..37322c7eb7 --- /dev/null +++ b/doc/guide/config.sdf @@ -0,0 +1,76 @@ +H1: The Big Picture - Configuration Choices + +This section gives a brief overview of various LDAP directory +configurations, and how your LDAP server (either {{I:slapd}} or +{{I:ldapd}}) fits in with the rest of the world. + + + +H2: LDAP as a local service only + +In this configuration, you run a {{I:slapd}} which provides directory service +for your local domain only. It does not interact with other directory +servers in any way. This configuration is shown in Figure 2. + +!import "figure_2.gif"; align="center"; title="Local service via slapd configuration" +FT: Figure 2: Local service via slapd configuration. + +Use this configuration if you are just starting out (it's the one the +quick-start guide makes for you) or if you want to provide a local +service and are not interested in connecting to the rest of the world. +It's easy to upgrade to another configuration later if you want. + + + +H2: Local service with X.500 referrals + +In this configuration, you run a slapd which provides directory service +for your local domain and an ldapd which provides access to the +X.500 world (you don't have to run the ldapd yourself - you can just +point to somebody else who does and doesn't mind you pointing to +their service). This configuration is shown in Figure 3. + +!import "figure_3.gif"; align="center"; title="Local service via slapd + X.500 referrals configuration" +FT: Figure 3: Local service via slapd + X.500 referrals configuration + +Use this configuration if you want to provide local service but still want +to be connected to the rest of the X.500 world. Remember, you don't +necessarily have to be running the ldapd in this picture; you just need +to find one you can point to. + + + +H2: LDAP as a front end to X.500 + +In this configuration, you run an X.500 service which provides +directory service for your local domain and gatewaying service to the +rest of the X.500 world. LDAP clients gain access to the directory +through an ldapd which runs at your site. This configuration is shown +in Figure 4. + +!import "figure_4.gif"; align="center"; title="Local service via X.500 and ldapd configuration" +FT: Figure 4: Local service via X.500 and ldapd configuration + +Use this configuration if you are already running an X.500 service. +Slapd is not involved in this configuration, so you can probably stop +reading this guide. + + + +H2: Replicated slapd service + +The slurpd daemon is used to propagate changes from a master slapd +to one or more slave slapds. An example master-slave configuration +is shown in figure 5. + +!import "figure_5.gif"; align="center"; title="Master slapd with two slaves replicated with slurpd" +FT: Figure 5: Master slapd with two slaves replicated with slurpd + +This configuration can be used in conjunction with the first two +configurations in situations where a single slapd does not provide the +required reliability or availability. + +PB: + + + diff --git a/doc/guide/dbtools.sdf b/doc/guide/dbtools.sdf new file mode 100644 index 0000000000..b75170c8d0 --- /dev/null +++ b/doc/guide/dbtools.sdf @@ -0,0 +1,702 @@ +H1: Database Creation and Maintenance Tools + +This section tells you how to create a slapd database from +scratch, and how to do trouble shooting if you run into +problems. There are two ways to create a database. First, +you can create the database on-line using LDAP. With this +method, you simply start up slapd and add entries using the +LDAP client of your choice. This method is fine for relatively +small databases (a few hundred or thousand entries, +depending on your requirements). + +The second method of database creation is to do it off-line, +using the index generation tools. This method is best if you +have many thousands of entries to create, which would take +an unacceptably long time using the LDAP method, or if you +want to ensure the database is not accessed while it is +being created. + + + +H2: Creating a database over LDAP + + + +With this method, you use the LDAP client of your choice +(e.g., the ldapadd(1) tool) to add entries, just like you would +once the database is created. You should be sure to set the +following configuration options before starting slapd: + +E: suffix + +As described in the preceding section, this option says what +entries are to be held by this database. You should set this +to the DN of the root of the subtree you are trying to create. +For example + +E: suffix "o=University of Michigan, c=US" + +You should be sure to specify a directory where the index +files should be created: + +E: directory + +For example: + +E: directory /usr/local/openldap/slapd + +You need to make it so you can connect to slapd as +somebody with permission to add entries. This is done +through the following two options in the database definition: + +E: rootdn +E: rootpw + +These options specify a DN and password that can be used +to authenticate as the "superuser" entry of the database (i.e., +the entry allowed to do anything). The DN and password +specified here will always work, regardless of whether the +entry named actually exists or has the password given. This +solves the chicken-and-egg problem of how to authenticate +and add entries before any entries yet exist. + +Finally, you should make sure that the database definition +contains the index definitions you want: + +E: index { | default} [pres,eq,approx,sub,none] + +For example, to index the cn, sn, uid and objectclass +attributes the following index configuration lines could be +used. + +E: index cn,sn,uid +E: index objectclass pres,eq +E: index default none + +See Section 4 on the configuration file for more details on +this option. Once you have configured things to your liking, +start up slapd, connect with your LDAP client, and start +adding entries. For example, to add a the U of M entry +followed by a Postmaster entry using the {{I:ldapadd}} tool, you +could create a file called {{EX:/tmp/newentry}} with the contents: + + +E: o=University of Michigan, c=US +E: objectClass=organization +E: o=University of Michigan +E: description=University of Michigan at Ann Arbor +E: cn=Postmaster, o=University of Michigan, c=US +E: objectClass=organizationalRole +E: cn=Postmaster +E: description=U of M postmaster - postmaster@umich.edu + +and then use a command like this to actually create the +entry: + +E: ldapadd -f /tmp/newentry -D "cn=Manager, o=University of +E: Michigan, c=US" -w secret + +The above command assumes that you have set {{EX: rootdn}} to +"cn=Manager, o=University of Michigan, c=US" and {{EX: rootpw}} +to "secret". + +H2: Creating a database off-line + +The second method of database creation is to do it off-line, +using the index generation tools described below. This +method is best if you have many thousands of entries to +create, which would take an unacceptably long time using +the LDAP method described above. These tools read the +slapd configuration file and an input file containing a text +representation of the entries to add. They produce the LDBM +index files directly. There are several important configuration +options you will want to be sure and set in the config file +database definition first: + +E: suffix + +As described in the preceding section, this option says what +entries are to be held by this database. You should set this +to the DN of the root of the subtree you are trying to create. +For example + +E: suffix "o=University of Michigan, c=US" + +You should be sure to specify a directory where the index +files should be created: + +E: directory + +For example: + +E: directory /usr/local/openldap-slapd + +Next, you probably want to increase the size of the in-core +cache used by each open index file. For best performance +during index creation, the entire index should fit in memory. If +your data is too big for this, or your memory too small, you +can still make it pretty big and let the paging system do the +work. This size is set with the following option: + +E: dbcachesize + +For example: + +E: dbcachesize 50000000 + +This would create a cache 50 MB big, which is pretty big (at +U-M, our database has about 125K entries, and our biggest +index file is about 45 MB). Experiment with this number a bit, +and the degree of parallelism (explained below), to see what +works best for your system. Remember to turn this number +back down once your index files are created and before you +run slapd. + +Finally, you need to specify which indexes you want to build. +This is done by one or more index options. + +E: index { | default} [pres,eq,approx,sub,none] + +For example: + +E: index cn,sn,uid pres,eq,approx +E: index default none + +This would create presence, equality and approximate +indexes for the cn, sn, and uid attributes, and no indexes for +any other attributes. See the configuration file section for +more information on this option. + +H3: The {{EX: ldif2ldbm}} program + +Once you've configured things to your liking, you create the +indexes by running the ldif2ldbm program: + +E: ldif2ldbm -i -f +E: [-d ] [-j ] +E: [-n ] [-e ] + +The arguments have the following meanings: + +E: -i + +Specifies the LDIF input file containing the entries to add in +text form (described below in Section 8.3). + +E: -f + +Specifies the slapd configuration file that tells where to +create the indexes, what indexes to create, etc. + +E: -d + +Turn on debugging, as specified by {{EX: }}. The +debug levels are the same as for slapd (see Section 6.1). + +E: -j + +An optional argument that specifies that at most {{EX: }} +processes should be started in parallel when building the +indexes. The default is 1. If set to a value greater than one, +{{I: ldif2ldbm}} will create at most that many subprocesses at a +time when building the indexes. A separate subprocess is +created to build each attribute index. Running these +processes in parallel can speed things up greatly, but +beware of creating too many processes, all competing for +memory and disk resources. + +E: -n + +An optional argument that specifies the configuration file +database for which to build indices. The first database listed +is "1", the second "2", etc. By default, the first ldbm database +in the configuration file is used. + +E: -e + +An optional argument that specifies the directory where +{{EX: ldif2ldbm}} can find the other database conversion tools it +needs to execute ({{EX: ldif2index}} and friends). The default is the +installation {{EX: ETCDIR}}. + +The next sections describe the programs invoked by +{{I: ldif2ldbm}} when it is building indexes. Normally, these +programs are invoked for you, but occasionally you may +want to invoke them yourself. + + + +H3: The {{EX: ldif2index}} program + +Sometimes it may be necessary to create a new attribute +index file without disturbing the rest of the database. This is +possible using the {{EX: ldif2index}} program. {{EX: ldif2index}} is invoked +like this + +E: ldif2index -i -f +E: [-d ] [-n ] + +Where the -i, -f, -d, and -n options are the same as for the +{{I: ldif2ldbm}} program. {{EX: }} is the attribute to build an index for. +Which indexes are built (e.g., equality, substring, etc.) is +controlled by the corresponding index line in the slapd +configuration file. + +You can use the ldbmcat program to create a suitable LDIF +input file from an existing LDBM database. + + + +H3: The {{EX: ldif2id2entry}} program + +The {{EX: ldif2id2entry}} program is normally invoked from {{EX: ldif2ldbm}}. +It is used to convert an LDIF text file into an {{EX: id2entry}} index. +It is unlikely that you would need to invoke it yourself, but if +you do it works like this + +E: ldif2id2entry -i -f +E: [-d ] [-n ] + +The arguments are the same as for the {{EX: ldif2ldbm}} program. + + + +H3: The {{EX: ldif2id2children}} program + +The {{EX: ldif2id2children}} program is normally invoked from +{{EX: ldif2ldbm}}. It is used to convert an LDIF text file into +{{EX: id2children}} and {{EX: dn2id}} indexes. Occasionally, it may be +necessary to run this program yourself, for example if one of +these indexes has become corrupted. {{EX: ldif2id2children}} is +invoked like this + +E: ldif2id2children -i -f +E: [-d ] [-n ] + +The arguments are the same as for the {{EX: ldif2ldbm}} program. +You can use the ldbmcat program to create a suitable LDIF +input file from an existing LDBM database. + + + +H3: The {{EX: ldbmcat}} program + +The {{EX: ldbmcat}} program is used to convert an {{EX: id2entry}} index +back into its LDIF text format. This can be useful when you +want to make a human-readable backup of your database, +or as an intermediate step in creating a new index using the +{{EX: ldif2index}} program. The program is invoked like this: + +E: ldbmcat [-n] + +where {{EX: }} is the name of the {{EX: id2entry}} index file. The +corresponding LDIF output is written to standard output. + +The -n option can be used to prevent the printing of entry +IDs in the LDIF format. If you are creating an LDIF format for +use as input to {{EX: ldif2index}} or anything by {{EX: ldif2ldbm}}, you +should not use the -n option (because the entry IDs must +match those already in the id2entry file). If you are just +making a backup of your data, you can use the -n option to +save space. + + + +H3: The {{EX: ldif}} program + +The ldif program is used to convert arbitrary data values to +LDIF format. This can be useful when writing a program or +script to create the LDIF file you will feed into the ldif2ldbm +program, or when writing a SHELL backend. ldif takes an +attribute name as an argument, and reads the attribute +value(s) from standard input. It produces the LDIF formatted +attribute line(s) on standard output. The usage is: + +E: ldif [-b] + +where {{EX: }} is the name of the attribute. Without the +-b option, ldif considers each line of standard input to be a +separate value of the attribute. + +The -b option can be used to force ldif to interpret its input +as a single raw binary value. This option is useful when +converting binary data such as a {{EX: jpegPhoto}} or {{EX: audio}} +attribute. + + +H2: The LDIF text entry format + +The LDAP Data Interchange Format (LDIF) is used to +represent LDAP entries in a simple text format. The basic +form of an entry is: + +E: [] +E: dn: +E: : +E: : +E: +E: ... + +where {{EX: }} is the optional entry ID (a positive decimal +number). Normally, you would not supply the {{EX: }}, allowing +the database creation tools to do that for you. The ldbmcat +program, however, produces an LDIF format that includes +{{EX: }} so that new indexes created will be consistent. + +A line may be continued by starting the next line with a +single space or tab character. e.g., + +E: dn: cn=Barbara J Jensen, o=University of Michigan, c=US + +Multiple attribute values are specified on separate lines. e.g., + +E: cn: Barbara J Jensen +E: cn: Babs Jensen + +If an {{EX: }} contains a non-printing character, or +begins with a space or a colon `:', the {{EX: }} is followed +by a double colon and the value is encoded in base 64 +notation. e.g., the value " begins with a space" would be +encoded like this: + +E: cn:: IGJlZ2lucyB3aXRoIGEgc3BhY2U= + +Multiple entries within the same LDIF file are separated by +blank lines. Here's an example of an LDIF file containing +three entries. + +E: dn: cn=Barbara J Jensen, o=University of Michigan, c=US +E: cn: Barbara J Jensen +E: cn: Babs Jensen +E: objectclass: person +E: sn: Jensen +E: +E: +E: dn: cn=Bjorn J Jensen, o=University of Michigan, c=US +E: cn: Bjorn J Jensen +E: cn: Bjorn Jensen +E: objectclass: person +E: sn: Jensen +E: +E: dn: cn=Jennifer J Jensen, o=University of Michigan, c=US +E: cn: Jennifer J Jensen +E: cn: Jennifer Jensen +E: objectclass: person +E: sn: Jensen +E: jpegPhoto:: /9j/4AAQSkZJRgABAAAAAQABAAD/2wBDABALD +E: A4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQ +E: ERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVG +E: +E: ... + +Notice that the {{EX: jpegPhoto}} in Jennifer Jensen's entry is +encoded using base 64. The {{EX: ldif}} program (described in +Section 8.2.6) can be used to produce the LDIF format. + +Note: Trailing spaces are not trimmed from values in an +LDIF file. Nor are multiple internal spaces compressed. If +you don't want them in your data, don't put them there. + + +H2: Converting from QUIPU EDB format to LDIF format + +If you have directory data that is or was held in a QUIPU +DSA (available as part of the ISODE package), you will want +to convert the EDB files used by QUIPU into an LDIF file. +The edb2ldif program is provided to do most of the +conversion for you. Once you have an LDIF file, you should +follow the steps outlined in section 6.2 above to build an +LDBM database for slapd. + + + +H3: The {{EX: edb2ldif}} program + +The edb2ldif program is invoked like this: + +E: edb2ldif [-d] [-v] [-r] [-o] [-b ] +E: [-a ] [-f ] +E: [-i ] [] + +The LDIF data is written to standard output. The arguments +have the following meanings: + +E: -d + +This option enables some debugging output on standard +error. + +E: -v + +Enable verbose mode that writes status information to +standard error, such as which EDB file is being processed, +how many entries have been converted so far, etc. + +E: -r + +Recurse through child directories, processing all EDB files +found. + +E: -o + +Cause local .add file definitions to override the global addfile +(see -a below) + +E: -b + +Specify the Distinguished Name that all EDB file entries +appear below. + +E: -a + +The LDIF information contained in this file will be appended +to each entry. + +E: -f + +Specify a single directory where all file-based attributes +(typically sounds and images) can be found. If this option is +not given, file attributes are assumed to be located in the +same directory as the EDB file that refers to them. + +E: -i + +Specify an attribute that should not be converted. You can +include as many -i flags as necessary. + +E: + +Specify a particular EDB file (or files) to read data from. By +default, the EDB.root (if it exists) and EDB files in the current +directory are used. + +When {{EX: edb2ldif}} is invoked, it will also look for files named +.add in the directories where EDB files are found and append +the contents of the .add file to each entry. Typically, this +feature is used to include inherited attribute values (e.g., +{{EX: objectClass}}) that do not appear in the EDB files. + + + +H3: Step-by-step EDB to LDIF conversion + +The basic steps to follow when converting your EDB format +data to an LDIF file are: + +^ Locate the directory at the top of the EDB file hierarchy +.that your QUIPU DSA masters. The EDB file located there +.should contain the entries for the first level of your +.organization or organizational unit. If you are using an +.indexed database with QUIPU, you may need to create EDB +.files from your index files (using the synctree or qb2edb +.tools). +. + ++If you do not have a file named EDB.root in the same +.directory that contains your organizational or organizational +.unit entry, create it now by hand. Its contents should look +.something like this: +. +.{{EX: MASTER}} +.{{EX: 000001}} +.{{EX: }} +.{{EX: o=University of Michigan}} +.{{EX: objectClass= top & organization & domainRelatedObject &\}} +.{{EX: quipuObject & quipuNonLeafObject}} +.{{EX: l= Ann Arbor, Michigan}} +.{{EX: st= Michigan}} +.{{EX: o= University of Michigan & UMICH & UM & U-M & U of M}} +.{{EX: description= The University of Michigan at Ann Arbor}} +.{{EX: associatedDomain= umich.edu}} +.{{EX: masterDSA= c=US@cn=Woolly Monkey}} +.{{EX: }} + ++ (Optional) Create a global add file and/or local .add files to +.take care of adding any attribute values that do not appear in +.the EDB files. For example, if all entries in a particular EDB +.are person entries and you want to add the appropriate +.objectClass attribute value for them, create a file called .add +.in the same directory as the person EDB that contains the +.single line: +. +.{{EX: objectClass: person }} +. + ++ Run the edb2ldif program to do the actual conversion. +.Make sure you are in the directory that contains the root of +.the EDB hierarchy (the one where the EDB.root file resides). +.Include a -b flag with a base DN one level above your +.organizational entry, and include -i flags to ignore any +.attributes that are not useful to slapd. E.g., the command: +. +.{{EX: edb2ldif -v -r -b "c=US" -i iattr -i acl -i xacl -i sacl}} +.{{EX: -i lacl -i masterDSA -i slaveDSA > ldif}} +. +.will convert the entire EDB hierarchy to LDIF format and +.write the result to a file named ldif. Some attributes that are +.not useful when running slapd are ignored. The EDB +.hierarchy is assumed to reside logically below the base DN +."c=US". +. + ++ Follow the steps outlined in section 8.2 above to produce +.an LDBM database from your new LDIF file. + + + +H2: The ldbmtest program + +Occasionally you may find it useful to look at the LDBM +database and index files directly (i.e., without going through +slapd). The {{EX: ldbmtest}} program is provided for this purpose. It +gives you raw access to the database itself. {{EX: ldbmtest}} should +be run line this: + +E: ldbmtest [-d ] [-f ] + +The default configuration file in the {{EX: ETCDIR}} is used if you +don't supply one. By default, ldbmtest operates on the last +database listed in the config file. You can specify an +alternate database, or see the current database with the +following commands. + +E: b specify an alternate backend database +E: B print out the current backend database + +The {{EX: b}} command will prompt you for the suffix associated with +the database you want. The database you select can be +viewed and modified using a set of two-letter commands. +The first letter selects the command function to perform. +Possible commands and their meanings are as follows. + +E: l lookup (do not follow indirection) +E: L lookup (follow indirection) +E: t traverse and print keys and data +E: T traverse and print keys only +E: x delete an index item +E: e edit an index item +E: a add an index item +E: c create an index file +E: i insert an entry into an index item + +The second letter indicates which index the command +applies to. The possible index selections are as follows. + +E: c id2children index +E: d dn2id index +E: e id2entry index +E: f arbitrary file name +E: i attribute index + +Each command may require additional arguments which +ldbmtest will prompt you for. + +To exit {{EX: ldbmtest}}, type {{EX: control-D}} or {{EX: control-C}}. + +Note that this is a very raw interface originally developed +when testing the database format. It is provided and +minimally documented here for interested parties, but it is not +meant to be used by the inexperienced. See the next section +for a brief description of the LDBM database format. + + + +H2: The LDBM database format + +In normal operation, it is not necessary for you to know much +about the LDBM database format. If you are going to use the +ldbmtest program to look at or alter the database, or if you +want a deeper understanding of how indexes are maintained, +some knowledge of how it works could be useful. This +section gives an overview of the database format and how +slapd makes use of it. + + + +H3: Overview + +The LDBM database works by assigning a compact +four-byte unique identifier to each entry in the database. It +uses this identifier to refer to entries in indexes. The +database consists of one main index file, called id2entry, +which maps from an entry's unique identifier (EID) to a text +representation of the entry itself. Other index files are +maintained, for each indexed attribute for example, that map +values people are likely to search on to lists of EIDs. + +Using this simple scheme, many LDAP queries can be +answered efficiently. For example, to answer a search for +entries with a surname of "Jensen", slapd would first consult +the surname attribute index, look up the value "Jensen" and +retrieve the corresponding list of EIDs. Next, slapd would +look up each EID in the id2entry index, retrieve the +corresponding entry, convert it from text to LDAP format, and +return it to the client. + +The following sections give a very brief overview of each +type of index and what it contains. For more detailed +information see the paper "An X.500 and LDAP Database: +Design and Implementation," available in postscript format +from + +{{CMD[jump="ftp://terminator.rs.itd.umich.edu/ldap/papers/xldbm.ps"]ftp://terminator.rs.itd.umich.edu/ldap/papers/xldbm.ps}} + + + +H3: Attribute index format + +The LDBM backend will maintain one index file for each +attribute it is asked to index. Several sets of keys must +coexist in this file (e.g., keys for equality and approximate +equality), so the keys are prefixed with a character to ensure +uniqueness. The prefixes are given in the table below + +E: = equality keys +E: ~ approximate equality keys +E: * substring equality keys +E: \ continuation keys + +Key values are also normalized (e.g., converted to upper +case for case ignore attributes). So, for example, to look up +the surname equality value in the example above using the +ldbmtest program, you would look up the value "{{EX: =JENSEN}}". + +Substring indexes are maintained by generating all possible +N-character substrings for a value (N is 3 by default). These +substrings are then stored in the attribute index, prefixed by +"*". Additional anchors of "^" and "$" are added at the +beginning and end of words. So, for example the surname of +Jensen would cause the following keys to be entered in the +index: {{EX: ^JE, JEN, ENS, NSE, SEN, EN$}}. + +Approximate values are handled in a similar way, with +phonetic codes being generated for each word in a value +and then stored in the index, prefixed by "~". + +Large blocks in the index are split into smaller ones. The +smaller blocks are accessed through a level of indirection +provided by the original block. They are stored in the index +using the continuation key prefix of "\". + + + +H3: Other indexes + +In addition to the {{EX: id2entry}} and attribute indexes, LDBM +maintains a number of other indexes, including the {{EX: dn2id}} +index and the {{EX: id2children}} index. These indexes provide the +mapping between a DN and the corresponding EID, and the +mapping between an EID and the EIDs of the corresponding +entry's children, respectively. + +The {{EX: dn2id}} index stores normalized DNs as keys. The data +stored is the corresponding EID. + +The {{EX: id2children}} index stores EIDs as keys. The data stored +is a list of EIDs, just as for the attribute indexes. + + +PB: + + + diff --git a/doc/guide/figure_1.gif b/doc/guide/figure_1.gif new file mode 100644 index 0000000000..9fdfaf71b7 Binary files /dev/null and b/doc/guide/figure_1.gif differ diff --git a/doc/guide/figure_2.gif b/doc/guide/figure_2.gif new file mode 100644 index 0000000000..d518bca919 Binary files /dev/null and b/doc/guide/figure_2.gif differ diff --git a/doc/guide/figure_3.gif b/doc/guide/figure_3.gif new file mode 100644 index 0000000000..c986d865e1 Binary files /dev/null and b/doc/guide/figure_3.gif differ diff --git a/doc/guide/figure_4.gif b/doc/guide/figure_4.gif new file mode 100644 index 0000000000..916a26eae3 Binary files /dev/null and b/doc/guide/figure_4.gif differ diff --git a/doc/guide/figure_5.gif b/doc/guide/figure_5.gif new file mode 100644 index 0000000000..d0b70bba48 Binary files /dev/null and b/doc/guide/figure_5.gif differ diff --git a/doc/guide/figure_6.gif b/doc/guide/figure_6.gif new file mode 100644 index 0000000000..70814033e5 Binary files /dev/null and b/doc/guide/figure_6.gif differ diff --git a/doc/guide/install.sdf b/doc/guide/install.sdf new file mode 100644 index 0000000000..22d7355d2f --- /dev/null +++ b/doc/guide/install.sdf @@ -0,0 +1,268 @@ +H1: Building and Installing slapd & slurpd + +Building and installing slapd requires three simple steps: configuring; +making; and installing. The following sections describe each step in +detail. If you are reading this guide, chances are you have already +obtained the software, but just in case, here's where you can get the +latest version of the OpenLDAP package, which includes all of the +software discussed in this guide: + +{{CMD[jump="ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release.tgz"]ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release.tgz}} + +There is also an OpenLDAP homepage accessible from the World +Wide Web. This page contains the latest OpenLDAP news, release +announcements, and pointers to other resources. You can access it +at: + +{{CMD[jump="http://www.OpenLDAP.org/"]http://www.OpenLDAP.org/}} + + +H2: Pre-Build Configuration + +Before building slapd, be sure to take a look at the README file in the +top level directory in the distribution so that you are familiar with the +general configuration and make process. + +Briefly, you should edit the include/ldapconfig.h.edit and +Make-common files to contain the site-specific configuration your site +requires before making. The next sections discuss these steps in +more detail. + + + +H3: Editing the {{EX: Make-common}} file + +All of the general Make-common configuration variables (e.g., +ETCDIR, BINDIR, etc.) apply to both slapd and slurpd. There are +additional Make-common configuration variables that also affect how +slapd and slurpd are built. They are: + +H4: MAKE_SLAPD + +This option controls whether slapd and slurpd get built at all. You +should set it to yes, like this: + +E: MAKE_SLAPD = yes + +H4: SLAPD_BACKENDS + +This option controls which slapd backend databases get built. You +should set it to one or more of the following: + +*{{EX: DLDAP_LDBM}} This is the main backend. It is a high-performance +disk-based database suitable for handling up to a million entries or so. +See the LDBMBACKEND and LDBMLIB options below. + +*{{EX: DLDAP_PASSWD}} This is a simple search-only backend that can be +pointed at an {{EX: /etc/passwd}} file. It is intended more as an example than +as a real backend. + +*{{EX: DLDAP_SHELL}} This backend allows the execution of arbitrary +system administrator-defined commands in response to LDAP +queries. The commands to execute are defined in the configuration file. +See Appendix B for more information on writing shell backend +programs. + +Example to enable the LDBM and SHELL backends only: + +E: SLAPD_BACKENDS= -DLDAP_LDBM -DLDAP_SHELL + +The default is to build all three backends. Note that building a backend +only means that it can be enabled through the configuration file, not +that it will automatically be enabled. + +H4: LDBMBACKEND + +This option should only be defined if you have enabled the LDBM +backend as described above. The LDBM backend relies on a +low-level hash or B-tree package for its underlying database. This +option selects which package it will use. The currently supported +options in order of preference are: + +*{{EX: DLDBM_USE_DBBTREE}} +. +.This option enables the Berkeley DB package btree database as the +.LDBM backend. You can get this package from +. +.{{CMD[jump="ftp://ftp.cs.berkeley.edu/ucb/4bsd/db.tar.Z"]ftp://ftp.cs.berkeley.edu/ucb/4bsd/db.tar.Z}} +. + +*{{EX: DLDBM_USE_DBHASH}} +. +.This option enables the Berkeley DB package hash database as the +.LDBM backend. You can get this package from +. +.{{CMD[jump="ftp://ftp.cs.berkeley.edu/ucb/4bsd/db.tar.Z"]ftp://ftp.cs.berkeley.edu/ucb/4bsd/db.tar.Z}} +. + +*{{EX: DLDBM_USE_GDBM}} +. +.This option enables GNU dbm as the LDBM backend. You can get this +.package from +. +.{{CMD[jump="ftp://prep.ai.mit.edu/pub/gnu/gdbm-1.7.3.tar.gz"]ftp://prep.ai.mit.edu/pub/gnu/gdbm-1.7.3.tar.gz}} +. + +*{{EX: DLDBM_USE_NDBM}} +. +.This option enables the standard UNIX ndbm(3) package as the +.LDBM backend. This package should come standard on your UNIX +.system. man ndbm for details. +. + +Example to enable the Berkeley DB Btree backend: + +E: LDBMBACKEND= -DLDBM_USE_DBBTREE + +The default is -DLDBM_USE_NDBM, since it is the only one available +on all UNIX systems. NDBM has some serious limitations, though (not +thread-safe, severe size limits), and you are strongly encouraged to +use one of the other packages if you can. + +Note[label='Note to Solaris users: '] If you are running under Solaris 2.x +and linking in an external database package (e.g., db or gdbm) it is +very important that you compile the package with the {{EX: D_REENTRANT}} +flag. If you do not, bad things will happen. + +If you are using version 1.85 or earlier of the Berkeley db package, you +will need to apply the patch found in build/db.1.85.patch to the db +source before compiling it. You can do this with a command like this +from the db source area: + +E: patch -p < ldap-source-directory/build/db.1.85.patch + +H4: LDBMLIB + +This option should only be defined if you have enabled the LDBM +backend as described above, and the necessary library for the +LDBMBACKEND option you chose above is not part of the standard C +library (i.e., anything other than NDBM). This option specifies the library +to link containing the package you selected, and optionally, its location. + +Example to link with {{EX: libdb.a}}, contained in {{EX: /usr/local/lib}}: + +E: LDBMLIB= -L/usr/local/lib -ldb + +H4: THREADS + +This option is normally set automatically in the {{EX: Make-platform}} file, +based on the platform on which you are building. You do not normally +need to set it. If you want to use a non-default threads package, you +can specify the appropriate {{EX: -Ddefine}} to enable it here. + +H4: THREADSLIB + +This option is normally set automatically in the {{EX: Make-platform}} file, +based on the platform on which you are building. You do not normally +need to set it. If you have set {{EX: THREADS}} to a non-default threads +package as described above, you can specify the appropriate + {{EX: -Ldirectory}} flag and {{EX: -llibname}} flag needed to link the package here. + +H4: PHONETIC + +This option controls the phonetic algorithm used by {{I: slapd}} when doing +approximate searches. The default is to use the metaphone algorithm. +You can have {{I: slapd}} use the soundex algorithm by setting this variable +to {{EX: -DSOUNDEX}}. + + +H3: Editing the {{EX: include/ldapconfig.h}} file + +In addition to setting the {{EX: LDAPHOST}} and {{EX: DEFAULT_BASE}} defines +near the top of this file, there are some slapd-specific defines near the +bottom of the file you may want to change. The defaults should be just +fine, unless you have special needs. + +H4: SLAPD_DEFAULT_CONFIGFILE + +This define sets the location of the default slapd configuration file. +Normally, it is set to {{EX: $(ETCDIR)/slapd.conf}}, where +{{EX: ETCDIR}} comes from Make-common. + +H4: SLAPD_DEFAULT_SIZELIMIT + +This define sets the default size limit on the number of entries returned +from a search. This option is configurable via the tailor file, but if you +want to change the default, do it here. + +H4: SLAPD_DEFAULT_TIMELIMIT + +This define sets the default time limit for a search. This option is +configurable via the tailor file, but if you want to change the default, do it +here. + +H4: SLAPD_PIDFILE + +This define sets the location of the file to which slapd will write its +process ID when it starts up. + +H4: SLAPD_ARGSFILE + +This define sets the location of the file to which slapd will write its +argument vector when it starts up. + +H4: SLAPD_MONITOR_DN + +This define sets the distinguished name used to retrieve monitoring +information from {{I: slapd}}. See section 7 for more details. + +H4: SLAPD_LDBM_MIN_MAXIDS + +This define is only relevant to the LDBM backend. It sets the minimum +number of entry IDs that an index entry will contain before it becomes +an allIDs entry. See Section 9.1 for more details. + + + +H2: Making the Software + +Once you have edited the {{EX: include/ldapconfig.h.edit}} file and the +Make-common file (see the top level {{EX: README}} file in the distribution), +you are ready to make the software. From the top level LDAP source +directory, type + +E: make + +You should examine the output of this command carefully to make sure +everything is built correctly. Note that this command builds the LDAP +libraries and associated clients as well as slapd and slurpd. + +Note that the LDAP distribution can support making for multiple +platforms from a single source tree. If you want to do this, consult the +{{EX: INSTALL}} file in the top level distribution directory. + + + +H2: Installing the Software + +Once the software has been properly configured and successfully +made, you are ready to install it. You will need to have write permission +to the installation directories you specified in the {{EX: Make-common}} file. +Typically, the installation is done as root. From the top level LDAP +source directory, type + +E: make install + +You should examine the output of this command carefully to make sure +everything is installed correctly. Slapd, slurpd, and their configuration +files, {{EX: slapd.conf}}, {{EX: slapd.at.conf}}, and {{EX: slapd.oc.conf}} + will be installed in the {{EX: ETCDIR}} directory you specified +in the {{EX: Make-common}} file. + +This command will install the entire LDAP distribution. If you only want +to install slapd and slurpd, you could do something like this: + +E: (cd servers/slapd; make install) + +E: (cd servers/slurpd; make install) + +Note: The installation process installs configuration files as well as +binaries. Existing configuration files are first moved to a name with a +dash '-' appended, e.g., {{EX: slapd.conf}} is moved to {{EX: slapd.conf-}}. +If you install things twice, however, you can lose your existing configuration +files. + + +PB: + + diff --git a/doc/guide/intro.sdf b/doc/guide/intro.sdf new file mode 100644 index 0000000000..e9d47a815c --- /dev/null +++ b/doc/guide/intro.sdf @@ -0,0 +1,227 @@ +H1: Introduction to slapd and slurpd + +This document describes how to build, configure, and run the stand-alone +LDAP daemon ({{I:slapd}}) and the stand-alone LDAP update replication +daemon ({{I:slurpd}}). It is intended for newcomers and experienced +administrators alike. This section provides a basic introduction to directory +service, and the directory service provided by {{I:slapd}} in particular. + + + +H2: What is a directory service? + +A directory is like a database, but tends to contain more descriptive, +attribute-based information. The information in a directory is generally read +much more often than it is written. As a consequence, directories don't +usually implement the complicated transaction or roll-back schemes regular +databases use for doing high-volume complex updates. Directory updates +are typically simple all-or-nothing changes, if they are allowed at all. +Directories are tuned to give quick-response to high-volume lookup or +search operations. They may have the ability to replicate information widely in +order to increase availability and reliability, while reducing response time. +When directory information is replicated, temporary inconsistencies between +the replicas may be OK, as long as they get in sync eventually. + +There are many different ways to provide a directory service. Different +methods allow different kinds of information to be stored in the directory, +place different requirements on how that information can be referenced, +queried and updated, how it is protected from unauthorized access, etc. +Some directory services are {{I:local}}, providing service to a restricted +context (e.g., the finger service on a single machine). Other services are +global, providing service to a much broader context (e.g., the entire Internet). +Global services are usually {{I:distributed}}, meaning that the data they contain +is spread across many machines, all of which cooperate to provide the +directory service. Typically a global service defines a uniform {{I:namespace}} +which gives the same view of the data no matter where you are in relation to +the data itself. + + + +H2: What is LDAP? + +{{I:Slapd}}'s model for directory service is based on a global directory model +called LDAP, which stands for the Lightweight Directory Access Protocol. +LDAP is a directory service protocol that runs over TCP/IP. The nitty-gritty +details of LDAP are defined in RFC 1777 "The Lightweight Directory Access +Protocol." This section gives an overview of LDAP from a user's perspective. + +{{I:What kind of information can be stored in the directory?}} The LDAP directory +service model is based on {{I:entries}}. An entry is a collection of +attributes that has a name, called a {{I:distinguished name}} (DN). +The DN is used to refer to the entry unambiguously. Each of the +entry's attributes has a {{I:type}} and one or +more {{I:values}}. +The types are typically mnemonic strings, like "{{EX:cn}}" for common +name, or "{{EX:mail}}" for email address. The values depend on what type of +attribute it is. For example, a {{EX:mail}} attribute might contain the value +"{{EX:babs@umich.edu}}". A {{EX:jpegPhoto}} attribute would contain +a photograph in binary JPEG/JFIF format. + +{{I:How is the information arranged?}} In LDAP, directory entries are arranged in +a hierarchical tree-like structure that reflects political, geographic and/or +organizational boundaries. Entries representing countries appear at the top +of the tree. Below them are entries representing states or national +organizations. Below them might be entries representing people, +organizational units, printers, documents, or just about anything else you can +think of. Figure 1 shows an example LDAP directory tree, which should help +make things clear. + + +!import "figure_1.gif"; align="center"; title="An example LDAP directory tree" +FT: Figure 1: An example LDAP directory tree. + + +In addition, LDAP allows you to control which attributes are required and +allowed in an entry through the use of a special attribute called +{{I:objectclass}}. +The values of the {{I:objectclass}} attribute determine +the {{I:schema}} rules the entry +must obey. + +{{I:How is the information referenced?}} An entry is referenced by its +distinguished name, which is constructed by taking the name of the entry +itself (called the relative distinguished name, or RDN) and concatenating the +names of its ancestor entries. For example, the entry for Barbara Jensen in +the example above has an RDN of "{{EX:cn=Barbara J Jensen}}" and a DN of +"{{EX:cn=Barbara J Jensen, o=U of M, c=US}}". The full DN format is described in +RFC 1779, "A String Representation of Distinguished Names." + +{{I:How is the information accessed?}} LDAP defines operations for interrogating +and updating the directory. Operations are provided for adding and deleting +an entry from the directory, changing an existing entry, and changing the +name of an entry. Most of the time, though, LDAP is used to search for +information in the directory. The LDAP search operation allows some portion +of the directory to be searched for entries that match some criteria specified +by a search filter. Information can be requested from each entry that matches +the criteria. + +For example, you might want to search the entire directory subtree below the +University of Michigan for people with the name Barbara Jensen, retrieving +the email address of each entry found. LDAP lets you do this easily. Or you +might want to search the entries directly below the c=US entry for +organizations with the string "Acme" in their name, and that have a fax +number. LDAP lets you do this too. The next section describes in more detail +what you can do with LDAP and how it might be useful to you. + +{{I:How is the information protected from unauthorized access?}} Some directory +services provide no protection, allowing anyone to see the information. LDAP +provides a method for a client to authenticate, or prove its identity to a +directory server, paving the way for rich access control to protect the +information the server contains. + + + +H2: How does LDAP work? + +LDAP directory service is based on a {{I:client-server}} model. One or more +LDAP servers contain the data making up the LDAP directory tree. An LDAP +client connects to an LDAP server and asks it a question. The server +responds with the answer, or with a pointer to where the client can get more +information (typically, another LDAP server). No matter which LDAP server a +client connects to, it sees the same view of the directory; a name presented +to one LDAP server references the same entry it would at another LDAP +server. This is an important feature of a global directory service, like LDAP. + + + + +H2: What is slapd and what can it do? + +{{I:Slapd}} is an LDAP directory server that runs on many different UNIX +platforms. You can use it to provide a directory service of your very own. +Your directory can contain pretty much anything you want to put in it. You +can connect it to the global LDAP directory service, or run a service all by +yourself. Some of slapd's more interesting features and capabilities include: + +{{B:Choice of databases}}: {{I:Slapd}} comes with three different backend +databases you can choose from. They are LDBM, a high-performance disk-based +database; SHELL, a database interface to arbitrary UNIX commands or shell +scripts; and PASSWD, a simple password file database. + +{{B:Multiple database instances}}: {{I:Slapd}} can be configured to serve +multiple databases at the same time. This means that a single {{I:slapd}} +server can respond to requests for many logically different portions +of the LDAP tree, using the same or different backend databases. + +{{B:Generic database API}}: If you require even more customization, {{I:slapd}} +lets you write your own backend database easily. {{I:Slapd}} +consists of two distinct parts: a front end that handles protocol +communication with LDAP clients; and a backend that handles database +operations. Because these two pieces communicate via a well-defined +C API, you can write your own customized database backend to {{I:slapd}}. + +{{B:Access control}}: {{I:Slapd}} provides a rich and powerful access +control facility, allowing you to control access to the information +in your database(s). You can control access to entries based on +LDAP authentication information, IP address, domain name and other criteria. + +{{B:Threads}}: {{I:Slapd}} is threaded for high performance. A +single multi-threaded {{I:slapd}} process handles all incoming +requests, reducing the amount of system overhead required. {{I:Slapd}} +will automatically select the best thread support for your platform. + +{{B:Replication}}: {{I:Slapd}} can be configured to maintain replica +copies of its database. This master/slave replication scheme is +vital in high-volume environments where a single {{I:slapd}} just +doesn't provide the necessary availability or reliability. + +{{B:Configuration}}: {{I:Slapd}} is highly configurable through a +single configuration file which allows you to change just about +everything you'd ever want to change. Configuration options have +reasonable defaults, making your job much easier. + +{{I:Slapd}} also has its limitations, of course. It does not +currently handle aliases, which are part of the LDAP model. The +main LDBM database backend does not handle range queries or negation +queries very well. These features and more will be coming in a future release. + + + +H2: What about X.500? + +LDAP was originally developed as a front end to X.500, the OSI directory +service. X.500 defines the Directory Access Protocol (DAP) for clients to +use when contacting directory servers. DAP is a heavyweight protocol that +runs over a full OSI stack and requires a significant amount of computing +resources to run. LDAP runs directly over TCP and provides most of the +functionality of DAP at a much lower cost. + +This use of LDAP makes it easy to access the X.500 directory, but still +requires a full X.500 service to make data available to the many LDAP clients +being developed. As with full X.500 DAP clients, a full X.500 server is no +small piece of software to run. + +The stand-alone LDAP daemon, or {{I:slapd}}, is meant to remove much of the +burden from the server side just as LDAP itself removed much of the burden +from clients. If you are already running an X.500 service and you want to +continue to do so, you can probably stop reading this guide, which is all +about running LDAP via {{I:slapd}}, without running X.500. If you are not running +X.500, want to stop running X.500, or have no immediate plans to run X.500, +read on. + +It is possible to replicate data from a {{I:slapd}} directory +server to an X.500 DSA, which allows your organization to make your +data available as part of the global X.500 directory service on a +"read-only" basis. This is discussed in section 11.6. + +Another way to make data in a {{I:slapd}} server available to the X.500 +community would be by using a X.500 DAP to LDAP gateway. At this time, no +such software has been written (to the best of our knowledge), but hopefully +some group will see fit towrite such a gateway. + + + +H2: What is slurpd and what can it do? + +{{I:Slurpd}} is a UNIX daemon that helps {{I:slapd}} provide +replicated service. It is responsible for distributing changes made +to the master {{I:slapd}} database out to the various {{I:slapd}} +replicas. It frees {{I:slapd}} from having to worry that some +replicas might be down or unreachable when a change comes through; +{{I:slurpd}} handles retrying failed requests automatically. +{{I:Slapd}} and {{I:slurpd}} communicate through a simple text +file that is used to log changes. + +PB: + + diff --git a/doc/guide/master.sdf b/doc/guide/master.sdf new file mode 100644 index 0000000000..88aa6e97ae --- /dev/null +++ b/doc/guide/master.sdf @@ -0,0 +1,20 @@ +# master.sdf +# SDF master file for the OpenLDAP Administrator's guide +# + +!include "preamble.sdf" + +!include "intro.sdf" +!include "quickstart.sdf" +!include "config.sdf" +!include "install.sdf" +!include "slapdconfig.sdf" +!include "runningslapd.sdf" +!include "monitoringslapd.sdf" +!include "dbtools.sdf" +!include "tuning.sdf" +!include "referrals.sdf" +!include "replication.sdf" + + + diff --git a/doc/guide/monitoringslapd.sdf b/doc/guide/monitoringslapd.sdf new file mode 100644 index 0000000000..884ff4831e --- /dev/null +++ b/doc/guide/monitoringslapd.sdf @@ -0,0 +1,112 @@ +H1: Monitoring Slapd + +Slapd supports a monitoring interface you can use to find out +many useful bits of information about what slapd is currently +doing, how many connections it has, how many threads are +working, etc. You can access the monitor feature by doing a +base object search of the SLAPD_MONITOR_DN from +include/ldapconfig.h with any kind of valid filter (e.g., +"(objectclass=*)"). By default, this DN is set to "cn=monitor". +You will get one entry returned to you, with the following +attributes: + +E: version: slapd () + +This attribute identifies the slapd server software by name, +version, and build date, e.g., {{EX: slapd 3.3 (Thu May 21 14:19:03 +EDT 1996)}} + +E: threads: + +This attribute indicates the number of threads (operations) +currently outstanding in slapd. + +E: connection: : : : +E: : : [ ] + +This multi-valued attribute summarizes information for each +open connection. The information given is {{EX: }}, the file +descriptor; {{EX: }}, the time the connection was opened +in UTC format; {{EX: }}, the number of operations +initiated over the connection; {{EX: }}, the number +of operations completed over the connection; {{EX: }}, the +DN currently bound to the connection; and optionally {{EX: }}, +indicating whether the connection is currently blocked for +read or write.. + +E: currentconnections: + +The current number of connections. + +E: totalconnections: + +The total number of connections handled by slapd since it +started. + +E: dtablesize: + +The size of slapd's file descriptor table. + +E: writewaiters: + +The number of threads blocked waiting to write data to a +client. + +E: readwaiters: + +The number of threads blocked waiting to read data from a +client. + +E: opsinitiated: + +The total number of operations initiated by slapd since it +started. + +E: opscompleted: + +The total number of operations completed by slapd since it +started. + +E: entriessent: + +The total number of entries sent to clients by slapd since it +started. + +E: bytessent: + +The total number of bytes sent to clients by slapd since it +started. + +E: currenttime: + +Slapd's idea of the current time. + +E: starttime: + +The time slapd was started. + +E: nbackends: + +The number of backends currently being served by slapd. + +E: concurrency: + +Under Solaris 2.x only, an indication of the current level of +thread concurrency. + +Note that slapd takes a snapshot of this information and +returns it to you. No attempt is made to ensure that the +information is consistent (i.e., if an operation thread is +modifying one of these things when the monitor thread is +reading it, strange results could be returned). + +You should be able to use any LDAP client to retrieve this +information. Here's how you might do it using the +{{I: ldapsearch}}(1) client: + +E: ldapsearch -s base -b cn=monitor 'objectclass=*' + +PB: + + + diff --git a/doc/guide/preamble.sdf b/doc/guide/preamble.sdf new file mode 100644 index 0000000000..2ccda413db --- /dev/null +++ b/doc/guide/preamble.sdf @@ -0,0 +1,11 @@ +# file: guide.sdf +# + +!define DOC_NAME "OpenLDAP Administrators Guide" +!define DOC_AUTHOR "OpenLDAP" +!define DOC_LOGO "LDAPwww.gif" +!define DOC_LOF_TITLE "Figures" +!define DOC_LOF +!build_title + + diff --git a/doc/guide/quickstart.sdf b/doc/guide/quickstart.sdf new file mode 100644 index 0000000000..4d97407a73 --- /dev/null +++ b/doc/guide/quickstart.sdf @@ -0,0 +1,123 @@ + +H1: A Quick-Start Guide to Running slapd + +This section provides a quick step-by-step guide to building, +installing and running {{I:slapd}}. It is intended to provide users with a +simple and quick way to get started only. If you intend to run slapd +seriously, you should read the rest of this guide. + + +^ {{B:Get the software}}. {{I:Slapd}} is part of the OpenLDAP distribution, which +you can retrieve using this URL: +. +.{{CMD[jump="ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release.tgz"]ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release.tgz}} +. +.If you are reading this guide, you have probably already done this. +. + ++ {{B:Untar the distribution}}. Pick a place for the LDAP source to live, cd +there, and untar it. For example: +. +.{{EX: cd /usr/local/src}} +.{{EX: gunzip -c openldap-release.tgz | tar xvfB -}} +.{{EX: cd ldap}} ++{{B: Configure the software}}. +. +.You will have to edit two files to configure things for your site. +. +.{{EX: vi Make-common}} +.{{EX:vi include/ldapconfig.h.edit}} +. +.Read the comments in Make-common and configure things +.appropriately. If you have the Berkeley DB package installed, or the +.GDBM package, you should set the LDBMBACKEND variable +.accordingly. Otherwise, the defaults should be OK to get you started. +. +.In the include/ldapconfig.h.edit file, be sure to set the DEFAULT_BASE +.and LDAPHOST variables to something appropriate for your site. +.Other than that, the defaults should work OK. + + ++{{B:Install the software}}. From the top level LDAP source directory, +type: +. +.{{EX: su}} +.{{EX: make install}} +. +.Examine the output of this command carefully to ensure everything is +.installed properly. +. + + ++{{B:Make a configuration file}}. Create a file called myslapd.conf and +.enter the following lines into it. See Section 5 for more details on this +.file. +. +.{{EX:referral ldap://ldap.itd.umich.edu}} +.{{EX:database ldbm}} +.{{EX:suffix "o=, c=US"}} +.{{EX:rootdn "cn=, o=, c=US"}} +.{{EX:rootpw secret}} +. +.Be sure to replace "" with the name of your +.organization and "" with your name. If you are not in +.the US, replace "US" with your two-letter country code. The rootdn +.and rootpw lines are only required if later you want to easily add or +.modify entries via LDAP. +. + + ++{{B:Create a database}}. This is a two-step process. Step A is to create +.a file (we'll call it myldif) containing the entries you want your database +.to contain. Use the following example as a guide, or see Section 7.3 for +.more details. +. +.{{EX:dn: o=, c=US}} +.{{EX:o: }} +.{{EX:objectclass: organization}} +. +.{{EX:dn: cn=, o=, c=US}} +.{{EX:cn: }} +.{{EX:sn: }} +.{{EX:mail: }} +.{{EX:objectclass: person}} +. +.You can include additional entries and attributes in this file if you want, +.or add them later via LDAP. +. +.Step B is to run this file through a tool to create the slapd database. +. +.{{EX:$(ETCDIR)/ldif2ldbm -f myslapd.conf -i myldif}} +. +.Where myslapd.conf is the configuration file you made in step 6, and +.myldif is the file you made in step 7A above. By default, the database +.files will be created in /usr/tmp. You may specify an alternate directory +.via the directory option in the slapd.conf file. +. + + ++{{B:See if it works}}.You can use any LDAP client to do this, but our +.example uses the ldapsearch tool. +. +.{{EX:ldapsearch -h 127.0.0.1 'objectclass=*'}} +. +.This command will search for and retrieve every entry in the database. +.Note the use of single quotes around the filter, which prevents the "*" +.from being interpreted by the shell. +. +.You are now ready to add more entries (e.g., using {{I:ldapadd}}(3) or +.another LDAP client), experiment with various configuration options, +.backend arrangements, etc. Note that by default, the {{I:slapd}} database +.grants {{EX:READ}} access to everybody. So if you want to add or modify +.entries over LDAP, you will have to bind as the rootdn specified in the +.config file (see Section 5.2.2), or change the default access control +.(see Section 5.3). +. + + +The following sections provide more detailed information on making, +installing, and running slapd. + +PB: + + diff --git a/doc/guide/referrals.sdf b/doc/guide/referrals.sdf new file mode 100644 index 0000000000..00477e9c37 --- /dev/null +++ b/doc/guide/referrals.sdf @@ -0,0 +1,41 @@ +H1: Distributing {{I: slapd}} DATA + +For many sites, running one or more {{I: slapds}} that hold an +entire subtree of data is sufficient. But sometimes it may be +desirable to have one slapd refer to other {{I: slapds}} for a +certain part of the tree. This can be accomplished by +creating a referral entry in one {{I:slapd}}'s database pointing +to another {{I: slapd}}. For those familiar with X.500, a {{I:slapd}} +{{I: referral}} entry is similar to an X.500 knowledge reference. + +The referral entry acts as a mount point, glueing two slapd +databases together. A referral entry has an {{I: objectclass}} of +"referral" and is named by a {{I: ref}} attribute containing a URL +pointing to the slapd holding the data below the mount +point. This mechanism is very general and allows slapd +databases that are not normally hierarchical to be grafted +together. + +An example should help illustrate things. Suppose your +company is running a slapd and just purchased a new +company, also running a slapd. You can easily connect +the two databases by creating an entry like this in your +slapd's database. + +E: dn: ref="ldap://new.host/o=New Company,c=US", o=Your + +company, c=US + +E: objectclass: referral + +Now any subtree search that has this entry in its scope +will return a referral to the new company, in addition to any +entries matched in your database. Referral-aware clients +will continue the search at the new company's server. + +A mechanism similar to this is used to support distributed +indexing, described in Appendix C. + + +PB: + diff --git a/doc/guide/replication.sdf b/doc/guide/replication.sdf new file mode 100644 index 0000000000..b25b89e5a6 --- /dev/null +++ b/doc/guide/replication.sdf @@ -0,0 +1,407 @@ +H1: Replication with slurpd + +In certain configurations, a single slapd instance may be +insufficient to handle the number of clients requiring +directory service via LDAP. It may become necessary to +run more than one slapd instance. At the University of +Michigan, for instance, there are four slapd servers, one +master and three slaves. A DNS lookup of the name +ldap.itd.umich.edu returns the IP addresses of those four +servers, distributing the load among them. This +master/slave arrangement provides a simple and effective +way to increase capacity, availability and reliability. + +Slurpd provides the capability for a master slapd to +propagate changes to slave slapd instances, +implementing the master/slave replication scheme +described above. Slurpd runs on the same host as the +master slapd instance. + + + +H2: Overview + +Slurpd provides replication services "in band". That is, it +uses the LDAP protocol to update a slave database from +the master. Perhaps the easiest way to illustrate this is +with an example. In this example, we trace the propagation +of an LDAP modify operation from its initiation by the LDAP +client to its distribution to the slave slapd instance. + + +{{B: Sample replication scenario:}} + +* Step 1: An LDAP client starts up and connects to a slave +slapd. + +* Step 2: The LDAP client submits an LDAP modify +. operation to the slave slapd. + +* Step 3: The slave slapd returns a referral to the LDAP +. client, which causes the client to send the modify +. operation to the master slapd. + +* Step 4: The master slapd performs the modify operation, +. writes out the change to its replication log file and returns +. a success code to the client. + +* Step 5: The slurpd process notices that a new entry has +. been appended to the replication log file, reads the +. replication log entry, and sends the change to the slave +. slapd via LDAP. + +* Step 6: The slave slapd performs the modify operation and +. returns a success code to the slurpd process. + +Note: if the LDAP client happened to connect to the +master slapd to begin with, Step 3 is omitted, but the rest +of the scenario remains the same. + + + +H2: Replication Logs + +When slapd is configured to generate a replication logfile, +it writes out a file in a format which is a variant of the LDIF +format. The replication log gives the replication site(s), a +timestamp, the DN of the entry being modified, and a series +of lines which specify the changes to make. In the +example below, "Barbara Jensen" has replaced a line of +her multiLineDescription. The change is to be propagated +to the slapd instance running on truelies.rs.itd.umich.edu. +The lastModifiedBy and lastModified Time attributes are +also propagated to the slave slapd. + +E: replica: truelies.rs.itd.umich.edu:389 +E: time: 809618633 +E: dn: cn=Barbara Jensen, ou=People, o=University of Michigan,c=US +E: changetype: modify +E: delete: multiLineDescription +E: multiLineDescription: I enjoy sailing in my spare time +E: - +E: add: multiLineDescription +E: multiLineDescription: A dreamer... +E: - +E: delete: lastModifiedBy +E: - +E: add: lastModifiedBy +E: lastModifiedBy: cn=Barbara Jensen, ou=People, o=University of Michigan, c=US +E: - +E: delete: lastModifiedTime +E: - +E: add: lastModifiedTime +E: lastModifiedTime: 950825073308Z +E: - + +The modifications to {{EX: lastModifiedBy}} and {{EX: lastModifiedTime}} +were initiated by the master {{I: slapd}}. + + + +H2: Command-Line Options + +Slurpd supports the following command-line options. + +E: -d | ? + +This option sets the slurpd debug level to {{EX: }}. When +level is a `?' character, the various debugging levels are +printed and slapd exits, regardless of any other options +you give it. Current debugging levels (a subset of slapd's +debugging levels) are + + +E: 4 heavy trace debugging +E: 64 configuration file processing +E: 65535 enable all debugging + +Debugging levels are additive. That is, if you want heavy +trace debugging and want to watch the config file being +processed, you would set level to the sum of those two +levels (in this case, 68). + +E: -f + +This option specifies an alternate slapd configuration file. +Slurpd does not have its own configuration file. Instead, all +configuration information is read from the slapd +configuration file. + +E: -r + +This option specifies an alternate slapd replication log file. +Under normal circumstances, slurpd reads the name of +the slapd replication log file from the slapd configuration +file. However, you can override this with the -r flag, to +cause slurpd to process a different replication log file. See +section 10.5, Advanced slurpd Operation, for a discussion +of how you might use this option. + +E: -o + +Operate in "one-shot" mode. Under normal +circumstances, when slurpd finishes processing a +replication log, it remains active and periodically checks to +see if new entries have been added to the replication log. +In one-shot mode, by comparison, slurpd processes a +replication log and exits immediately. If the -o option is +given, the replication log file must be explicitly specified +with the -r option + +E: -t + +Specify an alternate directory for slurpd's temporary +copies of replication logs. The default location is /usr/tmp. + +E: -k + +When slurpd uses kerberos to authenticate to slave slapd +instances, it needs to have an appropriate srvtab file for +the remote slapd. This option allows you to specify an +alternate filename containing kerberos keys for the remote +slapd. The default filename is /etc/srvtab. You can also +specify the srvtab file to use in the slapd configuration +file's replica option. See the documentation on the srvtab +directive in section 5.2.2, General Backend Options. A +more complete discussion of using kerberos with slapd +and slurpd may be found in Appendix D. + + + +H2: Configuring slurpd and a slave slapd instance + +To bring up a replica slapd instance, you must configure +the master and slave slapd instances for replication, then +shut down the master slapd so you can copy the +database. Finally, you bring up the master slapd instance, +the slave slapd instance, and the slurpd instance. These +steps are detailed in the following sections. You can set +up as many slave slapd instances as you wish. + + +H3: Set up the master slapd + +Follow the procedures in Section 4, Building and Installing +slapd. Be sure that the slapd instance is working properly +before proceeding. Be sure to do the following in the +master slapd configuration file. + +^ Add a replica directive for each replica. The binddn= +. parameter should match the updatedn option in the +. corresponding slave slapd configuration file, and should +. name an entry with write permission to the slave database +. (e.g., an entry listed as rootdn, or allowed access via +. access directives in the slave slapd configuration file). + ++ Add a replogfile directive, which tells slapd where to log +. changes. This file will be read by slurpd. + + + +H3: Set up the slave slapd + +Install the slapd software on the host which is to be the +slave slapd server. The configuration of the slave server +should be identical to that of the master, with the following +exceptions: + +^ Do not include a replica directive. While it is possible to +. create "chains" of replicas, in most cases this is +. inappropriate. + ++ Do not include a replogfile directive. + ++ Do include an updatedn line. The DN given should +. match the DN given in the {{EX: binddn=}} parameter of the +. corresponding {{EX: replica=}} directive in the master slapd +. config file. + ++ Make sure the DN given in the {{EX: updatedn}} directive has +. permission to write the database (e.g., it is listed as rootdn +. or is allowed access by one or more access directives). + + + +H3: Shut down the master slapd + +In order to ensure that the slave starts with an exact copy +of the master's data, you must shut down the master +slapd. Do this by sending the master slapd process an +interrupt signal with {{EX: kill -TERM }}, where {{EX: }} is the +process-id of the master slapd process. + +If you like, you may restart the master slapd in read-only +mode while you are replicating the database. During this +time, the master slapd will return an "unwilling to perform" +error to clients that attempt to modify data. + + + +H3: Copy the master slapd's database to the slave + +Copy the master's database(s) to the slave. For an +LDBM-based database, you must copy all index files as +well as the "NEXTID" file. Index files will have a different +suffix depending on the underlying database package +used. The current possibilities are + +* {{EX: dbb}} Berkeley DB B-tree backend +* {{EX: dbh}} Berkeley DB hash backend +* {{EX: gdbm}} GNU DBM backend +* {{EX: pag}} UNIX NBDM backend +* {{EX: dir}} UNIX NBDM backend + +You should copy all files with such a suffix that are located +in the index directory specified in your slapd config file. + + + +H3: Configure the master slapd for replication + +To configure slapd to generate a replication logfile, you +add a "{{EX: replica}}" configuration option to the master slapd's +config file. For example, if we wish to propagate changes +to the slapd instance running on host +truelies.rs.itd.umich.edu: + +E: replica host=truelies.rs.itd.umich.edu:389 +E: binddn="cn=Replicator, o=U of M, c=US" +E: bindmethod=simple credentials=secret + +In this example, changes will be sent to port 389 (the +standard LDAP port) on host truelies. The slurpd process +will bind to the slave slapd as +"cn=Replicator, o=U of M, c=US" +using simple authentication with password "secret". +Note that the entry given by the binddn= directive must +exist in the slave slapd's database (or be the rootdn +specified in the slapd config file) in order for the bind +operation to succeed. + + + +H3: Restart the master slapd and start the slave slapd + +Restart the master slapd process. To check that it is +generating replication logs, perform a modification of any +entry in the database, and check that data has been +written to the log file. + + + +H3: Start slurpd + +Start the slurpd process. Slurpd should immediately send +the test modification you made to the slave slapd. Watch +the slave slapd's logfile to be sure that the modification +was sent. + +{{EX: slurpd -f }} + + + +H2: Advanced slurpd Operation + +H3: Replication errors + +When slurpd propagates a change to a slave slapd and +receives an error return code, it writes the reason for the +error and the replication record to a reject file. The reject +file is located in the same directory with the per-replica +replication logfile, and has the same name, but with the +string ".rej" appended. For example, for a replica running +on host truelies.rs.itd.umich.edu, port 389, the reject file, if it +exists, will be named + +E: /usr/tmp/truelies.rs.itd.umich.edu:389. + +A sample rejection log entry follows: + +E: ERROR: No such attribute +E: replica: truelies.rs.itd.umich.edu:389 +E: time: 809618633 +E: dn: cn=Barbara Jensen, ou=People, o=University of Michigan, c=US +E: changetype: modify +E: delete: multiLineDescription +E: multiLineDescription: I enjoy sailing in my spare time +E: - +E: add: multiLineDescription +E: multiLineDescription: A dreamer... +E: - +E: delete: lastModifiedBy +E: - +E: add: lastModifiedBy +E: lastModifiedBy: cn=Barbara Jensen, ou=People, o=University of Michigan, c=US +E: - +E: delete: lastModifiedTime +E: - +E: add: lastModifiedTime +E: lastModifiedTime: 950825073308Z +E: - + +Note that this is precisely the same format as the original +replication log entry, but with an ERROR line prepended to +the entry. + + + +H3: {{I:Slurpd}}'s one-shot mode and reject files + +It is possible to use slurpd to process a rejection log with +its "one-shot mode." In normal operation, slurpd watches +for more replication records to be appended to the +replication log file. In one-shot mode, by contrast, slurpd +processes a single log file and exits. Slurpd ignores +ERROR lines at the beginning of replication log entries, so +it's not necessary to edit them out before feeding it the +rejection log. + +To use one-shot mode, specify the name of the rejection +log on the command line as the argument to the -r flag, +and specify one-shot mode with the -o flag. For example, +to process the rejection log file +/usr/tmp/replog.truelies.rs.itd.umich.edu:389 and exit, use the +command + +E: slurpd -r /usr/tmp/truelies.rs.itd.umich.edu:389 -o + + +H2: Replication from a slapd directory server to an X.500 DSA + +In mixed environments where both X.500 DSAs and slapd +are used, it may be desirable to replicate changes from a +slapd directory server to an X.500 DSA. This section +discusses issues involved with this method of replication, +and describes the currently-available facilities. + +To propagate changes from a slapd directory server to an +X.500 DSA, slurpd runs on the master slapd host, and +sends changes to an ldapd which acts as a gateway to +the X.500 DSA: + +!import "figure_1.gif"; align="center"; title="Replication from slapd to an X.500 DSA" +FT: Figure 6: Replication from slapd to an X.500 DSA + +Note that the X.500 DSA must be a read-only copy. Since +the replication is one-way, updates from DAP clients +connecting to the X.500 DSA simply cannot be handled. + +A problem arises where attribute names differ between the +slapd directory server and the X.500 DSA. At present, +slapd and slurpd do not support selective replication of +attributes, nor do they support translation of attribute +names and values. For example, slurpd will attempt to +update the "modifiersName" and "modifyTimeStamp" +attributes on the slave it connects to. However, the X.500 +DSA may expect these attributes to be named +"lastModifiedBy" and "lastModifiedTime". + +A solution to this attribute naming problem is to have the +ldapd read oidtables that map "modifiersName" to the +objectID (OID) for the "lastModifiedBy" attribute and +"modifyTimeStamp" to the OID for the "lastModifiedTime" +attribute. Since attribute names are carried as OIDs over +DAP, this should perform the appropriate translation of +attribute names. + + diff --git a/doc/guide/runningslapd.sdf b/doc/guide/runningslapd.sdf new file mode 100644 index 0000000000..f591f92d72 --- /dev/null +++ b/doc/guide/runningslapd.sdf @@ -0,0 +1,123 @@ +H1: Running slapd + +Slapd can be run in two different modes, stand-alone or from +inetd(8). Stand-alone operation is recommended, especially if you +are using the LDBM backend. This allows the backend to take +advantage of caching and avoids concurrency problems with the +LDBM index files. If you are running only a PASSWD or SHELL +backend, running from inetd is an option. How to do this is +described in the next section, after the command-line options and +stand-alone daemon operation are described. + + + +H2: Command-Line Options + +{{I:Slapd}} supports the following command-line options. + +E: -d | ? + +This option sets the slapd debug level to . When level is a +`?' character, the various debugging levels are printed and slapd +exits, regardless of any other options you give it. Current +debugging levels are + +E: 1 trace function calls +E: 2 debug packet handling +E: 4 heavy trace debugging +E: 8 connection management +E: 16 print out packets sent and received +E: 32 search filter processing +E: 64 configuration file processing +E: 128 access control list processing +E: 256 stats log connections/operations/results +E: 512 stats log entries sent +E: 1024 print communication with shell backends +E: 2048 print entry parsing debugging +E: 65535 enable all debugging + +Debugging levels are additive. That is, if you want to trace function +calls and watch the config file being processed, you would set +level to the sum of those two levels (in this case, 65). Consult +{{EX: }} for more details. + +Note: slapd must have been compiled with {{EX:-DLDAP_DEBUG}} +defined for any debugging information beyond the two stats levels +to be available. + +E: -f + +This option specifies an alternate configuration file for slapd. + +E: -i + +This option tells slapd that it is running from inetd instead of as a +stand-alone server. See the next section on running slapd from +inetd for more details. + +E: -p + +This option specifies an alternate TCP port on which slapd should +listen for connections. The default port is 389. + + + +H2: Running slapd as a Stand-Alone Daemon + +In general, slapd is run like this: + +E: $(ETCDIR)/slapd [