2 # Copyright 1999-2007 The OpenLDAP Foundation, All Rights Reserved.
3 # COPYING RESTRICTIONS APPLY, see COPYRIGHT.
4 H1: Replication with slurpd
6 In certain configurations, a single {{slapd}}(8) instance may be
7 insufficient to handle the number of clients requiring
8 directory service via LDAP. It may become necessary to
9 run more than one slapd instance. At many sites,
10 for instance, there are multiple slapd servers: one
11 master and one or more slaves. {{TERM:DNS}} can be setup such that
12 a lookup of {{EX:ldap.example.com}} returns the {{TERM:IP}} addresses
13 of these servers, distributing the load among them (or
14 just the slaves). This master/slave arrangement provides
15 a simple and effective way to increase capacity, availability
18 {{slurpd}}(8) provides the capability for a master slapd to
19 propagate changes to slave slapd instances,
20 implementing the master/slave replication scheme
21 described above. slurpd runs on the same host as the
22 master slapd instance.
28 {{slurpd}}(8) provides replication services "in band". That is, it
29 uses the LDAP protocol to update a slave database from
30 the master. Perhaps the easiest way to illustrate this is
31 with an example. In this example, we trace the propagation
32 of an LDAP modify operation from its initiation by the LDAP
33 client to its distribution to the slave slapd instance.
36 {{B: Sample replication scenario:}}
38 ^ The LDAP client submits an LDAP modify operation to
41 + The slave slapd returns a referral to the LDAP
42 client referring the client to the master slapd.
44 + The LDAP client submits the LDAP modify operation to
47 + The master slapd performs the modify operation,
48 writes out the change to its replication log file and returns
49 a success code to the client.
51 + The slurpd process notices that a new entry has
52 been appended to the replication log file, reads the
53 replication log entry, and sends the change to the slave
56 + The slave slapd performs the modify operation and
57 returns a success code to the slurpd process.
60 Note: {{ldapmodify}}(1) and other clients distributed as part of
61 OpenLDAP Software do not support automatic referral chasing
62 (for security reasons).
68 When slapd is configured to generate a replication logfile, it
69 writes out a file containing {{TERM:LDIF}} change records. The
70 replication log gives the replication site(s), a timestamp, the DN
71 of the entry being modified, and a series of lines which specify
72 the changes to make. In the example below, Barbara ({{EX:uid=bjensen}})
73 has replaced the {{EX:description}} value. The change is to be
74 propagated to the slapd instance running on {{EX:slave.example.net}}
75 Changes to various operational attributes, such as {{EX:modifiersName}}
76 and {{EX:modifyTimestamp}}, are included in the change record and
77 will be propagated to the slave slapd.
79 > replica: slave.example.com:389
81 > dn: uid=bjensen,dc=example,dc=com
83 > replace: multiLineDescription
84 > description: A dreamer...
86 > replace: modifiersName
87 > modifiersName: uid=bjensen,dc=example,dc=com
89 > replace: modifyTimestamp
90 > modifyTimestamp: 20000805073308Z
93 The modifications to {{EX:modifiersName}} and {{EX:modifyTimestamp}}
94 operational attributes were added by the master {{slapd}}.
98 H2: Command-Line Options
100 This section details commonly used {{slurpd}}(8) command-line options.
104 This option sets the slurpd debug level to {{EX: <level>}}. When
105 level is a `?' character, the various debugging levels are printed
106 and slurpd exits, regardless of any other options you give it.
107 Current debugging levels (a subset of slapd's debugging levels) are
109 !block table; colaligns="RL"; align=Center; \
110 title="Table 13.1: Debugging Levels"
112 4 heavy trace debugging
113 64 configuration file processing
114 65535 enable all debugging
117 Debugging levels are additive. That is, if you want heavy trace
118 debugging and want to watch the config file being processed, you
119 would set level to the sum of those two levels (in this case, 68).
123 This option specifies an alternate slapd configuration file. Slurpd
124 does not have its own configuration file. Instead, all configuration
125 information is read from the slapd configuration file.
129 This option specifies an alternate slapd replication log file.
130 Under normal circumstances, slurpd reads the name of the slapd
131 replication log file from the slapd configuration file. However,
132 you can override this with the -r flag, to cause slurpd to process
133 a different replication log file. See the {{SECT:Advanced slurpd
134 Operation}} section for a discussion of how you might use this
139 Operate in "one-shot" mode. Under normal circumstances, when slurpd
140 finishes processing a replication log, it remains active and
141 periodically checks to see if new entries have been added to the
142 replication log. In one-shot mode, by comparison, slurpd processes
143 a replication log and exits immediately. If the -o option is given,
144 the replication log file must be explicitly specified with the -r
145 option. See the {{SECT:One-shot mode and reject files}} section
146 for a discussion of this mode.
150 Specify an alternate directory for slurpd's temporary copies of
151 replication logs. The default location is {{F:/usr/tmp}}.
154 H2: Configuring slurpd and a slave slapd instance
156 To bring up a replica slapd instance, you must configure the master
157 and slave slapd instances for replication, then shut down the master
158 slapd so you can copy the database. Finally, you bring up the master
159 slapd instance, the slave slapd instance, and the slurpd instance.
160 These steps are detailed in the following sections. You can set up
161 as many slave slapd instances as you wish.
164 H3: Set up the master {{slapd}}
166 The following section assumes you have a properly working {{slapd}}(8)
167 instance. To configure your working {{slapd}}(8) server as a
168 replication master, you need to make the following changes to your
171 ^ Add a {{EX:replica}} directive for each replica. The {{EX:binddn=}}
172 parameter should match the {{EX:updatedn}} option in the corresponding
173 slave slapd configuration file, and should name an entry with write
174 permission to the slave database (e.g., an entry allowed access via
175 {{EX:access}} directives in the slave slapd configuration file).
176 This DN generally {{should not}} be the same as the master's
179 + Add a {{EX:replogfile}} directive, which tells slapd where to log
180 changes. This file will be read by slurpd.
183 H3: Set up the slave {{slapd}}
185 Install the slapd software on the host which is to be the slave
186 slapd server. The configuration of the slave server should be
187 identical to that of the master, with the following exceptions:
189 ^ Do not include a {{EX:replica}} directive. While it is possible
190 to create "chains" of replicas, in most cases this is inappropriate.
192 + Do not include a {{EX:replogfile}} directive.
194 + Do include an {{EX:updatedn}} line. The DN given should match the
195 DN given in the {{EX:binddn=}} parameter of the corresponding
196 {{EX:replica=}} directive in the master slapd config file. The
197 {{EX:updatedn}} generally {{should not}} be the same as the
198 {{EX:rootdn}} of the master database.
200 + Make sure the DN given in the {{EX:updatedn}} directive has
201 permission to write the database (e.g., it is is allowed {{EX:access}}
202 by one or more access directives).
204 + Use the {{EX:updateref}} directive to define the URL the slave
205 should return if an update request is received.
208 H3: Shut down the master server
210 In order to ensure that the slave starts with an exact copy of the
211 master's data, you must shut down the master slapd. Do this by
212 sending the master slapd process an interrupt signal with
213 {{EX:kill -INT <pid>}}, where {{EX:<pid>}} is the process-id of the master
216 If you like, you may restart the master slapd in read-only mode
217 while you are replicating the database. During this time, the master
218 slapd will return an "unwilling to perform" error to clients that
219 attempt to modify data.
222 H3: Copy the master slapd's database to the slave
224 Copy the master's database(s) to the slave. For an {{TERM:BDB}} and
225 {{TERM:LDBM}} databases, you must copy all database files located
226 in the database {{EX:directory}} specified in {{slapd.conf}}(5).
227 In general, you should copy each file found in the database {{EX:
228 directory}} unless you know it is not used by {{slapd}}(8).
230 Note: This copy process assumes homogeneous servers with identically
231 configured OpenLDAP installations. Alternatively, you may use
232 {{slapcat}} to output the master's database in LDIF format and use
233 the LDIF with {{slapadd}} to populate the slave. Using LDIF avoids
234 any potential incompatibilities due to differing server architectures
235 or software configurations. See the {{SECT:Database Creation and
236 Maintenance Tools}} chapter for details on these tools.
239 H3: Configure the master slapd for replication
241 To configure slapd to generate a replication logfile, you add a
242 "{{EX: replica}}" configuration option to the master slapd's config
243 file. For example, if we wish to propagate changes to the slapd
244 instance running on host {{EX:slave.example.com}}:
246 > replica uri=ldap://slave.example.com:389
247 > binddn="cn=Replicator,dc=example,dc=com"
248 > bindmethod=simple credentials=secret
250 In this example, changes will be sent to port 389 (the standard
251 LDAP port) on host slave.example.com. The slurpd process will bind
252 to the slave slapd as "{{EX:cn=Replicator,dc=example,dc=com}}" using
253 simple authentication with password "{{EX:secret}}".
255 If we wish to perform the same replication using ldaps on port 636:
257 > replica uri=ldaps://slave.example.com:636
258 > binddn="cn=Replicator,dc=example,dc=com"
259 > bindmethod=simple credentials=secret
261 The host option is deprecated in favor of uri, but the following
262 replica configuration is still supported:
264 > replica host=slave.example.com:389
265 > binddn="cn=Replicator,dc=example,dc=com"
266 > bindmethod=simple credentials=secret
268 Note that the DN given by the {{EX:binddn=}} directive must exist
269 in the slave slapd's database (or be the rootdn specified in the
270 slapd config file) in order for the bind operation to succeed. The
271 DN should also be listed as the {{EX:updatedn}} for the database
272 in the slave's slapd.conf(5). It is generally recommended that
273 this DN be different than the {{EX:rootdn}} of the master database.
275 Note: The use of strong authentication and transport security is
279 H3: Restart the master slapd and start the slave slapd
281 Restart the master slapd process. To check that it is
282 generating replication logs, perform a modification of any
283 entry in the database, and check that data has been
284 written to the log file.
289 Start the slurpd process. Slurpd should immediately send
290 the test modification you made to the slave slapd. Watch
291 the slave slapd's logfile to be sure that the modification
294 > slurpd -f <masterslapdconfigfile>
298 H2: Advanced slurpd Operation
300 H3: Replication errors
302 When slurpd propagates a change to a slave slapd and receives an
303 error return code, it writes the reason for the error and the
304 replication record to a reject file. The reject file is located in
305 the same directory as the per-replica replication logfile, and has
306 the same name, but with the string "{{F:.rej}}" appended. For
307 example, for a replica running on host {{EX:slave.example.com}},
308 port 389, the reject file, if it exists, will be named
310 > /usr/local/var/openldap/replog.slave.example.com:389.rej
312 A sample rejection log entry follows:
314 > ERROR: No such attribute
315 > replica: slave.example.com:389
317 > dn: uid=bjensen,dc=example,dc=com
319 > replace: description
320 > description: A dreamer...
322 > replace: modifiersName
323 > modifiersName: uid=bjensen,dc=example,dc=com
325 > replace: modifyTimestamp
326 > modifyTimestamp: 20000805073308Z
329 Note that this is precisely the same format as the original replication
330 log entry, but with an {{EX:ERROR}} line prepended to the entry.
334 H3: One-shot mode and reject files
336 It is possible to use slurpd to process a rejection log with its
337 "one-shot mode." In normal operation, slurpd watches for more
338 replication records to be appended to the replication log file. In
339 one-shot mode, by contrast, slurpd processes a single log file and
340 exits. Slurpd ignores {{EX:ERROR}} lines at the beginning of
341 replication log entries, so it's not necessary to edit them out
342 before feeding it the rejection log.
344 To use one-shot mode, specify the name of the rejection log on the
345 command line as the argument to the -r flag, and specify one-shot
346 mode with the -o flag. For example, to process the rejection log
347 file {{F:/usr/local/var/openldap/replog.slave.example.com:389}} and
348 exit, use the command
350 > slurpd -r /usr/tmp/replog.slave.example.com:389 -o