2 # Copyright 1999-2012 The OpenLDAP Foundation, All Rights Reserved.
3 # COPYING RESTRICTIONS APPLY, see COPYRIGHT.
7 This is perhaps one of the most important chapters in the guide, because if
8 you have not tuned {{slapd}}(8) correctly or grasped how to design your
9 directory and environment, you can expect very poor performance.
11 Reading, understanding and experimenting using the instructions and information
12 in the following sections, will enable you to fully understand how to tailor
13 your directory server to your specific requirements.
15 It should be noted that the following information has been collected over time
16 from our community based FAQ. So obviously the benefit of this real world experience
17 and advice should be of great value to the reader.
20 H2: Performance Factors
22 Various factors can play a part in how your directory performs on your chosen
23 hardware and environment. We will attempt to discuss these here.
28 Scale your cache to use available memory and increase system memory if you can.
30 See {{SECT:Caching}} for BDB cache tuning hints.
31 Note that MDB uses no cache of its own and has no tuning options, so the Caching
32 section can be ignored when using MDB.
37 Use fast filesystems, and conduct your own testing to see which filesystem
38 types perform best with your workload. (On our own Linux testing, EXT2 and JFS
39 tend to provide better write performance than everything else, including
40 newer filesystems like EXT4, BTRFS, etc.)
42 Use fast subsystems. Put each database and logs on separate disks
43 (for BDB this is configurable via {{DB_CONFIG}}):
46 > set_data_dir /data/db
48 > # Transaction Log settings
54 http://www.openldap.org/faq/data/cache/363.html
59 H3: Directory Layout Design
61 Reference to other sections and good/bad drawing here.
71 H3: Understanding how a search works
73 If you're searching on a filter that has been indexed, then the search reads
74 the index and pulls exactly the entries that are referenced by the index.
75 If the filter term has not been indexed, then the search must read every single
76 entry in the target scope and test to see if each entry matches the filter.
77 Obviously indexing can save a lot of work when it's used correctly.
81 You should create indices to match the actual filter terms used in
84 > index cn,sn,givenname,mail eq
86 Each attribute index can be tuned further by selecting the set of index types to generate. For example, substring and approximate search for organizations (o) may make little sense (and isn't like done very often). And searching for {{userPassword}} likely makes no sense what so ever.
88 General rule: don't go overboard with indexes. Unused indexes must be maintained and hence can only slow things down.
90 See {{slapd.conf}}(8) and {{slapdindex}}(8) for more information
95 If your client application uses presence filters and if the
96 target attribute exists on the majority of entries in your target scope, then
97 all of those entries are going to be read anyway, because they are valid
98 members of the result set. In a subtree where 100% of the
99 entries are going to contain the same attributes, the presence index does
100 absolutely NOTHING to benefit the search, because 100% of the entries match
101 that presence filter.
103 So the resource cost of generating the index is a
104 complete waste of CPU time, disk, and memory. Don't do it unless you know
105 that it will be used, and that the attribute in question occurs very
106 infrequently in the target data.
108 Almost no applications use presence filters in their search queries. Presence
109 indexing is pointless when the target attribute exists on the majority of
110 entries in the database. In most LDAP deployments, presence indexing should
111 not be done, it's just wasted overhead.
113 See the {{Logging}} section below on what to watch our for if you have a frequently searched
114 for attribute that is unindexed.
119 H3: What log level to use
121 The default of {{loglevel stats}} (256) is really the best bet. There's a corollary to
122 this when problems *do* arise, don't try to trace them using syslog.
123 Use the debug flag instead, and capture slapd's stderr output. syslog is too
124 slow for debug tracing, and it's inherently lossy - it will throw away messages when it
127 Contrary to popular belief, {{loglevel 0}} is not ideal for production as you
128 won't be able to track when problems first arise.
130 H3: What to watch out for
132 The most common message you'll see that you should pay attention to is:
134 > "<= bdb_equality_candidates: (foo) index_param failed (18)"
136 That means that some application tried to use an equality filter ({{foo=<somevalue>}})
137 and attribute {{foo}} does not have an equality index. If you see a lot of these
138 messages, you should add the index. If you see one every month or so, it may
139 be acceptable to ignore it.
141 The default syslog level is stats (256) which logs the basic parameters of each
142 request; it usually produces 1-3 lines of output. On Solaris and systems that
143 only provide synchronous syslog, you may want to turn it off completely, but
144 usually you want to leave it enabled so that you'll be able to see index
145 messages whenever they arise. On Linux you can configure syslogd to run
146 asynchronously, in which case the performance hit for moderate syslog traffic
147 pretty much disappears.
149 H3: Improving throughput
151 You can improve logging performance on some systems by configuring syslog not
152 to sync the file system with every write ({{man syslogd/syslog.conf}}). In Linux,
153 you can prepend the log file name with a "-" in {{syslog.conf}}. For example,
154 if you are using the default LOCAL4 logging you could try:
157 > LOCAL4.* -/var/log/ldap
159 For syslog-ng, add or modify the following line in {{syslog-ng.conf}}:
161 > options { sync(n); };
163 where n is the number of lines which will be buffered before a write.
168 We all know what caching is, don't we?
170 In brief, "A cache is a block of memory for temporary storage of data likely
171 to be used again" - {{URL:http://en.wikipedia.org/wiki/Cache}}
173 There are 3 types of caches, BerkeleyDB's own cache, {{slapd}}(8)
174 entry cache and {{TERM:IDL}} (IDL) cache.
177 H3: Berkeley DB Cache
179 There are two ways to tune for the BDB cachesize:
181 (a) BDB cache size necessary to load the database via slapadd in optimal time
183 (b) BDB cache size necessary to have a high performing running slapd once the data is loaded
185 For (a), the optimal cachesize is the size of the entire database. If you
186 already have the database loaded, this is simply a
190 in the directory containing the OpenLDAP ({{/usr/local/var/openldap-data}}) data.
192 For (b), the optimal cachesize is just the size of the {{id2entry.bdb}} file,
193 plus about 10% for growth.
195 The tuning of {{DB_CONFIG}} should be done for each BDB type database
196 instantiated (back-bdb, back-hdb).
198 Note that while the {{TERM:BDB}} cache is just raw chunks of memory and
199 configured as a memory size, the {{slapd}}(8) entry cache holds parsed entries,
200 and the size of each entry is variable.
202 There is also an IDL cache which is used for Index Data Lookups.
203 If you can fit all of your database into slapd's entry cache, and all of your
204 index lookups fit in the IDL cache, that will provide the maximum throughput.
206 If not, but you can fit the entire database into the BDB cache, then you
207 should do that and shrink the slapd entry cache as appropriate.
209 Failing that, you should balance the BDB cache against the entry cache.
211 It is worth noting that it is not absolutely necessary to configure a BerkeleyDB
212 cache equal in size to your entire database. All that you need is a cache
213 that's large enough for your "working set."
215 That means, large enough to hold all of the most frequently accessed data,
216 plus a few less-frequently accessed items.
218 For more information, please see: {{URL:http://www.oracle.com/technology/documentation/berkeley-db/db/ref/am_conf/cachesize.html}}
220 H4: Calculating Cachesize
222 The back-bdb database lives in two main files, {{F:dn2id.bdb}} and {{F:id2entry.bdb}}.
223 These are B-tree databases. We have never documented the back-bdb internal
224 layout before, because it didn't seem like something anyone should have to worry
225 about, nor was it necessarily cast in stone. But here's how it works today,
228 A B-tree is a balanced tree; it stores data in its leaf nodes and bookkeeping
229 data in its interior nodes (If you don't know what tree data structures look
230 like in general, Google for some references, because that's getting far too
231 elementary for the purposes of this discussion).
233 For decent performance, you need enough cache memory to contain all the nodes
234 along the path from the root of the tree down to the particular data item
235 you're accessing. That's enough cache for a single search. For the general case,
236 you want enough cache to contain all the internal nodes in the database.
240 will tell you how many internal pages are present in a database. You should
241 check this number for both dn2id and id2entry.
243 Also note that {{id2entry}} always uses 16KB per "page", while {{dn2id}} uses whatever
244 the underlying filesystem uses, typically 4 or 8KB. To avoid thrashing,
245 your cache must be at least as large as the number of internal pages in both
246 the {{dn2id}} and {{id2entry}} databases, plus some extra space to accommodate
247 the actual leaf data pages.
249 For example, in my OpenLDAP 2.4 test database, I have an input LDIF file that's
250 about 360MB. With the back-hdb backend this creates a {{dn2id.bdb}} that's 68MB,
251 and an {{id2entry}} that's 800MB. db_stat tells me that {{dn2id}} uses 4KB pages, has
252 433 internal pages, and 6378 leaf pages. The id2entry uses 16KB pages, has 52
253 internal pages, and 45912 leaf pages. In order to efficiently retrieve any
254 single entry in this database, the cache should be at least
256 > (433+1) * 4KB + (52+1) * 16KB in size: 1736KB + 848KB =~ 2.5MB.
258 This doesn't take into account other library overhead, so this is even lower
259 than the barest minimum. The default cache size, when nothing is configured,
262 This 2.5MB number also doesn't take indexing into account. Each indexed
263 attribute results in another database file. Earlier versions of OpenLDAP
264 kept these index databases in Hash format, but from OpenLDAP 2.2 onward
265 the index databases are in B-tree format so the same procedure can
266 be used to calculate the necessary amount of cache for each index database.
268 For example, if your only index is for the objectClass attribute and db_stat
269 reveals that {{objectClass.bdb}} has 339 internal pages and uses 4096 byte
270 pages, the additional cache needed for just this attribute index is
272 > (339+1) * 4KB =~ 1.3MB.
274 With only this index enabled, I'd figure at least a 4MB cache for this backend.
275 (Of course you're using a single cache shared among all of the database files,
276 so the cache pages will most likely get used for something other than what you
277 accounted for, but this gives you a fighting chance.)
279 With this 4MB cache I can slapcat this entire database on my 1.3GHz PIII in
280 1 minute, 40 seconds. With the cache doubled to 8MB, it still takes the same 1:40s.
281 Once you've got enough cache to fit the B-tree internal pages, increasing it
282 further won't have any effect until the cache really is large enough to hold
283 100% of the data pages. I don't have enough free RAM to hold all the 800MB
284 id2entry data, so 4MB is good enough.
286 With back-bdb and back-hdb you can use "db_stat -m" to check how well the
287 database cache is performing.
289 For more information on {{db_stat}}: {{URL:http://www.oracle.com/technology/documentation/berkeley-db/db/utility/db_stat.html}}
291 H3: {{slapd}}(8) Entry Cache (cachesize)
293 The {{slapd}}(8) entry cache operates on decoded entries. The rationale - entries
294 in the entry cache can be used directly, giving the fastest response. If an entry
295 isn't in the entry cache but can be extracted from the BDB page cache, that will
296 avoid an I/O but it will still require parsing, so this will be slower.
298 If the entry is in neither cache then BDB will have to flush some of its current
299 cached pages and bring in the needed pages, resulting in a couple of expensive
300 I/Os as well as parsing.
302 The most optimal value is of course, the entire number of entries in the database.
303 However, most directory servers don't consistently serve out their entire database, so setting this to a lesser number that more closely matches the believed working set of data is
304 sufficient. This is the second most important parameter for the DB.
306 As far as balancing the entry cache vs the BDB cache - parsed entries in memory
307 are generally about twice as large as they are on disk.
309 As we have already mentioned, not having a proper database cache size will
310 cause performance issues. These issues are not an indication of corruption
311 occurring in the database. It is merely the fact that the cache is thrashing
312 itself that causes performance/response time to slowdown.
315 H3: {{TERM:IDL}} Cache (idlcachesize)
317 Each IDL holds the search results from a given query, so the IDL cache will
318 end up holding the most frequently requested search results. For back-bdb,
319 it is generally recommended to match the "cachesize" setting. For back-hdb,
320 it is generally recommended to be 3x"cachesize".
322 {NOTE: The idlcachesize setting directly affects search performance}
325 H2: {{slapd}}(8) Threads
327 {{slapd}}(8) can process requests via a configurable number of threads, which
328 in turn affects the in/out rate of connections.
330 This value should generally be a function of the number of "real" cores on
331 the system, for example on a server with 2 CPUs with one core each, set this
332 to 8, or 4 threads per real core. This is a "read" maximized value. The more
333 threads that are configured per core, the slower {{slapd}}(8) responds for
334 "read" operations. On the flip side, it appears to handle write operations
335 faster in a heavy write/low read scenario.
337 The upper bound for good read performance appears to be 16 threads (which
338 also happens to be the default setting).