Scale your cache to use available memory and increase system memory if you can.
-More info here.
+See {{SECT:Caching}}
H3: Disks
-Use fast subsystems. Put each database and logs on separate disks.
+Use fast subsystems. Put each database and logs on separate disks configurable
+via {{DB_CONFIG}}:
-Example showing config settings
+> # Data Directory
+> set_data_dir /data/db
+>
+> # Transaction Log settings
+> set_lg_dir /logs
H3: Network Topology
The most common message you'll see that you should pay attention to is:
-> "<= bdb_equality_candidates: (foo) index_param failed (18)"
+> "<= bdb_equality_candidates: (foo) index_param failed (18)"
That means that some application tried to use an equality filter ({{foo=<somevalue>}})
and attribute {{foo}} does not have an equality index. If you see a lot of these
you can prepend the log file name with a "-" in {{syslog.conf}}. For example,
if you are using the default LOCAL4 logging you could try:
-> # LDAP logs
-> LOCAL4.* -/var/log/ldap
+> # LDAP logs
+> LOCAL4.* -/var/log/ldap
For syslog-ng, add or modify the following line in {{syslog-ng.conf}}:
-> options { sync(n); };
+> options { sync(n); };
where n is the number of lines which will be buffered before a write.
-H2: BDB/HDB Database Caching
+H2: Caching
We all know what caching is, don't we?
H3: Berkeley DB Cache
-BerkeleyDB's own data cache operates on page-sized blocks of raw data.
+There are two ways to tune for the BDB cachesize:
+
+(a) BDB cache size necessary to load the database via slapadd in optimal time
+
+(b) BDB cache size necessary to have a high performing running slapd once the data is loaded
+
+For (a), the optimal cachesize is the size of the entire database. If you
+already have the database loaded, this is simply a
+
+> du -c -h *.bdb
+
+in the directory containing the OpenLDAP ({{/usr/local/var/openldap-data}}) data.
+
+For (b), the optimal cachesize is just the size of the {{id2entry.bdb}} file,
+plus about 10% for growth.
+
+The tuning of {{DB_CONFIG}} should be done for each BDB type database
+instantiated (back-bdb, back-hdb).
Note that while the {{TERM:BDB}} cache is just raw chunks of memory and
configured as a memory size, the {{slapd}}(8) entry cache holds parsed entries,
That means, large enough to hold all of the most frequently accessed data,
plus a few less-frequently accessed items.
-ORACLE LINKS HERE
+For more information, please see: {{URL:http://www.oracle.com/technology/documentation/berkeley-db/db/ref/am_conf/cachesize.html}}
H4: Calculating Cachesize
you're accessing. That's enough cache for a single search. For the general case,
you want enough cache to contain all the internal nodes in the database.
-> db_stat -d
+> db_stat -d
will tell you how many internal pages are present in a database. You should
check this number for both dn2id and id2entry.
internal pages, and 45912 leaf pages. In order to efficiently retrieve any
single entry in this database, the cache should be at least
-> (433+1) * 4KB + (52+1) * 16KB in size: 1736KB + 848KB =~ 2.5MB.
+> (433+1) * 4KB + (52+1) * 16KB in size: 1736KB + 848KB =~ 2.5MB.
This doesn't take into account other library overhead, so this is even lower
than the barest minimum. The default cache size, when nothing is configured,
With back-bdb and back-hdb you can use "db_stat -m" to check how well the
database cache is performing.
+For more information on {{db_stat}}: {{URL:http://www.oracle.com/technology/documentation/berkeley-db/db/utility/db_stat.html}}
-H3: {{slapd}}(8) Entry Cache
+H3: {{slapd}}(8) Entry Cache (cachesize)
The {{slapd}}(8) entry cache operates on decoded entries. The rationale - entries
in the entry cache can be used directly, giving the fastest response. If an entry
cached pages and bring in the needed pages, resulting in a couple of expensive
I/Os as well as parsing.
+The most optimal value is of course, the entire number of entries in the database.
+However, most directory servers don't consistently serve out their entire database, so setting this to a lesser number that more closely matches the believed working set of data is
+sufficient. This is the second most important parameter for the DB.
+
As far as balancing the entry cache vs the BDB cache - parsed entries in memory
are generally about twice as large as they are on disk.
itself that causes performance/response time to slowdown.
-MOVE BELOW AROUND:
-
-
-If you want to setup the cache size, please read:
-
- (Xref) How do I configure the BDB backend?
- (Xref) What are the DB_CONFIG configuration directives?
- http://www.sleepycat.com/docs/utility/db_recover.html
-
-A default config can be found in the answer:
-
- (Xref) What are the DB_CONFIG configuration directives?
-
-just change the set_lg_dir to point to your .log directory or comment that line.
-
-Quick guide:
-* Create a DB_CONFIG file in your ldap home directory (/var/lib/ldap/DB_CONFIG) with the correct "set_cachesize" value
-* stop your ldap server and run db_recover -h /var/lib/ldap
-* start your ldap server and check the new cache size with:
-
- db_stat -h /var/lib/ldap -m | head -n 2
-
-* this procedure is only needed if you use OpenLDAP 2.2 with the BDB or HDB backends; In OpenLDAP 2.3 DB recovery is performed automatically whenever the DB_CONFIG file is changed or when an unclean shutdown is detected.
-
-
---On Tuesday, February 22, 2005 12:15 PM -0500 Dusty Doris <openldap@mail.doris.cc> wrote:
-
- Few questions, if you change the cachesize and idlecachesize entries, do
- you have to do anything special aside from restarting slapd, such as run
- slapindex or db_recover?
-
-
- Also, is there any way to tell how much memory these caches are taking up
- to make sure they are not set too large? What happens if you set your
- cachesize too large and you don't have enough available memory to store
- these? Will that cause an issue with openldap, or will it just not cache
- those entries that would make it exceed its available memory. Will it
- just use some sort of FIFO on those caches?
-
-
-It will consume the memory resources of your system, and likely cause issues.
-
- Finally, what do most people try to achieve with these values? Would the
- goal be to make these as big as the directory? So, if I have 400,000 dn's
- in my directory, would it be safe to set these at 400000 or would
- something like 20,000 be good enough to get a nice performance increase?
-
+H3: {{TERM:IDL}} Cache (idlcachesize)
-I try to cache the most actively used entries. Unless you expect all 400,000 entries of your DB to be accessed regularly, there is no need to cache that many entries. My entry cache is set to 20,000 (out of a little over 400,000 entries).
+Each IDL holds the search results from a given query, so the IDL cache will
+end up holding the most frequently requested search results. For back-bdb,
+it is generally recommended to match the "cachesize" setting. For back-hdb,
+it is generally recommended to be 3x"cachesize".
-The idlcache has to do with how many unique result sets of searches you want to store in memory. Setting up this cache will allow your most frequently placed searches to get results much faster, but I doubt you want to try and cache the results of every search that hits your system. ;)
+{NOTE: The idlcachesize setting directly affects search performance}
---Quanah
+H3: {{slapd}}(8) Threads
-H3: {{TERM:IDL}} Cache
+{{slapd}}(8) can process requests via a configurable number of thread, which
+in turn affects the in/out rate of connections.
+This value should generally be a function of the number of "real" cores on
+the system, for example on a server with 2 CPUs with one core each, set this
+to 8, or 4 threads per real core. This is a "read" maximized value. The more
+threads that are configured per core, the slower {{slapd}}(8) responds for
+"read" operations. On the flip side, it appears to handle write operations
+faster in a heavy write/low read scenario.
-http://www.openldap.org/faq/data/cache/1076.html
+The upper bound for good read performance appears to be 16 threads (which
+also happens to be the default setting).
projects / openldap / commitdiff