[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
ldapadd: hopelessly slow loading due to high disk iowait
- To: OpenLDAP-bugs@OpenLDAP.org
- Subject: ldapadd: hopelessly slow loading due to high disk iowait
- From: Kelley Hu <khu@nlm.nih.gov>
- Date: Tue, 28 Sep 1999 10:00:29 -0400
- Organization: U.S. National Library of Medicine
Dear Openldap user:
We are using a Sun Ultra-2 running under Solaris 2.6, with dual
internal disk
drives and a number of external drives.
Here is the problem: we are running openLDAP, and trying to populate
a database using gdbm_1.7.3. The input file (IN) is a million-record
file,
each record containing 3 lines like:
dn: cn=1000/nl4/78654328,dc=NLM,dc=gov
cn: 1000/nl4/78654328
sn: z39.50r://z3950.nlm.nih.gov/medline?78654328
The goal is to be able to use LDAP to create a service that
can provide the location of the object's sn, given the cn. The output
file (OUT)
is the database file created by gdbm.
When I load the record using
./ldapadd -D "cn=superuser, dc=NLM, dc=gov" -wrules <
/IN/ldap_list.txt > /dev/null
I noticed that the insertion of records is hopelessly slow! it takes
overnight to get 10% of one million records in!
If we run the "top" command we observe very high percentage in
"iowait" parameter it displays. Here is typical output from "top"
during a high I/O wait experiment:
last pid: 24683; load averages: 0.12, 0.31,
0.28 15:33:37
83 processes: 82 sleeping, 1 on cpu
CPU states: 0.4% idle, 0.4% user, 1.2% kernel, 98.0% iowait,
0.0%
swap
Memory: 256M real, 4720K free, 169M swap in use, 88M swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
24677 root 7 29 0 3760K 2368K sleep 0:01 1.10% slapd
24612 root 1 33 0 2104K 1608K cpu/1 0:01 0.25% top
24683 root 1 33 0 1504K 1168K sleep 0:00 0.06% vi
19240 root 1 33 0 1272K 1064K sleep 0:00 0.06% csh
19029 root 1 33 0 1912K 1592K sleep 0:03 0.02% sshd
24680 root 4 33 0 2760K 2280K sleep 0:00 0.01% ldapadd
I have played with different configuration in which IN (the record txt
file) and OUT (the database directory) is in different disk-drive, or
the same disk drive, they all have ~98% cpu waiting on IO, and the
./ldapadd only consumes 0.01% CUP time!
I wonder if this is a openldap bug, or perhaps a bug in GDBM? I
appreciate very much if someone could shed light on this!
Thanks in advance!
--
Kelley Hu
U.S. National Library of Medicine, Computer Science Branch
Bethesda, MD
(301) 435-3205 khu@nlm.nih.gov