[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapadd performance degradation from 2.3.43 to 2.4.12

Ralf Haferkamp wrote:
Am Freitag 21 November 2008 14:21:06 schrieb Ralf Haferkamp: 
I did some profiling (with valgrinds callgrind tool) to find out where all
the time is spend and it revealed that 2.4 spend a significantly larger
amount of systime in the pwrite() function than 2.3. Most of that seemed to
come from the bdb_tool_trickle_task() that calls libdb's memp_trickle()
function. Just to verify this I ran a testbuild with the trickle_task
disabled(). And slapadd's performance was back to a normal level,
comparable to the 2.3.43 release.

Something else came to mind. The trickle_task obviously cannot increase the overall volume of I/O, so there's no good reason for it to make things more than twice as slow. Except, that if you're getting reads mixed with writes you will lose a lot in disk seek time.

Since you said that your BDB cache size is large enough to contain the entire DB in each case, try this: preload the LDIF into the FS cache before running slapadd. (dd if=<ldif> of=/dev/null bs=1024k)

That will eliminate any seek overhead during the run.

AFAIK the trickle_task() was introduced into 2.4 to increase slapadd
throughput but has exactly the opposite effect on my test system. Did
anybody else make similar experiences? Or do you see anything thats
obviously wrong with my testcases?


Btw, I just re-ran the test with an unmodified 2.4.12 compiled against
db-4.7.25. With that combination slapadd is still a bit slower than the
OpenLDAP 2.3/db-4.5.20 combination, but the difference not quite as huge:
16m8s compared to 13m54s for the 500k Entries.

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/