[Date Prev][Date Next]
RE: (ITS#3565) Programatic Insert Scaleability Problem (#2)
I'll try to test sometime in the near future.
From: Howard Chu [mailto:email@example.com]
Sent: Monday, April 18, 2005 10:49 AM
To: Armbrust, Daniel C.
Subject: Re: (ITS#3565) Programatic Insert Scaleability Problem (#2)
I believe this is fixed by ITS#3666.
>Full_Name: Dan Armbrust
>OS: Fedora Core 3
>Submission from: (NULL) (126.96.36.199)
>This bug is very similar to this bug
>http://www.openldap.org/its/index.cgi?findid=3564 - maybe the same bug.
>But, since that one could be classified as a feature request, and I think this
>one is more clearly a bug, I'm filing them separately.
>I tried to do the same type of load as I detailed in the above bug report -
>except this time, I removed all of the extra indexes that we were adding, and
>disabled schema checking (on a hunch) Since the indexing seemed to be the source
>of the memory management issue - I had hopes of being able to load the whole
>thing without indexes, then slapcat it, and then reload it with slapadd with the
>indexes enabled - since I know that works for large datasets.
>This time, I got about twice as far in the load process (roughly around 1.2
>million dn's - each with about 10 attributes). Unlike the load process in the
>other bug report - this time it did not get slower as it progressed. It was
>running fine up until the point that it kicked out the following error:
>LDAP: error code 80 - entry store failed
>At this point, the server was using about 4 GB of ram, which I though was rather
>excessive. (Nothing else is being done on the server other than the load - no
>searches at all)
>I tried to connect to the server with a ldap browser, and slapd segfaulted.
>Database size only made it up to 4.3 GB.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
Symas: Premier OpenSource Development and Support