[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: Slow adding attributes to a entry with many attributes
On Fri, 2005-12-02 at 09:30 -0200, Ariel Malves wrote:
> Dear list.
>
> In the past 4 days I have had problems with OpenLdap, running a Samba
> PDC solution.
>
> My PDC has about 3000 users, migrated from a Netware 5 with
> eDirectory. Each user is in a primary and a secondary group too
> (Students and Domain Users groups). I could see a very fast increasing
> of log size (log.000000*) when adding users to secondary groups. For
> each new memberUID attribute added to the group, the log grows about 7
> mb! The time for this operation is very bad too, about 3 seconds.
>
> Searching in archive of this list I found a message from other user
> with same problem (
> http://www.openldap.org/lists/openldap-software/200407/msg00446.html),
> and the recommendation was to upgrade the version of OpenLdap to 2.2.*
> and BDB to 4.2.52, but I already have those versions in my Debian
> Sarge (stable).
>
> Well, my question is: Is this normal?
>
> I have tried changing the backend from bdb to hdb with slapcat &
> slapadd, but nothing changed.
>
> The only little performance improvement was when I create a DB_CONFIG
> with the following attributes:
>
> set_cachesize 0 300000000 10
> set_lg_regionmax 262144
>
> set_lg_bsize 2097152
>
>
> set_flags DB_TXN_WRITE_NOSYNC
>
> I tried another simulation in a clean ldap installation, creating a
> unique person (inetOrgPerson) and adding many (2000+) mail addresses
> to it and the results was similar: an intensive I/O on logs and the
> operation of adding new attributes slowing down each time.
>
> I have thought about a workaround: don't put these students in a
> secondary group, but manage rights access to files would become
> difficult a bit.
>
> Please, if anyone has an idea, tell me.
>
> Thanks to all, and sorry my poor English.
OpenLDAP's slapd destroys and recreates indices on __all__ values when
modifying multivalued attributes. I do not recall if this behavior
changed. When using Samba, for each user insertion the newly inserted
DN gets added to a number of groups. This causes the repetitive
deletion and recreation of a number of data in the database that grows
linearly with the number of users; as a result, if you consecutively add
N users, N*(N+1)/2 index deletions and recreations actually occurs.
This bumps up the size of the logs quadratically for a large import. If
you can accept the performance degradation (using OpenLDAP on a decent
hardware allows to add thousands of users in reasonable time) and you
keep checkpointing and archiving the logs (see checkpoint in slapd-bdb
(5) and db_archive(1)), the disk occupation will not be impacted (apart
for the storage where you archive the log files, of course).
In case of massive operations like this (don't know if Samba can deal
with it, though) I suggest to collect all values and add them within a
single write operation. In the specific case of Samba, Samba tools
could be modified, or a new tool could be implemented to provide a
massive load command that defers group membership at the end of the
import, but we're going OT now.
p.
Ing. Pierangelo Masarati
Responsabile Open Solution
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati@sys-net.it
------------------------------------------