[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: parsing logs, trace replay

On 7/16/07, Kelly, Terence P <terence.p.kelly@hp.com> wrote: 
Hi,

Question 1:

Is there any documentation available on
the OpenLDAP access log format?  I wasn't
able to find any via Google or on the
OpenLDAP Web site.

I looked at how ldap-stats.pl version 5.2
parses logs but I see discrepancies between
the operation counts returned by ldap-stats
and counts obtained by independent means
(e.g., simple AWK scripts), so I'm not very
confident that it's doing the right thing.

I can try to reverse-engineer the log
format but if there's good documentation
and/or a good parser out there I'd prefer
that.

Question 2:

My goal is to replay traces of accesses
to an LDAP server to a replica of the
server itself.  I want the exercise to
be reasonably realistic, e.g., I want
to achieve the same throughputs and
level of concurrency as the real server.
At the moment I'm planning to write my
own trace-replay tool.

I have investigated two trace-replay
tools:  dupetrace.pl and slamd.  The
former passes the buck to a shell
utility which probably won't achieve
the performance I need; the latter
seems formidably difficult to set up
and use.

Are you aware of any LDAP trace replay
tools that are as capable yet as simple
as httperf?

Many thanks for any pointers you can
provide!



slamd isn't that difficult to setup and I can recommend it for load
testing.  I've never used the log-replay stuff, however.

The openldap transaction logs are going through syslog, a single
transaction can go over many lines, and the return code for that
transaction is on yet another line; So you have to be careful when
parsing them with line-oriented tools such as awk and perl.  :) (be
especially careful when syslog rotates and you have conn's that span
over the rotation)