[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: Some more Intense LDAP write operations
I am an old timer - we (ICL) used to run a bank or London Transport on 128Kb
-
The Four Yorkshiremen - Monty Python :-)
Re the flat search times? (engineering report required) - well is this all
about single server tuning, search prediction, caching and selection of
index mechanisms.. ie does one gets the "spanners" out when you add more
schema configuration, more objects, more attributes, more access controls,
more applications, more clients, etc and when one adds more replicated
servers (because the system is not distributed) then its not just one
mechanic - its a team of them working with miners lamps (night time) and
being totally synchronised..
Is this a new olympic sport in the making - synchronised LDAP server
management? :-)
Distributed directories provide a distributed scaleable, context (country or
application) designed capacity directory for globally distributed
organisations who also want the ability to manage their own part of the
system - and let administrators do a global search and say.. how many
customers across the world have I got with this service..how many of my
customers with certs have their certs expire next week..
See the telephone service... ie it does not replicate everything to every
where..
A Virtual ISP service is users and services working via internet, telephones
and television access onto the public and secure private global internets. -
Sorry I just cannot engineer these things with LDAP replicate everything to
everywhere servers..
A database is 30 year old technology, it is good for storing what has
happened in a transaction - at a central place.. a service.
Directories are distributed - and are ideal for storing what things are and
what they can do..
However, a directory needs to use underlying database engineering in each
server to enable the same integrity and product tools.. ie a directory (ours
) is a distributed OO application that uses databases as its storage,
indexing and data retrieval system. note 32 patents on this OO directory
application of RDBs.
My role is to design large scale EC infrastructure for businesses. If a
solution requires more people than the servers being installed then it is
not a solution.
regards alan
-----Original Message-----
From: Subbu K. k. [mailto:KKSUBRAMANIAM@novell.com]
Sent: Friday, June 09, 2000 7:41 PM
To: alan.lloyd@ca.com; ietf-ldapext@netscape.com
Subject: RE: Intense LDAP write operations
[Moderator, I am not sure if I am violating the intent of this list in
continuing this discussion. If so, pls. let me know and I can take this on a
separate discussion group. Thanks in advance]
Alan,
Actually, the exercise was taken to see how far we could push the limits
before eDirectory breaks down (It didnt, in this case. We had to stop the
exercise at 1+ billion objects to show it in an event).
Billion objects may sound like a lot today but so did 640KB of address space
for PCs in 1984!
Not all entries need necessarily be personal secrets. In your specific
example, it does makes sense to have a global directory for mobile phone
numbers (projected to cross 1 billion by 2003) that is updated by regional
admins but seen as one by the mobile phone owners. Global knowledge bases
like Human Genome will contain about 3 billion codes. Replication may take a
'long' time and may even be filtered, but since they dont change much, it
makes sense to have the directory automatically replicate this where
scientists actually need them.
I dont think the assumption that there is a trade-off between
(capacity,concurrency,search time) is necessarily true. With proper design
(as in NDS eDirectory) that exploits the usage pattern for directories, it
is possible to retain essentially flat search times thru hundreds of
millions of entries.
Subbu K. K.
>>> "Lloyd, Alan" <Alan.Lloyd@ca.com> 06/09/00 11:45AM >>>
If you are worried about capacity and scaleability - well replicated systems
dont scale - Just consider the telephone system - we log on where we are and
use local calls, national calls and long distance calls - ie the
authentication base is distributed. X.500 is a distributed system in that it
forms a distributed system infrastructure.. ie one does not need to
replicate AU to the US, US to the UK and so on..
It seems odd that one would put a billion objects in one server and then
have to replicate this all over the place? Becuase one could not have a
distributed system.
In addition the real test for a directory is not lots of objects that
replicate - its the ability to have these objects distributed - as well as
having a distributed service that can handle high user concurrency - eg 100s
of Ms of users - all logging on in seconds and all wanting directory service
information...in seconds.
In terms of design - with multi LDAP/DAP/DSP/DISP backbone load balancing /
alternate DSA modules - coupled to DSA processes, with one or more DBs, etc
- we can do fault tolerant, load balancing distributed/replicated meshed
based global directory services and integrate LDAP servers. So the issues of
read/write requirements is met in the operation system design of the
directory service..
Replicated systems... one only does this for fault tolerance.. If you
replicate names/passworrd/certs/CRLs, ACI for millions of entries all over
the world then - one will have a considerable security issue to deal with..
regards alan
-----Original Message-----
From: Subbu K. k. [mailto:KKSUBRAMANIAM@novell.com]
Sent: Thursday, June 08, 2000 7:33 PM
To: TGullotta@access360.com; ietf-ldapext@netscape.com;
Martin.Rahm@nokia.com
Subject: RE: Intense LDAP write operations
Not really. It is just that when you have millions of objects, they tend to
be read much more often than modified. For instance, when a user object is
created, it is read thousands of times before it gets updated.
Operationally, NDS does update its database internally for its replication
status etc. The last login time attribute does get updated for every login,
but this is an operational attribute.
NDS itself uses a transactional database to ensure full integrity and
incremental replication to ensure scalability. It has tested to hold more
than 1 billion objects on a single E450. As installed, it just takes less
than a megabyte for its database.
Subbu K. K.
-------------------
K. K. Subramaniam,
Product Manager, NDS eDirectory (UNIX/Linux)
Novell Software, Bangalore Ph: +91 (80)
572-1856x2212
>>> <TGullotta@access360.com> 06/07/00 09:08PM >>>
Subbu,
Has Novell done any benchmarks on read vs. write operations? It seems that
in a lot of directory implementations, although not every entry is being
updated that frequently, if you have a million or so users, just periodic
updates to entries could add up when you look at the total number of users.
I would think this is realistic. Is it a problem with NDS?
Tony Gullotta
Lead System Architect
access360
-----Original Message-----
From: Subbu K. k. [mailto:KKSUBRAMANIAM@novell.com]
Sent: Wednesday, June 07, 2000 12:15 AM
To: ietf-ldapext@netscape.com; Martin.Rahm@nokia.com
Subject: Re: Intense LDAP write operations
I dont think this was the sweet spot planned for directories. Writes would
entail applying locks on the data store which would impact the regular
search, retrieval and any sync operations. You also need to deal with
atomicity of updates to the group of attributes and the latency of
propagation of changes across multiple copies of the data store. A
transactional database may be a better choice.
Are you sure you need a directory service and not a transactional database
engine that supports LDAP in addition to SQL?
Subbu K. K
?---------------------------------------------
K. K. Subramaniam
Product Manager, Novell Directory Services (UNIX)
Novell
>>> <Martin.Rahm@nokia.com> 06/07/00 12:27PM >>>
Hi,
I am working on a problem where LDAP would be used to keep a directory with
user data. One concern I have is that LDAP is supposed to be less efficient
when it comes to intense write operations.
Can LDAP handle millions of users data in a directory where some of the data
(a few attributes) needs to be updated very frequently? How would that
affect performance and is LDAP effective when the searched data must be
returned prompty with very little delay?
If anyone has any comments on this, I would appreciate it,
Martin Rahm