Re: db stick in RAM ?

[DT Piotr Wadas <pwadas@dtpw.pl>]

Hello,
I've tried this ramdisk concept a few months ago - speed increase is huge,
anyway, at least with slapd version from that time, there are some
unexpected data integrity problems ( sometimes for unknown reason
some data are likely to become corrupted ).

Regarding SSD disks - for now devices with bigger transfer rate are quite 
expensive, especially bigger capacities, IMO that should work. Note bdb 
checkpoint cache (depending on your configuration ), may contain more 
data, than data itself, e.g. with ~128MB of data one may have 300-500MB of 
log000.* files. Checkpointing configuration and amount of data should be 
taken into consideration with planning ramdisk storage capacity.

Regarding local syncrepl - slave-initiated gives performance impact,
while syncrepl master-initiated scenario (master + syncrepl + 
ldap-backend on one side ) with current version has some problem with 
proxying/detectking dsa read-only attributes like contextDSN etc.

Note - ramdisk storage also brings power failure problem - if server goes 
down unexpectedly, after power back there's no datadir at all, and before 
you get everything up, you need to restore it (automatically ? ), or use 
external source ( backup ldap server ).

The only thing which works fine with such ideas is to use ramdisk
as temporary-in-the-middle while restoring slapcat backup. Restoring full
backup with several thousands of entries (including reindexing), in
to-disk scenario takes at least 15-20 minutes, while restoring to ramdisk,
and copying the whole data dir from ramdisk to storage media ( with 
dedicated script ), takes about 2-3 minutes ( also including reindexing,
before moving restored backup from ramdisk to HDD ).

Regards,
DT 

On Tue, 19 Oct 2010, Frank Bonnet wrote:

> 
> Perhaps running syncrpl between ramdisk and hardisk ?
> 
> 
> On 10/19/2010 11:00 AM, Benjamin Griese wrote:
> > Hi Frank,
> > 
> > by rsync to hdd, do you mean copying the db files or slapcat the
> > output to a file?
> > The first one seems to be a db killer if the db is in use while copying it.
> > :)
> > I think the only safe way is by doing a slapcat after every write to
> > the db (accesslog?).
> > 
> > Anyway, sounds like a nice idea to me to get a big performance boost.
> > Have you considered SSDs instead of ramdisks?
> > Imo its the more safe and nearly as fast as ramdisk alternative. :)
> > 
> > Bye, Benjamin.
> > 
> > On Tue, Oct 19, 2010 at 10:40, Frank Bonnet<f.bonnet@esiee.fr>  wrote:
> > > Hello
> > > 
> > > I'm thinking to put db files in RAM instead of hard disk
> > > for performance reasons.
> > > 
> > > FreeBSD ( and Linux thought ) provide some utility to build
> > > RAMDISK ( md )
> > > 
> > > I did some tests in rebuilding db files with the slapcat/slapadd
> > > commands
> > > 
> > > with RAMDISK
> > > 
> > > ldap3# time slapadd -l test.ldif
> > > /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted
> > > unlimited privileges.
> > > #################### 100.00% eta   none elapsed             20s spd 633.9
> > > k/s
> > > 
> > > WITH HARDDISK
> > > 
> > > ldap3# time slapadd -l test.ldif
> > > /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted
> > > unlimited privileges.
> > > #################### 100.00% eta   none elapsed          04m22s spd  48.4
> > > k/s
> > > 
> > > Does anyone runs production servers with ramdisk ?
> > > 
> > > I know it is risky but running rsyncd between ramdisk and a hardisk depot
> > > would be safe huh ?
> > > 
> > > Thanks for any advices
> > > 
> > > 
> > 
> > 
>