[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
Re: LMDB mmap usage
Ok, so the page cache, that's exactly what I want to read more about.
Not for tuning, but just to understand how the page cache works with
the different subsystems in order to dimension the hardware properly.
For example what IO devices affect the page cache? Why are memory
pages reshuffled between different NUMA nodes? etc..
On Sat, Feb 6, 2016 at 7:15 PM, Jay Booth <jaybooth@gmail.com> wrote:
> Mmapped memory goes through the exact same page cache as all other
> filesystem data -- you probably want to do exactly nothing for optimal
> performance, the kernel will be smart about what it keeps paged in
>
> On Sat, Feb 6, 2016 at 12:44 PM, Kristoffer Sjögren <stoffe@gmail.com>
> wrote:
>>
>> I'm not looking for fairies :-) I was just curious exactly how mmap
>> interacts with the OS in order to understand performance metrics
>> better.
>>
>> On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu <hyc@symas.com> wrote:
>> > Kristoffer Sjögren wrote:
>> >>
>> >> Thanks Howard. What's the best documentation on mmap you know, aside
>> >> from read kernel source code?
>> >
>> >
>> > Not sure what you're asking. Read the mmap(2) manpage and its related
>> > pages.
>> > If you're thinking there's something fancy hidden in there for you to
>> > take
>> > advantage of, I don't think you should be using mmap in the first place.
>> >
>> >
>> >> On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu <hyc@symas.com> wrote:
>> >>>
>> >>> Kristoffer Sjögren wrote:
>> >>>>
>> >>>>
>> >>>> Hi
>> >>>>
>> >>>> Our application do lots of caching using vmtouch, up to a point where
>> >>>> there isn't a lot of memory left on the machine. We would like to use
>> >>>> LMDB on the same machine to store around 40GiB data of a few hundred
>> >>>> million entries.
>> >>>>
>> >>>> How can we best understand the interaction and behavior of the OS
>> >>>> cache and sharing of memory between processes? Is LMDB doing
>> >>>> something
>> >>>> to help the OS?
>> >>>
>> >>>
>> >>>
>> >>> Nope, LMDB does nothing special with its mmap.
>> >>>
>> >>> --
>> >>> -- Howard Chu
>> >>> CTO, Symas Corp. http://www.symas.com
>> >>> Director, Highland Sun http://highlandsun.com/hyc/
>> >>> Chief Architect, OpenLDAP http://www.openldap.org/project/
>> >>
>> >>
>> >
>> >
>> > --
>> > -- Howard Chu
>> > CTO, Symas Corp. http://www.symas.com
>> > Director, Highland Sun http://highlandsun.com/hyc/
>> > Chief Architect, OpenLDAP http://www.openldap.org/project/
>>
>