Re: (ITS#8813) MDB_VL32 causes MDB_TXN_FULL

[markus@greenrobot.de]

On 24.10.19 03:49, Howard Chu wrote:
> You're free to define MDB_TRPAGE_MAX to a larger value. It just means
> you increase the chance of overrunning the 2GB available address space.
> There's no magic, you can't fit every 64bit database workload into only
> a 32bit address space. When your transactions are too large, the normal
> thing to do is commit more often so they don't grow so large.

I think you meant MDB_TRPAGE_SIZE? At least that seemed to work, while 
MDB_TRPAGE_MAX ended up in another MDB_TXN_FULL (tl[0].mid < 
MDB_TRPAGE_SIZE check failed).

Doubling MDB_TRPAGE_SIZE also doubled the threshold of the object count 
where it starts failing: using 8192 it was able to remove 4M entries 
while the limit with 4096 was 2M entries.

MDB_TRPAGE_SIZE is only used to malloc txn->mt_rpages (and some checks), 
as far as I can tell.

To make a reasonable decision here, could you please confirm:

1. txn->mt_rpages is RAM only and has no impact on mmaped sections 
(and/or file format).

2. RAM consumption is defined by number of transactions only: per 
transaction MDB_TRPAGE_SIZE bytes is allocated. Other than that it does 
not increase memory consumption.

3. There is no other disadvantage other than higher memory consumption 
per transaction as written in the previous point.


In that case, I think I'll go with 16K (or even 32K). Does not seem too 
much for an transaction, given that it quadruples the amount of data 
allowed to be processed. 32 bit devices are not servers, so I do not 
expect a high number of concurrent transactions anyway.

Thanks!

Markus