Kristoffer Sjögren wrote:
My idea is to have one bit index per logical column+value stored in a value where the key also have some means of partitioning the data, maybe over time. So high cardinality columns will generate lots of keys+values. I was thinking of storing multiple indexes in each key+value, making values bigger but fewer. Say around a few hundreds kilobytes each. Are there any trade offs between many smaller key+value vs fewer larger ones? I'm more concerned about read performance.
There would be a speed advantage to using fewer keys+larger values. Search performance is O(logN) where N is the number of keys...
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/