Geek Answers Handbook

Tokyo Cabinet - Slow insertions after hitting 1 million

I appreciate the Tokyo closet. Input speed slows significantly after reaching 1 million records. The lot size is 100,000 and is executed in a transaction. I tried installing xmsiz, but it is useless anyway. Has anyone encountered this problem in a tokyo office?

More details

Tokyo office - 1.4.3
Perl bindings - 1.23
OS: Ubuntu 7.10 (VMWare player on top of Windows XP)

+5
source share
3 answers

1 ( , ). ttserver, , , , ,

echo 80 > /proc/sys/vm/dirty_ratio

( 10) - ( 8 , node) :

total:  14238792  records,  27.5881 GB size
total:  14263546  records,  27.6415 GB size
total:  14288997  records,  27.6824 GB size
total:  14309739  records,  27.7144 GB size
total:  14323563  records,  27.7438 GB size
(here I changed the dirty_ratio setting for all shards)
total:  14394007  records,  27.8996 GB size
total:  14486489  records,  28.0758 GB size
total:  14571409  records,  28.2898 GB size
total:  14663636  records,  28.4929 GB size
total:  14802109  records,  28.7366 GB size

, , 7-8 . 4,5 node ( ), 8 ( dirty_ratio 10 , ​​ 800 ).

, , ext2 ( : ext3) noatime, ramdisk (, , , ).

+4

, .

+2

I think that changing the bnum parameter in the dbtune function will also give a significant speed improvement.

+1
source

More articles:


All Articles