time to insert 100,000 key,data pairs into DB_BTREE is 2s!!!

berkeleydb_user · *nix forums beginner Joined: 16 Mar 2005 Posts: 17

Probably once. It could go upto half million or more. I have a memory
only implementation of B+tree and I could do about 400000 under 2s. But
my program is still nascent. I understand that Berkeley DB is more
generic. Is there some tuning that I can do to make DB faster?

thanks

Patrick Schaaf · *nix forums beginner Joined: 28 May 2005 Posts: 36

"berkeleydb_user" <prasad.koya@gmail.com> writes:

berkeleydb_user · *nix forums beginner Joined: 16 Mar 2005 Posts: 17

I tried this with sorted input and the response is about 600-700ms
faster. But I don't expect my input (keys) to be sorted.

berkeleydb_user · *nix forums beginner Joined: 16 Mar 2005 Posts: 17

Given that key type (data) is opaque, what would you rather recommend
for insertion sort?

Patrick Schaaf · *nix forums beginner Joined: 28 May 2005 Posts: 36

"berkeleydb_user" <prasad.koya@gmail.com> writes:

berkeleydb_user · *nix forums beginner Joined: 16 Mar 2005 Posts: 17

I want the database to sort it. Thats why i want to use BTree. I want
fast searching, sorting, insertion, read keys in order. Thats why I
have to use BTree based database.

Michael Cahill · *nix forums Guru Wannabe Joined: 26 May 2005 Posts: 219

Try loading all the words into memory and sorting with qsort before
doing the insert.

Regards,
Michael.

berkeleydb_user · *nix forums beginner Joined: 16 Mar 2005 Posts: 17

Hi

I have a simple memory-only DB, configured to use DB_BTREE during
open() time. I am inserting 100,000 unsorted dictionary words into this
with no transactions and cache size set to 10MB. It takes 2s on a 1GHz
x86 processor. The average key size is about 10 bytes. Can I make it
faster? I am expecting in the order of 200 or 300ms not this high.

thanks

Google