java collections api + out of memory on load

sunfishy@gmail.com · *nix forums beginner Joined: 03 May 2006 Posts: 1

Hi,

I'm a newbie to berkeley db, so apologies in advance for any silly
questions...

I'm reading a large file into my berkeley db using the collections API.
I am using JVM settings of 512 MB, but I keep running out of memory,
perhaps because I am loading it in a very simple way: I am currently
doing a map.put() for each line. Why would there be such a large memory
footprint if I am just adding entries to the db? Is there a faster way
to load in a large, sorted, formatted file?

Thanks very much!

Michael Cahill · *nix forums Guru Wannabe Joined: 26 May 2005 Posts: 219

Hi,

Out-of-memory errors can be caused by lots of things. The first thing
to do is to run "db_stat -e" in your environment directory, and look at
the cache and locking regions to see whether some resource has been
exhausted.

Also check how much cache you have allocated, the amount of memory
available to the JVM, and so on. The defaults are quite conservative
and shouldn't cause problems, but if you've allocated a bigger cache
than you have memory for (for example), that could cause problems.

Regards,
Michael.

Google