BDB and 4GB+ RAM on 32bit system

Howard Chu · *nix forums beginner Joined: 14 Feb 2005 Posts: 15

studdugie@gmail.com · *nix forums beginner Joined: 21 Jun 2005 Posts: 28

I'm trying to figure out what is the best way to give BerkeleyDB
4.3.27NC on Linux 2.6.xx, 4GB or more of RAM w/o the amount of RAM
given to BDB counting against the process that started BDB. I've read
the documentation at least 100x times and so far the only thing I'm
sure about is using DB_PRIVATE would cause any memory BDB allocated to
count against the process that started BDB. I don't fully understand
the interaction between BDB and the OS' memory management facilities so
I'm not quite sure how the shared memory and/or file system backed
memory options work insofar as counting against my process. It may be
useful to note that I'm running a single process/threaded application
that does not use transactions, locking, replication, etc.

If I had to express what I'm trying to do as a series of questions, it
would go like this:

Is it possible for a process to open and use BDB w/o the memory that
BDB uses counting as part of the memory of the process that opened BDB?

What is maximum amount of memory BDB can use, where memory consists of
the BDB cache in an Environment and database cache(s) outside of an
Environment?

I'm pasting some sample code of how I've got it setup now.
Unfortunately, I'm not able to test it because the memory that I've
ordered for the box hasn't arrived yet. So while I wait for it to get
here I figured I would ask what limitations I'm going to run up
against.

<code>
/*Note to helpful group reader:
* The tails, indexDb, and rindexDb are the databases the need the 4GB
cache.
* They are going to be the only databases that will participate in an
Environment.
* All other databases will exist outside of any Environment so I set
the cache directly
* on the database object.*/

BDB_PAGE_SIZE = Bytes.KILOBYTE << 6;

EnvironmentConfig envc = new EnvironmentConfig();
envc.setAllowCreate( true );
envc.setInitializeRegions( true );
envc.setInitializeCache( true );
envc.setCacheSize( Bytes.GIGABYTE << 2 );
sysenv = new Environment( SYSTEM_DIR, envc );

DatabaseConfig bdbConfig = new DatabaseConfig();
bdbConfig.setAllowCreate( true );
bdbConfig.setType( DatabaseType.HASH );
tails = sysenv.openDatabase( null, "tails", null, bdbConfig );

bdbConfig.setPageSize( BDB_PAGE_SIZE );
indexDb = sysenv.openDatabase( null, "indexDb", null, bdbConfig );

bdbConfig.setType( DatabaseType.BTREE );
rindexDb = sysenv.openDatabase( null, "rindexDb", null, bdbConfig );

SequenceConfig seqc = new SequenceConfig();
seqc.setAllowCreate( true );
seqc.setWrap( false );
seqc.setAutoCommitNoSync( true );
seqc.setInitialValue( 1 );
seqc.setRange( 1, 1L << 32 );
sequence = rindexDb.openSequence( null, new BigEndianIntegerEntry( 0 ),
seqc );

bdbConfig.setUnsortedDuplicates( true );
bdbConfig.setCacheSize( BDB_PAGE_SIZE << 1 );
logLines = new Database( new File( STAGING, name.concat( dbSuffix )
).getAbsolutePath(), null, bdbConfig );
</code>

Thanxs,

Dane

Google