Berkeley DB 4.2 slower on Linux 2.6.10?

sudarshan.purohit@gmail.c · *nix forums beginner Joined: 05 Apr 2005 Posts: 3

Hi Michael,
I tried using the DB_NOSYNC flag in the DB->close calls, and this
immediately brought the performance up to the same levels as the BDB
2.7.7 version. Thanks for the help.
I'm a little confused about why this should make a difference,
though. Is there someplace in the documentation which explains when
this flag should be used? The DB->close page talks of it as a pretty
dangerous option to be used, which could result in loss of data.
Oddly, I used the same option in the 2.7.7 version of the app, and
it made no difference at all to performance.
Would be great if you could point me to some documentation/books
that explain such issues in detail.
In any case, thanks for the help. This post is mainly to let others
know what worked.

Regards,
Sudarshan

Michael Cahill · *nix forums Guru Wannabe Joined: 26 May 2005 Posts: 219

One more question: what do your DB->open calls look like? In
particular, is the subdb parameter NULL?

Regards,
Michael.

Michael Cahill · *nix forums Guru Wannabe Joined: 26 May 2005 Posts: 219

Hi Sudarshan,

Opening and closing lots of little databases is never going to be
optimal -- is it possible to restructure the data so it is in a smaller
number of larger databases?

If not, can you try adding the DB_NOSYNC flag to DB->close calls? The
sync shouldn't be required as you're keeping the environment open for
longer than the databases.

Regards,
Michael.

sudarshan.purohit@gmail.c · *nix forums beginner Joined: 05 Apr 2005 Posts: 3

Hi,
WHOOPS. My bad : I retested the system on kernel 2.4.20 with higher
loads, and there is a slowdown (50%) from BDB 2.7 to 4.2.52 as well. So
this has nothing to do with the kernel, sorry. It is only related to
the BDB upgrade. To restate: When I upgrade my app from BDB 2.7.7 to
4.2.52, the app slows down.
The application is a DB server daemon, a layer over the Berkeley DB
API. It is continously processing DB Open, Close, Get and Put commands,
accessing several thousand small database files. The slowdown is
observed for all of these commands. The CPU usage goes up from about
4-5% to about 40% when I moved from BDB 2.7.7 to BDB 4.2.52. (which
makes me think the problem is with I/O; CPU usage is not maxing out).
The application is not multithreaded, there is only a simple for-loop,
processing DB commands from multiple clients.
The file system is ext3 in all cases.
I wasn't able to run db_stat because the logging is disabled; we
use our own logging mechanism. But an OProfile run revealed the
following figures: [All runtime parameters etc. are the same;
'dbserver' is my app]

[With BDB 4.2.52 on Linux 2.6.10, top 10 ]
samples % app name symbol name
226088 36.3760 no-vmlinux (no symbols)
123360 19.8478 dbserver __dblist_get
114421 18.4095 dbserver __memp_close_flush_files
92465 14.8770 dbserver __db_dbenv_setup
24527 3.9462 dbserver __memp_sync_int
20142 3.2407 dbserver __memp_fopen
7392 1.1893 dbserver __memp_bhwrite
724 0.1165 libc-2.3.2.so _IO_vfprintf_internal
505 0.0813 libc-2.3.2.so _int_malloc
364 0.0586 dbserver __memp_fget

[for comparison, with BDB 2.7.7 on Linux 2.6.10, top 10 ]
samples % app name symbol name
618510 98.1734 no-vmlinux (no symbols)
715 0.1135 libc-2.3.2.so _IO_vfprintf_internal
419 0.0665 libc-2.3.2.so _int_malloc
344 0.0546 libc-2.3.2.so __GI___strtol_internal
326 0.0517 dbserver Client::HandleConnection()
309 0.0490 libc-2.3.2.so _IO_default_xsputn_internal
249 0.0395 dbserver memp_fget
235 0.0373 dbserver Client::FlushDBT()
235 0.0373 dbserver minimal_diff(char const*,
char const*, unsigned, iovec*, int, int)
231 0.0367 libc-2.3.2.so __GI_memcpy

Note that in the 2.7.7 version, the 'memp_fget' method is the only BDB
method that even shows up in the top 10. The total CPU taken by all my
own methods is hardly 4% in both cases. The increase is due to the
__dblist_get, __memp_close_flush_files, __db_dbenv_setup,
__memp_sync_int, and __memp_fopen methods. These look like they're all
related to I/O.

Hope this clarifies.

Thanks for the help...
Sudarshan

Michael Cahill · *nix forums Guru Wannabe Joined: 26 May 2005 Posts: 219

Hi,

That's not much to go on -- can you explain what your application is
doing when you are measuring this slowdown? Is it using 100% of CPU,
or I/O bound, or are there a lot of threads blocking on locks? Are you
using the same filesystem type after the upgrade? Can you post the
environment statistics by running "db_stat -e" in the environment
directory?

Have you tried a profiling tool such as oprofile
(http://oprofile.sf.net)?

Regards,
Michael.

sudarshan.purohit@gmail.c · *nix forums beginner Joined: 05 Apr 2005 Posts: 3

Hi folks,
I recently started work on an app using Berkeley DB 2.7.7, on RH
Linux 9, with 2.4.20 kernel, on an x86 system. I ported the app to use
BDB 4.2.52, and saw no performance degradation.
Then, I upgraded my system to Kernel 2.6.10. The 2.7.7 version
shows comparable performance to the older version, but the 4.2 version
has a noticeable slowdown (nearly 50%).
I'm not really sure what more info to give. I went through the
environment settings for 4.2 and tried modifying cache sizes, page
sizes, whatever flags seemed relevant. Nothing seems to help.

Has someone else seen this? Is this a known problem?

Thanks in advance!

Sudarshan Purohit

Google