|
|
|
|
|
|
| Author |
Message |
fabrice-pas-despame.bacch
*nix forums beginner
Joined: 01 Nov 2005
Posts: 8
|
 Posted: Mon Jun 26, 2006 9:08 pm Post subject:
TCP stack of Solaris
|
|
|
Sun says that they improved their TCP stack. But I think their is
still a lot of work to do.
See those 2 graph :
http://www.box.net/public/tdi16rjlsy
http://www.box.net/public/a4pa9cdpd4
I used NetPipe to generate them (http://www.scl.ameslab.gov/netpipe/).
They have been generated using two DL385, linked by a Cisco 3750
switch.
Can someone explains to me why the Solaris graphs are so ugly ? |
|
| Back to top |
|
 |
Rick Jones
*nix forums Guru
Joined: 24 Feb 2005
Posts: 492
|
| Posted: Mon Jun 26, 2006 9:32 pm Post subject:
Re: TCP stack of Solaris
|
|
|
fabrice-pas-despame.bacchella@worldonline.fr wrote:
Might be nice to label each axis and provide a bit more detail about
things like socket buffer sizes, send sizes etc etc.
It would also be good to include "service demand" figures. Service
demand is what netperf calculates based on the throughput and the CPU
utilization - it says how much CPU was used to perform a given unit of
work. I've not looked closely at netpipe, but would expect it to have
some sort of similar thing available.
| Quote: | They have been generated using two DL385, linked by a Cisco 3750
switch.
|
Using the core interface(s) on the DL385? I am not from the ProLiant
side of the house, but I believe there are several variations on
"DL385" so more details would be goodness.
| Quote: | Can someone explains to me why the Solaris graphs are so ugly ?
|
For the latency, that rather looks like results I've gotten in other
situations with netperf TCP_RR tests when the NIC (typically gigabit -
is that what you were using?) has been hard set to a certain interrupt
limit. Latency tests will not often hit the "packets to trigger an
interrupt" component which then often leaves waiting for the timer on
the NIC to expire. That then does arguably nasty things to the
minimum latency reported by single-instance tests. Often as not the
tradeoff was for (potentially) a higher aggregate packet-per-second
limits. Basically, tuning for bandwidth rather than latency.
Of course, I've no idea that is exactly what is going-on here, but it
would be an avenue to explore. On the Linux side I suspect you would
manipulate the "coalescing" settings via ethtool or perhaps
modules.config (?). For Solaris I suspect it would be ndd.
rick jones
--
The glass is neither half-empty nor half-full. The glass has a leak.
The real question is "Can it be patched?"
these opinions are mine, all mine; HP might not want them anyway... 
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH... |
|
| Back to top |
|
|
fabrice-pas-despame.bacch
*nix forums beginner
Joined: 01 Nov 2005
Posts: 8
|
| Posted: Mon Jun 26, 2006 10:39 pm Post subject:
Re: TCP stack of Solaris
|
|
|
On Mon, 26 Jun 2006 21:32:15 GMT, Rick Jones <rick.jones2@hp.com>
wrote:
| Quote: | fabrice-pas-despame.bacchella@worldonline.fr wrote:
Sun says that they improved their TCP stack. But I think their is
still a lot of work to do.
See those 2 graph :
http://www.box.net/public/tdi16rjlsy
http://www.box.net/public/a4pa9cdpd4
I used NetPipe to generate them (http://www.scl.ameslab.gov/netpipe/).
Might be nice to label each axis and provide a bit more detail about
things like socket buffer sizes, send sizes etc etc.
|
I've just updated them with more informations. The new URLS are :
http://www.box.net/public/8i95tcavfr
http://www.box.net/public/urxqioy5hs
Solaris 11's build is 41, not 44, don't dream about a secret release,
my fault.
socket buffer size where default one :
16384 and 87380 for send and receive buffers on Linux
49152 for both on Solaris 10 & 11
The program says : A bug in Linux doubles the requested buffer sizes.
Send size is what NetPipe change
| Quote: |
It would also be good to include "service demand" figures. Service
demand is what netperf calculates based on the throughput and the CPU
utilization - it says how much CPU was used to perform a given unit of
work. I've not looked closely at netpipe, but would expect it to have
some sort of similar thing available.
Nop, but I will have a look using netperf. |
| Quote: |
They have been generated using two DL385, linked by a Cisco 3750
switch.
Using the core interface(s) on the DL385? I am not from the ProLiant
side of the house, but I believe there are several variations on
"DL385" so more details would be goodness.
|
The NIC :
03:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 10)
The processors : 2 x Opteron 265 ( 2 x 1.8 Ghz)
The RAM : 16 GBytes
By the way, could please tell the Proliant guys to finish porting all
the tools to Solaris ? We're still missing hpacucli and things like
that here.
| Quote: | Of course, I've no idea that is exactly what is going-on here, but it
would be an avenue to explore. On the Linux side I suspect you would
manipulate the "coalescing" settings via ethtool or perhaps
modules.config (?). For Solaris I suspect it would be ndd.
I don't think there is this kind of tuning on Solaris this ndd, but |
playing with /etc/system might be the way.
|
|
| Back to top |
|
|
Rick Jones
*nix forums Guru
Joined: 24 Feb 2005
Posts: 492
|
| Posted: Mon Jun 26, 2006 11:40 pm Post subject:
Re: TCP stack of Solaris
|
|
|
fabrice-pas-despame.bacchella@worldonline.fr wrote:
| Quote: | socket buffer size where default one :
16384 and 87380 for send and receive buffers on Linux
49152 for both on Solaris 10 & 11
The program says : A bug in Linux doubles the requested buffer sizes.
|
While I agree with the netpipe people that Linux's behaviour WRT
SO_*BUF in setsockopt/getsockopt is annoying, my understanding is that
it isn't so much doubling the requested size as it is setting the
"overhead" (size of the skb's (eg mbufs or mblks depending on your
history)) limit to 2X what one asks to be the data limit in a
setsockopt() call, and then getsockopt() returns the overhead limit
not necessarily the data limit.
I could though be wrong there.
| Quote: | Send size is what NetPipe change
It would also be good to include "service demand" figures. Service
demand is what netperf calculates based on the throughput and the CPU
utilization - it says how much CPU was used to perform a given unit of
work. I've not looked closely at netpipe, but would expect it to have
some sort of similar thing available.
Nop, but I will have a look using netperf.
|
BTW, I should warn you that this may be the first time the "kstat10"
CPU method in netperf has been used for Solaris x86. It may only have
been used on SPARC to this point. It would be good for you to take a
close look at the caveats in the comments in src/netcpu_kstat10.c
| Quote: | The NIC :
03:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 10)
The processors : 2 x Opteron 265 ( 2 x 1.8 Ghz)
The RAM : 16 GBytes
|
All good things to include in a report.
| Quote: | By the way, could please tell the Proliant guys to finish porting all
the tools to Solaris ? We're still missing hpacucli and things like
that here.
|
What, and take all the fun away from Sun?-) If I happen across some of
those folks I will try to let them know.
| Quote: | Of course, I've no idea that is exactly what is going-on here, but it
would be an avenue to explore. On the Linux side I suspect you would
manipulate the "coalescing" settings via ethtool or perhaps
modules.config (?). For Solaris I suspect it would be ndd.
I don't think there is this kind of tuning on Solaris this ndd, but
playing with /etc/system might be the way.
|
try:
ndd -get /dev/<basename> ?
where <basename> is whatever Solaris decided to call the interfaces
with the PPA stripped - eg ipge if you have ipge0 and ipge1
rick jones
--
The glass is neither half-empty nor half-full. The glass has a leak.
The real question is "Can it be patched?"
these opinions are mine, all mine; HP might not want them anyway...
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH... |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Tue Dec 02, 2008 1:27 pm | All times are GMT
|
|
Apply for Credit Card | Loans | 0 Credit Cards | Web Advertising | Mobile Phones
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
*nix
»
Solaris
»
x86
