|
|
|
|
|
|
| Author |
Message |
Doug V
*nix forums beginner
Joined: 05 Jun 2005
Posts: 8
|
 Posted: Thu Jul 20, 2006 4:16 pm Post subject:
Slow Connectivity via Gigabit
|
|
|
Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
Netgear GS108 Gigabit Switch
Comp1
---------
P3 500 mhz
Fedora Core 5 2.6.17 kernel
Reiserfs
Netgear GA311 gigabit PCI card
[root@comp1]# ifconfig
eth1 Link encap:Ethernet HWaddr 00:14:6C:33:42:D8
inet addr:192.168.1.1 Bcast:192.168.1.255
Mask:255.255.255.0
inet6 addr: fe80::216:6cff:fe35:40d8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:7200 Metric:1
RX packets:4690518 errors:0 dropped:0 overruns:0 frame:0
TX packets:8788694 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:248804322 (237.2 MiB) TX bytes:2073137731 (1.9 GiB)
Interrupt:3 Base address:0xa800
[root@comp1]# ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
Link detected: yes
Comp2
--------
P3 500 mhz
Fedora Core 5 2.6.17 kernel
ext3
Netgear GA311 gigabit PCI card
[root@comp2]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:14:6C:35:49:4D
inet addr:192.168.1.2 Bcast:192.168.1.255
Mask:255.255.255.0
inet6 addr: fe80::216:6cff:fe35:494d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:7200 Metric:1
RX packets:4095533 errors:0 dropped:0 overruns:0 frame:0
TX packets:1540683 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4246062689 (3.9 GiB) TX bytes:114723964 (109.4 MiB)
Interrupt:3 Base address:0x2400
[root@comp2]# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
Link detected: yes |
|
| Back to top |
|
 |
Michael Heiming
*nix forums Guru
Joined: 19 Feb 2005
Posts: 1423
|
| Posted: Thu Jul 20, 2006 5:41 pm Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
In comp.os.linux.networking dbv@hotmail.com:
| Quote: | Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
|
Playing with MTU size might not be that helpful if the switch
doesn't support them?
| Quote: | RX packets:4690518 errors:0 dropped:0 overruns:0 frame:0
TX packets:8788694 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
[ stuff ] |
| Quote: | [root@comp2]# ethtool eth0
Speed: 1000Mb/s
Duplex: Full
|
Looks good. I'd try to rule out the switch at first, connect the
boxes directly, with GB nics you might not even need a
cross-over, many nics seem to adjust that. And see if this gives
more speed?
To rule out CPU/etc try ftp/nfs to transfer files.
Good luck
--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 405: Sysadmins unavailable because they are in
a meeting talking about why they are unavailable so much. |
|
| Back to top |
|
|
Unruh
*nix forums Guru
Joined: 22 Mar 2005
Posts: 1166
|
| Posted: Thu Jul 20, 2006 8:44 pm Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
Michael Heiming <michael+USENET@www.heiming.de> writes:
| Quote: | In comp.os.linux.networking dbv@hotmail.com:
Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
Playing with MTU size might not be that helpful if the switch
doesn't support them?
RX packets:4690518 errors:0 dropped:0 overruns:0 frame:0
TX packets:8788694 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
[ stuff ]
[root@comp2]# ethtool eth0
Speed: 1000Mb/s
Duplex: Full
Looks good. I'd try to rule out the switch at first, connect the
boxes directly, with GB nics you might not even need a
cross-over, many nics seem to adjust that. And see if this gives
more speed?
To rule out CPU/etc try ftp/nfs to transfer files.
|
I would worry that the actual transfer by the computers themselves is the
limitation here. scp requires that all of th epackets be encrypted and
decrypted. That takes time and effort. Plus the load on the machine. Since
this cannot simply be piped out ( the data is transformed) it is going to
slow things down due to time slicing as well. |
|
| Back to top |
|
|
Rick Jones
*nix forums Guru
Joined: 24 Feb 2005
Posts: 492
|
| Posted: Fri Jul 21, 2006 12:11 am Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
dbv@hotmail.com wrote:
| Quote: | Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
|
7200 byte MTU? Where did that come from? "Proper" JumboFrame support
in GbE is a 9000 byte MTU. What happens if you keep the MTU like that
of 100BT?
FWIW, you now have experienced firsthand that changing just one
component of a multicomponent system may not have as large an effect
as the difference in the changed components themselves. File transfer
scp isn't "just" networking.
Things you need to investigate:
*) How much idle CPU did you have when transferring over 100BT? In
and of itself, gigabit ethernet does _nothing_ to make transfer any
easier on a host than 100BT. Just as 100BT did nothing in and of
itself to make transfer any easier on the host than 10BT. In
broadhandwaving terms, for "ethernet" links it takes just as many
CPU cycles to transfer a KB of data over 10BT as it does for 100BT,
or Gigabit, or even 10 Gigabit Ethernet.
Now, there can be differences in how NICs _implement_ those things
that will alter the level of effort required to transfer data, but
none of them are fundamental to the underlying "Ethernet" specs. Eg
ChecKsum Offload, Transport Segmentation Offload, non-standard MTUs.
So, if you were at 60ish percent CPU util for your 4.5 mb/s (did
you really mean to say 'B' for bytes instead of 'b' for bits?)
transfer, and the network were the bottleneck, it would stand to
reason that switching from 100BT to 1Gbit would not let you go more
than 50% faster than you did before.
*) How busy were your discs when you were transferring over 100BT?
The same sort of logic applies there.
*) What is the bandwidth-delay product of your connection? One limit
to the performance of a TCP connection (and IIRC scp uses TCP) is
the window size divided by the Round-Trip-Time (RTT) of the
connction. Going from 100BT to 1Gig does not mean the RTT will be
1/10th what it was before, so perhaps a larger TCP window is
required to achieve higher throughput over the Gigabit link. This
can be true even over a LAN.
*) What manner of I/O bus(ses) do you have in the system and what was
their level of utilization when transferring over 100BT?
If you want to see the network-level differences in isolation from the
rest of the components in the system you need to run a networking
benchmark - eg netperf - http://www.netperf.org/
--
firebug n, the idiot who tosses a lit cigarette out his car window
these opinions are mine, all mine; HP might not want them anyway... 
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH... |
|
| Back to top |
|
|
Allen McIntosh
*nix forums Guru Wannabe
Joined: 22 Mar 2005
Posts: 160
|
| Posted: Fri Jul 21, 2006 1:52 am Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
Rick Jones wrote:
| Quote: | dbv@hotmail.com wrote:
Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
....
Things you need to investigate:
*) How much idle CPU did you have when transferring over 100BT?
|
Indeed. Don't forget also that scp requires CPU cycles at each end for
encryption/decryption. If you are somewhere relatively secure, try
using FTP and see if that makes a difference.
There is also the matter of window scaling. You really need it working
in order to get the full benefit of gigabit speeds. Some versions of
the kernel won't do it unless you tweak some of the TCP parameters. (I
haven't looked at recent kernels, so this may have been fixed.) I don't
think you are running into this, but you may once you remove other
things that are keeping the transfer rate down. |
|
| Back to top |
|
|
Thomas Zajic
*nix forums beginner
Joined: 21 Jul 2006
Posts: 2
|
| Posted: Fri Jul 21, 2006 6:46 am Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
* Michael Heiming <michael+USENET@www.heiming.de>:
| Quote: | In comp.os.linux.networking dbv@hotmail.com:
Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. [...]
Playing with MTU size might not be that helpful if the switch
doesn't support them?
|
Should be fine, provided that its S/N is within the supported range:
http://kbserver.netgear.com/kb_web_files/n101539.asp
Thomas
--
=-------------------------------------------------------------------------=
- Thomas "ZlatkO" Zajic <zlatko@gmx.at> Linux-2.6.17 & slrn-0.9.8.1 -
- "It is not easy to cut through a human head with a hacksaw." (M. C.) -
=-------------------------------------------------------------------------= |
|
| Back to top |
|
|
Thomas Zajic
*nix forums beginner
Joined: 21 Jul 2006
Posts: 2
|
| Posted: Fri Jul 21, 2006 6:49 am Post subject:
Re: Slow Connectivity via Gigabit
|
|
|
* Rick Jones <rick.jones2@hp.com>:
| Quote: | dbv@hotmail.com wrote:
Recently I upgraded to gigabit, however the increase in speed I got was
far less than I thought. Copying a 20G tar file using scp transfers at
around 6.5 mb/sec, which while greater than the 4.5 mb/sec I used to
get using 100 mbps connection, is far less than the 15-20 mb/sec I
though I'd get. The gigabit lights are displayed correctly on both PCI
cards, as well as the switch. I put in new 3 ft cat 6 cables for both
machines. There are other computers connected to the switch which are
running at 100 mbps, including a separate 100mbps card on comp1. MTU is
set to 7200 on both machines. Does anyone have any idea how to speed it
up?
7200 byte MTU? Where did that come from? "Proper" JumboFrame support
in GbE is a 9000 byte MTU. What happens if you keep the MTU like that
of 100BT?
|
The GA311's maximum supported frame size is 7500 bytes, according to
http://kbserver.netgear.com/kb_web_files/n101539.asp, so it seems to
be a harware limitation in this case.
Thomas
--
=-------------------------------------------------------------------------=
- Thomas "ZlatkO" Zajic <zlatko@gmx.at> Linux-2.6.17 & slrn-0.9.8.1 -
- "It is not easy to cut through a human head with a hacksaw." (M. C.) -
=-------------------------------------------------------------------------= |
|
| Back to top |
|
|
Google
|
|
| Back to top |
|
|
|
|
The time now is Sun Nov 23, 2008 3:03 pm | All times are GMT
|
|
Loans | Loans | Names | Israel Perry | Car Finance
|
|
Copyright © 2004-2005 DeniX Solutions SRL
|
|
|
|
Other DeniX Solutions sites:
Unix/Linux blog |
electronics forum |
medicine forum |
science forum |
|
|
Privacy Policy
|
Powered by phpBB © 2001, 2005 phpBB Group
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Forum index
»
*nix
»
Linux
»
networking
