Ethernet Performance issues

[Lnx2Mac]

I've been researching the performance issues on AFP and SMB some of you were reporting...

Well, it seems you are not alone, and this is not a hackintosh nor a 0.0.67 related issue...

Several Apple users have been experiencing this.

I found some pointers which might be of interest to you (some have already been mentioned):

1) The reason... Straight from an Apple Researcher's desk
TCP Performance problems caused by interaction between Nagle's Algorithm and Delayed ACK
http://www.stuartcheshire.org/papers/NagleDelayedAck/
and more terse: http://www.netafp.com/tn003-slow-afp-re ... rmance-90/

2) Fixing the delayed_ack...
See: http://www.small-tree.com/kb_results.asp?id=1
Note than instead of modifying the /etc/sysctl.conf and rebooting, you can issue a similar command in the terminal.
For example:

• sudo sysctl -w net.inet.tcp.delayed_ack=0

Note however, that the results will not persist your next reboot.
For that you'll have to edit the /etc/sysctl.conf file.

3) Tuning send/receive buffers, etc.
See: http://www.small-tree.com/kb_results.asp?ID=20 viewtopic.php?p=73005#p73005
Again, you can use the terminal command as above instead of editing the /etc/sysctl.conf and rebooting.

So... what parameter sets bring you the best results ?

 

[freituk]

Hello Lnx2Mac, thanks again for the support

Ok, just to clarify, this only happen with RTL8111E Chips?
Because i have 2 more macs here (MAc book pro and another hackintosh), and none have this network issues.

Another thing is, i dont use windows, so my problem is only on mac to mac.

Teste part:

Some info about my system
GA-X58A-UD3R (Rev 2.0)
Lan: RTL8111E
OSX 10.6.5 (Beta)
Router D-link dir-665 (Jumbo frame) with cat 6 cables.

Some SS from my first test playing around with delayed_ack:

delayed_ack=0

delayed_ack=1

delayed_ack=2

delayed_ack=3

About this

3) Tuning send/receive buffers, etc.
See: http://www.small-tree.com/kb_results.asp?ID=20
Again, you can use the terminal command as above instead of editing the /etc/sysctl.conf and rebooting.

Here i dont get it... what suppose to be se server and the client?

thanks Lnx2Mac
[]

 

[Lnx2Mac]

Hello Lnx2Mac, thanks again for the support

Ok, just to clarify, this only happen with RTL8111E Chips?

No, this seems to happen on every NIC, and as Stuart Cheshire describes in his post, it seems to be due to "a TCP performance problem resulting from a little-known interaction between Nagle's Algorithm and Delayed ACK".

Because i have 2 more macs here (MAc book pro and another hackintosh), and none have this network issues.

Another thing is, i dont use windows, so my problem is only on mac to mac.

I experienced this between my hack (OSX 10.6.4) and my MBP (OSX 10.5.8).

3) Tuning send/receive buffers, etc.
See: http://www.small-tree.com/kb_results.asp?ID=20
Again, you can use the terminal command as above instead of editing the /etc/sysctl.conf and rebooting.

Here i dont get it... what suppose to be se server and the client?

In principle, the server would be the one sharing the disk, however, the main difference between these in the article are just 2 parameters:

• kern.maxnbuf: Maximum number of I/O buffer headers metadata[/*:m:3d4bl37d]
• kern.maxvnodes: Maximum number of files (inodes) cache metadata[/*:m:3d4bl37d]

This will influence how much RAM is *allowed* to be used for cache buffers, but it's a soft-limit (the kernel will weigh the need for this vs. your apps needs).

Should be safe applying this to both machines.

Another interesting link (albeit outdated):
- http://www.macgeekery.com/tips/configur ... _revisited

It should be better testing this on RAM disks, so as to cancel the effect of HDD access...
See: http://www.macgeekery.com/tips/cli/ram_disks_made_easy

Enjoy,

 

[martinkou]

iperf is probably a better throughput performance test than SMB/AFP, in light of their protocol bugs. It can be installed via MacPorts on the command line and it works like this...

On one machine, you open an iperf server (can be Windows, Linux or another Mac)

Martin-Kous-MacBook-Pro:~ martinkou$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:   256 KByte (default)
------------------------------------------------------------

And on the test Hackintosh, you use iperf to connect to the server.

Martin-Kous-Mac:~ martinkou$ iperf -c 192.168.1.112
------------------------------------------------------------
Client connecting to 192.168.1.112, TCP port 5001
TCP window size:   129 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.110 port 49518 connected with 192.168.1.112 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.08 GBytes    931 Mbits/sec

For a good GE interface, the iperf throughput should be >800Mbps.

 

[Lnx2Mac]

martinkou,

Thanks for this.
See also: viewtopic.php?p=39355#p39355

 

[svar]

Can anyone tell me about what lines to use in step 3?
The articel link is ether deleted or moved.

 

[Lnx2Mac]

The page http://www.small-tree.com/kb_results.asp?ID=20 seems to be removed from their site, but I found it in Google's cache
http://webcache.googleusercontent.com/s ... %3FID%3D20

Quoted here for posterity

http://www.small-tree.com/kb_results.asp?ID=20[/url]]Suggested tuning parameters for a AFP client and AFP Server
Last Updated 9/6/2010
Here are some suggested tuneables we have found to increase AFP
performance between Mac clients and a Mac server.

-Bring up a Terminal/shell window

-become root user with:

sudo sh

(supply the password...it will not display as you type)

The result should be that the "prompt" changes from
a $ to a #

Copy these lines with your mouse and execute them (right-click/paste)
them into the terminal/shell window you became root on. The list
of commands are different based on the amount of memory
you have on your system as well as if it is a client or server.

For an AFP server:


For systems with >= 16GBytes of memory,

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "kern.maxnbuf=60000" >> /etc/sysctl.conf
echo "kern.maxvnodes=280000" >> /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

For systems with >= 8 GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "kern.maxnbuf=42000" >> /etc/sysctl.conf
echo "kern.maxvnodes=150000" >> /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

For systems with >=4 GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "kern.maxnbuf=60000" >> /etc/sysctl.conf
echo "kern.maxvnodes=280000" >> /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

or for systems with < 4GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "kern.maxvnodes=60000" >> /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf


For an AFP client:


For systems with >= 16GBytes of memory,

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

For systems with >= 8 GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

For systems with >=4 GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

or for systems with < 4GBytes of memory:

echo "net.inet.tcp.delayed_ack=2" > /etc/sysctl.conf
echo "net.inet.tcp.sendspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.recvspace=2097152" >> /etc/sysctl.conf
echo "net.inet.tcp.maxseg_unacked=32" >> /etc/sysctl.conf

The above changes take effect after a reboot.

Just keep in mind that you'll probably have to play with these, to find the best values for your systems.

Good Luck !
Lnx2Mac

 

[svar]

HI ang thanks
I now using this lines:

net.inet.tcp.delayed_ack=1
net.inet.tcp.sendspace=2097152
net.inet.tcp.recvspace=2097152
net.inet.tcp.maxseg_unacked=32
kern.maxnbuf=60000
kern.maxvnodes=280000

The first one has most impact on download speed, I cant say if the rest helped that mutch.
Anyone know what the max-min of the rest of the lines is?
Then I can try to change them

 

[Belmore]

WOW! The results of doing this are DRAMATIC on my build (i7 2600K, a Gigabyte P67A-UD3, 8G RAM). I initially added the following to /etc/sysctl.conf file and rebooted as suggested earlier in the thread:

net.inet.tcp.delayed_ack=2
net.inet.tcp.sendspace=2097152
net.inet.tcp.recvspace=2097152
net.inet.tcp.maxseg_unacked=32

I didn't perceive a big difference. Of course, I wasn't using any metric other than "feelings" I hesitated to play with these values because I didn't want to hassle with building netperf, etc.

But then I just decided to use the "time" command and copy a 852MB file from my OpenSolaris server (using NFS) using various delayed_ack settings. Here are the commands that I used, changing the delayed_ack value for each run.

rm junk.avi
sudo sysctl -w net.inet.tcp.delayed_ack=0
time cp /Volumes/Movies/junk.avi junk.avi

Here are the results:

delayed_ack: 0 4m14.490s
delayed_ack: 1 39.592s
delayed_ack: 2 2m15.545s
delayed_ack: 3 1m24.969s

Using a delayed_ack of 1 increased my throughput over 6 times!

THANK YOU!

 

[Lnx2Mac]

WOW! The results of doing this are DRAMATIC on my build

...

Using a delayed_ack of 1 increased my throughput over 6 times!