Skip to content

Linux: Setting speed and duplex on NIC

March 17, 2012

TLDR? Click to jump.

At the office, one of our two Internet connections gives roughly 15mbps of symmetric throughput (it’s metro Ethernet). The other Internet connection (a cable line) only gives 7/2.5 so I’d like to use the faster option for site-site and road warrior VPNs. Unfortunately that metro-E line is also our most reliable and lowest latency option, so we need to use it for some VoIP traffic too. We found these goals were mutually exclusive without some sort of rate limiting on the VPN, because when usage went up the call quality would suffer (the provider does highly prioritize RTP, but depending on what else you’re trying to do it isn’t always enough). Our LAN gateway has an interface for both WAN connections and the VPN server stays behind its firewall, but the voice server is not on the LAN and is only connected directly to the metro-E.

My co-worker had the idea that we could set our gateway interface for the metro-E connection to 10mbps instead of the auto-negotiated 100mbps, and it would be a ridiculously simple rate limiter that would leave plenty of capacity to the VoIP server. This is working well so far, but I did learn that there is a right and wrong way to do it and that most recommendations out there seem to advocate the wrong way.

Without stepping too far into one of the many IT ‘holy wars’, it should suffice to say that for quite a while after auto-negotiation came about, most mission critical connections still had their speed and duplex manually configured on both ends. This was to help avoid cross-vendor issues on the still changing auto-neg standards. There are still a number of old hands that would give countless examples from the “Old Days”(tm) in which auto-neg caused problems and would swear that manual is still the way to go. Then there are newer guys, like myself, who swear they haven’t seen that happen in over a decade (but as most admit, some of us were just finishing up high school a decade ago).

The problem with this is that a lot of connections end up with one side on auto-neg and the other side on manual, whether it is due to a different person configuring each end or because one device or the other does not allow specifying manual settings. The thing I previously did not know is that when only one side is manually configured, the other end will connect at the ‘forced’ speed but will consider the auto-negotiation to have failed and will drop to half duplex. Again, I won’t delve too deep into the details, partially because I’m still reading about them, but if you force full duplex on one end and the other end fails back to half it can change the link status to ‘Suck’ real quick-like. Furthermore, gigE connections use the auto-negotiation process to work through many of the extra details needed to maintain 1000mbps, and so auto-neg is required for that speed. I did read in one place that some vendor implementations still let you turn off auto-neg for gigE via one sort of voodoo or another, but that doesn’t make it a good idea.

My solution (for Linux hosts):
Tell ethtool to make the NIC only advertise the auto-negotiation rates that you want it to.

From man ethtool:

advertise N
Sets the speed and duplex advertised by autonegotiation. The argument is a hexidecimal value using one or a combination of the following values:
0x001 10 Half
0x002 10 Full
0x004 100 Half
0x008 100 Full
0x010 1000 Half(not supported by IEEE standards)
0x020 1000 Full
0x8000 2500 Full(not supported by IEEE standards)
0x1000 10000 Full
0x03F Auto

So we ran the command:
ethtool -s eth1 advertise 0x003

…and then tacked this onto /etc/sysconfig/network-scripts/ifcfg-eth1:
ETHTOOL_OPTS="advertise 0x003"

I was wondering for a while on a theoretical level whether 10mbps full duplex would mean the line could still pull 20 (making this attempt futile), but I’m pretty sure now that the full duplex speed refers to traffic going in both directions. In any case it has been a month now and there have been no issues with call quality, and the remote users whose capacity from us to them increased 5X have only recently stopped offering to name their children and pets after us. Ok, so that last thing isn’t true but they were really happy.

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: