Hack and / - Linux Troubleshooting, Part II: Local Network
This column is the second in a series dedicated to one of my favorite subjects: troubleshooting. Because my column is generally aimed more at tips and tricks and less on philosophy and design, I'm not going to talk much about overall approaches to problem solving. Instead, in this series, I describe some general classes of problems you might find on a Linux system, and then I discuss how to use common tools, most of which probably already are on your system, to isolate and resolve each class of problem.
In the first column, I talked about how to diagnose high-load issues on a server, but the fact is that these days, just about every Linux computer is connected to a network, and a large number of the problems you have are based in the network. This month, I focus on local network troubleshooting, and although I am writing from the perspective of servers, most of these steps will apply to any Linux machine on a network. Also, because the goal of this article is to show how to become better at troubleshooting, I list each step from the lowest level on up. In real life, I'd probably skip ahead here and there to make the troubleshooting process faster.
The generic problem I cover here is how to track down the root cause when one machine can't communicate with another machine on the same network. For this example, let's assume I have two servers named bill and shawn. The server shawn is trying to communicate with bill over port 25 (port 25 is used for sending e-mail over SMTP), but wouldn't you know it, bill isn't responding.
One of the first things I might do in a scenario like this is find another machine on the same network and try to connect with bill from there. If I can talk to bill from another machine on the same network, the problem is most likely with shawn or with the network in between shawn and bill. If I have the same problem from another machine on the same network, it's more likely that the problem is with bill, so I would start troubleshooting from there. Just so I can discuss more troubleshooting steps, let's start troubleshooting from shawn.
One of the most embarrassing things in troubleshooting is to waste an hour only to find out that something wasn't plugged in. So the first step I perform is to make sure that shawn is plugged in to the network. Although I could inspect the port physically on the server, if the server were in a different city, I might run a program like ethtool. ethtool gives you a lot of different diagnostics on your Ethernet devices. By default, all you have to do is run ethtool as root and pass the Ethernet device you want to check as an argument. In many cases this will be eth0:
$ sudo ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full Advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pg Wake-on: d Current message level: 0x000000ff (255) Link detected: yes
As you can see, ethtool gives all sorts of information, including the fact that this machine supports 10 base T, 100 base T and gigabit networking speeds, but it currently communicates at 100 base T, full duplex. To check for a link, just look at the very last line that says “Link detected”. As you can see in my example, link is detected, so my cable is plugged in and I can move on.
Before I move past ethtool completely, it's worth mentioning that it does a lot more than just diagnose link problems. A common problem I've found on networks is a host with slower-than-normal network speeds. Often you'll see this crop up after a reboot or a power outage. What often happens is that when the interface connects to the network, it will try to auto-negotiate the fastest speed it can. Sometimes auto-negotiation doesn't work correctly, in which case the interface might fail back to half duplex mode or might even fail back to 10 base T! If you know that your network can support 100 base T at full duplex, you can use ethtool to disable auto-negotiation and force full duplex. To do this for eth0, you would type:
$ sudo ethtool -s eth0 autoneg off duplex full
Kyle Rankin is a VP of engineering operations at Final, Inc., the author of a number of books including DevOps Troubleshooting and The Official Ubuntu Server Book, and is a columnist for Linux Journal. Follow him @kylerankin.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- Google's Abacus Project: It's All about Trust
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Back to Backups
- Seeing Red and Getting Sleep
- Secure Desktops with Qubes: Introduction
- Fancy Tricks for Changing Numeric Base
- Working with Command Arguments
- Secure Desktops with Qubes: Installation
- Linux Mint 18
- CentOS 6.8 Released
Until recently, IBM’s Power Platform was looked upon as being the system that hosted IBM’s flavor of UNIX and proprietary operating system called IBM i. These servers often are found in medium-size businesses running ERP, CRM and financials for on-premise customers. By enabling the Power platform to run the Linux OS, IBM now has positioned Power to be the platform of choice for those already running Linux that are facing scalability issues, especially customers looking at analytics, big data or cloud computing.
￼Running Linux on IBM’s Power hardware offers some obvious benefits, including improved processing speed and memory bandwidth, inherent security, and simpler deployment and management. But if you look beyond the impressive architecture, you’ll also find an open ecosystem that has given rise to a strong, innovative community, as well as an inventory of system and network management applications that really help leverage the benefits offered by running Linux on Power.Get the Guide