Top Five Linux Security Tools
First, there are no magic programs you can run that will make your network or server secure forever. Security is an ongoing process of improving, evaluating, and then improving again. Fortunately, many good tools are available for Linux that will help you in this process. In this article, I present five of my favorites, which will help you prevent, detect and respond to intrusions. Although it is often much easier to prevent problems whenever possible, at some stage you'll have a problem you'll need to detect and respond to. This means it's a good idea to familiarize yourself with these programs before you need them in a crisis situation.
To evaluate how vulnerable a machine is, you need to know how many services are exposed to attackers. One excellent tool for this is Fyodor's Network MAPper Nmap. Debian users can fetch it via apt-get install nmap. Make sure you run it to check what services are available from a server—even if you think you already know. Obviously, you won't be affected by SSH password guessing if the SSH port is closed.
One of the simplest uses of Nmap is host discovery on your local network; in this instance, we ask Nmap to send ICMP echo request packets (pings) to each host in the IP address range:
$ nmap -sP 10.0.0.1-254 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2006-11-01 14:46 NZDT Host 10.0.0.25 appears to be up. MAC Address: 00:0C:F1:AE:E6:08 (Intel) Host 10.0.0.51 appears to be up. MAC Address: 08:00:09:9A:1A:AA (Hewlett Packard) Host 10.0.0.70 appears to be up. MAC Address: 00:0F:EA:64:4E:1E (Giga-Byte Tech Co.) ...
More often though, Nmap is used to discover which services are running. Because of the way that TCP establishes a connection using a three-way handshake, we can detect open ports without actually connecting to them. This is known as a SYN or half-open scan and is the default mode when used as root. If executed as a normal user, Nmap attempts to do a full connect to test if each port is open. (Technical aside: in the half-open mode, we send the initial SYN packet and listen for a reply. An RST indicates the port is closed; an SYN+ACK means it is open. If no reply is received, Nmap marks the port as filtered. Some firewalls will drop unwanted packets, leading to a status of filtered, and others will send RSTs, which make the port appear as closed.) Typically, if a server is listening on ports you don't expect it to, you should investigate:
#nmap -sS 10.0.0.89 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2006-11-01 14:52 NZDT Interesting ports on 10.0.0.89: (The 1637 ports scanned but not shown below are in state: closed) PORT STATE SERVICE 21/tcp open ftp 22/tcp open ssh 42/tcp open nameserver 80/tcp open http 110/tcp open pop3 ...
Fyodor also has added a number of service fingerprints, and you can ask Nmap to identify particular services running by using the -sV option:
# nmap -sV 10.0.0.89 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2006-11-01 14:47 NZDT Interesting ports on 10.0.0.89: (The 1637 ports scanned but not shown below are in state: closed) PORT STATE SERVICE VERSION 21/tcp open ftp? 22/tcp open ssh OpenSSH 3.8.1p1 Debian-8.sarge.4 (protocol 2.0) 42/tcp open nameserver? 80/tcp open http Apache httpd 1.3.33 ((Debian GNU/Linux) mod_gzip/184.108.40.206a PHP/4.3.10-16) 110/tcp open pop3? ...
The other incredibly useful option is OS detection, which can be invoked with -O. If the machine has at least one port open and at least one closed, you can get a fairly accurate idea as to what the operating system is:
# nmap -O -sS 10.0.0.89 Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) at 2006-11-02 09:02 NZDT Interesting ports on 10.0.0.89: (The 1637 ports scanned but not shown below are in state: closed) PORT STATE SERVICE 21/tcp open ftp ... Device type: general purpose Running: Linux 2.4.X|2.5.X|2.6.X OS details: Linux 2.5.25 - 2.6.3 or Gentoo 1.2 Linux 2.4.19 rc1-rc7), Linux 2.6.3 - 2.6.8 Uptime 30.906 days (since Mon Oct 2 11:18:59 2006)
So, please use Nmap to check that you don't have any machines or services on your network that have been “temporarily” installed and forgotten. You also should use it from outside your network perimeter to verify that your firewall configuration is as it should be.
While you are looking after your network, you need a secure way to administer your machines. This means not using Telnet or rcp, or any other protocols that transmit passwords and data without any protection. OpenSSH is most people's favourite replacement for Telnet/rcp, which encrypts all data as it travels and makes some effort to verify that no one is spoofing the identity of the remote end of the connection. Debian users can fetch it via apt-get install openssh-server.
First, I recommend switching to an alternate port, using passphrases only, not passwords, or auditing your passwords on a regular basis. Remember that SSH secures the transmission of data—this means if your access control is poor, an attacker can control your computer in a secure manner, which is probably not what you want. I've seen too many Linux machines compromised through “temporary” accounts, such as upload/upload. To change the port, edit /etc/ssh/sshd_config, alter the line #Port 22 to Port 12345, and restart the service.
In the following example, we show how to use passphrases instead of passwords. Let's call the computer you want to execute the scp/ssh command on the client and the remote computer to ssh to, or copy from, the server. What you're doing here is allowing your user account on the client computer to read and write to any of your files on the remote computer. Doing this as root makes the server just as insecure as the client—you have been warned.
You can omit the passphrase and not use ssh-agent (the ssh-add command), but it's better not to. Log in to the client and enter the commands:
client% ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/usr/local/sss/jriden/.ssh/id_rsa): Enter passphrase (empty for no passphrase): MY PASSPHRASE Enter same passphrase again: MY PASSPHRASE Your identification has been saved in /usr/local/sss/jriden/.ssh/id_rsa. Your public key has been saved in /usr/local/sss/jriden/.ssh/id_rsa.pub. The key fingerprint is: 75:65:36:2b:ed:38:9f:4a:6d:c4:d8:ec:25:ed:ff:31 jriden@its-dev2 client% ssh-add Enter passphrase for /usr/local/sss/jriden/.ssh/id_rsa: MY PASSPHRASE Identity added: /usr/local/sss/jriden/.ssh/id_rsa (/usr/local/sss/jriden/.ssh/id_rsa) client%
Now, get ~/.ssh/id_rsa.pub, and add the contents as one entry in ~/.ssh/authorized_keys or ~/.ssh/authorized_keys2 on the server. Make sure to remove extraneous newlines that may creep in during the copy and paste:
client% scp server:~/testfile . The authenticity of host 'server (220.127.116.11)' can't be established. RSA key fingerprint is 97:7b:e0:12:c2:f8:8e:05:cc:2b:74:50:9b:00:28:0e. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'server,18.104.22.168' (RSA) to the list of known hosts. testfile |***************************************************| 81940 00:00
The prompt here is because the server is not a known host, and this is the way SSH attempts to combat host spoofing. The next time you do it, there should be no prompt.
tcpdump is probably the best known application for examining network traffic in its raw form. Debian users can fetch it using apt-get install tcpdump. tcpdump files are sometimes known as pcap files, because pcap is the library that is doing the packet capturing.
In this example, we dump all the traffic that is coming from either source port 53 or going to destination port 53—meaning, all DNS traffic. If you wanted to be more specific, you could use src port 53 or dst port 53. I have used the -n flag; otherwise, tcpdump tries to resolve the IP addresses to domain names to print-friendlier names, and those DNS requests also will show up in our capture:
# tcpdump -n 'port 53' tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 11:19:58.302298 IP 192.168.0.8.1037 > 192.168.128.1.53: 36224+ A? www.slashdot.org. (34) 11:19:58.360227 IP 192.168.128.1.53 > 192.168.0.8.1037: 36224 1/5/5 A 22.214.171.124 (239) ...
Among other things, Snort can store tcpdump format captures, and you need to be able to read and write stored files with the -r <filename> and -w <filename> options.
Snort is the leading open-source network intrusion detection system. This basically means it looks out for bad traffic and alerts you to it, which saves you having to read raw tcpdump output all the time. Debian users can get it via apt-get install snort. By default, it puts the network interface into promiscuous mode—that is, all packets on the wire are examined, not only the ones with the appropriate MAC address.
If you're trying to protect a network, a common place to put your Snort sensor is on a SPAN port on a major router, so it has a view of all the network traffic traversing that router. (A SPAN port presents an aggregation of all the traffic that is passing through the router, within certain limits.) If you are trying to protect only a single host, simply install it on that host.
Snort works by checking packets against a database of rules. Here's an example from /etc/snort/rules/bad-traffic.rules that flags invalid traffic having a tcp port number of zero:
alert tcp $EXTERNAL_NET any <> $HOME_NET 0 (msg:"BAD-TRAFFIC tcp port 0 traffic"; flow:stateless; classtype:misc-activity; sid:524; rev:8;)
Snort logs its alerts to a file, typically /var/log/snort/alert, which looks like this, indicating a policy rule has fired because Web traffic has been observed coming from the Google Desktop user agent:
[**] [1:2002801:3] BLEEDING-EDGE WEB Google Desktop User-Agent Detected [**] [Classification: Potential Corporate Privacy Violation] [Priority: 1] 11/02-11:27:58.855143 10.0.0.82:3449 -> 126.96.36.199:80 TCP TTL:128 TOS:0x0 ID:35935 IpLen:20 DgmLen:399 DF ***AP*** Seq: 0x847921EE Ack: 0x1A7D5C20 Win: 0xFFFF TcpLen: 20 [Xref => http://news.com.com/2100-1032_3-6038197.html]
Snort also can be configured to log matching packets to tcpdump format, as shown here:
# log_tcpdump: log packets in binary tcpdump format # ------------------------------------------------- # The only argument is the output file name. # output log_tcpdump: tcpdump.log
When the above option is in effect, snort will record matching packets to tcpdump files as well as storing them in the alert file. This allows you to determine more easily which alerts you need to worry about and which alerts are false alarms. You can read them as follows:
# tcpdump -r /var/log/snort/tcpdump.log.1161106015 reading from file /var/log/snort/ ↪tcpdump.log.1161106015, link-type EN10MB (Ethernet) 06:37:50.839942 IP 10.0.0.82.1410 > 10.10.218.95.www: P 2352360050:2352360119(69) ack 1723804156 win 65535 06:53:07.792492 IP 10.0.0.82.1421 > 10.10.218.95.www: P 2124589760:2124589829(69) ack 2684875377 win 65535 ...
If you want to see the packet contents, use the -X flag, as in tcpdump -X -r /var/log/snort/tcpdump.log.1161106015. Snort also has some functionality to disrupt connections based on rule matches (flexible response), and there are many useful third-party rules, available at Bleeding Edge Threats (see Resources).
iptables is a statefull firewall that ships with nearly all Linux distributions. What this means is you can use it to control access to your server based on the IP address of the remote machine, as well as the type of connection requested. (The old stateless firewalls let you make decisions based only on the contents of the packet, so you were restricted to port numbers and couldn't track the existence of conversations, such as the FTP data stream.) Debian users can get iptables via apt-get install iptables conntrack.
Please do your initial testing when you have console access to the machine. It is very easy to lock yourself out with an incorrect rule, and then you have to visit it in person to recover. With this script, there will be an instant when all packets are allowed through, so if this is not acceptable, look into the iptables-save and iptables-restore commands. Here is an annotated example script for setting up a firewall. It's very basic, but it should give you a basic idea of how iptables works:
#!/bin/bash # example iptables script # flush the old rules iptables -F # set the default policy of the chain to accept iptables -P INPUT ACCEPT # create a new table for logging and discarding # unwanted packets iptables -N LOGDROP # use rate limiting on the logging, and # add a prefix of 'filter: ' iptables -A LOGDROP -m limit -j LOG ↪--log-prefix "filter: " # drop unwanted TCP connections with a # TCP ReSeT packet iptables -A LOGDROP -p tcp -j REJECT ↪--reject-with tcp-reset # drop other packets by sending an ICMP # port unreachable in response iptables -A LOGDROP -j REJECT ↪--reject-with icmp-port-unreachable # now drop the packet iptables -A LOGDROP -j DROP #allow anything on the local interface iptables -A INPUT -i lo -j RETURN # allow packets that are related to # an on-going conversation iptables -A INPUT -p tcp -m conntrack ↪--ctstate RELATED,ESTABLISHED -j RETURN iptables -A INPUT -p udp -m conntrack ↪--ctstate RELATED,ESTABLISHED -j RETURN # allow SSH traffic iptables -A INPUT -p tcp -m tcp ↪--dport 22 -j RETURN # allow HTTP and HTTPS traffic iptables -A INPUT -p tcp -m tcp ↪--dport 443 -j RETURN iptables -A INPUT -p tcp -m tcp ↪--dport 80 -j RETURN # accept the following ICMP types - # echo, echo reply, source quench, ttl exceeded, # destination unreachable - and drop the rest iptables -A INPUT -p icmp -m icmp ↪--icmp-type 0 -j RETURN iptables -A INPUT -p icmp -m icmp ↪--icmp-type 3 -j RETURN iptables -A INPUT -p icmp -m icmp ↪--icmp-type 4 -j RETURN iptables -A INPUT -p icmp -m icmp ↪--icmp-type 8 -j RETURN iptables -A INPUT -p icmp -m icmp ↪--icmp-type 11 -j RETURN # if we haven't accepted it, drop and log it. iptables -A INPUT -j LOGDROP
If you're wondering where the logs go and what they look like, this an example entry from /var/log/messages (syslog). Here, a UDP packet from 192.168.50.40, source port 30766 to the server 192.168.0.8, destination port 1026 was discarded. It was probably Messenger service spam intended for a Windows machine:
Nov 22 06:24:00 localhost kernel: filter: IN=eth0 OUT=MAC=00:0a:e6:4e:6d:49:00:14:6c:67:cc:9a:08:00 SRC=192.168.50.40 DST=192.168.0.8 LEN=402 TOS=0x00 PREC=0x00 TTL=47 ID=3345 PROTO=UDP SPT=30766 DPT=1026 LEN=382
The log output looks a little scary, but you easily can pick out the data you want. First, you have the date and the hostname. IN and OUT describe which interface the packet comes in from and exits through (if any). MAC gives the source and destination MAC addresses. SRC and DST are (obviously) the source and destination IP addresses. PROTO is UDP, TCP, ICMP and so forth. SPT and DPT are the source and destination port numbers. For example, most connections to a service such as SSH will have an ephemeral source port, for example 35214, and a destination port of 22. As mentioned, the example above is probably Messenger service spam which was intended for a Windows machine, so you can safely ignore it.
When you have set Up the firewall, make sure you run Nmap against the machine from another host and check that only the correct ports are shown as open.
Security is all about a constant cycle of enhancing and evaluating your protection measures. The tools above allow you to protect your servers, ensure it is working as required and analyse the traffic on your network for unexpected events. Remember, the unexpected is usually the same as undesirable! As one of Larry Niven's characters says, “anything you don't understand is dangerous until you do understand it”.
When you are using these tools, try to think about the three phases of security: prevention, detection and response. It's best to prevent incidents if you can, as cleaning up can be costly and you can lose valuable data during the intrusion. However, you will see intrusions at some point, so be prepared for them as best you can.
TCP Three-Way Handshake
UDP is a connectionless protocol and is a “best effort” service. In contrast, TCP will deal with resending data that gets lost in transit. To do this, it needs a more complicated setup than UDP does, which begins with the three-way handshake. To initiate a connection, the client sends an initial SYNchronize packet. The server then replies with a SYNchronize+ACKnowledge packet if the port is open, in which case the client replies with an ACKnowledge, and the connection is established. These packets carry data about sequence numbers that allows the protocol to detect and resend lost packets. If the port is closed or firewalled, the reply may be a ReSeT (RST) packet or no answer at all. More information can be found in the documentation for Nmap.
Jamie Riden studied math and computer science at Oxford, worked in software development for a few years and then took a Masters' degree in artificial intelligence at Edinburgh. Since then, he has worked in IT security for Massey University, New Zealand, where he was involved in incident response, forensics and intrusion detection. During this time, he developed an interest in using honeypots to discover more about attackers and methods used to compromise machines, particularly Linux servers. He obtained a CISSP in September 2006 and is a member of the New Zealand honeynet group (www.nz-honeynet.org).