Understanding IDS for Linux

Do you feel your network is safe? Do you really know what is happening on your network right now? Once upon a time, there were network administrators who thought that the solution to their security was a simple firewall. In the past few years, we have verified that this is not true anymore. The need for some element that could alert and inform administrators about something strange in near real time resulted in intrusion detection systems (IDSes). In this article we discuss the types and models of IDSes: the host-based intrusion detection system (HBIDS), the network intrusion detection system (NIDS) and the new concept of hybrid-IDS. How to analyze the data generated and how to create signatures (the patterns that identify the attacks) also are discussed, as well as some examples of IDSes for Linux, like the open-source NIDS Snort.

An IDS is a program that tries to detect strange packets and behaviors that may compromise a network. The first IDS was the host-based IDS, but the one that really got the market was the NIDS, the network-based. There is usually some software or appliance, called a sensor or agent, that has one or two network interfaces (as we will see later, it may work perfectly with one network interface), which works in promiscuous mode. In other words, it will catch all the packets that come to the interface and not just those with its particular destination IP address. In this way, the IDS can analyze all the packets that cross the network, check if they contain, for example, any suspicious strings and then decide how to perform a reaction, such as interacting with the firewall to create new rules to block the IP address, sending pager/e-mail alerts to the security administrator and so on. One important topic about the NIDS is where to deploy the sensor, inside or outside the firewall. I like to quote an interesting passage about this from SANS' GIAC Director Stephen Northcutt's book, Network Intrusion Detection: An Analyst's Handbook:

Where is the best place? We may have a long discussion about this since there are defenders of both points, but undoubtedly all agree that the best option is the use of sensors inside and outside the firewall.

To understand the IDS better, first we need to know how it works. Basically we have two models of IDSes: the misuse or signature-based model and the anomaly model.

The misuse or signature-based is the most-used IDS model. Signatures are patterns that identify attacks by checking various options in the packet, like source address, destination address, source and destination ports, flags, payload and other options. The collection of these signatures composes a knowledge base that is used by the IDS to compare all packet options that pass by and check if they match a known pattern. Later we will discuss a Nimda worm signature example in the Snort IDS.

The anomaly model tries to identify new attacks by analyzing strange behaviors in the network. To make this possible, it first has to “learn” how the traffic in the network works and later try to identify different patterns to then send some kind of alert to the sensor or console. The disadvantage of this model is that you will never know if your network has produced all types of behavior in the IDS learning phase, so it may cause a high number of false-positive alerts.

False-positives are false alerts produced by the IDS to inform of an attack when in fact it is just nonconfigured variables or an application that sent some packet to a different port than usual, instead of a real backdoor, for example. To solve this, the security administrator has to observe the alerts generated by the IDS for some time and then fine-tune it.

Host-based intrusion detection systems usually are located in servers and only detect events related to the machine in which it is installed. The main purpose of the HBIDS is to avoid changes that may compromise the machine and detect malicious queries. Examples of changes that can prove the importance of this kind of IDS are web defacement and rootkits installed in the machine to attack other machines.

Rootkits are packages installed in the compromised machine by the cracker, which contain files used to open backdoors, erase log files to hide their presence and replace binaries like ps and netstat, and also hide any dæmon or open port.

Besides this, the HBIDS has the function of trying to detect attacks before they happen, analyzing logs to point out strange behaviors and also detecting port scans.