SQL Comes to Nmap: Power and Convenience
I recently was exchanging e-mail with someone who regularly needs to port scan his own network for vulnerability trending. The port scanning tool of choice for this task is Nmap, but managing the data from Nmap was an entirely different beast. A few weeks later, a patch to Nmap that allows it to log the results directly to MySQL was ready. Although Nmap supports machine-parseable format as well as XML output, the ability to log directly to an SQL database far outruns XML or even machine-parseable output. For one, nmapsql does not involve an extra step in the shell to feed the output to a back end.
nmapsql is a direct patch applied to Fyodor's venerable Nmap v3.48 port scanning tool (at the time of this writing, Nmap v3.50 had just been released; an updated version of nmapsql for v3.50 is available from the Web site). It adds MySQL support, but it goes beyond merely adding the results; it also does target tagging, scanner tagging and simple trending. Once the data has been captured in an SQL database, a whole new set of tasks is possible. nmapsql can be downloaded from sourceforge.net/projects/nmapsql. At the moment, it relies on MySQL's client interface for data manipulation.
As security administrators aren't necessarily database wizards, nmapsql was designed to be simple to use. It's simple enough that most of the information one might want in a network scan can be obtained from a single table. Simplicity is also why IP addresses are stored as plain text instead of with inet_aton() notation. I'm aware of the performance penalties of text manipulation, but the focus is to demonstrate the convenience with a small data set. The target tags, runtime and scanner IDs are there for numeric searches in large data sets where performance is critical.
In this article, we concentrate first on running an SQL-enabled scan to establish a baseline of open ports and live targets on a network. Later, we take a look at the data captured in SQL and find ways of comparing the results.
nmapsql starts out by reading the ~/nmpsql.rc file in the effective user's home directory. So, if you used su to get to root before running nmapsql, ~root/nmapsql.rc is read. At this time, only four items are read from nmapsql.rc, each on a line by itself and in the item=value format common to many other utilities. The items are server=localhost, db=nmaplog, user=nmap and passwd=scanamanga.
The server is the DNS name of the host where MySQL is running, and db is the name of the database on that server. The user and password items are used to connect to the database, and the user listed must have at least SELECT, INSERT and UPDATE rights to the database.
On the command line, nmapsql introduces four new options to those Nmap already provides: --mysql, --runid, --targetid and --scannerid. When the nmapsql binary is executed without any of these options, it behaves exactly as normal Nmap does. None of these options interfere with Nmap's existing output abilities, so it's entirely possible to log to SQL as well as to produce machine-parseable output from the same scan.
The --mysql option, without any of the other nmapsql options on the command line, triggers MySQL logging, with all tags and IDs auto-assigned. All other nmapsql options automatically assume --mysql. Auto-assignment always picks the maximum available value in the respective table and increments by one.
The scanner ID feature, initiated by the --scanner-id xxx option, where xxx is the ID value, is intended for scenarios where more than one scanner is deployed, perhaps in a multisubnet environment. The scanner ID, along with the runtime ID, is stored in the portstat table to allow separation of result sets by the scanning host. It would be simple to separate the results of scanner ten, for instance, using a query like this:
mysql> select * from portstat -> where scannerid = 10 and runid = 100;
The --run-id xxx option is used to specify a specific ID for the current nmapsql run. If this option is not specified, a system-generated ID is used. If the runid specified already exists in the database, it is reused. This feature allows results of multiple scans to be grouped conveniently under a single runid.
The runtime ID and its associated information are stored in the runlist table. See the “Tables Used by nmaplog” sidebar for a summary of the tables used. Some of the runtime information is updated at the end of the scan, including the total number of possible targets specified on the command line and the total number found alive. Similarly, the scanner ID and related information go to the scanners table.
|Happy Birthday Linux||Aug 25, 2016|
|ContainerCon Vendors Offer Flexible Solutions for Managing All Your New Micro-VMs||Aug 24, 2016|
|Updates from LinuxCon and ContainerCon, Toronto, August 2016||Aug 23, 2016|
|NVMe over Fabrics Support Coming to the Linux 4.8 Kernel||Aug 22, 2016|
|What I Wish I’d Known When I Was an Embedded Linux Newbie||Aug 18, 2016|
|Pandas||Aug 17, 2016|
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Happy Birthday Linux
- The Great Software Schism
- New Version of GParted
- What I Wish I’d Known When I Was an Embedded Linux Newbie
- Updates from LinuxCon and ContainerCon, Toronto, August 2016
- All about printf
- ContainerCon Vendors Offer Flexible Solutions for Managing All Your New Micro-VMs
- A New Project for Linux at 25
- Tor 0.2.8.6 Is Released
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide