At the Forge - Checking Your Ruby Code with metric_fu

Among programmers, there has long been a dispute between those who want a language to constrain them and those who want great flexibility.

If you have been programming for a while, you'll understand the benefits that each side touts. A rigid language can help check your code, often using a compiler and a strict type system, to find potential problems before they make their way into production systems. By contrast, a more flexible language is designed with the knowledge that compiler and strict typing don't find all bugs and often force programmers to work around the system's constraints, rather than benefit from them.

This brief description is little more than a caricature of modern programmer attitudes. But, it does point to a tension programmers often face when choosing a language. How much do you want the language to constrain you, and what trade-offs are you willing to make? Would you rather have a strict language that doesn't let you express yourself the way you want or a flexible language that won't stop you from doing something foolish or dangerous?

Like many Web developers, I have come to prefer dynamic, flexible languages. I don't want the language to stop me preemptively from doing things, even if what I'm doing might seem crazy or weird. I've become quite a fan of Ruby over the last few years because of the balance it tries to strike.

However, the lack of a compiler or other tool to perform regular sanity checks does bother me somewhat. I wouldn't ever claim that a compiler is the only tool a programmer should use to test the code, but it does perform a first-pass inspection that can provide some useful feedback.

Fortunately, the Ruby community encourages the use of regular automated testing to ensure that code works in the way you expect. Done correctly, testing actually can be better than a compiler and strict typing. It can check the code at multiple levels, reflect actual use cases and serve as a sanity check not only for the code's syntax, but also for its logic and specification. Moreover, writing tests forces programmers to reflect on their work, chewing over how they have implemented a particular feature. Such reflection is an essential part of the learning process, and it offers programmers a chance to become better at their craft, as well as to write better programs.

Automated testing, accompanied by automated analysis, thus, can help improve programmers, as well as improve the programs they write. So, I was delighted to discover metric_fu, a Ruby gem from Jake Scruggs and others that pulls together some of the best-known analysis tools in one convenient package for Rails programmers. The combination of these various tools—including rcov, Flay and Flog—makes it easy to locate potential problems in code you've written and improve it. Automated analysis tools won't ever provide you with 100%-accurate feedback, but it's always good to get this sort of input.

This month, I look at metric_fu and some of the code-analysis tools it makes available to Rails programmers. It's true that metric_fu is “just” a wrapper for these individual tools, but by making them so easily available and integrated with the rest of your testing, you'll constantly be in a position to understand where potential problems might lie and to fix issues before they cause you any real trouble.

metric_fu is a Ruby gem, which means you can download and install it with:

sudo gem install metric_fu

The metric_fu gem specification automatically requires a number of other gems that it uses, including rcov and Flog. So installing the metric_fu gem should mean your system is ready, without the need for additional downloads and installations.

Assuming you are using metric_fu with Rails, you probably will want to tell Rails that it should look for and include the metric_fu gem. You can do this in modern versions of Rails by adding the following line to config/environment.rb:

config.gem 'jscruggs-metric_fu', :version => '0.9.0',
    :lib => 'metric_fu', :source => 'http://gems.github.com'

In other words, you want Rails to load the gem known as metric_fu, which can be downloaded from Github as jscruggs-metric_fu, version 0.9.0. If this gem does not exist, Rails will exit with an error.

Finally, you must add a line to your Rails application's Rakefile, telling it you want to load the Rake tasks associated with metric_fu:

require 'metric_fu'

Once this is complete, you should find a number of new tasks, all of whose names start with metric, available in Rake. You can list them with:

rake -T | grep metrics

I typically run all the tests, which you can invoke with:

rake metrics:all

This runs all of the software metric_fu works with, a list that has grown somewhat in the last year. At the time of this writing, running metrics:all includes:

I cover a number of these tests in greater detail below. But, before continuing, it's important to note that metrics:all will fail to run all the tests if the rcov coverage tool encounters one or more errors. This isn't a problem if you test frequently, but it can bite you if you break a test and then run metrics:all.

When you run the full report with rake metrics:all, metric_fu puts all the output files under your application's tmp/metric_fu directory. Each test has its own separate subdirectory and produces output in HTML for easy reading with a Web browser. The fact that the files are put in tmp/metric_fu makes them easy to find and view on a local system, but it requires that you move them into a Web-accessible directory (for example, public/tmp/metric_fu) if you want to view them from a remote machine. It should go without saying that you don't want this information to appear on a Web site that is publicly viewable, so be sure to password-protect or delete these reports to avoid unpleasantness.

Although metric_fu's defaults work for most initial cases, you may find yourself wanting to customize one or more of its tests. You can do this within your Rakefile by adding a MetricFu::Configuration block and invoking config.*, where * is one of the tests that metric_fu brings in. For example, you can customize which tests run for :all with:

MetricFu::Configuration.run do |config|
  config.metrics          = [:coverage, :flog]
end

If you modify config.metrics to include only a subset of metric_fu's tests, you may find yourself puzzled when other tests fail. For example, if you were to set config.metrics to the above value of [:coverage, :flog], invoking rake metrics:reek would fail, with Rake complaining that it wasn't able to find such a task.