Seven Criteria for Evaluating Open-Source Content Management Systems

If you're new to open-source CMSes, here are some guidelines to help you find the best one for your project.
Stability and Development Activity

Open-source CMSes frequently start out as an experiment by a single programmer. Promising CMSes may attract other programmers, while less interesting projects are abandoned. CMSes that continue to grow may evolve into mature applications. One indicator of a healthy open-source project is constant development activity. Check how often new versions have been released. Regular or frequent releases are a good sign.

Do not base your decision solely on the version number of the open-source CMS, however. Some projects tend to inflate version numbers while others are modest. For example, a CMS sporting a 5.3 version number may not be necessarily more stable than one whose latest release is 1.2. Look for the project's Changelog or Release Notes documents. These usually include a list of new features, fixed bigs, improvements and known issues for each release. Scanning through this list may give you an idea of the pace of development activity and the project's stability.

Don't be discouraged by open-source projects that display a list of software bugs on their Web site. Good open-source projects acknowledge bugs and work openly to fix them. Projects that work quickly to address reported bugs are likely to be a good choice. Some may use bug-tracking software that allows developers and users to track the status of each bug. Try to get a sense of how long bugs stay open before they are addressed by looking at recently closed bugs. Also, assess if known issues or currently open bugs may make the CMS unsuitable for your needs. For a closer look at the development activity behind an open-source CMS project, take a look at the development forums or e-mail list archives, if they are available. This level of transparency is a strength of the open-source development process and enables you to make more informed decisions.

Repositories of open-source software try to make it easier to evaluate an open-source project by displaying indicators of its development activity and popularity. Figure 1 shows statistics compiled by the Freshmeat.net repository for Moodle, an open-source CMS. It includes scores and rankings by popularity, vitality and user ratings. Figure 2 shows similar statistics compiled by the development Web site Sourceforge.net, for the Plone CMS. Sourceforge.net also provides an activity percentile along with projects statistics that include downloads, page views, bugs, patches and project rank.

Figure 1. Statistics for the Moodle CMS

Figure 2. Statistics for the Plone CMS

User Community

The size and involvement of the user community is a good indicator of interest in the CMS. Look for active user discussion forums and/or mailing lists. Some projects proudly display a list of Web sites that are using its CMS. Visiting some of these Web sites can give you can idea of how the CMS is being used and how it can adapted.

The functionality of some open-source CMSes can be extended with the help of plugins. Others offer the ability to change presentations and appearances easily by using different themes. A large number of plugins and themes created by the user community demonstrates how easy it is to the customize the CMS. Also, scan user discussion forums and mailing lists to get a sense of how the developers respond to users' concerns and suggestions. Some projects also may offer user-contributed tips and tutorials. All are indications of an involved user community that may be able to help you. As open-source offerings have matured, commercial support for open-source CMS also is becoming available. Search for consultants that may be able to provide commercial technical support if you need it.

Documentation and Source Code

Successful open-source CMSes always are evolving, and it is a challenge to keep documentation up-to-date and useful. The Open Source user community often lends a helping hand in creating and updating documentation. Well-developed and current documentation would indicate an organized open-source CMS with an active community. Documentation may come in many forms; review changelogs, release notes, installation instructions and user manuals. Does the documentation address your concerns and questions?

Check if the open-source CMS has a developers' guide, coding standards and information on how others can contribute to the project. Established coding standards can help maintain consistent code even when a large number of programmers contribute to its development. Even if you do not have any programming experience, it may help to glance at the source code for the open-source CMS. Look for comments within the code written in English. Well-commented code can be easier to maintain and customize. If you are familiar with the programming language, you may be able to gain valuable insights into how the CMS works and the quality of its code.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Social application CMS

Social Application's picture

Hey its seem informative post.

Evaluating Open-Source Content Management Systems

Open source CMS 's picture

Hey ,Nice and helpful post about open source CMS.

They have to be SIMPLE

Peter Tandrer's picture

After working here - in Phuket - the only condition they have to fulfill here - they have to be SIMPLE. You really don't have a clue, how "simple-thinking" the people here are. Oh man - 8-)

Usability should be higher on the list

Matt Nuzum's picture

I have looked at many, many CMS systems and one thing I can say for certain is that few consider a polished, easy-to-learn interface as a high priority.

If you make frequent changes to your website then spend some extra time up front finding a CMS that facilitates quick and easy changes. It would be better in the long run to spend a little extra [time|money|etc] up front to find a CMS that will save you hours a week in the long run.

I'd say that usability is far more important in the long-run than the platform.

Choosing a CMS.

Taran's picture

While these 7 steps are important, the more practical aspects are just as important - and we could squeeze them under accessibility, or we could squeeze them under usability, or what have you. The ability to maximize the use of the content for the purpose of the owner of website is really what the whole thing is about.

For example - taxonomy/category handling is important. Intelligent use of categories allows data to be more intuitively handled on a site, and that means that the person designing the category has to have the maximum amount of flexibility through multiple and single hierarchies, as well as cross pollination between the categories.

That leads to documentation. If you're checking out CMS's, take a read of the documentation as well, and the support site (if available) and make sure that you understand what you are getting into.

And that leads to the most important thing about CMS choices: You really should know what you wish to do before you choose one so you select the right one for the right job. Having a proper definition of what a site is supposed to do comes first.

Apples and Oranges

Anonymous Coward's picture

A quick visit to Moodle's web site (http://moodle.org/) reveals that Moodle is a "Course Management System", NOT a "Content Management System". Using it as an example in an article about "Content Management Systems" is misleading.

You are right that Moodle is

Anonymous's picture

You are right that Moodle is a "course management system", also known as a "learning management system". However...

The term "content management system" is broad and includes various types of CMSes. According to Wikipedia, a "learning management system" such as Moodle is considered to be a type of CMS.

Moodle can certainly be used for non-teaching purposes such as community building or knowledge management and the Moodle website itself is a good example of such use.

The Moodle screenshot here is used as an example of the info you can get on Freshmeat and Sourceforge about an open source project. It is not being compared to anything else.

still misleading (imho)

Anonymous coward's picture

Yes, I noticed that the screenshot was posted to show information that can be obtained from Freshmeat, but there are thousands of other projects that could have been used for the same purpose (i.e. show the information that can be obtained). I don't think the choice of "Moodle" was mere coincidence. Come on.

Good point on Wikipedia, though. But even though Wikipedia's definition includes it, I still maintain that using a Course Management System as an example of a Content Management System in an article about choosing a Content Management System is misleading. Before I let the Wikipedia definition rest, though, I must say that we *do* work in an industry that sometimes has a bad habit of lumping things together that don't belong together. For example, there are those who call HTML a programming language.

I work with a commercial Content Management System every day (and I'll be the first to admit this in no way makes me an expert). It is used (quite simply) to "manage content" of two websites. That is, we (the "experts" (haha)) set up templates so that we have control over the look and feel of the entire site. "Users", then put in their "content" and viola, they've got a web page.

I would hesitate to call a Course Management System even a subset of Content Management. I mean, there is certainly some intersection between the two problem domains, but they still remain two distinct problems (in my mind, anyway). Maybe if I was in a good mood I would yield that a Course Management System is a highly specialized Content Management System... but even then I would start to think about how course management might have things like schedules; "students", "teachers", and "administrators" (as opposed to "editors", "reviewers", and "authors", which I would consider the users in the Content Management System domain); classes; campuses; etc. (none of which I believe to be part of a Content Management System).

I picture the person interested in reading this article as someone who is trying to choose a system that will allow them delagate management of a very large website to different groups of people within an agency or company. Or perhaps someone who is trying to maintain a consistent look and feel for a news website that has 20 different authors who regularly post articles. I don't think someone looking for a Course Management System would see this article on their rss feed and think "Hey, that might help me solve my problem; I should read it."

CMS vs LMS

haas's picture

Since this topic became a heated argument, this may encourage the need for another article that covers the definitions of LMS vs CMS, their feature and functionalities, and after that we can have a much more interesting debate whether LMS can be used as a CMS and vice-versa. (Just a suggestion.)

But in my humble opinion, I would agree with Anonymous Coward that using Moodle as an example is misleading even if it can be customized and used as a CMS. I would have used Drupal, Joomla, or others as examples because they are legitimate CMS platforms and it would have been much more appropriate and accurate.

Take care guys.
Haas

CMS as CMS?

Anonymous's picture

So it's a CMS with some extra features for managing "classes" -- but according to an independent site it works as a regular CMS too.

RE: CMS as CMS?

Anonymous coward's picture

okay, I'll bite... what is this "independent site" that you speak of?

CMS or not CMS

Anonymous's picture

children!

come on... the LMS is managing content that an instructor wishes to pass to her students, correct? Does it manage content, or not?

Besides, the refrence to the LMS was only to illustrate an example of viewing project activity on freshmeat. For that example, who cares what program is used?

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix