The Scalable Test Platform
The STP is a hardware and software configuration for automated testing. To run the back-end control and scripting of the tests, we developed Brimstone, a combined batch control system and automated test harness. Requests are submitted using Eidetic, a user-friendly web front end, or brim-gate, the e-mail gateway. Using these front ends, entire test sequences can be requested in less than two minutes.
How STP works begins with developers checking patches in to our kernel CVS tree and requesting a test sequence using their patch. After the tests are completed, detailed results are returned to the developer via e-mail and are also archived on our web site. To simplify the process even further, a developer could write a short shell script that, in less than five lines, would check their patch into CVS and submit a preformatted test request via e-mail. Then all it would take to check the effectiveness of their patch would be a single command. Everything involved in a full-scale test run would then be taken care of without a second thought.
The hardware dedicated to the STP by the OSDL includes a 1.8TB storage away network setup connected to each server (four-CPU and up), via multiple Fibre Channel connections. Servers include two each of 2-CPU, 4-CPU and 8-CPU boxes, as well as a single 16-CPU IBM NUMA-Q server. A second 16-CPU NEC AzusA server, containing Itanium CPUs, is on order. We also have over 50 client-load machines that can be moved into the STP at the press of a key. We are also looking into the possibility of including a few single-CPU machines to ensure kernel modifications don't adversely affect the vast majority of current installs.
Eidetic, Brimstone and the e-mail gateway are all under the GPL, so interested parties can use them when setting up their own labs for specialized testing interests.
The first step is to go through the free sign-up to become an OSDL Lab Associate, available at www.osdlab.org. Next, enter a test request through the web page, which will involve something like this: choose the kernel tag to run (2.4.8 for instance), choose the distribution to use, choose the test to run, list the CPU details, list the various hardware restrictions (optional) and enter an optional LILO command line (allows for restriction on the RAM used). After submitting the base items, you need to spend a moment filling out the setup page for the test you selected, then submit the final request.
It's as simple as that. Depending on the length of time required to complete the type of test requested, you could have your response back in less than 25 minutes. Full environment documentation and the resulting data sets will be archived on the web site for you. Short tests will take at least 15 minutes because a fresh copy of the OS is installed prior to every test sequence.
Since the entire process is automated, testing does not stop when our office closes. This means the quick response will be possible at any time, for developers located anywhere in the world.
Since the STP is currently in the initial rollout phase, the number of scripted tests is still low. At the time of this writing, the following is a sample list of possible workloads: Juan Quitela's “memtest” VM abuse test suite, dbench (Samba) filesystem punishment, the ever-popular scripted kernel compile, simulated real-world CVS punishment and lmbench.
A long list of potential tests including scenarios involving multiple servers and applications such as Apache and MySQL is currently being evaluated. Of course, as an open-source project, we welcome assistance in the automation of these tests. Getting a full range of tests ready for use with the STP is going to be a major undertaking, but we believe the benefits to Linux will be worth it. For me personally, this is where I hope to give something back to the Linux community that will make a positive impact.
We are also in active cooperation with developers from SGI and IBM in the Linux Test Project (LTP). One current goal is to enlarge the LTP's coverage to include both targeted and general workload simulation tests. The LTP kernel features test is almost ready to be automated, which will provide us with a large regression test suite, as well as a solid base for stability research. The LTP's future plans include the development of a number of self-contained tests that will make great testing targets for the STP.
Nathan Dabney (email@example.com) has been working with Linux since Slackware in 1994. He enjoys breaking bad concepts and going for walks in the rain with his fiancee.
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
|Introduction to MapReduce with Hadoop on Linux||Jun 05, 2013|
- Containers—Not Virtual Machines—Are the Future Cloud
- Non-Linux FOSS: libnotify, OS X Style
- Validate an E-Mail Address with PHP, the Right Way
- RSS Feeds
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Introduction to MapReduce with Hadoop on Linux
- Help with Designing or Debugging CORBA Applications
- New Products
- Returning Values from Bash Functions
- Linux Systems Administrator
- notifier shortcomings
18 min 25 sec ago
1 hour 55 min ago
- Android User
1 hour 56 min ago
- Reply to comment | Linux Journal
3 hours 50 min ago
6 hours 39 min ago
- This is a good post. This
11 hours 52 min ago
- Great, This is really amazing
11 hours 54 min ago
- These posts are really good
11 hours 55 min ago
- It’s a really great site you
11 hours 58 min ago
- Beautiful ... I love your
12 hours 24 min ago
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?