One Box. Sixteen Trillion Bytes.

 in
Build your own 16-Terabyte file server with hardware RAID.
RAID Card Battery

I wanted the best possible RAID performance, which means using the “write-back” setting in the RAID controller, as opposed to “write-through”. The advantage of write-back cache is that it should improve write performance by writing to RAM first and then to disk later, but the disadvantage is that data could be lost if the system crashes before the data was actually written to disk.

The battery backup unit (BBU) option for the 3ware 9560 RAID controllers protects this cached data from being lost by preserving it across reboots.

Ordering Process

I had no problems finding all the hardware using the various price-comparison Web sites, although I was unable to find a single vendor that had every component I needed in stock. Beware that the in-stock indications on those price-comparison Web sites are unreliable. I followed up with a phone call for the big-ticket items to make sure they actually were in stock before ordering on-line. Table 1 shows the details.

Table 1. Parts List

QuantityDescriptionSourcePrice per unitTotal price
1Intel Xeon 5110 Woodcrest 1.6GHzNewegg$211$211
1Supermicro MB X7DBE-ONewegg$426$426
8ATP AP28K72S8BHE6S 1GB RAM modulesATP$65$520
1Supermicro Chassis SC836TQ-R800BSuper Warehouse$923$923
13ware 9650SE-16MLNewegg$919$919
13ware BBU-Module-04The Nerds$109$109
1Supermicro Heat Sink SNK-P0018Wired Zone$30$30
16Seagate ST31000340ASNewegg$274$4,384
   Grand total:$7,311

As you can see from Table 1, the hardware RAID components are about $1,000 of the total system cost.

Hardware Assembly

The chassis is pretty much pre-assembled. I had to insert some additional motherboard stand-offs and put on the rackmounting rails. I also snapped off some of the material on the plastic cooling shroud to fit around the motherboard power cables.

Figure 2. Inside View of the Server Chassis

The process of assembling the motherboard, CPU, heat sink, disks and memory was conventional, so I don't cover it here.

RAID Card Installation

Most of the 3ware 9650 controllers use “multi-lane” SATA cables with a single connector on the controller fanning out into four individual SATA cables. As this is a 16-port controller, four of the multi-lane cables connect to the SATA backplane. I made the process of connecting the SATA cables much easier by first removing the chassis cooling fans—they pop out quite easily. I also had to remove a couple of the disk backplane power connectors to access the bottom-most SATA connector.

Figure 3. SATA Backplane with Cooling Fans Removed

Be sure to connect the correct SATA cable to the correct SATA port, as a mistake here would be a disaster. You will need to determine the physical location of a disk with certainty when it comes time to replace one, or you will risk destroying the entire array. Familiarize yourself with the cable and disk numbering schemes before proceeding. For example, in the set of four multi-lane cables that came with my controller, one cable was labeled with the first port at 0 (and ending at 3), and the other three had the first port at 1 (and ending at 4), while the backplane ports were numbered starting at 0 (and ending at 15), with the lowest numbered port at the bottom left (as viewed from the front). This seems to be a chassis-specific scheme, as other Supermicro chassis models number the ports from the top left down. SATA ports are numbered starting at 0 within the 3ware administrative interfaces. The 3ware administrative tools have a feature to “blink” a drive LED for locating a specific drive, but that is not supported in this particular Supermicro chassis.

The 3ware BBU typically is mounted on the controller card, but I have found that the controller starts complaining about battery temperature being too high unless there is generous airflow over the battery. I purchased the remote BBU option, which is a dummy PCI card that carries the battery and an extension cable that runs from the remote BBU to the main RAID controller card. I mounted the battery a couple PCI slots away from the RAID controller so it would be as cool as possible.

Figure 4. 3ware RAID Controller with Remote Battery Option

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

A Problem with device driver programming

mety's picture

hello
i am writing a driver for a board which uses AMCC s5935 and a EEPROM.
i use lspci -x to see the card`s information but i see the wrong values. i have tested it on MS DOS and Windows but i see this wrong values again.
once i prepared a Windriver for this card and i saw the correct values . i wanna write a program on linux and i need help .what should i do?
i see a wrong number on Base Address Register0.
but it must be something else.
i must add BaseAddressRegister0 with 0x3c and read its address but i read something wrong and i am confused.
i would be very great full if u could help me.
thanx

speed test on similar system

hjmangalam's picture

Readers of this might be interested in some benchmarks on a similar system broken down by filesystem, types of apps, #s of disks, types of RAID, etc.

The Storage Brick - Fast, Cheap, Reliable Terabytes

http://moo.nac.uci.edu/~hjm/sb/index.html

Cheers
Harry

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState