Wednesday, September 10, 2008

The HP AutoRAID Hierarchical Storage System

The HPAutoRAID is a redundancy-level storage hierarchy implemented through the (SCSI) block-level device interface that employs background garbage collecting and load balancing. It was created to solve the difficult problem of manually configuring a RAID system. The storage hierarchy consists of mirrored storage for active data and RAID5 for inactive data, and the data can only be kept in one of these levels at a time. Data space on the disks is split up into physical extents (PEXes), which are grouped into PEGs on different disks in such a way as to keep data on the disks balanced, and are further broken down into segments, which serve as the stripe and duplication units. The logical space, on the other hand, is divided into relocation blocks (RBs) which serve as units of data migration.

Reads and writes to the mirrored storage are fairly simple, as are RAID reads. RAID writes, on the other hand, can use the per-RB scheme, that is, to write one RB (and also its stripe's parity) at a time, or they can use batched writes. If these batched writes fail, the per-RB scheme will be undertaken. The authors made a good decision when they decided to use RBs demoted during idle periods to plug the holes left by previously promoted RBs; not using hole-plugging may be faster and easier initially, but enacting it nearly cut down on any need for a RAID 5 cleaner in the simulations.

1 comment:

CalvinZ said...

I was part of the HP Storage marketing team that introduced HP AutoRAID. I'll never forget the day we showed this to a leading industry analyst at IDC. He was amazed at what AutoRAID could do.

Over the life of the technology, HP shipped a lot of these disk arrays. The technology was very cool.

There was initially a lot of resistance to the idea especially from database administrators that wanted complete control over where data is placed. However, back in the day, HP used AutoRAID in many server performance benchmarking efforts (e.g. TPC-C) and had industry leading numbers.

Anyway, it's interesting to see people blogging about AutoRAID 14 years after we introduced the first product.