Comparing Storage Arrays Apples-to-Apples

One of the biggest challenges facing the enterprise IT professional is to vet out the best data storage solution (SAN, NAS, etc) for their business. Often a review committee of five or six individuals are tasked with comparing and recommending a particular storage array from the many vying for their business. These, in turn, build a relatively complicated flow sheet and process for giving weighted scores to different features from those invited to quote and commonly ask each vendor to supply a given amount of capacity, often defined by drive spindle count or raw space.

Compounding the difficulty is that all storage array vendors have different capacity utilization levels due to feature holdbacks (snapshot reserve, threshold sets, performance limitations, software upgrades, data movement and optimization requirements, thin provisioning, etc.). If a team asks for quotes based upon the raw capacity of a fixed number of drives they’ll find a significant variation from vendor to vendor in the amount of capacity that is ultimately available for data storage (typically the primary reason for the purchase in the first place).

Those who buck the trend and stipulate usable capacity instead of raw capacity often have no idea of how to verify whether the stipulated capacity is contained in the quote and they almost never clarify to the vendor that he/she will be required to provide any capacity deficiency free of charge if the implementation comes up short. Instead the team relies on the vendors’ understanding the concept of usable capacity in the same way as the customer. This is almost always off because it’s easier for the storage array vendor to ask for forgiveness after the deal has been awarded than clarification before the deal is done and risk having to raise his/her price. This method works because no IT professional wants to admit to their management that they didn’t verify the vendors’ claims and so the whole thing is quietly covered over to the benefit of the storage array vendor who unfairly won the deal.

So how do you boil it down and make a fair comparison based upon the merits rather than the marketing? The answer is to compare all storage arrays from the standpoint of usable capacity. Because of the issues mentioned above, it is critical to define what usable capacity means to you. Stipulate that usable capacity means capacity that exists after all feature holdbacks, hot spares, Base10/Base2 conversion, RAID penalties, and best practice usage levels are subtracted from the total capacity. It means that if you had a single volume of data that would consume all the capacity you asked for, the storage array would not only accommodate it but the vendor would approve it as meeting its own published best practices for utilization in the presence of included feature sets. When all storage arrays are compared in this way they are reduced to a common denominator which removes marketing fluff from the equation and allows you to compare them on a dollar-for-dollar basis.

Another key to comparing storage arrays is to compare performance analytics from unbiased sources such as the SPC-1 and SPC-2 tests found at http://www.storageperformance.org. Although no test is a perfect duplication of a company’s unique environment, these tests are unbiased, audited, and all vendors are subjected to the same series of tests based upon average response time giving the end user a standard means of comparison. Take the total amount of IOPS or MBps and divide it by the number of hard drives it took to accomplish the task. That will give you an IOPS per drive or a MBps per drive number. Compare each vendor by this number and you’ll have another common denominator you need for your apples-to-apples comparison of the cost per IO or cost per MB. This test works well in comparing all of the major storage array vendors in the market today with the exception of EMC and Compellent as these two vendors apparently prefer not to submit their array performance to public scrutiny (they must be fast!!!). ;0)

Agree? Disagree? I would love to hear from you either way. Cheers!

About these ads