I’m thinking here that a better metric might by “Watts per IO” and “BTUs per IO”, if that information was available. Hopefully soon we’ll see companies submitting results to the SPC-1/E.

One interesting little tidbit I noticed about the AMS 2500. The system was configured without any spares. That’s definitely not a real world configuration. There’s probably more examples of that kind of stuff but I don’t have time to look through all the listed SPC results.

And one final note on your comment that “some vendors are vastly under populating their arrays indicating that the controllers are the bottleneck.” I think cost is the more likely factor, not controller bottlenecks! The vendor has to cover the cost of the test system, and some vendors, even large ones like Hitachi, are not going to be able to put all necessary resources towards a fully-maxed system. Sometimes those lab drives are just too valuable!

I find the SPC-1 numbers to be very helpful. Same for SPEC SFS. It provides common ground and full disclosure. It didn’t help me during my last array purchase since I already knew what I needed to know.

I recall one vendor a couple of years ago touting their IOPS performance and the numbers they were talking about were clearly from cache, not from disk, but those were the only numbers they would give (few months later they posted SPC-1 numbers). I don’t mind cache numbers but really do want full disclosure, in any situation.

thanks for reading!

“I’ve yet to come across anything more complete” – huh?

Since when is “the only available data” accepted as accurate or valid?

BP told the world their well was only leaking 3-5000 barrels a day. We know know that the amount was AT LEAST 10 times that amount, and perhaps even more than 20.

It is not easy to build a fair, accurate and representative benchmark – that I will not argue. But when people start accepting a benchmark as fact solely because it is all they have – well, that’s a travesty of logic.

Perhaps we’re already seeing the negative implications of “teaching to the test” that is so rampant in our schools…logics, analytics and the scientific method seem to be abandoned – replaced by meaningless data in pretty bar charts.

(No insult intended, but the derived charts only serve to further obfuscate the fact that the SPC benchmarks bear no definitive relationship to ANY real-world use case – and “random workload” is anything but definitive).

I updated the # of disks graph.

The main point of this article was about the more scalable arrays rather than the smaller ones, say minimum of 100 disks? If the system can’t scale then, well that’s a whole other can of worms

Take IBM XIV for example, very nice software feature set, but doesn’t go beyond 180 drives, really limits it’s use cases. Since it’s an IBM, expand support to 1000 drives and it sounds like a nice system. Though the core architecture of the system as-is prevents that, which is why they are limited to 180.

Bottom line, in IOPS you cannot go faster than disk IOPS. My rule of thumb is 200IOPS per disk no matter what capacity or storage controller you have.

SPC do not represent reality. The benchmark do not assume “real” SAN configuration. Look at the number of hosts connected during the tests. One or two. In practical world we have up to hundreds of them with various workload and not just one type.

In my world they is no such thing of sequential. Everything is random considering all severs activities on the array. Same is true for SPC-2. No one have SAN with one type of workload for the entire array.

My 2 cents!