Pedal to the metal: HP 3PAR 7450
[NOTE: I expect to revise this many times - I'm not at HP Discover (maybe next year!), so I am basing this post off what info I have seen elsewhere, I haven't yet got clarification on what NDA info specifically I can talk about yet so am trying to be cautious !]
[Update: HP's website now has the info]
I was hoping they would announce the SPC-1 results of this new system, and I was going to wait until that happens, but I am not sure if they have them finalized yet, I've heard the ballpark figures, but am waiting for the official results.
The upside is I am on the east coast so I am up bright and early relative to my normal Pacific time zone morning.
I thought it would be announced later in the week but my first hint was this Russian blog (google translated), which I saw on LinkedIn a few minutes ago(relative to the time I started the blog post which took me a good two hours to write), also came across this press release of sorts, and there is the data sheet for the new system.
In a nut shell the 7450 is the system that HP mentioned at the launch event for the 7400 last December - though the model number was not revealed they said
In addition to mixed SSD/HDD and all-SSD configurations across the HP 3PAR StoreServ family, HP has announced the intent to develop an SSD-optimized hardware model based on the 3PAR operating system.
As fast as the all-SSD 7400 was, that was not the "optimized" hardware model - this one is (the one that was mentioned last December). I think the distinction with the word optimized vs using the phrase purpose built is important to keep in mind.
The changes from a hardware perspective are not revolutionary, 3PAR has, for the first time in their history (as far as I know anyway) has fairly quickly leveraged the x86 processors and upgraded both the processors and the memory (ASIC is the same as 7400) to provide the faster data ingest rate. I had previously (incorrectly of course) assumed that the ASIC was tapped out with earlier results and perhaps they would need even more ASICs to drive the I/O needs of an all-SSD system. The ASIC will be a bottleneck at some point but it doesn't seem to be today - the bottleneck was the x86 CPUs.
They also beefed up the cache, doubling what the 7400 has.
- 4-Node 7400: 4 x Intel Xeon 6-core 1.8 Ghz w/64GB Cache
- 4-Node 7450: 4 x Intel Xeon 8-core 2.3Ghz w/128GB Cache
Would of been nice to have seen them use the 10-core chips, maybe the turnaround for such a change would of been too difficult to pull off in a short time frame. 8-core Intel is not bad though.
The Russian blog above touts a 55% increase in performance on the 7450 over the 7400, and the cost is about 6% more (the press release above quotes $99,000 as entry level pricing)
Throughput is touted as 5.5 Gigabytes/second, which won't win any SPC-2 trophies, but is no slouch either - 3PAR has always been more about random IOPS than sequential throughput (though they often tout they can do both simultaneously within a single array - more effectively than other platforms).
The new system is currently tested (according to press release) at 540,000 read IOPS @ 0.6ms of latency. Obviously SPC-1 will be less than the 100% random read. This compares to the 7400 which was tested(under the same 100% read test I believe) to run at 320,000 IOPS @ 1.6ms of latency. So a 59.2% improvement in read IOPS and about 62% less latency.
Maybe we could extrapolate that number a bit here, the 7400 achieved 258,000 SPC-1 IOPS. 59.2% more would make the 7450 look like it would score around 413,000 SPC-1 IOPS, which is nearly the score of an 8-node P10800 which has 16 ASICs and 16 x Quad core Xeon processors! (that P10800 requires basically a full rack for just the controllers vs 4U for the 7450 (assuming they can get the full performance out of the controllers with only 48 SSD drives).
The blog also talks about the caching improvements targeted to improve performance and lifetime of the SSDs. The new 3PAR software also has a media wear gauge for the SSDs, something I believe the HP Lefthand stuff got in a year or two ago (better late than never!). The graphics the Russian blog has are quite good, I didn't want to too shamelessly rip them from their blog to re-post here so I encourage you to go there to see the details on the caching improvements that are specific to SSD).
The competition
This system is meant to go head to head with the all-flash offerings from the likes of EMC, IBM NetApp (not aware of any optimized flash systems from HDS yet - maybe they will buy one of those new startups to fill that niche - they do have an optimized flash module for their VSP but I'd consider that a different class of product which may retain the IOPS constraints of the VSP platform).
However unlike the competition who has had to go outside of their core technology, HP 3PAR has been able to bring this all flash offering under the same architecture as the spinning rust models, basically it's the same system with some tweaked software and faster processors with more memory. The underlying OS is the same, the features are the same, the administrative experience is the same. It's the same, which is important to keep in mind. This is both good and bad, though for the moment I believe more good (Granted of course HP had to go to 3PAR to get all of this stuff, but as this blog has had a lot of 3PAR specific things I view this more in a 3PAR light than in a HP light if you get what I mean).
Of the four major competitors, EMC is the only one that touts deduplication (which, IMO is only really useful for things like VDI in transactional workloads)
3PAR is the only one with a mature enterprise/service provider grade operating system. On top of that obviously 3PAR is the only one that has a common platform amongst all of it's systems from the 7200 all the way to the 10800.
3PAR and IBM are the only ones that are shipping now. Just confirmed from El Reg that the 7450 is available immediately.
None of big four tout compression, which I think would be a greater value add than deduplication for most workloads. I'm sure it's on all of their minds though, it could be a non trivial performance hit, and in 3PAR's case they'd likely need to implement it in the ASIC, if so, it means having to wait until the next iteration of the ASIC comes out. There has been gzip compression available in hardware form for many years so I imagine it wouldn't be difficult to put into the silicon to keep the performance up under such conditions.
The new system also supports a 400GB MLC self encrypting drive (along with other SEDs for other 3PAR platforms as well) - 3PAR finally has a native encryption option, for those that need it.
Who should buy this
This isn't an array for everyone (nor are the ones from the other big storage players). It's a specialized system for specific very high performance workloads where latency is critical, yet at the same time providing the availability and manageability of the 3PAR platform to an all SSD solution.
You can probably go buy a server and stuff it with a few PCIe flash boards and meet or exceed the IOPS at a similar latency and maybe less price. If your workload is just dumb IOPS and you care about the most performance at the least price then there are other options available to you (they probably won't work as well but you get what you (don't) pay for).
There clearly is a market for such a product though, the first hint of this was dropped when HP announced an all flash version of it's P10000 about a year ago. Customers really wanted an all flash system and they really wanted the 3PAR OS on it. If your not familiar with the high end 3PAR systems well from a form factor perspective driving 400k+ SPC-1 IOPS on a P10800 vs a 7450 you would probably get a good chuckle out of how much floor space and power circuits are required for the P10800 (power draw would be light on the SSDs of course, but they have hard requirements for power provisioning - most customers would pay per circuit regardless of draw).
I think a lot of this may be in the banking sector, where folks are happy to buy tons of fancy low latency stuff to make sure their transactions are processed in milliseconds.
Fifteen milliseconds may not seem like a significant amount of time—it is literally shorter than a human blink of an eye, which takes 300 to 400 milliseconds. But in the age of super-high-speed computerized trading, Wall Street firms need less than a millisecond to execute a trade.
[..]
All told, Nanex calculated that $28 million worth of shares were exchanged in a short time[15 milliseconds] before the official release of the ISM data.
The skeptics
There have been a lot of skeptics out there wondering whether or not the 3PAR architecture could be extended to cover an all flash offering (you can actually sort of count me in the skeptical camp as well, I was not sure even after they tried to re-assure me, I want to see the numbers at the end of the day). I believe with this announcement they have shown that even more so than the 7400, they have a very solid all flash offering that will, in most cases beat the tar out of the competition, not only on performance, not only on latency, not only on enterprise grade availability and functionality, but on price as well.
Even with this high performance system, these all SSD systems illustrate quite well how a modern storage controller is not able to scale anywhere nearly as well with SSDs as with spinning rust. Most of the SSD offerings have a small number of SSDs before they tap out the controllers. No single controller(that I've seen) supports the multi millions of IOPS that would be required to drive many hundreds of SSDs at line rate simultaneously(like regular storage arrays would drive hundreds of disks today).
It is just interesting to me to see the massive bottleneck shift continues to be the controller, and will be for some time to come. I wonder when the processors will get fast enough that they might shift the bottleneck back to the storage media, a decade? Or perhaps by that time everyone will be running on some sort of mature grid storage technology, and the notion of controllers as most of us know them today will be obsolete as a concept. Certainly several cloud providers are already trying to provide grid storage as an alternative, though in most cases, while the cost can be low, the performance is very poor as well (relative to an HP 3PAR anyway).
There is always more work to do (in this case mainly dedupe and compression), and as you might expect HP, along with the other big storage companies are constantly working to add more, I am very excited about what the future holds for 3PAR, really have never been so excited since the launch of the 7000 series last year(as a customer now for almost seven years) and am very pleased with what HP has managed to accomplish with the technology thus far.
Other 3PAR announcements
- 3PAR Priority Optimization is made available now (first announced last December) - this is basically fine grained QoS for IOPS and throughput, something that will be a welcome enhancement to those running true multi tenant systems.
- 3PAR Recovery Manager for Hyper-V - sounds like they are bringing Hyper-V up to the same level of support as VMware.
- As mentioned earlier, Self encrypting drive options are cited on the Russian blog include - 400GB MLC SSD, 450GB 10k, 900GB 10k, 1TB 7.2k 2.5 "
Side note: there are a few other things to write about later, such as the IBM XIV SPC-1, the HP StoreOnce VSA, and probably whatever else comes out at Discover. For sure I won't get to those today(or maybe even tomorrow, I am on a semi vacation/working week this week).
Real life Sim City
[WARNING: Non technical content directly ahead]
I've looked at Google maps a lot over the years, but don't remember ever seeing something quite like this. I played tons of Sim City many years ago and when I saw this I was immediately reminded of Sim City. It just seems so ..familiar.
I'm planning on staying at a hotel in this town in Nevada in a couple of weeks to visit a friend who is coming in from out of town(in case you were wondering how I stumbled upon this).
This first picture reminds me of many times when I would build out a neighborhood in Sim City with the roads, zone it with light (or medium) residential, perhaps put neighborhood school near by - then watch the houses pop up one by one:
Real life Sim City Part 1
You can see a few individual houses here and there, and it's pretty easy to make out what look a lot like Sim City zoned plots of land(semi square shaped), obviously with a bunch of roads that are already complete. For the most part very clean empty plots of land. Much different than what I have seen many times in the past where perhaps there is a big real estate project under development and many houses are being built simultaneously with the road being laid out.
There is another part of the town that is quite similar, again eerily reminds me of Sim City:
Sim City in real life part 2
In this case I'm again reminded of some low density residential, along with a park in the middle(well in this case the other half of the middle is not yet laid down (not in the picture above, see the google maps link). The plots are so uniform, the houses remind me so much of Sim City.
Break out the bubbly, 100k SPAM comments
It seems we crossed the 100k spam comments blocked by Akismet mark. (see right side of the page)
Saved for history: crossing the 100k comment spam marker
That is just insane. 100k. I don't even know what I would do without them. Well I guess I do -- I'd have to keep comments off. That low cost annual subscription pays for itself pretty quick.
I verified on archive.org that on October 10, 2012 this site was at ~32k spam comments blocked. On May 17, 2012 only ~23k.
75,000 spam comments in about one year? For this tiny site ?
*shakes head*
Side note: for some reason HP employees are always blocked by Akismet, I don't know why. I think they are the only ones who have contacted me saying their comments were (incorrectly) blocked.
Infoworld suggests radical Windows 8 changes
Saw this come across on slashdot, an article over at Infoworld on how Microsoft can fix Windows 8.
They suggest ripping out almost all of the new stuff (as defaults) and replacing it with a bunch of new options that users can pick from.
Perhaps I am lucky in that I've never used Windows 8 (I briefly held a MS Surface RT in my hands, a friend who is an MS employee got one for free(as did all employees I believe) and handed it to me to show me some pictures on it).
Some of the suggestions from Infoworld sound pretty good to me, though hard to have a firm opinion since I've never used the Metro UI (oh, sorry they changed the name to something else).
Windows 8 (as it stands today) certainly sounds pretty terrible from a UI standpoint. The only positives I have read on Windows 8 is people say it is faster. Which isn't much these days, machines have been fast enough for many years(which at least in part has led to the relative stagnation of the PC market). My computers have been fast enough for years(the laptop I am typing on is almost 3 years old, I plan to keep it around for at least another year as my primary machine -- I have another year of on site support so I'm covered from that angle).
It has been interesting to see, that really since XP was released, there haven't been anything really exciting on the Windows desktop front, it's a mature product(the results have shown, much like the economy pretty much every OS launch they've done has had weaker reception than the previous - Windows 7 sort of an exception from the hard core community but from a broader sense it still seemed weak). It's come a long way from the mess many of us dealt with in the 90s (and instability in NT4 was one big driver for me to attempt Linux on my primary desktop 15 years ago and I'm still with Linux today).
I don't use Windows enough to be able to leverage the new features. I'm still used to using the XP interface, so am not fond of many of the new UI things that MS has come up with over the years. Since I don't use it much, it's not critical.
The last time I did use Windows seriously was at a few different companies I had windows as my primary desktop. But you probably wouldn't know it if you saw it. It was customized with cygwin, and Blackbox for windows. Most recently was about three years ago (company was still on XP at the time). Most of the time my screen was filled with rxvt X terminals (there is a native Windows port for rxvt in cygwin that works wonderfully), and firefox. Sometimes had Outlook open or Visio or in rare cases IE.
Not even the helpdesk IT guy could figure my system out "Can you launch control panel for me?". It gave it a nice Linux look & feel(I would of killed for proper virtual desktop edge flipping but I never found a solution for that) with the common windows apps.
Ironically enough I've purchased more copies of Windows 7 (I think I have 7 now - 2 or 3 licenses are not in use yet - stocked up so I wouldn't have to worry about Windows 8 for a long time) than all previous MS operating systems combined. I've bought more Microsoft software in the past 3-4 years (Visio Pro 2010 is another one) than in the previous decade combined. As my close friends will attest I'm sure - I have not been a "hater" of Microsoft for some time now (12 years ago I threatened to quit if they upgraded from NT4 to Windows 2000 - and they didn't at least not as long as I was there - those were the days when I was pretty hard core anti MS - I was working on getting Samba-tng and LDAP to replace NT4 - I never deployed the solution, and today of course I wouldn't bother)
Some new Linux UIs suck too
Microsoft is not alone in crappy UIs though. Linux is right up there too (many would probably argue it always was, that very well could be true, though myself I was fine with what I have used over the years from KDE 0.x to AfterStep to GNOME 1.x/2.x). GNOME 3 (and the new Unity stuff from Ubuntu) looks at least as terrible as the new Microsoft stuff (if not more so).
I really don't like how organizations are trying to unify the UI between mobile and PC. Well maybe if they did it right I'd like it (not knowing what "right" would be myself).
By the same notion I find it ludicrous that LG would want to put WebOS on a TV! Maybe they know something I don't though, and they are actually going to accomplish something positive. I love WebOS (well the concept - the implementation needs a lot of work and billions of investment to make it competitive) don't get me wrong but I just don't see how there is any advantage to WebOS on a device like a TV. The one exception is ecosystem - if there is an ecosystem of WebOS devices that can seamlessly inter-operate with each other. There isn't such an ecosystem today, what's left has been a rotting corpse for the past two years (yes I still use my HP Pre3 and Touchpad daily). There's no sign LG has a serious interest in making such an ecosystem, and even if they did, there's no indication they have the resources to pull it off (I'd wager they don't).
I haven't used Unity but last weekend I did install Debian 7 on my server at home (upgraded from 6). 99% of the time from a UI perspective this system just cycles through tens of thousands of images as a massive slide show (at some point I plan to get a 40"+ screen and hang it on my wall as a full sized digital picture frame, I downloaded thousands of nice 1080p images from interfacelift as part of the library).
I was happy to see Debian 7 included a "GNOME 2 like" option, as a moderately customized Gnome 2 is really what I am used to, and I have absolutely positively no interest to change it.
It gets mostly there, maybe 50-75% of the way. First thing I noticed was the new Gnome did not seem to import any of the previous settings. I got a stock look - stock wallpaper, stock settings, and no desktop icons(?). I tried to right click on the desktop to change the wall paper - that didn't work either. I tried to right click on the menu bar to add some widgets, that didn't work either. I went from 0 to very annoyed almost immediately. This was with the "compatibility" gnome desktop! Imagine if I had tried to login to regular GNOME 3, I probably would of thrown my laptop against the wall before it finished logging in! 
(fortunately for my laptop's sake I have never gotten to that point)
Eventually I found the way to restore the desktop icons and the right click on the desktop, I managed to set one of my wonderful high res NSFW desktop backgrounds. I still can't add widgets to the menu bar I assume it is not possible. I haven't checked if I can do virtual desktop edge flipping with brightside (or with something built in), I'd wager that doesn't work either.
I'm not sure what I will do on my main laptop/desktop which are Ubuntu 10.04 which is now unsupported. I hear there are distros/packages out there that are continuing to maintain/upgrade the old Gnome 2 stuff (or have replaced Gnome 3's UI with Gnome 2), so will probably have to look into that, maybe it will be easy to integrate into Debian or Ubuntu 12.04(or both).
I saw a fantastic comment on slashdot recently that so perfectly describes the typical OSS developer on this stuff
[..]
What X11 is, is old. And developers are bored with it. And they want something new and shiny and a chance to play with the hardware without abstraction throwing a wet blanket over their benchmark scores.
The benchmark of success for Wayland is that _users_ don't actually notice that anything changed. They'll fall short of that benchmark because too many people like using X11, and even the backward compatibility inevitably will cause headaches.
But developers will enjoy it more, and in the FOSS world those are the only consumers that matter.
(the last sentence especially)
That was in a conversation related to replacing X11 (the main GUI base for Linux) with something completely different (apparently being developed by some of the same folks that worked on X11) that has been under development for many, many years. Myself I have no issues with X11, it works fine for me. Last time I had major issues with X11 was probably 10+ years ago.
As someone who has worked closely with developers for the past 13 years now I see a lot of this first hand. Often times the outcome is good, many other times not so much.
One system I worked with was so architecturally complex that two people on my team left the company within a year of starting and their primary complaint was the application was too complicated to learn (they had been working with it almost daily for their time there). It was complex for sure(many many sleepless nights and long outages too) - though it didn't make me want to throw my laptop against the wall like Chef does.
In the case of Microsoft, I found it really funny that one of(if not the) main managers behind Windows 8 suddenly resigned mere weeks after the Windows 8 launch.
Travel tips for Washington DC area?
I am planning on being in the Washington DC area next week to visit a a friend I haven't seen in a couple of years. I have another friend that I know that is in that area and will visit them too.
I've never been there before, so if anyone knows something/place cool to visit please send a note my way! Doesn't have to be in DC - but within say a 2 or maybe even 3 hour drive is fine.
I arrive at 10 AM on Saturday and can't check in till 3PM so my first thought to kill some time is to drive to Philadelphia to grab one of those famous original Cheese steak sandwiches at either one of Geno's Steaks or Pat's King of Steaks (which are apparently right across the street from each other).
Five hours round trip is a long ways to drive but in the grand scheme of life, who knows if I'll ever be in the area again, so I figure it's worth it. If I could find some place else to visit and take some pictures(preferably with a nice view, my camera has a 42X zoom). I'm not much for historical stuff or museums (though I may make some exceptions on this trip).
I browsed ~250 potential places on Trip Advisor in Philadelphia but sadly did not see anything that really interested me(except maybe this). Given the sheer number of ideas on that site I figure it may be difficult to find things on other sites that aren't just repeats.
One day during the week(perhaps Sunday the 9th) I plan to visit Norfolk, VA - a full six hours round trip. But it looks like it will be worth it too -- mainly to see the military stuff there. Of the three major locations it's the one I am most looking forward to.
Possibilities in Norfolk include
- USS Wisconsin
- Norfolk Naval Base
- Cape Henry Lighthouse
- Victory Rover Naval Base Cruises (hope I don't get sea sick)
- Former TechOpsGuy Dave's wife suggested The Mariner's Museum in Newport News, VA (fortunately in between DC and Norfolk)
One day hit Baltimore up for their Blue Crab, possibilities for this trip include
- Ripley's Believe it or Not! - I was a fan of a TV show of the same name when I was a kid
- Federal Hill Park
- Washington Monument and Mount Vernon Park
- Canton Waterfront Park
- Crab from a place like Captain James Crabhouse or Costas Inn
Then the rest of the time in DC, most of these places to visit just so I can say I visited them, really not excited about any of them (specifically avoided any places that don't allow pictures like the Mint) -
- Smithsonian Institution Building - perhaps the National Air and Space Museum
- Newseum
- National Geographic Museum
- Lincoln Memorial and Reflecting Pool
- Jefferson Memorial
- Capitol Hill
- National Mall
- Washington Monument
Hopefully I can easily hit all the above sites in less than one day.
- Sweetwater Tavern - local friend says there is good food there
- Ted's Montana Grill - enjoyed this place when I was in Atlanta - it was the first time I had Buffalo (that I can recall).
- Tilted Kilt would be nice but is ~70 miles away so probably will wait till I'm in Atlanta next to hit that place up.
I'll likely be working remotely for a couple days while there, not sure yet.
3PAR up 82% YoY – $1 Billion run rate
I came across this article on The Register which covered some of HP's storage woes (short story: legacy storage is nose diving and 3PAR is shining). El Reg linked to the conference call transcript and I just ran a quick keyword search for 3PAR and saw this
This has been one of our most successful product introductions and 3PAR has now exceeded the $1 billion run-rate revenue mark.
[..]
Converged storage products were up 48% year-over-year and within that 3PAR was up 82%
Congratulations 3PAR! Woohoo! All of us over here at Techopsguys are really proud of you - keep up the good work! <voice="Scotty">Almost brings a tear to me eye.</voice>
For a comparison, I dug up the 3PAR results on archive.org for the quarter immediately previous to them being acquired:
3PAR® (NYSE: PAR), the leading global provider of utility storage, today reported results for the first quarter of fiscal year 2011, which ended June 30th, 2010. Revenue for the first quarter was $54.3 million, an increase of 22% as compared to revenue of $44.5 million for the same period in the prior year, and an increase of 1% as compared to $53.7 million in the prior quarter, which ended March 31st, 2010.
I can't help but wonder how well Compellent is doing for Dell these days by contrast, since Dell withdrew from the bidding war for 3PAR with HP and went for them instead.. (side note: I once saw some value in Compellent as an alternative to 3PAR but that all went away with the 3PAR 7000-series.). I looked at the transcript for Dell's latest conference call and the only thing they touch about storage is declines of 10%, no mention of any product lines as far as I could tell.
3PAR 7400 SSD SPC-1
I've been waiting to see these final results for a while, and now they are out! The numbers(performance + cost + latency) are actually better than I was expecting.
You can see a massive write up I did on this platform when it was released last year.
(last minute edits to add a new Huawei results that was released yesterday)
(more last minute edits to add a HP P6500 EVA SPC-1E)
SPC-1 Recap
I'll say this again in case this happens to be read by someone who is new here. Myself, I see value in the SPC-1 as it provides a common playing field for reporting on performance in random transactional workloads (the vast majority of workloads are transactional). On top of the level playing field the more interesting stuff comes in the disclosures of the various vendors. You get to see things like
- Cost (SpecSFS for example doesn't provide this and the resulting claims from the vendors showing high performance relative to others at a massive cost premium but not disclosing the costs is very sad)
- Utilization (SPC-1 minimum protected utilization is 55%)
- Configuration complexity (only available in the longer full disclosure report)
- Other compromises the vendor might of made (see the note about disabling cache mirroring)
- 3 year 24x7 4 hour on site hardware support costs
There is a brief executive summary as well as what is normally a 50-75 page full disclosure report with the nitty gritty details.
SPC-1 also has maximum latency requirements - no I/O request can take longer than 30ms to serve or the test is invalid.
There is another test suite - SPC-2, which tests throughput with various means. Much fewer systems participate in that test (3PAR never has, though I'd certainly like them to).
Having gone through several storage purchases over the years I can say from personal experience it is a huge pain to try to evaluate stuff under real workloads - often times vendors don't even want to give evaluation gear (that is in fact in large part why I am a 3PAR customer today). Even if you do manage to get something in house to test, there are many things out there, with wide ranging performance / utilization ratios. At least with something like SPC-1 you can get some idea how the system performs relative to other systems at non trivial utilization rates. This example is rather extreme but is a good illustration.
I have no doubt the test is far from perfect, but in my opinion at least it's far better than the alternatives, like people running 100% read tests with IOMeter to show they can get 1 million IOPS.
I find it quite strange that none of the new SSD startups have participated in SPC-1, I've talked to a couple different ones and they don't like the test, they give the usual it's not real world, customers should take the gear and test it out themselves. Typical stuff. Usually means they would score poorly - especially those that leverage SSD as a cache tier, with high utilization rates of SPC-1 you are quite likely to blow out that tier, once that happens performance tanks. I have heard reports of some of these guys getting their systems yanked out of production because they fail to perform after utilization goes up. System shines like a star during brief evaluation - then after several months of usage and utilization increasing, performance no longer holds up.
One person said their system is optimized for multiple workloads and SPC-1 is a single workload. I don't really agree with that, SPC-1 does a ton of reads and writes all over the system, usually from multiple servers simultaneously. I look back to 3PAR specifically, who have been touting multiple workload (and mixed workload) support since their first array was released more than a decade ago. They have participated in SPC-1 for over a decade as well, so arguments saying testing is too expensive etc doesn't hold water either. They did it when they were small, on systems that are designed from the ground up for multiple workloads (not just riding a wave of fast underlying storage and hoping that can carry them), these new small folks can do it too. If they can come up with a better test with similar disclosures I'm all ears too.
3PAR Architecture with mixed workloads
The one place where I think SPC-1 could be improved is in failure testing. Testing a system in a degraded state to see how it performs.
The below results are from what I could find on all SSD SPC-1 results. If there is one/more I have missed(other than TMS, see note below), let me know. I did not include the IBM servers with SSD, since those are..servers.
Test Dates
| System Name | Date Tested |
|---|---|
| HP 3PAR 7400 | May 23, 2013 |
| HP P6500 EVA (SPC-1E) | February 17, 2012 |
| IBM Storwize V7000 | June 4, 2012 |
| HDS Unified Storage 150 | March 26, 2013 |
| Huawei OceanStor Dorado2100 G2 | May 22, 2013 |
| Huawei OceanStor Dorado5100 | August 13, 2012 |
I left out the really old TMS (now IBM) SPC-1 results as they were from 2011, too old for a worthwhile comparison.
Performance / Latency
| System Name | SPC-1 IOPS | Avg Latency (all utilization levels) | Avg Latency (Max utilization) | # of times above 1ms latency | # of SSDs |
|---|---|---|---|---|---|
| HP 3PAR 7400 | 258,078 | 0.66ms | 0.86ms | 0 / 15 | 32x 200GB |
| HP P6500 EVA (SPC-1E) | 20,003 | 4.01ms | 11.23ms | 13 / 15 | 8x 200GB |
| IBM Storwize V7000 | 120,492 | 2.6ms | 4.32ms | 15 / 15 | 18x 200GB |
| HDS Unified Storage 150 | 125,018 | 0.86ms | 1.09ms | 12 / 15 | 20x 200GB |
| Huawei OceanStor Dorado2100 G2 | 400,587 | 0.60ms | 0.75ms | 0 / 15 | 50x 200GB |
| Huawei OceanStor Dorado5100 | 600,052 | 0.87ms | 1.09ms | 7 / 15 | 96x 200GB |
A couple of my own data points:
- Avg latency (All utilization levels) - I just took aggregate latency of "All ASUs" for each of the utilization levels and divided it by 6 (the number of utilization levels)
- Number of times above 1ms of latency - I just counted the number of cells in the I/O throughput table for each of the ASUs (15 cells total) that the test reported above 1ms of latency
Cost
| System Name | Total Cost | Cost per SPC-1 IOP | Cost per Usable TB |
|---|---|---|---|
| HP 3PAR 7400 | $148,737 | $0.58 | $133,019 |
| HP P6500 EVA (SPC-1E) | $130,982 | $6.55 | $260,239 |
| IBM Storwize V7000 | $181,029 | $1.50 | $121,389 |
| HDS Unified Storage 150 | $198,367 | $1.59 | $118,236 |
| Huawei OceanStor Dorado2100 G2 | $227,062 | $0.57 | $61,186 |
| Huawei OceanStor Dorado5100 | $488,617 | $0.81 | $77,681 |
Capacity Utilization
| System Name | Raw Capacity | Usable Capacity | Protected Application Utilization |
|---|---|---|---|
| HP 3PAR 7400 | 3,250 GB | 1,159 GB | 70.46% |
| HP P6500 EVA (SPC-1E) | 1,600 GB | 515 GB | 64.41% |
| IBM Storwize V7000 | 3,600 GB | 1,546 GB | 84.87% |
| HDS Unified Storage 150 | 3,999 GB | 1,717 GB | 85.90% |
| Huawei OceanStor Dorado2100 G2 | 10,002 GB | 3,801 GB | 75.97% |
| Huawei OceanStor Dorado5100 | 19,204 GB | 6,442 GB | 67.09% |
Thoughts
The new utilization charts in the latest 3PAR/Huawei tests are quite nice to see, really good illustrations as to where the space is being used. They consume a full 3 pages in the executive summary. I wish SPC would go back and revise previous reports so they have these new easier forms of disclosure in them. The data is there for users to compute on their own.
HP EVA
This is a SPC-1e result rather than SPC-1 - I believe the work load is the same(?) they just measure power draw in addition to everything else. The stark contrast between the new 3PAR and the older P6500 is remarkable from every angle whether it is cost, performance, capacity, latency. Any way you slice it (well except power I am sure 3PAR draws more power )
It is somewhat interesting in the power results for the P6500 that there is only a 16 watt difference between 0% load and 100% load.
I noticed that the P6500 is no longer being sold (P6550 was released to replace it - and the 3PAR 7000-series was released to replace the P6550 which is still being sold).
Huawei
While I don't expect Huawei to be a common rival for the other three outside of China perhaps, I find their configuration very curious. On the 5100 with such a large number of apparently low cost SLC(!) SSDs, and "short stroking" (even though there are no spindles I guess the term can still apply) they have managed to provide a significant amount of performance at a reasonable cost. I am confused though they claim SLC but yet they have so many disks(would think you'd need fewer with SLC), at the same time at a much lower cost. Doesn't compute..
No software
Huawei appears to have absolutely no software options for these products - no thin provisioning, no snapshots, no replication, nothing. Usually vendors don't include any software options as part of the testing since they are not used. In this case the options don't appear to exist at all.
They seem to be more in line with something that LSI/NetApp E-series, or Infortrend or something like that rather than an enterprise storage system. Though looking at Infortrend's site earlier this morning shows them supporting thin provisioning, snapshots, and replication on some arrays. Even NetApp seems to have thin provisioning on their E-series included.
3PAR
3PAR's metadata
3PAR's utilization in this test is hampered by (relatively) excessive metadata, the utilization results say only 7% unused storage ratio which on the surface is an excellent number. But this number excludes metadata which in this case is 13%(418GB) of the system. Given the small capacity of the system this has a significant impact on utilization (compared to 3PAR's past results). They are working to improve this.
The next largest meta data size in the above systems is IBM which has only 1GB of metadata (about 99.8% less than 3PAR). I would be surprised if 3PAR was not able to significantly slash the metadata size in the future.
3PAR 7400 SSD SPC-1 Configured Storage Capacity (one of the new charts from the SPC-1 report)
In the grand scheme of things this problem is pretty trivial. It's not as if the meta data scales linearly with the system.
Only quad controller system
3PAR is the only SSD solution above tested with 4 controllers(totalling 4 Gen4 ASICs, 24 x 1.8Ghz Xeon CPU cores, 64GB of data cache, and 32GB of control cache), meaning with their persistent cache technology(which is included at no extra cost) you can lose a controller and keep a fully protected and mirrored write cache. I don't believe any of the other systems are even capable of such a configuration regardless of cost.
3PAR Persistent Cache mirrors cache from a degraded controller pair to another pair in the cluster automatically.
The 7400 managed to stay below 1 millisecond response times even at maximum utilization which is quite impressive.
Thin provisioning built in
The new license model of the 3PAR 7000 series means this is the first SPC-1 result to include thin provisioning for a 3PAR system at least. I'm sure they did not use thin provisioning(no point when your driving to max utilization), but from a cost perspective it is something good to keep in mind. In the past thin provisioning would add significant costs onto a 3PAR system. I believe thin provisioning is still a separate license on the P10000-series (though would not be surprised if that changes as well).
Low cost model
They managed to do all of this while remaining a lower cost offering than the competition - the economics of this new 7000 series are remarkable.
IBM's poor latency
IBM's V7000 latency is really terrible relative to HDS and HP. I guess that is one reason they bought TMS. Though it may take some time for them to integrate TMS technology (assuming they even try) to have similar software/availability capabilities as their main enterprise offerings.
Conclusion
With these results I believe 3PAR is showing well that they too can easily compete in the all SSD market opportunities without requiring excessive amounts of rack space or power circuits as some of their previous systems required. All of that performance(only 32 of the 48 drive bays are occupied!), in a small 4U package. Previously you'd likely be looking at a absolute minimum of half a rack!
I don't know whether or not 3PAR will release performance results for the 7000 series on spinning rust, it's not too important at this point though. The system architecture is distributed and they have proven time and again they can drive high utilization, so it's just a matter of knowing the performance capacity of the controllers (which we have here), and just throwing as much disk as you want at it. The 7400 series tops out at 480 disks at the moment - even if you loaded it up with 15k spindles you wouldn't come close to the peak performance of the controllers.
It is, of course nice to see 3PAR trouncing the primary competition in price, performance and latency. They have some work to do on utilization as mentioned above.
SHOCKER! Power grid vulernable to Cyberattack!
Yeah, it shouldn't be news.. but I guess I am sort of glad it is making some sort of headline. I have written in the past how I think the concept of a smart grid is terrible due to security concerns. I just have no confidence in today's security technology to properly secure such a system. If we can't properly secure our bank transactions(my main credit card was compromised for at least the 2nd or 3rd time this year and I am careful), how can anyone expect to secure the grid?
Just saw a new post on Slashdot which points to a new report being released that covers some of how vulnerable we are to attack on our grid.
The report, released ahead of a House hearing on cybersecurity by Congressmen Edward Markey (D-Mass.) and Henry Waxman (D-Calif.), finds that cyberattacks are a daily occurrence, with one power company claiming it fights off 10,000 attempted intrusions each month.
[..]
Such attacks could cut power to large sections of the country and take months to repair.
Oh how I miss the days of early cyber security where the threat was little more than kids poking around and learning. These days there is really little defense against the organized military of the likes of China, sigh.
If they want to get you, most likely they are going to get you.
I've had a discussion or two with a friend who works with industrial control systems and the security on those is generally worse than I had heard about with the various breaches around the world.
I don't see any real value the so called smart grid has, anything remotely resembling gains that would offset the massive growth of the network access points that are connected to the grid.
It's probably already too late. All security is some form of obscurity at the end of the day whether it is a password, or encryption or physical isolation. Obscuring the grid by reducing the network connections to it has got to provide some level of benefit...
When a server is a converged solution
Thought this was kind of funny/odd/ironic/etc...
I got an email a few minutes ago which is talking about HP App System for Vertica. Which, among other things HP describes as being able to
This solution delivers system performance and reduces implementation from months to hours.
I imagine they are referring to competing solutions and not comparing to running Vertica on bare metal. In fact it may be kind of misleading as Vertica is very open - you can run it on physical hardware (any hardware really), virtual hardware, and even some cloud services (it is supported in *shudder*Amazon even..). So you can get implementation of a basic Vertica system without buying anything new.
But if you are past the evaluation stage, and perhaps outgrew your initial deployment and want to grow into something more formal/dedicated, then you may need some new hardware.
HP pitches this as a Converged Solution. So I was sort of curious what HP solutions are they converging here?
Basically it's just a couple base configurations of HP DL380G8s with internal storage (these 2U servers support up to 25 2.5" disks). They don't even install Vertica for you --
HP Vertica Analytics Platform software installation is well documented and can be installed by customers.
They are kind enough to install the operating system though (no mention of any special tuning, other than they say it is "Standard" so I guess no tuning).
No networking included(outside of the servers as far as I can tell), the only storage is the internal DAS. Minimum of three servers is required so some sort of 10GbE switches are required (since the severs are 10GbE, you can run Vertica fine on 1GbE too for smaller data sets).
I would of expected the system to come with Vertica pre-installed, or automatically installed as part of setup and have a trial license built into the system.
Vertica is very easy to install and configure the basics, so in the grand scheme of things this AppSystem might save the average Vertica customer a few minutes.
Vertica is licensed normally by the amount of data stored in the cluster (pre-compression / encoding). The node count, CPU count, memory, spindles doesn't matter. There is a community edition that goes up to 3 nodes, and 3TB (it has some other software limitations - and as far as I know there is no direct migration path from community to enterprise without data export/import).
Don't get me wrong I think this is a great solution, very solid server, with a lot of memory and plenty of I/O to provide a very powerful Vertica experience. Vertica's design reduces I/O requirements by up to ~90% in some cases, so you'd be probably shocked the amount of performance you'd get out of just one of these 3 node clusters, even without any tuning at the Vertica level.
Vertica does not require a fancy storage system, it's really built with DAS in mind. Though I know there are bunches of customers out there that run it on big fancy storage because they like the extra level of reliability/availability.
I just thought it was kind of strange some of the marketing behind it, saving months of time, being converged infrastructure and what not..
It makes me think(if I had not installed Vertica clusters before) that if I want Vertica and don't get this AppSystem then I am in a world of hurt when it comes to setting Vertica up. Which is not the right message to send.
Here is this wonderful AppSystem that is in fact -- just a server with RHEL installed.
For some reason I expected more.
Big pop in Tableau IPO
I was first introduced to Tableau (and Vertica) a couple of years ago at a local event in Seattle. Both products really blew me away(and still do to this day). Though it's not an area I spend a lot of time in - my brain struggles with anything analytics related (even when using Tableau, same goes for Splunk, or SQL). I just can't make the connections, when I come across crazy Splunk queries that people write I just stare at it for a while in wonder(as in I can't possibly imagine how someone could of come up with such a query even after working with Splunk for the past six years).. then I copy+paste it and hope it works.
Sample Tableu reports pulled from google images
But that doesn't stop me from seeing an awesome combination that is truly ground breaking both in performance and ease of use.
I've seen people try to use Tableau with MySQL for example and they fairly quickly give up in frustration at how slow it is. I remember being told that Tableau used to get a bunch of complaints from users years ago saying how slow it seemed to be -- but it really wasn't Tableau's fault it was the slow back end data store.
Vertica unlocks Tableau's potential by providing a jet engine to run your queries against. Millions of rows? hundreds of millions? No problem.. billions ? It'll take a bit longer but shouldn't be an issue either. Try that with most other back ends and well you'll be waiting there for days if not weeks.
Tableau is a new generation of data visualization technology that is really targeted at the Excel crowd. It can read in data from practically anything(Excel files included), and it provides a seamless way to analyze your data and provide fancy charts and graphs, tables and maps..
It's not really for the hard core power users who want to write custom queries. Though I still think it is useful for those folks. A great use case for Tableau is for the business users to play around with it, and come up with the reports that they find useful, then the data warehouse people can take those results and optimize the warehouse for those types of queries (if required). It's a lot simpler and faster than the alternative..
I remember two years ago I was working with a data warehouse guy at a company and we were testing Tableau with MySQL at the time actually (small tables), just playing around, he poked around, created some basic graphs and drilled down into them. In all we spent about 5 minutes on this task and we found some interesting information. He said if he had to do that in MySQL queries himself it would of taken him roughly two days. Running query after query and then building new queries based on results etc. From two days to roughly five minutes -- for a very experienced SQL/data warehouse person.
Tableau has a server component as well, which you can publish your reports for others to see with a web browser or mobile device, the server can also of course directly link to your data to get updates as frequently as you want them.
You can have profiles and policies, one example Tableau gave me last year was one big customer enforces certain color codes across their organization so no matter what they are looking at they know Blue means X and Orange means Y. This is enforced at the server level, so it's not something people have to worry about remembering. They can also enforce policies around reporting so that the term "XYZ" is always the result of "this+that", so people get consistent results every time -- not a situation where someone interprets something one way, and another person another way. Again this is enforced at the server level, reducing the need for double checking and additional training.
They also have APIs - and users are able to embed Tableau reports directly into their applications and web sites(through the server component). I know one organization where almost all of their customer reporting is presented with Tableau - I'm sure it saved them a ton of time trying to replicate the behavior in their own code. I've seen folks try to write reporting UIs in past companies and usually what comes out is significantly sub par because it's a complicated thing to get right. Tableau makes it easy, and probably very cost effective relative to full time developers taking months/years to try to do it yourself.
It's one of the few products out there that I am really excited about, and I've seen some amazing stuff done with the software in a very minimal amount of time.
Tableau has a 15 day evaluation period if you want to try it out -- it really should be more, but whatever. Vertica has a community edition which you can use as a sort of long term evaluation - it's limited to 1TB of data and 3 cluster nodes. You can get a full fledged enterprise evaluation from Vertica as well if you want to test all of the features.
I wrote some scripts at my current company to refresh/import about 150GB of data from our MySQL systems to Vertica each night. It is interesting to see MySQL struggle to read the data out, and Vertica is practically idle as it ingests it (I'd otherwise normally think that the writing of the data would be more intensive than the reading). In order to improve performance I compiled a few custom MySQL binaries that allowed me to run MySQL queries and pipe the results directly into Vertica (instead of writing 10s of GBs to disk only to read it back again). The need for the custom binaries is MySQL by default only supports tab delimited results which was not sufficient for this data set (I actually compiled 3-4 different binaries with different delimiters depending on the tables - managed to get ~99.99999999% of the rows in without further effort). Also wrote a quick perl script to fix some of the invalid data like invalid time stamps which MySQL happily allows but Vertica does not.
Sample command:
$MYSQL --raw --batch --quick --skip-column-names -u $DB_USERNAME --password="${DB_PASSWORD}" --host=${DB_SERVER} $SOURCE_DBNAME -e "select * from $MY_TABLE" | $DATA_FIX | vsql -w $VERTICA_PASSWORD -c "COPY ${VERTICA_SCHEMA}.${MY_TABLE} FROM STDIN DELIMITER '|' RECORD TERMINATOR '##' NULL AS 'NULL' DIRECT"
Oh and back to the topic of the post - Tableau IPO'd today (ticker is DATA) - as of last check it is up 55%.
So, congrats Tableau on a great IPO!
About this site
Note on Comment SPAM
Spam Blocked
Follow Nate
Tags
Categories
- Datacenter (33)
- Events (11)
- General (77)
- linux (14)
- Monitoring (8)
- Networking (48)
- News (53)
- Random Thought (74)
- Security (7)
- Storage (95)
- Uncategorized (18)
- Virtualization (52)
Archives
- June 2013 (5)
- May 2013 (9)
- April 2013 (7)
- March 2013 (2)
- February 2013 (1)
- December 2012 (4)
- November 2012 (6)
- October 2012 (9)
- September 2012 (5)
- August 2012 (11)
- July 2012 (5)
- June 2012 (13)
- May 2012 (5)
- April 2012 (11)
- March 2012 (5)
- February 2012 (5)
- December 2011 (8)
- November 2011 (14)
- October 2011 (10)
- September 2011 (9)
- August 2011 (11)
- July 2011 (11)
- June 2011 (7)
- May 2011 (10)
- April 2011 (5)
- March 2011 (7)
- February 2011 (10)
- January 2011 (1)
- December 2010 (4)
- November 2010 (12)
- October 2010 (24)
- September 2010 (22)
- August 2010 (19)
- July 2010 (4)
- June 2010 (8)
- May 2010 (5)
- April 2010 (10)
- March 2010 (15)
- February 2010 (7)
- January 2010 (2)
- December 2009 (5)
- November 2009 (10)
- October 2009 (6)
- September 2009 (6)
- August 2009 (11)
- July 2009 (4)
Calendar
Blogroll
- Chuck's Blog
- Data Center Knowledge
- Planet V12n
- Storage Effect
- StorageMojo
- Thankfully the RAID is Gone