Diggin' technology every day


Dell and Exanet: MIA

TechOps Guy: Nate

The thoughts around Dell buying Compellent made me think back to Dell's acquistiion of the IP and some engineering employees of Exanet, as The Register put it, a crashed NAS company.

I was a customer and user of Exanet gear for more than a year, and at least in my experience it was a solid product, very easy to use, decent performance and scalable. The back end architecture to some extent mirrored the 3PAR hardware-based architecture but in software, really a good design in my opinion.

Basic Exanet Architecture

Their standard server at the time they went under was a IBM x3650, dual proc quad core Intel Xeon 5500-based platform with 24GB of memory.

Each server had multiple software processes called fsds or File system daemons, that ran, they ran one fsd per core. Each fsd was responsible for a portion of the file system (x number of files), they load balanced it quite well I never had to manually re-balance or anything. Each fsd was allocated its own memory space used for itself as well as cache, if I recall right the default was around 1.6GB per fsd.

Each NAS head unit had back end connectivity to all of the other NAS units in the cluster(minimum 2, maximum tested at the time they went under was 16). A request for a file could come in on any node, any link. If the file wasn't home to that node it would transparently forward the request to the right node/fsd to service the request on the back end. Much like how 3PAR's backplane forwards requests between controllers.

Standard for back end network was 10Gbps on their last models.

As far as data protection, the use of "commodity" servers did have one downside, they had to use UPS systems as their battery backup to ensure enough time for the nodes to shut down cleanly in the event of a power failure. This could present problems at some data centers as operating a UPS in your own rack can be complicated from a co-location point of view(think EPO etc). Another similar design that Exanet had compared to 3PAR is their use of internal disks to flush cache to, which is something I suppose Exanet was forced into doing, other storage manufacturers use battery backed cache in order to survive power outages of some duration. But both Exanet and 3PAR dump their cache to an internal disk so that the power outage can last for a day, a week, or even a month and it won't matter, data itnegrity is not compromised.

32-bit platform

The only thing that held it back was they didn't have enough time or resources to make the system fully 64-bit before they went under, that would of unlocked a whole lot of additional performance they could of gotten. Being locked into a 32-bit OS really limited what they could do on a single node, and as processors became ever more powerful they really had to make the jump to 64-bit.

Exanet was entirely based on "commodity" hardware, not only were they using x86 CPUs but their NAS controllers were IBM 2U rackmount servers running CentOS 4.4 or 4.5 if I recall right.

To me, as previous posts have implied, if your going to base your stuff on x86 CPUs, go all out, it's cheap anyways. I would of loved to have seen a 32-48 core Exanet NAS controller with 512GB-1TB of memory on it.

Back to Dell

Dell originally went into talks with Exanet a while back because Exanet was willing to certify Equallogic storage as a back end provider of disk to an Exanet cluster, using iSCSI inbetween the Exanet cluster and the Equallogic storage. Since nobody else in the indusry seemed willing to have their NAS solution talk to a back end iSCSI system. As far as I know the basic qualifications for this solution was completed in 2009, quite a ways before they ran out of cash.

Why did Exanet go under? I believe primarily because the market they were playing in was too small with too few players in it, not enough deals to go around, so whomever had the most resources to outlast the rest would come out on top, in this case I believe it was Isilon, even though they too were taken out by EMC from the looks of their growth it didn't seem like they were in a fine position to continue to operate independently. With Ibrix and Polyserve going to HP, Onstor going to LSI, and I'm still convinced BlueArc will go to HDS at some point(they are once again filing for IPO but word on the street is they aren't in very good shape), I suspect after they fail to IPO and go under. They have a very nice NAS platform, but HDS has their hands tied in supporting 3rd party storage other than HDS product, BlueArc OEM's LSI storage like so many others.

About a year ago SGI OEM'd one of BlueArc's products though recently I have looked around the SGI site and see no mention of it. Either they have abandoned it (more likely) or are just really quiet. Since I know SGI is also a big LSI shop I wonder if they are making the switch to Onstor. One industry insider I know suspects LSI is working on integrating the Onstor technology directly into their storage systems rather than having an independent head unit, which makes sense if they can make it work.

But really my question is why hasn't Dell announced anything related to the Exanet technology? They could of, quite possibly within a week or two had a system running and certified on Dell PowerEdge equipment and selling to both existing Exanet customers as well as new ones. The technology worked fine, it was really easy to setup and use, and it's not as if Dell has another solution in house that competes with it. AND since it was an entirely software based solution there was really no costs involved in manufacturing. Exanet had more than one PB-sized deal in the works at the time they went under, that's a lot of good will Dell just threw away. But hey, what do you expect, it's Dell. Thankfully they didn't get their dirty paws on 3PAR.

When I looked at how a NetApp system was managed compared to the Exanet my only response was You're kidding, right?

Time will tell if anything ever comes of the technology.

I really wanted 3PAR to buy them of course, they were very close partners with 3PAR and both pitched each other's products at every opportunitiy. Exanet would go out of their way to push 3PAR storage whenever possible because they knew how much trouble the LSI storage could be, and they were happy to get double the performance per spindle off 3PAR vs LSI. But I never did get an adequate answer out of 3PAR as to why they did not pursue Exanet, they were in the early running but pulled out for whatever reason, the price tag of less then $15M was a steal.

Now that 3PAR is with HP we'll see what they can do with Ibrix, I knew of more than one customer that migrated off of things like Ibrix and Onstor to Exanet, HP has been pretty silent about Ibrix since they bought them as far as I know. I have no idea how much R&D they have pumped into it over the years or what their plans might be.

Tagged as: , 4 Comments

Dell going after Compellent

TechOps Guy: Nate

I know this first made news a couple of days ago but I can't tell you how busy I've been recently. It seems like after Dell got reamed by HP in the 3PAR bidding war they are going after Compellent,  one of the only other storage technology companies utilizing distributed RAID, and as far as I know the main pioneer of automagic storage tiering.

This time around nobody else is expected to bid, it seems the stock speculators were a bit disappointed when the talks were announced as they had already bid the stock up far higher than what is being discussed as being the acquisition price.

While their previous generation of controllers seemed rather weak, their latest and greatest look to be a pretty sizable step up, and apparently can be leveraged by their existing customers, no need to buy a new storage system.

I can't wait to see how EMC responds myself. Dell must be really frustrated with them to go after Compellent so soon after losing 3PAR.


OpenBSD installer: party like it’s 2000

TechOps Guy: Nate

[Random Thought] The original title was going to be "OpenBSD: only trivial changes in the installer in one heck of a long time" a take off of their blurb on their site about remote exploits in the default install.

I like OpenBSD, well I like it as a firewall -- I love pf. I've used ipchains, iptables, ipfwadm, ipf (which I think pf was originally based off of and was spawned due to a licensing dispute with the ipf author(s)), ipfw, Cisco PIX and probably one or two more firewall interfaces, and pf is far and away the best that I've come across.  I absolutely detest Linux's firewall interfaces by contrast, going all the way back almost 15 years now.

I do hate the OpenBSD user land tools though, probably as much as the *BSD folks hate the Linux user land tools. I mean how hard is it to include an init script of sorts to start and stop a service? But I do love pf, so in situations where I need a firewall I tend to opt for OpenBSD wherever possible (when not possible I don't resort to Linux, I'd rather resort to a commercial solution perhaps a Juniper Netscreen or something).

But this isn't about pf, or user land. This is about the OpenBSD installer. I swear it's had only the most trivial changes and improvements done to it in at least the past 10 years, when I first decided to try it out. To me it is sad, the worst part about it is of course the disk partitioning interface. It's just horrible.

I picked up my 2nd Soekris net5501 system and installed OpenBSD 4.8 on it this afternoon, and was kind of sadened, yet not surprised how it still hasn't changed. I have my other Soekris running OpenBSD 4.4 and has been running for a couple years now. First used pf I believe back in about 2004 or so, so have been running it quite a while, nothing too complicated, it's really simple to understand and manage. My first experience with OpenBSD was I believe back in 2000, I'm not sure but I want to say it was something like v2.8. I didn't get very far with it, for some reason it would kernel panic on our hardware after about a day or so of very light activity, so went back to Linux.

I know pf has been ported to FreeBSD, and there is soon to be a fully supported Debian kFreeBSD distribution with the next major release of Debian whenever that is, so perhaps that will be worth while switching to for my pf needs, I don't know. Debian is another system which has been criticized over the years for having a rough installer, though I got to say in the past 4-5 years it really has gotten to be a good installer in my opinion. As a Debian user for more than 12 years now it hasn't given me a reason to switch away from it, but I still do prefer Red Hat based distros for "work" stuff.

First impressions are important, and the installer is that first impression. While I am not holding out hope they will improve their installer, it would be nice.


Java fallout from Oracle acquisition intensifies

TechOps Guy: Nate

I was worried about this myself, almost a year ago to the day raised my concerns about Oracle getting control of Java, and the fallout continues. Oracle already had BEA's JRockit, it's too bad they had to get Sun's JVM too.

Apache seems to have withdrawn from most things related to Java today according to our friends at The Register.

On Thursday, the ASF submitted its resignation from JCP's Java Standard and Enterprise Edition (SE/EE) Executive Committee as a direct consequence of the Java Community Process (JCP) vote to approve Oracle's roadmap for Java 7 and 8.

The ASF said it's removing all official representatives from all JSRs and will refuse to renew its JCP membership and EC position.

Java was too important a technology to be put in the hands of Oracle.

Too bad..

Tagged as: , 2 Comments