Diggin' technology every day


ISilon gets taken out by EMC

TechOps Guy: Nate

Looks like EMC did it after all, buying Isilon for $2.25 Billion. Probably the biggest tech deal for the Seattle area for quite some time.

I haven't paid too much attention to Isilon recently but it does seem like they have a nice product for the scale out media space, lots of big files and high throughput. Isilon, along with Panasas seem to be unique in tightly integrating the storage controller with the NAS side, while other solutions were purely gateway approaches of one sort or another.

So who's next?


Tagged as: Comments Off

Red Hat jacks up RHEL pricing

TechOps Guy: Nate

I didn't think they would do this, but Red Hat, along with RHEL 6 introduced some pretty dramatic price hikes.

They seem to have done away with the "unlimited socket" licensing and have at least two tiers - two socket and four socket.

What used to cost $2,499 for Red Hat Advanced Server Premium for unlimited sockets, now costs $6,498 for four sockets, a 260% increase.

That is very Oracle-esque, maybe even worse than Oracle, the biggest hikes I recall Oracle doing was in the 30-50% range. Wonder if there will be any push back from customers.

They don't seem to mention socket licensing beyond 4 sockets.

Filed under: linux, News Comments Off

RHEL 6 Launched

TechOps Guy: Nate

I didn't even notice it, as The Register put it, it was a very quiet launch. While I have been using Debian on my home systems for more than twelve years now, I do much prefer to use Red Hat Enterprise at work.

And RHEL 6 looks like a pretty decent upgrade

  • Significantly improved power management (aka lower cpu usage for idle VMs) - hello higher consolidation ratios
  • Hot add CPU and memory (wish there was hot remove - if there is I don't see it mentioned)
  • 85% increase in number of packages in the distribution - yay, maybe there will be a lot less things I will have to compile on my own

Sorry I still can't help but laugh at the scalability claims

Red Hat Enterprise Linux 6 has been designed to deliver performance and scalability without sacrificing data integrity. It allows scaling to 4,096 CPUs and 64 terabytes of RAM, providing a solid foundation for supporting upcoming generations of hardware.

It is interesting that the max file system size for the ext4 file system is the same as ext3 - 16TB. Seems kind of dinky.

XFS goes to 100TB which also seems small, maybe just "tested" limits, I would expect XFS to scale far higher than that given it's SGI heritage. The XFS documentation says for 64-bit Linux you can go to 18 Exabytes, which I think is just as crazy as Red Hat's CPU claims but as long as you can safely do a few hundred TB that is more than enough for these days I think.

I can't imagine anyone committing a petabyte or more to a single file system for a good long while at least.

I'll let others play with KVM until at least RHEL 7, until then it's VMware for me.

Filed under: linux, News Comments Off

10% Tale of two search engines

TechOps Guy: Nate

Saw! an! article! today! and! thought! of! a! somewhat! sad! situation,! at! least! for! those! at! Yahoo!

Not long ago, Google announced they would be giving every employee in the company a 10% raise starting January 2011. One super bad ass engineer is apparently going to get a $3.5M retention bonus to not go to the competition. Lucky for him perhaps that Google is based in California and non competes are not enforceable in California.

Now Yahoo! has announced somewhat of the opposite, no raises, in fact they are going to give the axe to 10% of their employees.

It's too bad that Yahoo! lost it's way so long ago. There was a really good blog post about what went wrong with Yahoo! Going back  more than a decade, really interesting insight into the company.

Tagged as: , Comments Off

Next Gen Opterons — to 20 cores and beyond?

TechOps Guy: Nate

So I came across this a short time ago, but The Register has a lot more useful information here.

From AMD

The server products (“Interlagos” and “Valencia”) will first begin production in Q2 2011, and we expect to launch them in Q3 2011. [This includes the Opteron 6100 socket compatible 16-core Opteron 6200]


Since Bulldozer is designed to fit into the same power/thermal envelope as our current AMD Opteron™ 6100/4100 series processors we obviously have some new power tricks up our sleeve.  One of these is the new CC6 power state, which powers down an entire module when it is idle. That is just one of the new power innovations that you’ll see with Bulldozer-based processors.


We have disclosed that we would include AMD Turbo CORE technology in the past, so this should not be a surprise to anyone. But what is news is the uplift – up to 500MHz with all cores fully utilized. Today’s implementations of boost technology can push up the clock speed of a couple of cores when the others are idle, but with our new version of Turbo CORE you’ll see full core boost, meaning an extra 500MHz across all 16 threads for most workloads.


We are anticipating about a 50% increase in memory throughput with our new “Bulldozer” integrated memory controller.

From The register

Newell showed off the top-end "Terramar" Opteron, which will have up to 20 of a next-generation Bulldozer cores in a single processor socket, representing a 25 percent boost in cores from the top-end Interlagos parts, and maybe a 35 to 40 per cent boost in performance if the performance curve stays the same as the jump from twelve-core "Magny-Cours" Opteron 6100s to the Interlagos chips.


That said, AMD is spoiling for a fight about chip design in a way that it hasn't been since the mid-2000s.


with Intel working on its future "Sandy Bridge" and "Ivy Bridge" Xeon processors for servers, and facing an architecture shift in the two-socket space in 2011 that AMD just suffered through in 2010.

Didn't Intel just go through an architecture shift in the two socket space last year with the Xeon 5500s and their integrated memory controller? And they are shifting architectures again so soon? Granted I haven't really looked into what these new Intel things have to offer.

I suppose my only question is, will VMware come up with yet another licensing level to go beyond 12 cores per socket? It's kind of suspicious that both vSphere Advanced and Enterprise plus are called out at a limit of 12 cores per socket.

Tagged as: , , Comments Off

Compellent beats expectations

TechOps Guy: Nate

Earlier in the year Compellent's stock price took a big hit following lower expectations for sales and stuff, a bunch of legal stuff followed that, it seems yesterday they redeemed themselves though with their stock going up nearly 33% after they tripled their profits or something.

I've had my eye on Compellent for a couple of years now, don't remember where I first heard about them. They have similar technology to 3PAR, just it's implemented entirely in software using Intel CPUs as far as I know vs 3PAR leveraging ASICs (3PAR has Intel CPUs too but they aren't used for too much).

I have heard field reports that because of this that their performance is much more on the lower end of things, they have never published a SPC-1 result and I don't know anyone that uses them so don't know how they really perform.

They seem to use the same Xyratex enclosures that most everyone else uses. Compellent's controllers do seem to be somewhat on the low end of things, I really have nothing other to go on other than cache. With their high end controller coming in at only 3.5GB of cache (I assume 7GB mirrored for a pair of controllers?) it is very light on cache. The high end has a dual core 3.0Ghz CPU.

The lower amount of cache combined with their software-only design and only two CPUs per controller and the complex automated data movement make me think the systems are built for the lower end and not as scalable, but I'm sure perfectly useful for the market they are in.

Would be nice to see how/if their software can scale if they were to put say a pair of 8 or 12 core CPUs in their controllers. After all since they are leveraging x86 technology performing such an upgrade should be pretty painless! Their controller specs have remained the same for a while now(as far back as I can remember). The bigger CPUs will use more power, but from a storage perspective I'm happy to give a few hundred more watts if I can get 5x+ the performance, don't have to think once, yet alone twice.

They were, I believe the first to have automagic storage tiering and for that they deserve big props, though again no performance numbers posted (that I am aware of) that can illustrate the benefits this technology can bring to the table. I mean if anybody can prove this strategy works it should be them right? On paper it certainly sounds really nice but in practice I don't know, haven't seen indications that it's as ready as the marketing makes it out to be.

My biggest issue with automagic storage tiering is how fast the array can respond to "hot" blocks and optimize itself, which is why I think from a conceptual perspective I really like the EMC Fast Cache approach more (they do have FAST LUN and sub LUN tiering too). Not that I have any interest in using EMC stuff but they do have cool bits here and there.

Maybe Compellent the next to get bought out (as a block storage company yeah I know they have their zNAS), I believe from a technology standpoint they are in a stronger position than the likes of Pillar or Xiotech.

Anyway that's my random thought of the day


Rust in peace

TechOps Guy: Nate

So one of of the former TechOpsGuys Tycen emailed our little team of people yesterday mentioning that the company where we all met (Dave, Jason, Nate & Tycen) is now dead and buried according to an article on TechFlash. We were all on the same operations team in case it wasn't obvious., the Seattle online recruiting startup formerly known as Jobster, has quietly been sold in a deal of unknown size to Phoenix, Arizona-based Jobing, TechFlash has learned. The acquisition, which closed in July, marks the final chapter for one of Seattle's most heavily funded Internet companies of the past decade. Founded in 2004 by Internet entrepreneur Jason Goldberg, raised $55 million from a slate of investors that included Ignition Partners, Reed Elsevier Ventures, Trinity Ventures, Mayfield Fund and others.

Where did the money go? I don't know, we were really efficient in IT, and I think development was pretty good, so I can only think that it was the less technical areas of the company that spent most of the cash. $55 million doesn't seem like enough to say "one of the most heavily funded". Doesn't seem like much at all. I know it was a different era but I remember at (headquartered in Federal Way, near Seattle) they were spending $7 million/month on bandwidth they weren't even using (they didn't know they had it). That is just one of the little factoids.

The strategies the company had while I was there were typical? Nothing short of their original strategy made any sense to me and unfortunately for them they kept pulling resources out of that original strategy (the one generating the majority of the revenue but it wasn't shiny). We knew we were in trouble when we were denied the right to remove email addresses from our databases that were bouncing back causing us to get blacklisted on mail servers due to excessive bouncing. It would hurt our user count. Well those users don't exist anymore!!

Jobster certainly provided me with several opportunities to round out some of my skills and experience. I had learned a lot prior to that but often worked with dedicated people whether it was networking people, storage people etc. So while I knew what I was doing it was nice to be able to build things up from scratch. Did a data center move while I was there, moving out of a facility that was plagued with power outages in cabinets spread out all over the place to a better facility in a consolidated cage. Got a lot more hands on with our Oracle Databases as well, at previous companies we had dedicated Oracle folks and they pretty much ran everything. Speaking of Oracle even transitioned the company from Enterprise Edition to Standard Edition which of course saved a ton of greenbacks.

I went looking for my stock certificates I bought a few shares after I left, the price was cheap (~$0.15) just in case. For some reason I could not find them, maybe they didn't send them to me I don't remember it was a while ago. Not that I think the stock has any value but wanted to post a picture of what they looked like for some color :)

It was an interesting place to work for at the time, I joined shortly after they launched their main product and it was pretty fast downhill from there. What can we say, we built the system to scale to the growth expectations that they had and that growth never materialized.

At the peak we had almost three racks of production equipment (15kW active 15kW backup power), in my mac daddy 47U 28" wide 49" deep Rittal TS-8 racks. Those racks are bad ass I gotta say. I could only fit three of them in a five rack cage (but due to power we had more than enough space in three). The cable management ROCKED (each server had at least 5 cables, which adds up fast).  The extra 4" of width made all the difference. The Servertech PDUs were great too, loved the integrated environmental sensors, rear door fans to exhaust the hot air away from the equipment. I love big racks! (couldn't resist)

The different facility was really nice as well the only downside is it doesn't support high density stuff.  I was hosted at the same place at the company before that and due to power constraints we had to have 8 feet of space in between our rows of racks. Not that I minded too much 8 feet was a lot of area to stretch out, put tables and shelves in for storage etc. With ~30 foot ceilings it's one of the few data centers I've been to that didn't make me feel claustrophobic.

My strategy for the company at the time was to make their main product as good as possible and then partner with someone like linkedin to leverage their network with Jobster's technology. Jobster's main product was never visible to the public it was a subscription only product, so most probably never saw the technology that the company was built on (initially at least). Their strategy was OOH, SHINY! rinse and repeat (at least 3 times while I was there).

Filed under: News Comments Off

HP Launches new denser SL series

TechOps Guy: Nate

[domain name transfer still in progress but at least for now I managed to update the name servers to point to mine so the blog is being directed to the right server now]

Getting closer! Not quite there yet though.

Earlier in the year I was looking at the HP SL6000 series of systems for a project that needed high efficiency and density.

The biggest drawback to the system in my opinion is it wasn't dense enough, it was no denser than 1U servers for the configuration I was looking at (needing 4x3.5" drives per system). It was more power efficient though, and hardware serviceability was better.

The limitation was in the chassis, and HP acknowledged this at the time saying they were working on a new and improved version but it wasn't available at the time. Well looks like they have launched it today, in the form of the SL6500. It seems to deliver(on the statements HP gave to me earlier in the year), I don't see much info on the chassis itself on their site but looks significantly more dense, with the key here being the chassis is a lot deeper than the original 2U.

But they still have a ways to go, as far as I know the SGI Cloudrack C2 is the density leader in this space, at least from material that is publically available, who knows what the likes of IBM/Dell/HP come up with behind the scenes for special customers.

I did, what was to me a pretty neat comparison earlier this year comparing the power efficiency of the Cloudrack against the 3PAR T-class storage enclosures (granted the density technology behind the 3PAR is 8 years old at this point they haven't felt the need to go more dense, though HP may encourage them to since they waste up to 10U of space in each of the racks but weight and power can become issues in many facilities going even as dense as 3PAR can go).

Anyways, onto the comparison, this is one place where the picture tells the story, pretty crazy huh? Yeah I know the products are aimed at very diffierent markets, I just thought it was a pretty crazy comparison.

You can think of the Cloudrack as one giant chassis. The rack is the chassis(literally). So while HP has gone from a 2U chassis to a 4U chassis, SGI is waiting for them with a 38U chassis. Another nice advantage of the Cloudrack is you can get true N+1 power (3 diverse power sources), most systems can only support two power sources, the Cloudrack can go much, MUCH higher. And with the power supplies built into the chassis, the servers can benefit from that extra fault tolerance and high efficiency(no fans or power supplies in the servers! Same as the HP SL series)

Filed under: Datacenter, News Comments Off

Bye Bye 3PAR, Hello HP!

TechOps Guy: Nate

Wow that was fast! HP completed it's purchase of 3PAR this morning.

HP today announced that it has completed the acquisition of 3PAR Inc., a leading global provider of utility storage, for a price of $33 per share in cash, or an enterprise value of $2.35 billion.

3PAR technologies expand HP’s storage portfolio into enterprise-class public and private cloud computing environments, which are key growth markets for HP. Complementary with HP’s current storage portfolio, 3PAR brings market-differentiating technology to HP that will enable clients to maximize storage utilization, balance workloads and automate storage tiering. This allows clients to improve productivity and more efficiently operate their storage networks.

With a worldwide sales and channel network, coupled with extensive service operations, HP is uniquely positioned to rapidly expand 3PAR’s market opportunity. As part of the HP Converged Infrastructure portfolio, which integrates servers, storage, networking and management technologies, 3PAR solutions will further strengthen HP’s ability to simplify data center environments for clients.

Further details on product integration will be announced at a later date.

Certainly not messing around!

Tagged as: , , Comments Off

No more Cranky Geeks?

TechOps Guy: Nate

What!! I just noticed that it seems the only online video feed I watch, Cranky Geeks seems to be coming to an end? That sucks! I didn't stumble upon the series until about one and a half years ago on my Tivo. Been an big fan ever since. I rarely learned anything from the shows but I did like observing the conversations, it's not quite to the technical depth that I get into but it's a far cry from the typical "tech tv" videos/shows that don't seem to go beyond things like over clocking and what motherboard and video card to use for the latest games.

I know I'm a hell of a lot more cranky than anyone I ever saw on the show but they did bitch about some things. There seems to have been quite a few video blogs, for a lack of a better word, that have bitten the dust in recent months, I guess the economy is taking it's toll.

[Begin Another Tangent --]

I believe that we are entering the second phase of the great depression (how long until we are solidly in the second phase I'm not sure, won't know until we're there), the phase where states realize their budget shortfalls are too big for short term budget gimmicks and make drastic cuts and tax hikes which further damages the economy. I don't blame anyone in particular for our situation it's a situation that has been festering for more than thirty years, it's like trying to stop an avalanche with I don't know a snow plow?

This is what happens when you give people every incentive possible to pull demand forward, you run out of gimmicks to pull demand forward and are faced with a very large chasm that will only be healed with time, just look at Japan.

I have seen lots of folks say that this is not as bad as the real Great Depression, but they aren't taking into account the massive amount of social safety nets that have been deployed over the past 40-50+ years, I just saw a news report last night that said the rate of poverty among children is the same as it was in the 1960s. And to think the cost of living in the U.S. is so high that living in poverty here in many countries if you got paid that you'd be in the upper middle class.

Not sustainable, and as time goes on more and more people are realizing this, unfortunately too late for many they will be left behind, permanently.

My suggestion? Read the infrastructure report card. Yes I know infrastructure spending is not a short term stimulus, we need to take advantage of lower prices for wages, and materials, and rebuild the country, it will take years, maybe even a couple of decades but we need it. Long term problems call for long term solutions.

[End Another Tangent --]

I hope it doesn't go but it looks like it's essentially gone, and I just added the link to the blog roll a few days ago!

Noticed this from John in the comments -

The two companies couldn’t come to any agreement. This is a problem when you personally do not own the show. The fact is the show is not what advertising agencies want. They want two minute shows with a 15 second pre-roll ad at the beginning. They see no market for anything with a long format unless it is on network TV.

The irony is that the demographics for the show should be at $100/per k levels if they understood anything at all.

It’s amazing that we managed to get 4 1/2 years out of the show.



RIP Cranky Geeks, I shall miss you greatly.

Tagged as: Comments Off