TechOpsGuys.com Diggin' technology every day

27May/11Off

MS cashing in on Android

TechOps Guy: Nate

Oh how ironic, to me at least. Just came across this on Tech Flash.

Based on an estimate that HTC has shipped 30 million Android devices, Asymco calculates that Microsoft has seen $150 million in revenue from Android. With Microsoft selling 2 million Windows Phones licenses, its Windows Phone revenue comes in at $30 million.

Microsoft making more money off of Android than it is on it's own cutting edge mobile platform..

Tagged as: Comments Off
25May/11Off

RIP: Mark Haines

TechOps Guy: Nate

Not tech related, but a sad day for me, a great guy Mark Haines who was an anchor on CNBC for decades died recently at the age of 65. I have been watching his show for at least the past 5 years now and he really was my favorite guy, always honest, never afraid to confront someone on a topic, and never afraid to speak his own mind. He was with CNBC since the day they launched in 1989.

Mark Haines correctly called the top of the Nasdaq in 2000, also correctly called the bottom of the markets (known as Haines' bottom at the time) in 2008. With that in mind he called the recent tech IPOs (especially LinkedIn) a bubble (regardless of whether of whether or not the environment is different from the dot com days).

This sums him up pretty good in my eyes, from the words of Bob Pisani - "How do I feel about you as a person, do you make sense to me, does your argument make sense to me, if it doesn't make sense to me I'm not going to have that much respect for you - I don't care what your title is, I don't care what your position is. I don't care if your a famous economist, I don't care if your a world leader or not. If it makes sense to me, and I think you have a point to make, I'm going to give you the time and respect your opinion - if it doesn't - I'm going to come back at you".

There's a tribute show for him today at 4PM PDT on CNBC.

He was an awesome person, and will be greatly missed by me.

 

Filed under: General Comments Off
20May/11Off

LinkedIn IPO could ruin tech sector?

TechOps Guy: Nate

I've been seeing an increasing number of people (some of whom I at least know of and respect) saying how bad the LinkedIn IPO was yesterday.

A recent one I just came across is from John Dvorak (damn I miss Cranky Geeks), who has a column on the Wall Street Journal site saying how the LinkedIn IPO could ruin the tech sector.

Myself I don't agree that the IPO itself could ruin the tech sector, I think it's just another part of the frenzy in social media, LinkedIn is of course seen as a gateway into one of the leaders in the space and there has been so much hype being built up over the years. It's just a sign as to how rabid some of these people are(the fact that there is a whole second market for this kind of stuff that has opened up is far more concerning to me than the IPO). Whether or not they IPO'd wouldn't of changed that fact.

I just think back to my days at Jobster(closed up shop about two years ago) when they were running rampant on the social media stuff, I couldn't believe my eyes or ears. I knew the days were numbered when the management of the company wouldn't let us remove bad email addresses from our databases (the bad email addresses were causing us to get blacklisted, hampering abilities to do the amount of email traffic to users that we were doing).

We couldn't delete them because it would hurt our user count. I mean oh my god, are you kidding me? These users are not there anymore! Maybe they never were there! They are actually impacting other users by having their emails bounce! I suppose another approach we could of taken was somehow flag the accounts to not email them, but nobody seemed to come up with that idea at the time.

Here is a good video on some of the hot IPOs in the past year (many from China), and how poorly they have done since they debuted. I tell ya, the more I read and learn about stocks and investing the less interested I become in ever participating in it.

Maybe I'll get lucky and the world will end tomorrow and I won't have to worry about my home grown retirement plan :)

Tagged as: 1 Comment
19May/11Off

$76/subscriber for LinkedIn IPO

TechOps Guy: Nate

Has been kind of interesting to watch this hot IPO unfold on CNBC this morning, they are reporting that the current valuation of $8 billion prices their subscribers at about $76 a piece.

LinkedIn IPO

The LinkedIn CEO was so very careful not to wade into answering what he thinks the valuation of the company should be, trying to not be associated with the dot com bubble.

Now trading at around $88 per share..

I can see the rest of the social media bubble blowers huffing and puffing as fast as they can, not that it will do a whole lot of good for companies that aren't the leaders in the space, something the CEO of LinkedIn hinted at during an interview just after the IPO.

The pop in price certainly exceeded my wildest expectations.

I believe this is another sign that our economic cycle is nearing a peak, before it starts to decline again. Combine this with the uptrend in unemployment and "double dip" in housing along with other factors..

17May/11Off

LinkedIn IPO Pending

TechOps Guy: Nate

Not too much to say here, it seems LinkedIn will IPO in a couple of days with a share price north of $40/share, which to me seems kind of high, normally the IPOs I see have initial offerings at a much lower price, and they just offer more shares at the lower price (usually see mid teens or so).

I've never been a fan of social media (you won't find me on Facebook, Twitter or pretty much anything else), but LinkedIn  is one site I have gotten a ton of value out of.

I remember back when I worked for another social media company wannabe Jobster (long defunct) a few years ago they used to try to get the employees to invite their friends to join the social network, I never participated, there was no reason for anyone to join. I remember thinking to myself, I have more people in my LinkedIn network then Jobster has in it's entire community.

Funny screenshot from my LinkedIn profile from several years ago

 
I'm terrible at keeping in touch with people, and LinkedIn basically acts as an address book for me, I don't get overloaded with spam from their constant updates (I do wish I could turn off receiving twitter messages on linkedin though), don't have to be bothered with people's pictures or whatever, just basic contact information, no fuss, no obnoxious crap that you find on most other social media sites, very clean.
 
 
 
So, best wishes LinkedIn for your IPO, hope it goes well, you provide a good service, to some extent a service that Jobster had hoped to capitalize on several years ago but was never able to.

I won't be buying any of their stock, well, because I don't buy stock. period.

16May/11Off

MySQL Scalability with flash

TechOps Guy: Nate

Just a quick post, came across this on the MySQL Performance blog and thought it was a really well written paper. Talks about vertical scaling in the most current versions of MySQL, what the major bottlenecks are when scaling with more CPU cores, and how to extract the highest amount of I/O out of today's modern server hardware.

What I'd like to see just for comparison purposes is running the latest & greatest MySQL, vertically scale it to 48 cores, and compare it against Oracle Standard Edition on the same 48 cores. As far as I know the Oracle license agreement forbid publishing performance numbers so I'll probably never see this but it is a curiosity of mine, because sharding a database can make application development significantly more complex.

It is nice though that the latest versions of MySQL can scale beyond four cores.

Tagged as: Comments Off
11May/11Off

2000+ 10GbE ports in a single rack

TechOps Guy: Nate

The best word I can come up with when I saw this was

oof

What I'm talking about is the announcement of the Black Diamond X-Series from my favorite switching company Extreme Networks. I have been hearing a lot about other switching companies coming out with new next gen 10 GbE and 40GbE switches, more than one using Broadcom chips (which Extreme uses as well), so have been patiently awaiting their announcements.

I don't have a lot to say so I'll let the specs do the talking

Extreme Networks Black Diamond X-Series

 

  • 14.5 U
  • 20 Tbps switching fabric (up ~4x from previous models)
  • 1.2 Tbps fabric per line slot (up ~10x from previous models)
  • 2,304 line rate 10GbE ports per rack (5 watts per port) (768 line rate per chassis)
  • 576 line rate 40GbE ports per rack (192 line rate per chassis)
  • Built in support to switch up to 128,000 virtual machines using their VEPA/ Direct Attach system

 

 

 

This was fascinating to me:

Ultra high scalability is enabled by an industry-leading fabric design with an orthogonal direct mating system between I/O modules and fabric modules, which eliminates the performance bottleneck of pure backplane or midplane designs.

I was expecting their next gen platform to be a mid plane design (like that of the Black Diamond 20808), their previous 10GbE high density Enterprise switch Black Diamond 8800, by contrast was a backplane design (originally released about six years ago). The physical resemblance to the Arista networks chassis switches is remarkable. I would like to see how this direct mating system looks in a diagram of some kind to get a better idea on what this new design is.

Mini RJ21 adapters, 1 plug on the switch, goes to 6x1GbE ports

To put that port density in to some perspective, their older system (Black Diamond 8800), by comparison, has an option to use Mini RJ21 adapters to achieve 768 1GbE ports in a chassis (14U), so an extra inch of space gets you the same number of ports running at 10 times the speed, and line rate (the 768x1GbE is not quite to line rate but still damn fast). It's the only way to fit so many copper ports in such a small space.

 
 
 

It seems they have phased out the Black Diamond 10808 (I deployed a pair of these several years ago first released 2003), the Black Diamond 12804C (first released about 2007), the Black Diamond 12804R (also released around 2007) and the Black Diamond 20808 (this one is kind of surprising given how recent it was though didn't have anything approaching this level of performance of course, I think it was released in around 2009). They also finally seemed to drop the really ancient Alpine series (10+ year old technology) as well.

Also they seem to have announced a new high density stackable 10GbE switch the Summit X670, the successor to the X650 which was already an outstanding product offering several features that until recently nobody else in the market was providing.

Extreme Networks Summit X670

  • 1U
  • 1.28 Tbps switching fabric (roughly double that of the X650)
  • 48 x 10Gbps line rate standard (64 x 10Gbps max)
  • 4 x 40Gbps line rate (or 16 x 10Gbps)
  • Long distance stacking support (up to 40 kilometers)

The X670 from purely a port configuration standpoint looks similar to some of other recently announced products from other companies, like Arista and Force10, both of whom are using the Broadcom Trident+ chipset, I assume Extreme is using the same. These days given so many manufacturers are using the same type of hardware you have to differentiate yourself in the software, which is really what drives me to Extreme more than anything else, their Linux-based easy-to-use Extremeware XOS operating system.

Neither of these products appear to be shipping, not sure when they might ship, maybe sometime in Q3 or something.

40GbE has taken longer than I expected to finalize, they were one of the first to demonstrate 40GbE at Interop Las Vegas last year, but the parts have yet to ship (or if they have the web site is not updated).

For the most part, the number of companies that are able to drive even 10% of the performance of these new lines of networking products is really tiny. But the peace of mind that comes with everything being line rate, really is worth something !

x86 or ASIC? I'm sure performance boosts like the ones offered here pretty much guarantees that x86 (or any general purpose CPU for that matter) will not be driving high speed networking for a very long time to come.

Myself I am not yet sold on this emerging trend in the networking industry that is trying to drive everything to be massive layer 2 domains. I still love me some ESRP! I think part of it has to do with selling the public on getting rid of STP. I haven't used STP in 7+ years so not using any form of STP is nothing new for me!

8May/11Off

Social Shopping – weapon against small businesses

TechOps Guy: Nate

I'm not sure if Social Shopping is the right term or not, but what I am referring to is the Groupons of the world as well as those cell phone apps that help you hunt down specific deals at retailers.

I have been thinking about this off and on for quite a while, and it really is a troubling trend for me, what sparked my interest today was this article I saw on slashdot.

My concept around Groupon is kind of interesting in that you get a group of people a discount or special offer of sorts at your business. The businesses offering the discounts expect, in return to get repeat business from many of those consumers. However it seems in reality that is rarely what happens. The business becomes a victim of their own deal, and most of those consumers never return (unless they get another screamin' deal). So the business takes a big hit on the front end, and gets little or nothing in return.

Then there are the businesses that offer something that they cannot hope to possibly fulfill, such as the article about the photographer article above. I saw another news story last year where about a dance studio having similar issues, as well as a house cleaning service. This problem is probably addressable by better educating the business that is posting the offer to the site(s).

The other really troubling shopping trend are those cell phone apps that allow you to do stuff like scan the barcode of the item you want to buy and it will look at other shops in the area to try to find a better price.

On the surface it sounds like something good for the consumer, but in the longer term I believe strongly it will cause significant harm to the small businesses who can afford only so much to offer as loss leaders to bring people into their stores. Again there is some expectation that there will be return business, and I believe in this situation the likelihood of that happening is higher than that of Groupon but not nearly enough to make up for the losses incurred by the good deal they were offering.

I'm not sure what will happen with the group coupon sites, whether it is something like small businesses realize what is happening and stop participating in the sites altogether, or maybe they feel so much pressure from everyone else that they feel compelled to participate for some mutually assured destruction. Or maybe something entirely different.

As for the cell phone apps, it'll be a lot harder to deal with those, it could be that businesses are forced to stop doing loss leaders, or try to impose rules around taking pictures in the store(hard to enforce), I'm not sure.

It seems like a scary time to be a small business in this economy, with everything that is going on in the macro economy, having technology exploit your business even more is quite sad.

5May/11Off

Sony Compromised by Apache bug?

TechOps Guy: Nate

Came across an article from a friend that talks about how Sony thinks they were compromised.

According to Spafford, security experts monitoring open Internet forums learned months ago that Sony was using outdated versions of the Apache Web server software, which "was unpatched and had no firewall installed."

The firewall part is what gets me. Assuming of course this web server(s) were meant to be public, no firewall is going to protect you against this sort of thing since of course firewalls protecting public web servers have holes opened explicitly for the web server so all traffic is passed right through.

And I highly doubt those Apache web servers had confidential data as the article implies, obviously that data was on back end systems running databases of some sort.

Then there are people out there spouting stuff on PCI saying the automated external scans should of detected they were running outdated versions of software. In my experience such scans are really not worth much with Linux, primarily because they have no way to take into account patches that are back ported to the operating system. I've had a few arguments with security scanners trying to explain how a system is patched because the fix was back ported but them not being able to comprehend that because the major/minor version being reported by the server has not changed.

Then there was the company I worked for who had a web app that returned a HTTP/200 for pretty much everything, including things like 404s. This tripped every single alarm the scanners had, and they went nuts. And once again we had to explain that those windows exploits aren't going to work against our Apache Tomcat systems running Linux.

IDS and IPS are overrated as well, unless you really have the staff to watch and manage it full time. In all of the years I have worked at companies that deployed some sort of IDS (never IPS), I have seen it work, one time, back in I want to say 2002, I saw a dramatic upsurge in some type of traffic on our Snort IDS at the time from one particular host and turns out it had a virus on it. I worked at one company that was compromised at LEAST twice while I was there(on systems that weren't being properly managed). and of course the IDS never detected a thing. Then that company deployed(after I left) a higher end hardware-based IPS, and when they put it inline to the network (in passive, not enforcing mode) for some reason the IPS started dropping all SSL traffic for no reason.

They aren't completely useless though, they can help detect and sometimes protect against the more obvious types of attacks (SQL injection etc).  But in the grand scheme of things, especially when dealing with customized applications (not off the shelf like Exchange, Oracle or whatever), IDS/IPS and even firewalls provide only a tiny layer of additional security on top of good application design, good deployment practices(e.g. don't run as root, disable or remove subsystems that are not used, such as the management app in Tomcat, use encryption where possible), and a good authentication system for system level access (e.g. ssh keys). With regards to web applications, a good load balancer is more than adequate to protect the vast majority of applications out there, it is "firewall like" as in it only passes certain ports to the back end systems, but (for higher traffic sites this is important) vastly outperforms firewalls, which can be a massive bottleneck for front end systems.

With regards to the company that was compromised at least twice, the intrusion was minor and limited to a single system, the compromise occurred because the engineer who installed the system put it outside of the load balancers, it was a FTP server, or was it a monitoring server, I forgot.  Because it needed to be accessed externally the engineer thought hey let's just put it on the internet. Well it sat there for a good year or two, (never being patched in the meantime) before I joined the company, compromised in some fashion, and ssh was replaced with a trojaned copy (it was pretty obvious, I am assuming it was some sort of worm exploiting ssh). It had all sorts of services running on it. I removed the trojan'd ssh, asked the engineer if he thought there might be an issue, he said he didn't believe so. So I left it, until a few weeks later that trojan'd ssh came back. And at that point I shut the ethernet interfaces on the box off until it could be retired. There was no technical reason that it could not run behind the load balancer.

If you really need a front end firewall, consider a load balancer that has such functionality built in, because at least you have the ability to decrypt incoming SSL traffic and examine it, something very few firewall or IDS/IPS systems can do (another approach some people use is to decrypt at the load balancer than mirror the decrypted traffic to the IDS/IPS, but that is less secure of course).

It really does kind of scare me though that people seem to blindly associate a firewall with security, especially when it's a web server that is running. Now if those web servers were running RPC services and were hacked that way, a firewall very likely could of helped.

One company I worked at, my boss insisted we have firewalls in front of our load balancers, I couldn't convince him otherwise, so we deployed them. And they worked fine(for the most part). But the configuration wasn't really useful at all, basically we had a hole open in the firewall that pointed to the load balancer, which then pointed to the back end systems. So the firewall wasn't protecting anything that the load balancer wasn't doing already, a needless layer of complexity that didn't benefit anyone.

Myself I'm not convinced they were compromised via an Apache web server exploit, maybe they were compromised via an application running on top of Apache, but these days it's really rare to break into any web server directly via the web server software(whether it's Apache, IIS or whatever). I suspect they still don't really know how they were compromised and some manager at Sony pointed to that outdated software as the cause just so they could complete their internal processes on root cause and move on. Find something to tell congress, anything that sounds reasonable!!

Tagged as: , , Comments Off
4May/11Off

Microsoft Server Designs

TechOps Guy: Nate

I was out of town for most of last week so didn't happen to catch this bit of news that came out.

It seems shortly after Facebook released their server/data center designs Microsoft has done the same.

I have to admit when I first heard of the Facebook design I was interested, but once I saw the design I felt let down, I mean is that the best they could come up with? It seems there are market based solutions that are vastly superior to what Facebook designed themselves. Facebook did good by releasing in depth technical information but the reality is only a tiny number of organizations would ever think about attempting to replicate this kind of setup. So it's more for the press/geek factor than being something practical.

I attended a Datacenter Dynamics conference about a year ago, where the most interesting thing that I saw there was a talk by a Microsoft guy who spoke about their data center designs, and focused a lot on their new(ish) "IT PAC".  I was really blown away. Not much Microsoft does has blown me away but consider me blown away by this. It was (and still is) by far the most innovative data center design I have ever seen myself at least. Assuming it works of course, at the time the guy said there was still some kinks they were working out, and it wasn't on a wide scale deployment at all at that point. I've heard on the grape vine that Microsoft has been deploying them here and there in a couple facilities in the Seattle area. No idea how many though.

Anyways, back to the Microsoft server design, I commented last year on the concept of using rack level batteries and DC power distribution as another approach to server power requirements, rather than the approach that Google and some others have taken which involve server-based UPSs and server based power supplies (which seem much less efficient).

 

Google Server Design with server-based batteries and power supplies

Add to that rack-based cooling(or in Microsoft's case - container based cooling), ala SGI CloudRack C2/X2, and Microsoft's extremely innovative IT PAC containers, and you got yourself a really bad ass data center. Microsoft seems to borrow heavily from the CloudRack design, enhancing it even further. The biggest update would be the power system with the rack level UPS and 480V distribution.  I don't know of any commercial co-location data centers that offer 480V to the cabinets, but when your building your own facilities you can go to the ends of the earth to improve efficiency.

Microsoft's design permits up to 96 dual socket servers(2 per rack unit) each with 8 memory slots in a single 57U rack (the super tall rack is due to the height of the container). This compares to the CloudRack C2 which fits 76 dual socket servers in a 42U rack (38U of it used for servers).

SGI Cloudrack C2 tray with 2 servers, 8 disks (note no power supplies or fans, those are provided at the rack level )

My only question on Microsoft's design is their mention of "top of rack switches". I've never been a fan of top of rack switches myself. I always have preferred to have switches in the middle of the rack, better for cable management (half of the cables go up, the other half go down). Especially when we are talking about 96 servers in one rack. Maybe it's just a term they are using to describe what kind of switches, though there is a diagram which shows the switches positioned at the top of the rack.

SGI CloudRack C2 with top of rack switches positioned in the middle of the rack

I am also curious on their power usage, which they say they aim to have 40-60 watts/server, which seems impossibly low for a dual socket system, so they likely have done some work to figure out optimal performance based on system load and probably never have the systems run at anywhere near peak capacity.

Having 96 servers consume only 16kW of power is incredibly impressive though.

I have to give mad, mad, absolutely insanely mad props to Microsoft. Something I've never done before.

Facebook - 180 servers in 7 racks (6 server racks + 1 UPS rack)

Microsoft - 630 servers in 7 racks

Density is critical to any large scale deployment, there are limits to how dense you can practically go before the costs are too high to justify it. Microsoft has gone about as far as is achievable given current technology to accomplish this.

Here is another link where Microsoft provides a couple of interesting PDFs, the first one I believe is written by the same guy that gave the Microsoft briefing at the conference I was at last year.

(As a side note I have removed Scott from the blog since he doesn't have time to contribute any more)