TechOpsGuys.com Diggin' technology every day

17Nov/09Off

Affordable 10GbE has arrived

TechOps Guy: Nate

10 Gigabit Ethernet has been around for many years, for much of that time it has been for the most part(and with most vendors still is) restricted to more expensive chassis switches. For most of these switches the port density available for 10GbE is quite low as well, often maxing out at less than 10 ports per slot.

Within the past year Extreme Networks launched their X650 series of 1U switches, which currently consists of 3 models:

  • 24-port 10GbE SFP+
  • 24-port 10GbaseT first generation
  • 24-port 10GbaseT second generation (added link to press release, I didn't even know they announced the product yesterday it's been available for a little while at least)

For those that aren't into networking too much, 10GbaseT is an ethernet standard that provides 10 Gigabit speeds over standard CAT5e/CAT6/CAT6a cable.

All three of them are line rate, full layer 3 capable, and even have high speed stacking(ranging from 40Gbps to 512Gbps depending on configuration). Really nobody else in the industry has this ability at this time at least among:

  • Brocade (Foundry Networks) - Layer 2 only (L3 coming at some point via software update), no stacking, no 10GbaseT
  • Force10 Networks - Layer 2 only, no stacking, no 10GbaseT
  • Juniper Networks - Layer 2 only, no stacking, no 10GbaseT. An interesting tidbit here is the Juniper 1U 10GbE switch is an OEM'd product, does not run their "JunOS" operating system, and will never have Layer 3 support. They will at some point I'm sure have a proper 10GbE switch but they don't at the moment.
  • Arista Networks - Partial Layer 3(more coming in software update at some point), no stacking, they do have 10GbaseT and offer a 48-port version of the switch.
  • Brocade 8000 - Layer 2 only, no stacking, no 10GbaseT (This is a FCoE switch but you can run 10GbE on it as well)
  • Cisco Nexus 5000 - Layer 2 only, no stacking, no 10GbaseT (This is a FCoE switch but you can run 10GbE on it as well)
  • Fulcrum Micro Monte Carlo - I had not heard of these guys until 30 seconds ago, found them just now. I'm not sure if this is a real product, it says reference design, I think you can get it but it seems targeted at OEMs rather than end users. Perhaps this is what Juniper OEMs for their stuff(The Fulcrum Monaco looks the same as the Juniper switch). Anyways they do have 10GbaseT, no mention of Layer 3 that I can find beyond basic IP routing, no stacking. Probably not something you want to use in your data center directlty due to it's reference design intentions.

The biggest complaints against 10GbaseT have been that it was late to market(first switches appeared somewhat recently), and it is more power hungry. Well fortunately for it the adoption rate of 10GbE has been pretty lackluster over the past few years with few deployments outside of really high end networks because the cost was too prohibitive.

As for the power usage, the earlier 10GbaseT switches did use more power because well it usually requires more power to drive stuff over copper vs fiber. But the second generation X650-24T from Extreme has lowered the power requirements by ~30%(reduction of 200W per switch), making it draw less power than the SFP+ version of the product! All models have an expansion slot on the rear for stacking and additional 10GbE ports. For example if you wanted all copper ports on the front but needed a few optical, you could get an expansion module for the back that provides 8x 10GbE SFP+ ports on the rear. Standard it comes with a module that has 4x1GbE SFP ports and 40Gbps stacking ports.

So what does it really cost? I poked around some sites trying to find some of the "better" fully layer 3 1U switches out there from various vendors to show how cost effective 10GbE can be, at least on a per-gigabit basis it is cheaper than 1GbE is today. This is street pricing, not list pricing, and not "back room" discount pricing. YMMV

VendorModelNumber of ports on the frontBandwidth
for front
ports
(Full Duplex)
Priced
From
Street
Price
Cost per
Gigabit
Support
Costs?
Extreme NetworksX650-24t24 x 10GbE480 GbpsCDW$19,755 *$41.16Yes
Force10 NetworksS50N48 x 1GbE 96 GbpsInsight$5,078$52.90Yes
Extreme NetworksX450e-48p48 x 1GbE 96 GbpsDell$5,479$57.07Optional
Extreme NetworksX450a-48t48 x 1GbE 96 GbpsDell$6,210$64.69Yes
Juniper NetworksEX420048 x 1GbE 96 GbpsCDW$8,323$86.69Yes
Brocade (Foundry Networks)NetIron CES 2048C48 x 1GbE 96 GbpsPendingPendingPendingYes
Cisco Systems3750E-48TD48 x 1GbE 96 GbpsCDW$13,500$140.63Yes

* The Extreme X650 switch by default does not include a power supply(it has two internal power supply bays for AC or DC PSUs). So the price includes the cost of a single AC power supply.

17Nov/09Off

HP VirtualConnect for Dummies

TechOps Guy: Nate

Don't know what VirtualConnect is? Check this e-book out. Available to the first 2,500 people that register. I just browsed over it myself it seems pretty good.

I am looking forward to using the technology sometime next year(trying to wait for the 12-core Opterons before getting another blade system). Certainly looks really nice on paper, and the price is quite good as well compared to the competition. It was first introduced I believe in 2006 so it's fairly mature technology.

17Nov/09Off

Three thousand drives in the palm of your hand

TechOps Guy: Nate

I was poking around again and came across a new product from Fusion IO which looked really cool. Their new Iodrive Octal, which packs 800,000 IOPS on a single card with 6 Gigabytes/second sustained bandwidth. To put this in perspective, a typical high end 15,000 RPM SAS/Fiber Channel disk drive can do about 250 IOPS. As far as I/O goes this is roughly the same as 3,200 drives. The densest high performance storage I know of is 3PAR who can pack 320 15,000 RPM drives in a rack in their S-class and T-class systems (others can do high density SATA,  I'm not personally aware of others that can do high density 15,000 RPM drives for online data processing).

But anyways, in 3PAR's case that is 10 racks of drives, and three more racks for disk controllers(24 controllers), roughly 25,000 pounds of equipment(performance wise) in the palm of your hand with the Iodrive Octal. Most other storage arrays top out at between 200 and 225 disks per rack.

The Fusion IO solutions aren't for everyone of course they are targeted mostly at specialized applications with smaller data sets that require massive amounts of I/O. Or those that are able to distribute their applications amongst several systems using their PCIe cards.