I just came across this mention on AMD’s blog. They note that vSphere 4.1 Update 2 included a CPU licensing change –
For the AMD Opteron 6200 and 4200 series (Family 15h) processors, ESX/ESXi 4.1 Update 2 treats each core within a compute unit as an independent core, except while applying licenses. For the purpose of licensing, ESX/ESXi treats each compute unit as a core. For example, a processor with 8 compute units can provide the processor equivalent of 16 cores on ESX/ESXi 4.1 Update 2. However, ESX/ESXi 4.1 Update 2 only requires an 8 core license for each 16-core processor.
I had not heard of that before, so it’s news to me! So not only is the physical cost of the Opteron 6200 cheaper than the 6100, the licensing cost is half as much (per core). AMD’s blog post above shows some pretty impressive results where a pair of quad socket 6200 blades outperforming a pair of quad socket 10-core Intel blades(2 sockets populated per blade) and at the same time the 6200 solution costs half as much (per VM). Though it’s also comparing vSphere 4.1 vs 5.0, since the Opteron 6200 results seem to be the first vSphere 5.0 VMmark results posted. Also the Intel solution has twice the ram as the Opteron but still loses out.
Based on what I see it seems VMmark is more CPU bound than memory(capacity bound), which I suppose I can understand but still in the vast majority of situations the systems are not CPU bound. People tend to load up more on CPUs so they can get more memory capacity. I won’t have real numbers for probably two months but I’m expecting CPU usage on this new cluster I am building to be at least half the amount of memory usage.
The change sounds Oracle-esque in licensing where they have fairly complicated decisions they made to determine how many “Oracle cores” you have on your physical processor.
I am traveling tonight to Atlanta to deploy a new vSphere cluster with Opteron 6100s, I was going to go with vSphere 5 because of the license limits on vSphere 4.1 not supporting 16 core processors. Now I see 4.1 does support it so I have about 48 hours to think about whether or not I want to change my mind. I do like vSphere 5’s inclusion of LLDP support, more vCPUs per VM. Though really even now after I have been looking through what is in vSphere 5 I don’t see anything game changing, nothing remotely, in my opinion like the change to vSphere 4.0 from ESX 3.5.
Weigh the benefits of what’s new in vSphere 5 vs having the ability to have unlimited memory(well, up to 1TB, which for me is unlimited from a practical standpoint) in my hosts for no additional licensing cost…
I’m already licensed for vSphere 5 since we bought it after the deadline of the end of September.
Mad props to AMD for getting VMware to tweak their licensing.
Decisions, decisions..
Would be curious to see if there is any big difference in running both comparisons on the same version of vMark. One uses 2.1 the others using 2.1.1
Release notes really dont clarify if the fixes would affect testing or not: https://www.vmware.com/support/vmmark/doc/releasenotes_vmmark211.html
I would suggest reading this: http://www.vladan.fr/esxi-5-0-can-handle-33-more-vms/
regarding the differences in scores as they result from the number of tiles when going from 4.1 to 5.0
” The results show that, in an equivalent environment, vSphere 5 handled greater load than ESXi 4.1 before reaching saturation, and showed increased performance at lower levels of load as well. At saturation, vSphere 5 showed a 22% increase in overall VMmark 2 scores over ESXi 4.1. In this cluster, vSphere 5 supported 33% more VMs and twice the number of infrastructure operations while meeting Quality of Service requirements.”
Comment by gchapman — December 15, 2011 @ 12:07 pm
VMwares tests: http://blogs.vmware.com/performance/2011/11/comparing-esxi-41-and-esxi-50-scaling-performance.html
apples and apples is always best.
Comment by gchapman — December 15, 2011 @ 12:11 pm
I agree Apples to apples is best! Interesting link, the difference between 4.1 and 5.0 is a lot smaller than I might of expected. Though the CPU and memory in that test seemed to be fairly weak by comparison (4 cores/32GB per server — I was using 8 cores/32GB on ESX 3.5)
Comment by Nate — December 15, 2011 @ 12:16 pm
Something I would be curious to see is a pair of quad socket 64-core 6200 systems with 512GB/memory a piece on vSphere 4.1 (I emailed my vmware rep to confirm but I believe this new licensing means each system would need only 3 licenses of enterprise+ vs 4 before). Compared to both a comparable Intel box as well as another AMD box running both vSphere 4.1 and 5.0. Cost + performance metrics.
The newer Opterons, like the Xeons can go to 32GB DIMMs now so you can go beyond the original 512GB limit(on the blades), though looking at the 32GB pricing at over $2,000/chip I think the bang for the buck probably isn’t there yet. Though it seems crazy that you can get 1TB of ram for about $65,000, to me it seems like yesterday when a system with say 128GB or more would cost around a quarter million.
I’m thinkin the 64-core 4.1 box would really be the way to go 🙂
Comment by Nate — December 15, 2011 @ 12:25 pm
I’m 99% sure I’m going to use vSphere 4.1 U2 for my new cluster. Part is the lack of vRAM tax, another part is I really do like ESX more than ESXi, and so I believe I’m going to go with ESX 4.1U2 to get my big console back, install the stand alone HP management agents and go from there.
I believe my next major hypervisor upgrade will be to RHEV so I am not concerned about compatibility going from ESX to ESXi in the future (I’ve switched between ESX and ESXi in the past w/o issues, I just like having the full shell with all of the commands rather than a stripped down interface).
Comment by Nate — December 15, 2011 @ 1:24 pm
You will always need a license for each physical socket. Even with the new licensing scheme.
So far none of my workloads have ever really been CPU constrained. I think it would really depend on what you are putting on the box to determine the CPU/Core count. I can run 45 VM’s on a Dual Socket 16 Core IBM x3690 with 256GB of RAM. Those are mostly run of the mill type systems, but there is a pretty good mix of Share Point, Oracle Apps, Hyperion, Windows File, Print, AD, SQL, etc.
The general rule of thumb for me has been Memory, IO, then CPU in that order when it comes to constraints in virtual systems.
As for ESX vs ESXi, I’m in the latter camp. ESXi is lightweight and easier to manage and I dont have to worry about patching it every month. Support will eventually run its course for the ESX platform. Also, setting up a VMA box to do all the work of the service console seems like an easier fit for me.
Speaking of RHEV: http://www.redhat.com/promo/rhev3/?sc_cid=70160000000U4J5AAK&offer_id=70160000000Ty5NAAS&elq=8502461239d1403092b1758e28d08e2c
Beta 3 is available for DL right now.
Comment by gchapman — December 15, 2011 @ 2:56 pm
One thing I just caught on that blog was that AMD appears to be under the misconception that vSphere5 is not licensed by the socket any longer. That’s not the case, entitlements are based on each socket and a vRAM correlation to the level of vSphere that is purchased, Standard, Enterprise, Enterprise +
Comment by gchapman — December 16, 2011 @ 10:28 am
maybe they mean unlimited cores per socket in vsphere 5?
Comment by Nate — December 16, 2011 @ 7:28 pm
Could be, but the wording isn’t very clear. Funny, I’ve left comments there but 3 days later and they are still not approved.
Comment by gchapman — December 17, 2011 @ 6:58 pm
Somewhat unrelated, found this today: http://www.demartek.com/Demartek_Benchmark_Output_File_Formats.html
looks like IOmeter 2008 is utterly worthless now as a tool for bench marking.
Comment by gchapman — December 20, 2011 @ 9:48 am