Getting a bit drunk on enterprise dollars.

It's official - Gartner's Thomas Bittman thinks VMware might be drunk.

Denise Dubie has a great article on virtualization management capabilities and a great quote describing the upcoming battle between VMware and Microsoft for the x86 virtualization market, he is quoted as saying:

"The enterprise is going to be very leery of Microsoft, but the on-ramp to VMware is a bit steep for small businesses. VMware doesn't want to lose that potential business, but the company was getting a bit drunk on enterprise dollars," says Thomas Bittman, Gartner vice president and distinguished analyst.

The abundance of enterprise dollars spent on virtualization is because virtualization fixes so many problems, reduces power, reduces physical requirements, makes x86 hardware more efficient, increases uptime, allows resource management at the OS workload level, etc.

One of my favorite reports is one that IDC did in 2006 – it depicted– IT investment to be higher spending in Year 1 on a VMware / virtualization project but that in Year 2 and Year 3 and possibly Year 4 – IT departments would avoid spending on server hardware – you would just fill up the empty capacity of the system you built in 2006.

It looked good on paper, spend more now, avoid spending later.

Unfortunately, multiple issues caused IT departments to run out of capacity, VM sprawl occurred, single core and dual core servers could not hold as many VMs as the equivalent quad core servers.

Often P2V migrations went unchecked, servers have excess CPU capacity but not enough Memory, VMs are consuming too many resources and as a result enterprises are oversizing virtualization projects or not driving up VM density to get the biggest bang from their investment.

Avoid the hangover from “getting a bit drunk” and having to purchase new hardware, more memory, bigger servers, etc. by getting a resource management tool in place and understanding what resources your VMs are using and where you have capacity in virtualized environments.

Run OS Run...

The above title is a horrible attempt at a Forest Gump reference.

Either way - Novell announces support for VMWare's Virtual Machine Interface - which allows Suse Linux Enterprise Server (SLES) 10 Service Pack 2's kernel to have "increased performance and better interoperability".

Think of it this way - there will be preferred OS's to virtualize - Novell says you can virtualize Suse since its just x86 virtualization but if you do SLES 10SP2 - you can get VMI support.

In the paravirtualization space - this will mean increased density, better running VM's, better running ESX hosts, etc.

In other words - "the guest operating system is modified to work more closely with the underlying hardware and not just with the virtualized environment."

And its a brand differentiator as well - the OS and the VM platform both have to be tuned or tunable - and with VMI - VMWare is saying they are prepared to virtualize an OS like SUSE better than its competitors.

Killer VMs on the loose.

I have been hearing more and more about the "Killer VMs" - think of this as the lovechild of Virtualization and the industry term "Killer Apps".

Wikipedia has it as "is an application so compelling that someone will buy the hardware or software components necessary to run it."

Basically virtualize the "Killer App" and its becomes the "Killer VM".

This will make you visible, put your name on the map, your CIO will thank you, your CEO may even wave at you once in recognition of your stellar service.

Don't get cocky. These "Killer Apps" often have problems and are the application server than when it hiccups, goes down, causing your business pain and makes everyone go "We really should do something about that" but no one does.

Alot of IT departments may spend a ton of money on Active/Passive Clustering, etc - remember its a favorite child, it may get all sorts of resources, dollars spent on it, trying to ensure application SLA's or increase its uptime and performance.

I have seen new servers, more memory, faster CPUs, even SAN's purchased to manage "Killer Apps".

I have seen investment in high-end clustering, low-end clustering for "Killer Apps" - when at the end of the day the "Killer App" may just be a Windows 2000 Server running SQL with an application that is mission critical.

Enter VMWare, enter DRS, VMotion, HA, etc and you pick up some amazing tools to manage these "Killer Apps" and they become "Killer VMs".

Mark Brunnel writes about putting Navision and SQL 2005 into a VM, and had the v-piphany (virtualization epiphany):

"It is amazing to see VMWare running and the management and failover capabilities. For me it means the end of active passive clusters."

Mark's done some rough benchmarking and found that "VMWare is just slightly slower in posting but only 5% maximum." that's compared to the application running in Windows.

More and more people are going past their initial P2V consolidation effors, alot more are building VMs without every having a physical server, and now folks are optimizing environments, in this 2nd or 3rd phase of Virtualization, the Killer VMs are going to start showing up - these VMs will be more important than some others, require more attention, more care and feeding and better capacity management of the resources.

No CIO is going to like to hear that a VM used by 10 people took down a "Killer VM".

No CIO wants to hear that you could have prevented it but hadn't rolled out Resource Pools yet or don't have the right systems management tools to manage/monitor resource utilization.

Water Cooled Doors / Datacenters

On the third day of cloudy skies and rain, I figured it appropriate to talk about water and how its returning to provide cooling in datacenters.

With virtualization usage increasing, server capacity increasing, and VM densities on the rise (100:1 anyone?).

The amount of power in a rack is going to change from 7-8 kW to 30+ kW or more.

That's 300-400% increases in power densities inside a single rack. This explains the increase in discussion/slides on water cooling data centers - IBM has a Rear Door Heat eXchanger and I saw Vette Corp talking about their Rear Door Heat Exchanger.

I also think more folks start looking at Deep Lake Water Cooling (DLWC), which they are doing up at Cornell and Ontario.

Metro Hall, a 27 story office building in Toronto, went online with Enwave's Deep Lake Water Cooling system in June 2006. Energy consumption at Metro Hall will be reduced by 3 million kilowatt-hours per year and reduce CO2 emissions by 732 tons annually - equivalent to taking 160 cars off the road.

I am heading to Kitchener, Ontario (near Toronto) for a VMUG and I am going to try and get over to Toronto and check it out - Enwave distributes steam and chilled water to over 140 buildings via a 40km underground pipe network that covers most of the city's downtown core.

What's great is that they don't take water out of the lake, they use the "coldness" of the deep lake water to chill water in a pipe loop - this is done with a drastically lower carbon footprint that using electricity from coal-fed power plants.

One Quad or Two?

One of the best resources on the Internet for VMWare implementation is the VMTN community forums - its top notch.

This week there was a discussion about budgets and performance (where finance always mixes it up with IT (that and Chargeback)).

The post asks about the value of two medium-speed (1.6 - 2 Ghz) QuadCore CPU's ($$) vs. one high speed (3.33 Ghz) QuadCore CPU ($$$$).

I liked William Bishop from Huntsville Hospital response - "You'll get better density on the dual socket". He prefers the "dual proc, quad core setup" and has been "adminning vmware from some of the first dual cores to the newest quad cores."

I wonder if he has done anything with 4 socket x QuadCores?

Density is important - it's going to help you drive down your per VM costs and generate better ROI on the dollars invested in a virtualization product.

Data Rich, Insight Poor

This phrase has been gaining popularity.

It's a phrase used by CIO's to describe their business systems, the mountains of data generated by business applications.

This mountain has spawned new phrases and software to address business intelligence, dashboards, scorecards, key performance indicator's, etc.

At the recent MIT CIO Symposium, I heard a CIO talk about how easy it was to become "data rich, insight poor", she was talking primarily about how their BI product didn't make them that smart because the data was all over the place - they were engaged in new initiatives (ETL, Data Warehousing) to try and be data rich, insight rich.

I think a significant tertiary effect of virtualization is the increase of information.

Before you had individual piles of data, but with a virtualized environment you have the system and performance data from the ESX Server, the Host OS, the SAN, the Network, etc.

VKernel's Capacity Bottleneck Analyzer is a "ESX Intelligence" product - its a KPI, Dasboard, to do the analytics and automation to lift and load data out of Virtual Center and start analyzing resource constraints.

This problem is going to get worse.

The density of virtual machines is going to increase, we are going to see more CPU's, more RAM and more VMs per physical boxes.

It's time to look at the mountain of data and start working on the insight.

It's all about the ratio.

VKernel is an advocate of running your hardware at high levels of capacity. I know that we would see record-breaking ratios of virtual machines to server hardware.

• A major worldwide financial services organization achieved a 12:1 consolidation ratio and increased its central processing unit utilization by 30 percent.
  
  
• An Indian petroleum refining and distribution company achieved a 17:1 consolidation ratio and expects to increase that to 30:1 with additional CPUs and RAM.
  
  
• One of Italy's largest banks improved its server utilization rates by 100 percent.
  
  
• A leading US faucet manufacturer saved $250,000 in hardware costs by reallocating existing units instead of purchasing new, achieving a 10:1 consolidation ratio.
  
  
• A South American energy company consolidated its servers by a 20:1 ratio.
  
  
• A federation of trade unions in Singapore consolidated its servers by 46:1, achieving a 26 percent savings.
  

Smallest is 10:1 and largest is 46:1.

It's all about the ratio.

VMs can cost more than physical servers -- really!!

I am amazed how many virtual environments I have now seen that are severely under utilizing the new hardware and are afraid to increase VM density. They buy expensive server hardware, loaded it with 16Gigs or more for $30K to $50K and are running just a handful of VMs on it. This is analogous to driving a perfectly good Ferrari without ever getting out of first gear!! Say your are running 8 Vms on a $50K hardware. Add the cost of SANs, etc and you can quickly see how the cost of each VM can actually be higher than the physical server it replaced. This of course begs the question why do people underutilize the hardware?

As far as I can tell there are several reasons. Some are just being utilization "ignorant" about their environment, but the majority is simply afraid to "push the metal" and increase utilization because of concerns about running into ESX performance problems or worth yet -- downtime. Since finding capacity bottlenecks using Virtual Center is not trivial and time consuming, and predicting future capacity bottlenecks requires fairly advanced mathematical analysis of all core 4 resource types , disk I/O etc, most Vmware Admins lack the time and experience to do this exercise. So they keep the Ferrari in first gear, keep driving blindfolded, and hope that vm sprawl does not catch up with them. With availability of tools like the Vkernel Capacity Bottleneck Analyzer
VMware admins will gain visibility into current and future capacity problems and steer clear of performance issues. It heps driving with lights on!! Tell us what you think www.vkernel.com

Want awesome performance in VMWARE ESX?

Of course! we all do! So let's talk about how to get there. To achieve stellar VMware ESX performance you have to remove ALL bottlenecks in your environment. Remember your performance will only be as fast as the slowest "link" in your performance equation.

Here is the list to start removing performance bottlenecks:

1. Make sure that all of your hosts have sufficient RAM and do not over commit on memory utilization. If your workloads force the host to start swapping memory to disk, kiss your performance goodbye. There is a huge difference in speed between accessing memory internally and doing physical i/o to disk!

2. Make sure your CPU is not over utilized. Checks the processor ready queue to see how long threads are waiting to run. Extended period of time indicates a problem. Also check the overall processor utilization over a week or a month. Keep the overall utilization under 80%

3. As you scale up your ESX environment you will run into disk i/o bottlenecks. Make sure that you places your most mission critical VMs on the fastest storage available to you

4. Understand the timing of your workloads to identify when or if constraints develop in memory utilization, cpu, and disk i/o. Spread out the workloads by changing when they run i.e timing or move workloads to different hosts to mix it up.