Shrek, Onions, & Virtualized Application Architectures

What do the three things in the title have in common? Answer - they all have “layers”. If you do not have kids you may not know that Shrek, the ogre in the movie by the same name, has (contrary to popular opinion about ogres) layers to his personality. Onions have layers, and so do applications deployed via application virtualization technologies like Citrix, VMware, Microsoft Virtual Server, and Softricity. Why is this interesting or relevant to the topic of Applications Performance Management?

Because, when you use a virtualization technology to simplify some aspect of managing your applications, you at the minimum inject another layer or two of software into the mix. Let’s take VMware as a case in point. Before VMware you have the OS (Windows Server 200X) talking directly to the hardware. Applications talk more or less directly to the OS. So you have the hardware, the OS, and the applications to worry about.

Again using Windows as an example let’s look and see what happens when you deploy these applications in virtual machines hosted on VMware ESX. The OS that sits on top of the hardware is now the slimmed down version of Linux, upon which the VMware ESX host OS resides. Each VM consists of a VMware guest supporting a guest operating system. So now your layers consist of hardware, Linux, the VMware Host, and N instances of a VM Guest with the Guest OS and the applications. So you have gone from three layers to six. Do you feel as layered as an onion or an ogre yet? Or perhaps having to support all of this complexity makes you want to turn into an ogre so that you can deal with people who do not understand all of this the way that Shrek did (he could growl at people and they would all run away).

But seriously now, the move to virtualized applications architectures while laudable for reasons of flexibility and disaster recovery, has interjected a new level of complexity into the mix. And this new level of complexity is not just due to the fact that one more vendor’s product is running on your servers. It is due to the two facts:

VMware changes how certain aspects of Windows works. Specifically, if you are monitoring how a Windows OS is using resources to do performance management or capacity planning you cannot compare numbers gathered from a Windows OS running natively on its own hardware to numbers gathered from a Windows OS running inside of a VM. If you are running a web based application under IIS that typically used 50% of CPU on a dual-CPU server with Windows deployed on native hardware, you cannot compare that number to any number you get from the same application running on the same copy of IIS running on the same version of Windows Server inside of a VM. The reason is that the first number is the percentage of the total CPU available on the server that is being used by the application. The second number is the portion of the CPU that VMware has given that guest that is being used by the application. So, you cannot make apples to apples physical to virtual comparisons with Windows Perfmon counters that have to do with how much of a given resource or activity is being used or generated per unit of time. In other words all of the data generated by Windows Perfmon counters that pertain to time based resources (CPU Utilization per unit of time, page faults per second, etc.) is useless when that Windows OS is hosted inside of a VMware VM.

The second reason is that systems management tools and performance management tools have not (with some very rare exceptions) woken up to the fact that the world is getting more layered. Core infrastructure management products from vendors like HP, CA, IBM, and NetIQ are built around the assumption that there is an OS, some OS related middleware (IIS, COM+, .Net, etc.) and an application to manage. These products look at resources in a very simple way, by apportioning them between how they are used by either the OS, some middleware, or an instance of an application. These products do not understand the proliferation in layers that have occurred in our production server environments, nor do they understand how those layers are related to each other. It is the relationship between the layers that will prove to be the greatest challenge. In other words, what we need to know about a VMware hosted applications environment is not the VMware Guest’s opinion of how much of its resources an application is using, but the facts of how the server hardware resources and their utilization are being apportioned between all of these layers and the key applications running in each Guest OS. This requires a substantial change in how system management data is being gathered and presented, since neither the Window OS, nor the systems management products are built around the assumptions of these layered and virtualized environments.

If you are experiencing challenges in managing your virtualized applications environment, I would like to hear from you. Please send me an email at bernd.harzog@apmexperts.com, and let me know what your challenges are. Perhaps together we can work through them.

Bernd Harzog
CEO
Applications Performance Management Experts
www.apmexperts.com
bernd.harzog@apmexperts.com
770-475-4249