APM News, Reviews and Analysis

Liquidware Labs Addresses VDI Assessments with New Performance Metrics | The Virtualization Practice

2009-06-15T16:19:00.000-04:00

Liquidware Labs Addresses VDI Assessments with New Performance Metrics The Virtualization Practice:

David Bieneman, the founder of VizionCore has joined up with J. Tyler Rohrer formerly of the VMware VDI team and the team from VMsight lead by Jonathan Alexander to create a new company – Liquidware Labs. Liquidware Labs is focuses upon two aspects of the problem listed above. The first aspect is to define a methodology (Assess–>Design–>Deploy–>Manage) for VARs and service providers to use in VDI projects, and the second is to provide a toolkit that implements much of this methodology in software.

Quorum BC-V: Disaster Recovery for the SMB | The Virtualization Practice

2009-06-15T16:15:00.000-04:00

Quorum BC-V: Disaster Recovery for the SMB The Virtualization Practice:

One of the major benefits of virtualization for enterprises has been that through virtualization disaster recovery (DR) has become much easier and more cost effective to implement. This occurs because virtualization allows the entire environment for a server to be containerized in a VM which is nothing but a file that can easily be copied to a different location and brought up if servers in the main data center fail for some reason.

However when it comes to an SMB realizing these benefits, several factors still stand in the way. The first is that most SMB’s do not have the funds to virtualize their entire operation just to get the benefits of DR. Next is the fact that just virtualizing the servers does not really provide for DR, additional software is still needed to periodically update backup images, update backup data in databases, and most importantly replicate the most recent backup images to an off site location in case the primary site really does suffer a catastrophic event. The combination of these costs, processes, and logistics has kept real DR out of the reach of many SMB’s.

Cloud Computing and the End Run around IT – Here We Go Again - The Virtualization Practice

2009-06-15T16:08:00.000-04:00

Cloud Computing and the End Run around IT – Here We Go Again The Virtualization Practice

I think the early uses of Cloud Computing by enterprises (or any company with a significant IT department) will be again by users, departments, and business constituents who have some of their own reasonable level of technical expertise and who will find that they can bring up “tactical” or prototype business applications on external clouds much more easily than they can in the internal labs (even the ones based upon virtualization) offered by their internal IT departments. Cloud Computing offers these business constituents the opportunity to experiment with technology solutions that might fit various business cases and scenarios without having ask, explain, fill out forms, and wait for a response from IT. External Cloud vendors are structuring their entire businesses around being easy to work with on a technical and business level. Therefore, I think that Cloud Computing will be the basis of the next big end run around IT and IT’s structure and processes.

EMC Announces Acquisition of Configuresoft | The Virtualization Practice

2009-06-15T15:57:00.002-04:00

EMC Announces Acquisition of Configuresoft The Virtualization Practice

On 5/27, EMC announced that it was acquiring ConfigureSoft one of the two leading independent vendors of configuration, change, and compliance management software (the other leading independent vendor being TripWire). The stated goal of the acquisition is to “Empower Customers to Automate Visibility and Control Across their Physical and Virtual Data Centers”.

Virtualization Management – Enabling Dynamic Business Services - Virtualization Management Solutions

2008-11-19T11:12:00.002-05:00

Every leading edge IT organization is trying to provide effective IT services to its business constituents, in less time and at a lower cost. As tools for managing virtualized environments evolve, these tools will become the vehicles through which IT organizations deliver agile IT and business services to business constituents. This will occur as virtualization management tools evolve to allow the virtual environment to be provisioned, scaled, managed and priced in manners completely different from how this was done for the previous physical environment.

VMware, Citrix Xen, and Microsoft Hyper-V. Managing a Mix and Match Virtualization Platform Environment - VMware News Resources

2008-09-26T11:11:00.000-04:00

Enterprise IT shops will increasingly move to a multi-virtualization platform environment. Most enterprises will have more than one virtualization platform. In many cases, more than one product will be used within the implementation of one virtualization system. This will place a premium upon the proper use of cross-platform management tools.

Cloud Computing, Virtualization, and VMware - VMware News Resources

2008-09-24T09:31:00.001-04:00

This is my article up on DABCC.com about Cloud Computing, Virtualization, and Paul Martiz's Keynote at VMworld.

There Are No Free Hypervisors – Citrix Xen, Microsoft Hyper-V, and VMware ESXi will all Cost you in the End - Virtualization News Resources

2008-07-31T11:10:00.000-04:00

VMware has just made VMware ESXi “free”. Citrix makes a big deal out of the fact that its Xen hypervisor is “free”. Microsoft’s Hyper-V is included in Windows Server 2008, so it is “free”. However, none of these products is really free; you will pay for all of them in various ways.

Virtualization Management: vmSight Detailed Product Review

2008-07-18T11:17:00.000-04:00

Detailed product review of vmSight, a performance and access management solution for virtualized desktops on the VMware, Citrix Xen and soon Microsoft Hyper-V platforms.

Microsoft Hyper-V Virtualization: Resistance is Futile (Almost)

2008-06-27T11:08:00.000-04:00

Hyper-V does not mean the end of VMware. On the contrary, it means a dramatic growth in the entire virtualization market – enough growth so that Microsoft, VMware, and Citrix will all prosper in their own ways.

VMware Buys B-hive

2008-05-28T10:51:00.004-04:00

VMware announced today that they are buying B-hive, an Israeli based company that has an application mapping, application performance management, and end user experience management solution for virtualized applications systems. The B-hive product is interesting in that it is delivered as a virtual appliance that sits on a virtual mirror or spanned port on the virtual switch inside of the VMware host. This allows the product to see all of the application level flows between the layers of the applications systems, no matter how they are distributed between guests on one host, or multiple hosts.

This announcement changes the dynamics of the virtualization market in two very important and fundamental ways:

It is, at least for the VMware platform, a "game over" for many of the monitoring startups that were focused upon being "the" monitoring vendor for VMware as their business plan. Unless the remaining monitoring vendors have a really strong story as to why an enterprise customer should buy their product in addition to B-Hive, the startup monitoring vendors will now be selling against, and not with the VMware Sales organization. These vendors will now have to focus very hard on whatever their value add is with respect to B-Hive, and turn their attention to the Microsoft and Citrix virtualization platforms. Of course this also means waiting for Microsoft and Citrix to get to an installed base large enough to constitute a market, which certainly has not happened yet.
Taking ownership of applications performance is an extremely effective and strategic move on the part of VMware. This move is effective, because it says to customers that VMware understands that customers must be able to measure and ensure acceptable performance of virtualized applications in order to push virtualization beyond the "low hanging fruit" stage that currently exists. This move is strategic, because it says to Microsoft, "It does not matter if you make virtualization free. What matters is who can take virtualization the furthest and the fastest and thereby deliver the customer more ROI and flexibility". So, VMware is shifting the debate from "who has the best product", to "whose product can deliver the enterprise customer more ROI, more quickly". This is an extremely good move on the part of VMware because it will serve to accelerate deployments, and in turn drive more VMware license revenue.

This move on the part of VMware also significantly raises the "parity bar" on Citrix and just as they are respectively entering, or about to enter the market. VMware already has a signifant lead in the tools and products that surround the virtualization platform itself. HA, DRS, VMotion, the application mapping possible with the new release of SMARTS, and now application performance management and end user experience management with B-Hive consitute a significant difference in both manageability and functionality in VMware's favor.

Needless to way, Microsoft and Citrix are unlikely to stand idly by, and let VMware raise the bar in this manner with no responses. Since VMware has transformed the virtualization platform, into a virtualization suite, Microsoft and Citrix must to a significant degree follow suit. This means that remaining vendors that have significant pure play monitoring functionality for virtualized systems (see my Solutions Guide for the list), must now focus much more of their energies upon building a partnership with Microsoft and Citrix, and position themselves as candidates for acqusition. This also means that there is little likelihood of a long term independent market for application performance management and end user experience management for virtualized systems, as this will likely turn into war fought on the basis of who is building out their portfolio most effectively via acquisition.

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

Managing Virtualized Systems Solutions Guide

2008-03-24T09:14:00.000-04:00

A new Solutions Guide for the management of applications performance and end user experience is now available.

This guide consists of a White Paper that explains the challenges with using existing agent based resource monitors to manage applications performance, and that lays out the criteria by which applications performance and end user experience must be measured effectively in virtualized systems. The Guide also contains a detailed Product Comparison that analyzes nine different products that are targeting the virtualization management space, and drills into their relative strenths and weaknesses.

If you are interested in the Solution Guide, please hit the link, fill out the form and download it. It is free for you to use to help you address these issues.

Best Regards,

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

Symantec to Sell Application Performance Management Business to Vector Capital

2008-01-17T16:34:00.000-05:00

They're Baack! Veritas bought Precise Software who was at one point in time one of the best APM vendors in the market. Then Symantec bought Veritas. In the course of these two acquisitions, the Precise product line lost product focus, sales focus and marketing focus, and has lagged the industry as a result.

Now Precise exists again - proving that some people believe that the APM problem is still unsolved (I agree). We will just have to see what changes in strategy will be brought about by a newly independent and focused APM entity with serious venture capital backing.

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

The Virtualization Management Opportunity

2007-10-22T12:44:00.000-04:00

A very smart and experienced executive in the systems management industry once remarked to me that innovations in platforms always outstrip the ability of the major management vendors to keep up with them. The world needs management startups because corporate IT adopts new platforms before they can be managed by the incumbent frameworks from the major vendors like CA, IBM/Tivoli, HP/Mercury, and BMC. If you think about the major applications architecture and platform innovations that have occurred in our industry in the last 20 years, each has required and created a new set of management vendors. Some examples are:

Client/Server computing – Tivoli

Windows Servers for production applications – NetIQ

Web application response time management – Keynote

J2EE Web Application Management – Wily (now part of CA)

HTTP as the standard application level protocol – Coradient, and the other web appliances

TCP/IP as the standard transport – NetQos, Network General, and the other TCP/IP appliances

So, what is the next one of these, and what are the implications of this particular one? The answer is that virtualization, and in particular VMware has been widely adopted in corporate enterprise IT, and with this adoption has come a big problem for IT, and an even bigger opportunity for new application management vendors. However, this opportunity is not just about a new class of applications, or a new applications architecture, it is about changes to the management of every application that has been built and deployed since the start of business computing. That is because VMware is fundamentally changing things that have not changed in a long time, if ever. As a result of these changes, VMware raises the following questions:

Who budgets for and controls server capacity?

Who is responsible for applications performance?

How does the dynamic nature of virtualized environments change application performance management?

What metrics about server performance can be trusted?

How does virtualization impact root cause analysis?

What approaches to applications performance management stand a chance of working (and which ones do not)?

Let’s address each one of these in order:

Who budgets for and controls server capacity?

In most enterprise IT organizations, server budgets are split. IT controls the budget and capacity for commodity servers that provide horizontal services like file, print, and email. But the business units that own business critical applications (like SAP, CRM, etc.) own the budget for the servers that run their mission critical applications. Each business unit buys servers for “their” set of mission critical applications, which creates massive silos of application specific capacity within an enterprise.

With virtualization, this dynamic changes. As opposed to silos of servers that support each application (which is an incredible waste of server resources), IT provides a shared resource pool of server capacity. Each business units applications still run in their own OS, but instead of the OS being locked to one instance of server hardware, the interface between the OS and the server hardware is virtualized, and one server can support many instances of different types of operating systems. This allows server utilization to rise dramatically, and allows IT Operations to buy server capacity incrementally across all of the supported applications.

The change in who buys server capacity also creates a problem for both IT Operations and the application owner. That problem is that there needs to be rational way to know how much server capacity to buy to virtualize the next application, and when the lack of capacity is hurting applications performance. Related to this problem is that IT Operations needs to be able to prove to the application owner that the application will perform at least as well once virtualized as it did when it was running on its own hardware. These new problems are created by virtualization because virtualization invalidates the traditional metrics used in capacity planning (question #3).

Who is responsible for applications performance?

In the physical world of one application and one OS per server, there was often a clear line between IT Operations and the applications owner as to who was responsible for what. IT Operations was responsible for supporting the platform for the application (the hardware and the Windows OS), and the application owner was responsible for ensuring the performance of their own application. Since the application owner could add capacity whenever they wanted to, the application owner felt secure that sufficient resources were available to allow their application to run effectively. When there were problems with applications performance, IT Operations had a well understood process to prove their innocence, and to dump the problem in the lap of the applications owner. The process was basically, “My metrics prove that the OS is running fine, and that your application is using no more resources than it should, so it has to be a problem with your application, not my platform”.

Once an application is virtualized, IT Operations loses this defense, since the hardware is now shared between multiple instances of the OS, running multiple different applications. When the application does not perform well, the application owner immediately points the finger at the new virtulization layer as being at fault and IT Operations does not have the tools or the metrics to defend themselves. This brings up another problem created by the virtualization of applications. That problem is that tools do not exist that allow IT Operations to prove in a defensible manner (with numbers that people believe in) that the environment (including impacts upon application A by application B running in a different VMware Guest) are not at fault. In other words, IT Operations has lost its ability to defend itself in the blamestorming meetings that inevitably occur when applications do not perform well.

A related problem is that neither IT Operations nor the applications owners have any tools that can credibly compare the performance of applications across physical and virtual implementations. The primary reason for this is that the resource based applications performance metrics used by most application performance management (APM) vendors in the physical world, do not work in the virtual world, and therefore cannot be compared across applications running in physical and virtual environments.

How does the dynamic nature of virtualized environments change application performance management?

In physical implementations (dedicated implementations of physical servers, operating systems, and applications), APM products assume a specific and fixed set of hardware provides the resources that are used by the application. These same APM products assume via configuration that the web layer of an application is talking to a specific middle tier layer, which in turn is talking to a specific database server.

When problems are reported about the performance of an application, that report contains references to the physical environment of the application, like the name of the server, its CPU rating, the total amount of memory in the server, its IP address, etc. APM products assume that these physical elements are a fixed reference point through time against which utilization of resources and performance can be judged. These products often create baselines, or statistical representations of what is “normal” based upon how much of a resource an application is using at a point in time. These products also assume that relationships between layers of an appliation system are fixed, instead of dynamic.

The dynamic nature of virtualized applications creates two more points of pain. One is that baselines for normal performance related to the specific hardware upon which a portion of an application is running at a moment in time are invalidated. The other is that products that rely upon manual configuration to understand the mappings between applications components are just too brittle to be able to deal with the rate of change in the virtual platform underneath virtualized application components.

What metrics about server performance can be trusted?

The answer to this question needs to start with a discussion of what cannot be trusted, and then to see what is left. The basic problem is that an operating system that measures the performance and resource utilization of its own processes, and applications running on that operating system assumes it is the sole user of the hardware resources on the server or workstation it is running on. For any resource that involves time (CPU %, Disk Time, Page Faults per Second, Context Switches per second), the OS assumes that it is the sole user of the system clock. So, when the OS measures how much CPU a process (an application) has used in the last N milliseconds, the OS assumes that it is the only user of the CPU.

Once you virtualize an OS, all time based metrics collected by the OS about itself and the applications running on top of the OS are shifted by the degree to which that OS is now one of many OS’s sharing the same hardware. If there are 5 guests on a server, and one application running in each guest, and all are doing equal work at that moment in time, then the metrics reported by a guest OS will be off by the fact that each guest only sees one-fifth of the hardware resources at that moment in time. Of course if in the next second, a guest is shut down, then the degree of time shift changes.

The first conclusion about metrics in virtualized environments is that any metric about resources that is based upon the use of a resource over time which includes all of the ones listed directly above, is invalidated (made irrelevant and untrustworthy) by virtualization. The only resource based metrics that remain valid are ones like how many bytes of memory an individual process is using, and how big a file or database is in actual bytes.

However, the problem gets worse from here. The holy grail of applications performance metrics, response time, is also impacted by this time-shifting. If an APM product reports that transaction A as measured by the elapsed clock time from action B to response C is .5 seconds in a physical implementation, then it really took .5 seconds. If that same measurement occurs within a VMware Guest that is one of 10 guests on a server, and all guests are equally busy, then the response time monitor could well again report .5 seconds, but the actual clock time that elapsed could well have been 5 whole seconds. This is because the Guest OS only knows about the clock ticks that it gets, and as opposed to getting all of them in a physical environment, that guest is only getting a variable share of those clicks at each moment in time. So, virtualization can also invalidate the most valuable and credible of APM metrics (response time) if those metrics are collected from within a virtualized guest OS.

How does Virtualization impact root cause analysis?

Virtualization makes traditional root cause analysis much more difficult for all of the reasons mentioned above. By invalidating many of the metrics and their baselines that server and application support teams rely upon, virtualization makes it much more difficult to use those metrics to pinpoint abnormal behavior.

Virtualization also creates a whole new root cause problem. That problem is to answer the question, "Why does this application perform poorly when virtualized, but performs just fine when it is using completely normal amounts of CPU and memory on a physical server". The inability to know how well a particular application will perform once virtualized means that the only method that is feasible is to "try it and see how well it works (or does not)". Since IT does not inspire confidence with many business units and applications teams, having IT have to use the "trust me it will work" promise is a major roadblock to virtualizing the 80% of the applications that really matter in a corporate enterprise.

Which approaches to applications performance management stand a chance of working (and which ones do not)?

Before virtulization, IT Operations had resource based metrics to fall back upon when questions of applications performance arose. Now these metrics are either unavailable, or not credible. The shared and dynamic nature of virtualized environments makes APM approaches based upon how much CPU, or Disk I/O an application is generating at a moment in time untrustworthy and invalid. Furthermore, approaches that gather response time data via scripted synthetic agents, or real time passive agents are also impacted when those metrics are collected inside of guest OS.

So, what works today, and what new approaches are needed? There are two approaches that stand a chance of working. One is to rationalize the resource based metrics, by allowing metrics collected by the host OS, to be combined at any moment in time with metrics inside of the Guest to provide a true picture of resources utilization. This requires VMware to either collect and publish the host metrics in a usable form, or requires third party agents running in the host OS (something VMware is reluctant to get behind since it wants to keep the host OS as thin and efficient as possible). The second approach is to rely upon actual transaction response time metrics collected at actual end user workstations, from within the actual applications or upon response times collected by network appliances connected to mirror ports on switches. This has been the holy grail of APM for several years now, and this approach has now been made all the more valuable by the demise of the traditional resource based metrics. Whichever approach is used, it will have to be combined with an ability to dynamically understand changes to the enviroment of a guest OS, and to the components of an applications system at a point in time. So, changes in the environment of an application needs to be constantly auto-discovered by APM tools in order for the tools to be able to provide relevant information.

The race has started to allow IT Operations and applications owners to know how well their virtualized applications are really performing, and to be able to quantify that performance in ways that allow for more applications to be virtualized in denser implementations then are now possible (since no one knows how densely you can pack guests into a host without causing problems). Successful vendors of response time based APM solutions are effective today when they sell to applications owners. Virtualization creates many additional pain points for applications owners and IT Operations. The successful vendors will figure out how to make their products dynamic and based upon credible metrics and to tune their sales and marketing approaches to address these new points of pain and target audiences.

Bernd Harzog
CEO
Application Performance Management Experts
770-475-4249
bernd.harzog@apmexperts.com
http://www.apmexperts.com/

Product Review — Wily Introscope for Microsoft .NET @ SOA WORLD MAGAZINE

2007-06-01T20:25:00.000-04:00

CA Releases Wily Introscope for Microsoft .NET

2006-10-25T13:22:00.000-04:00

This whole web site is about the fact that despite the billions spent on Systems Management and Applications Performance Management, knowing that a user is having a problem with an application, and then knowing how to fix remains a largely unsolved problem. For the last few years, Wily Technology (now part of CA) has lead the market for addressing this problem with high-end production J2EE based applications.

Wily has now announced that they have extended support in their market leading Introscope product to Microsoft .Net based applications. The detail is that Wily ported the agent that runs inside of the J2EE applications server to C#, and did the remaining steps necessary to allow that agent to manage .Net applications as well as the Java agent manages J2EE applications. The .Net agent also leverages the rest of the Introscope product line. This will prove to be a boon for customers who have both J2EE based applications and .Net applications (or applications that are comprised of some of each environment), as it will allow for a common way of measuring performance, tracing transactions, and performing root cause analysis.

What this new release does not do (and does not claim to do) is solve the APM problem for the general case of Windows applications no matter how they are developed. This remains a really hard (and probably impossible) problem to solve since there have been and continue to be so many different ways to build and deploy Windows applications.

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

RIP Cesura

2006-06-10T22:01:00.000-04:00

Cesura, one of the more promising startups in the applications and performance monitoirng space shut down the week of June 5th, 2006. Cesura had a lot going for it including:

It was lead by Bob Fabbio, who was the founder of Tivoli, and who was highly skilled at raising enough venture capital enable a startup to achieve some real traction
It was able to raise $15m in a restart after a previous positioning failed in order to target the applications performance problem with web and Citrix hosted applications
It sported an impressive set of functionality covering some true measurement of end user experience, instrumentation of a broad set of infrastructure metrics and a highly differentiated set of analytics to pinpoint the areas in the infrastructure responsible for the end user or applications issue.

However, despite lots of money and significantly differentiated technology, Cesura was done in by two big mistakes. The first mistake was that the company did not maintain a focus upon a target market long enough to take market feedback and build something that truely met a market need. Cesura relaunched with a focus upon Citrix in the fall of 2005. By the middle of Q1/2006 the focus was upon health care applications and partners. From April of this year on, the focus shifted to value added hosting vendors and MSP's. Three target markets in three calendar quarters is enough thrashing to confuse customers, prospects, the press, analysts and even upper and middle management. A CEO with a pedigree, and $15m in the bank was apparently not enough to overcome the confusion caused by a strategy du jour operating plan.

The second big mistake was that things that the company throught were features of the product, were in fact viewed by prospects as impediments to adoption. For example:

The analysis engine and database were housed in an applicance, that had to be LAN connected to each switch that was a part of each unique subnet that hosted portions of the software that comprised the target applicaitons system. The feature here was the idea that the mangement product had its own network, and would not die if the production network died. The reality is that caused the sales team to have to get the network folks to approve the installation of something that was supposed to solve and applications and business level problem.
The product consisted of one and only one appliance. Each appliance could only support around 100 servers, and there were no cross appliance analytics or reporting. So, a product with a high end message and value proposition, could not scale up to high end enterprise applications systems.
The Citrix offering was unique in its ablity to measure true end user experience at the ICA client, but this only worked for a subset of the applications and transactions published through Citrix, and only worked on a subset of the Citrix client environments. Again, something that was truely unique in certain cases, did not support the breadth of environments to make cusotmers comfortable that this was an enterprise solution.

In short, the rules for startups targeting the user or applications performance space remain the same. Make sure that what you have to sell adds value to the monitoring products that customers have already invested in, and focus like a laser upon a set of prospects (otherwise known as a market segment), that have a common problem, and that can be accessed as a community or a market. It is amazing that after all of the money that has been invested in monitoirng startups, that companies still get funded and fail for these simple reasons.

CA, Wily and Round Two of the Applications Management Battle

2006-01-13T14:03:00.000-05:00

CA announced last week that they were acquiring Wily Technology for $375 million. I have to admit that when I first saw the announcement, my reaction was, "Wait a minute, CA is where old software goes to die, what is Wily (the leader in the business of managing J2EE applications in production) thinking selling themselves to those dinosaurs?" Then I listened into the analysts call and actually had a chance to talk to some of the executives who are involved from both companies. As a result of that conversation, I came away impressed with the commitment of CA's new management team to break with the past patterns of angering customers. I also realized that given who Wily competes with (IBM, Symantec/Veritas/Precise, Quest and Compuware) being able to leverage CA's 2500 salespeople and their customer base might just be the best strategy for Wily to continue leading the J2EE Applications Management market, and continuing on its torrid growth path.

It is the realization that the best path for Wily to continue to lead the J2EE Applications Management Market was for it to become part of a much larger and more entrenched company with a much broader product line, that leads me to the conclusion that something fundamental has now changed about the Applications Management Market. If you are talking about managing J2EE based applications in production (things running under Weblogic, Websphere, JBoss, etc.) this war is over from the perspective of startups, early adaptors, venture capitalists and people like me who make their money helping small companies be more successful. This is now a business where big companies go to one of a small number of vendors all of whom are selling a broad product line complemented by a broad set of services. Another way of looking at this is that Gartner has had a "Magic Quadrant" on this market for over a year, and by the time Gartner publishes one of those on a category, you can be pretty sure that the opportunity to start a company from scratch to attack that market is long gone.

So, where does that leave the rest of the Applications Management Market? First of all, what does the rest of the market consist of? Well it pretty much consists of the following groups of applications. There are several large vendors of line of business applications for which good Application Management still does not exist. For example if you are running SAP, Siebel, Oracle, etc. in production any pieces of those applications that are not J2EE are still black holes and management nightmares. The massive world of Windows applications remain largely unmanaged in production as well. This is a particularly hard problem because Windows applications have been built in so many different ways that one approach to managing them cannot work. So there are point solutions by applications type, which makes for lots of niche vendors and lots of confused customers. Microsoft is focused upon .Net since Microsoft knows that if .Net does not succeed, they will have lost the war for production applications platforms with the J2EE camp. Finally, Citrix Systems continues to gain traction with its method of centralizing the delivery of applications. However, once you use Citrix for its centralization benefits you compound your applications management challenges.

So, we are just going to have to see where the next category of interesting applications management solutions comes from. My bets are on .Net and Citrix for two very different reasons. .Net because it promises to be the next great thing from Microsoft and that always attracts entrepreneurs and venture money. And Citrix because it is just a big enough niche to get some smart entrepreneurs started in a market with big pain and little competition.

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

Shrek, Onions, & Virtualized Application Architectures

2005-09-30T08:52:00.000-04:00

What do the three things in the title have in common? Answer - they all have “layers”. If you do not have kids you may not know that Shrek, the ogre in the movie by the same name, has (contrary to popular opinion about ogres) layers to his personality. Onions have layers, and so do applications deployed via application virtualization technologies like Citrix, VMware, Microsoft Virtual Server, and Softricity. Why is this interesting or relevant to the topic of Applications Performance Management?

Because, when you use a virtualization technology to simplify some aspect of managing your applications, you at the minimum inject another layer or two of software into the mix. Let’s take VMware as a case in point. Before VMware you have the OS (Windows Server 200X) talking directly to the hardware. Applications talk more or less directly to the OS. So you have the hardware, the OS, and the applications to worry about.

Again using Windows as an example let’s look and see what happens when you deploy these applications in virtual machines hosted on VMware ESX. The OS that sits on top of the hardware is now the slimmed down version of Linux, upon which the VMware ESX host OS resides. Each VM consists of a VMware guest supporting a guest operating system. So now your layers consist of hardware, Linux, the VMware Host, and N instances of a VM Guest with the Guest OS and the applications. So you have gone from three layers to six. Do you feel as layered as an onion or an ogre yet? Or perhaps having to support all of this complexity makes you want to turn into an ogre so that you can deal with people who do not understand all of this the way that Shrek did (he could growl at people and they would all run away).

But seriously now, the move to virtualized applications architectures while laudable for reasons of flexibility and disaster recovery, has interjected a new level of complexity into the mix. And this new level of complexity is not just due to the fact that one more vendor’s product is running on your servers. It is due to the two facts:

VMware changes how certain aspects of Windows works. Specifically, if you are monitoring how a Windows OS is using resources to do performance management or capacity planning you cannot compare numbers gathered from a Windows OS running natively on its own hardware to numbers gathered from a Windows OS running inside of a VM. If you are running a web based application under IIS that typically used 50% of CPU on a dual-CPU server with Windows deployed on native hardware, you cannot compare that number to any number you get from the same application running on the same copy of IIS running on the same version of Windows Server inside of a VM. The reason is that the first number is the percentage of the total CPU available on the server that is being used by the application. The second number is the portion of the CPU that VMware has given that guest that is being used by the application. So, you cannot make apples to apples physical to virtual comparisons with Windows Perfmon counters that have to do with how much of a given resource or activity is being used or generated per unit of time. In other words all of the data generated by Windows Perfmon counters that pertain to time based resources (CPU Utilization per unit of time, page faults per second, etc.) is useless when that Windows OS is hosted inside of a VMware VM.

The second reason is that systems management tools and performance management tools have not (with some very rare exceptions) woken up to the fact that the world is getting more layered. Core infrastructure management products from vendors like HP, CA, IBM, and NetIQ are built around the assumption that there is an OS, some OS related middleware (IIS, COM+, .Net, etc.) and an application to manage. These products look at resources in a very simple way, by apportioning them between how they are used by either the OS, some middleware, or an instance of an application. These products do not understand the proliferation in layers that have occurred in our production server environments, nor do they understand how those layers are related to each other. It is the relationship between the layers that will prove to be the greatest challenge. In other words, what we need to know about a VMware hosted applications environment is not the VMware Guest’s opinion of how much of its resources an application is using, but the facts of how the server hardware resources and their utilization are being apportioned between all of these layers and the key applications running in each Guest OS. This requires a substantial change in how system management data is being gathered and presented, since neither the Window OS, nor the systems management products are built around the assumptions of these layered and virtualized environments.

If you are experiencing challenges in managing your virtualized applications environment, I would like to hear from you. Please send me an email at bernd.harzog@apmexperts.com, and let me know what your challenges are. Perhaps together we can work through them.

Bernd Harzog
CEO
Applications Performance Management Experts
www.apmexperts.com
bernd.harzog@apmexperts.com
770-475-4249

Beware of the Franken-Monitor

2005-09-12T11:00:00.000-04:00

As I have mentioned many times in my various blogs, it continues to amaze me that we as an industry have gone so far down the road of building and deploying management tools, and we have yet to solve the most basic and most important problem. That problem is knowing when a user of a business critical application is having a problem with that application, and then combining that knowledge with knowledge of the state of the application’s infrastructure to find the likely source of the problem.

Let’s focus on the response time part of that challenge first. Here is a little ad hoc list of vendors who in some way or another measure response time:

Mercury can measure response time to scripted synthetic agents (an approach that I am not a fan of), and also measure response time at the HTTP level via it Real User Manager (RUM)
Wily can measure the response time from the perspective of a browser used by a real production user via its Browser Response Time Adaptor which is a companion product to Introscope its J2EE deep dive product.
Cesura (formerly VEIO) can measure the response time for URL classes at the web server (this misses the part of the round trip out to the client, but is valuable nonetheless)
Premitech can measure the response time of the network (network latency) between a Citrix server and the network out to the clients
A variety of network management vendors (NetQos, Visual Networks, NetScout) can measure network latency in various hops of the network that supports the application in question.

Now for the second part of the challenge, figuring out where the source of the problem is. Unfortunately most enterprises have deployed infrastructure management products from vendors like IBM/Tivoli, HP, CA, BMC, and NetIQ that collect a variety of metrics about how servers are using various resources and use this information to report availability (which they are pretty good at), and to infer something about performance (which they are terrible at since you cannot infer response time of an application from its underlying resource utilization statistics).

So, if you have a utilization monitor, and if you have a separate response time monitoring tool (you should) then you are well on your way towards building and supporting your own Franken-Monitor, a hodge-podge of ad-hoc tools each designed to solve one very specific problem, which do not put their information into a common database, and which therefore cannot work together to help you really find the source of your response time problems.

If you feel bad about having your own custom Franken-Monitor that you have carefully assembled over the last ten years, do not despair; there are vendors who would happily sell you one. Vendors that grown their portfolio of systems management products through acquisition are especially guilty of this. Let’s pick on some people to make an example of them:

Mercury acquired SiteScope (an agentless infrastructure monitor), acquired Real User Manager (an HTTP level response time monitor), acquired its J2EE Deep Diagnostics product, and then converted is WinRunner test tool, into Topaz to do scripted response time monitoring of web-based applications. Guess what, none of these products can put their respective information into a common unified database to support an integrated problem resolution process.
IBM/Tivoli will happily sell you a combination of OMEGAMON for WebSphere Applications Server, Tivoli Web Response Monitor, and a variety of Tivoli monitors for your server infrastructure, but, guess what? Again, none of these products integrate their data with each other, nor do they feed any kind of integrated problem resolution process.
Compuware has aquired Adlex to bolster its ability to measure response time at the network level, and has recently acquired a J2EE deep dive tool as well. Do these new products integrate with Compuware Vantage, which is core systems management product from Compuware? Of course not.

So, is there a solution to this mess? Is there one thing you can buy that monitors true end user response time, collects the infrastructure metrics for the application system, and feeds all of that into some sort of automated process for finding the likely source of the problem? I know of a couple of vendors who are working on various aspects of this problem, so there is hope.

If you are an enterprise struggling with these issues, I would like to hear from you. If you send me an email, we can set up a one hour brainstorming session at no cost to you, and discuss your environment, your problems, and how they might be best addressed.

Best Regards,

Bernd Harzog
CEO
Applications Performance Management Experts
www.apmexperts.com
bernd.harzog@apmexperts.com
770-475-4249

BMC Software Introduces Performance Manager, Promises Agentless APM

2005-05-24T13:24:00.000-04:00

On May 23rd, BMC announced the integration of Patrol Express, its agentless offering with Patrol and rebranded the new product Performance Manager. This is both an extremely significant move on the part of BMC, and the extension of a marketing position to a new level of spin. The significance of this move lies in the fact that smaller infrastructure management vendors sporting agentless offerings (and even some large ones like Mercury with SiteScope) have been putting tremendous pressure upon the large incumbent vendors like BMC, IBM/Tivoli, CA, HP, and NetIQ. The pitch on the part of the agentless crowd has been that they can do between 80% and 100% of what the agent-based vendors can do, but with significantly reduced licensing costs, and (more importantly) significantly reduced complexity in terms of deployment and maintenance of the software solution. The agentless crowd has even has some notable successes in getting enterprises to unplug their complex agent-based enterprise management frameworks, and replace them with the simpler and cheaper agentless offerings. Mercury in particular has been very aggressive in going after large enterprises with a pitch that is based upon SiteScope licensing costs for less than the recurring maintenance charges for the incumbent enterprise frameworks. So, BMC (and the others will follow) had to do something if for no other reason that to protect its customer base.

The first significant effect of the move to agentless technology on the part of the infrastructure management vendors will be that it will sharpen the distinction between vendors who provide infrastructure management (management of the hardware, the network, the OS, and the middleware layers), and vendors who provide application management (or Application Performance Management). As the infrastructure management vendors become agentless, it will become clearer and clearer that they are not managing applications, nor are they managing or assuring the performance of applications. This will create enormous opportunities for the vendors who deliver real application management solutions to gain traction, as it will create room in the market for these solutions. It will also create some technical room for the applications management vendors, which they sorely need, since true management of applications and applications performance requires that an agent be present on the server. So, getting rid of a marginally valuable agent that monitors CPU utilization and disk space, creates room for a more useful agent that that actually helps manage the performance and availability of applications.

Now for the part about the increase in spin from BMC. If going agentless means that Patrol will become less useful in terms of managing applications performance you have to wonder what the marketing folks at BMC were thinking when they named the new offering Performance Manager. Seems like they picked the one name that evokes the exact opposite of where the product is actually going. Just goes to show you that marketing and reality continue to bear no resemblance to one another (at least at BMC).

Bernd Harzog
CEO
APM Experts
Bernd.Harzog@apmexperts.com

Wily Technology Releases Browser Response Time Adapter

2005-04-19T08:45:00.000-04:00

Wily continues to lead the market for J2EE Applications Performance Management by doing things the right way. A critical element of any APM solution is the measurement of actual end user experience. For J2EE applications which have a web server as the front end, one right way to measure end user experience is to measure how long the application system takes to respond to each user request. Since each user request comes in the form of a URL, measuring the arrival time of that URL and then the gap to the response on the part of the application system to that URL is one of the "right" way to do things. Of course, Wily does not stop at the measurement of just the response time, since the IntroScope product can then drill into the middle tier components on J2EE applications servers to find out why response time has degraded.

Wily is also correct in its approach of not relying upon synthetic transactions (scripted agents emulating real users). APM Experts believes that while a script may be useful during QA, and it might be useful as a beginning of the day check (to see if the system is working before real users log on), there is no way that scripts can be written to accurately emulate what real users do. It is also extremely difficult to use scripted synthetic agents to emulate the combination of actions that a set of production users take on a live application system. Please see the article "Why are Scripts and Synthetic Transactions Undesirable" in the Analysis section of the APM Experts web site for more details on this subject.

Bernd Harzog
CEO
APM Experts
bernd.harzog@ampexperts.com

Who will Grab the Brass Ring in Application Management?

2005-04-15T14:28:00.000-04:00

Here is an interesting disparity for you to think about. If you have invested a reasonable amount in network and systems management tools, your IT department should be able to:

Keep your servers and networks up and running
Monitor on an ongoing basis how utilized they are
Use change management software and processes to manage keeping them up to date
Use capacity planning tools to understand when to upgrade them
Commit to service levels in terms of the availability and capacity of these resources

On the other hand, you are highly unlikely to have an analogous set of capabilities for your applications. For example, you probably:

Cannot keep your applications up and running as smoothly as your servers and networks
Cannot know on an ongoing basis how utilized an application is relative to its capacity to support users and work
Do not know what the performance (response time) the application is delivering to each of your end users
Cannot quickly and effectively identify problems in the applications, resolve them, and get the changes into production
Cannot commit to application service levels that credibly promise availability and performance levels of the applications to your users.

There are three groups of vendors who collectively stand some chance of addressing this set of issues:

Traditional infrastructure monitoring vendors (BMC, CA, IBM/Tivoli, NetIQ, HP, etc.). While these vendors products are very widely installed in enterprise IT shops they have neither a robust ability to measure applications availability and performance, nor any ability to assist in the application problem resolution process. These vendors are therefore the least likely of the three groups to be the source of a meaningful solution to the problem.
APM focused monitoring vendors (Wily, Veritas, Mercury, Compuware, Quest, VIEO, ProactiveNet, eG Innovations, Premitech, etc.). These vendors all focus heavily upon measuring applications performance (response time), and offer in some cases (J2EE) a reasonable ability to pinpoint problems in middle tier applications. However, none of these vendors (as of yet) combines a comprehensive ability to monitor a variety of different applications architectures (J2EE, COM+, .Net, ASP, Citrix, etc.) for end user performance with an ability to help the problem resolution process once an application problem is identified.
Problem resolution vendors (Identify Software, Mercury, Compuware, AviCode, XtremeSoft, etc.) offer a variety of capabilities to drill into specific source code environments and either pinpoint the problem or at least get the developer close to the source of the problem. However, these products are for the most part difficult to use in a "monitor all of the time mode" since they usually consume to much resource and generate too much data to be left on all of the time on every production server.

In summary the market opportunity for management of applications while they are in production is wide open. In order to seize this opportunity, someone will meet the requirements below and seize the brass ring:

Monitor every aspect of the application no matter how it is architected (J2EE, .Net, etc.) and no matter where it runs.
Collect meaningful performance data (response time) about every user of every application
Operate in a manner consistent with the requirements of products that are running all of the time on all production servers (low resource utilization, and smart management of volumes of data produced)
Automatically map the application components across the application system for each user of the application.
Synthesize 1-4 above into true root cause reports that tell administrators and developers where the problems are and what to fix.

Someone will grab the brass ring and run off with a huge market opportunity. Who will it be?

Bernd Harzog
CEO
APM Experts
Bernd.harzog@APMExperts.com

NETSCOUT, QUANTIVA and the Role of Statistics in APM

2005-02-14T10:09:00.000-05:00

NETSCOUT SYSTEMS TO ACQUIRE QUANTIVA

One of the difficult things about implementing an effective APM solution (one that accurately measures all user transactions and accurately points the finger at the real cause of the problem) is that the number of things that can go wrong are large in number, and greatly varied in character. Furthermore, problems with end user experience often lack any kind of repeatable pattern in their cause (it is something different every time).

There are two basic approaches that one can take to figuring out what went wrong with application service level or end user experience. The first approach is a deterministic one. This means that the APM product has to know all of the interactions that an application has with its own components and its supporting infrastructure. Examples of a deterministic approaches that work are products from Wily Technology, Symantec/Veritas, Quest, Tivoli, Identify Software, and AviCode that all measure performance at the web server, and then time how long each module of code takes to do its job, and then point out the guilty line of code. This works just fine when the application is written to J2EE or .Net and there is specific support in the code inspection tool for applications written to that infrastructure.

There is a far larger problem of how to do APM when the application includes pieces that may not be J2EE or its Windows equivalent (previously COM+ and now .Net), and the infrastructure for the application includes many different types of servers. The problem is also expanded in complexity once you include the LAN and WAN components as part of the application infrastructure.

In order to deal with both the problem of complexity in applications systems and the fact that each problem tends to have a unique cause several vendors have deployed rich and complex statistical analysis of the data that is collected by both their own APM systems and by agents from other vendors. In the late 1990's CA tried this with Neugents (and failed), and ProactiveNet and Netuitive tried this with their technology. Today, ProactiveNet focuses upon predicting performance problems and is selling its product as a statistical overlay on top of existing infrastructure management products from HP, CA, IBM/Tivoli, NetIQ and BMC. Netuitive is focusing its statistics technology upon analyzing alarms from BMC and NetIQ to determine which ones are valid (alarm reduction). VIEO employs statistical analysis in its APM solution for the Citrix market which helps its offering do problem determination for applications hosted in the Citrix MetaFrame environments.

Augmenting deterministic features in an APM offering with statistical capabilities allows these vendors to bring the advantages of statistical analysis to their customers. Some of the advantages of statistics in APM are:

A statistics engine can learn normal behavior patterns, and can stop alerts related to "normal busyness" from flooding the administrators of systems. So, if a particular application system is busy at 10 AM and 2 PM on weekdays, and CPU regularly spikes to 70% on applications servers during those times, admin's can be spared a large number of "false positive" alerts.
A statistics engine can detect emerging correlations between degradations in performance (user experience) and resource bottlenecks somewhere in the applications system. The ability to figure out that contention for a particular resource on a particular server is the cause of a performance problem is probably the single most valuable aspect of applying statistics to APM. However this is only true if the statistical engine can dynamically and automatically evolve its model as the behavior of the system changes.

On the other hand there are issues with certain implementations of certain statistical systems that constrain their usefulness, and increase the cost and complexity of their use. These issues are:

Some statistical systems need regular daily patterns of utilization to learn and establish a baseline of normal operations. This is fine for a stock trading web site, but may be a deal killer for many internal applications that have much more varied usage patterns.
Some statistical systems need an up front training period to build a model (this was the flaw in the neural network technology that was the basis of CA Neugents), and this model must be rebuilt whenever demand and resource utilization patterns change.

The acquisition of Quantiva by Netscout is another example of how rich statistical analysis can be applied to large amounts of data in order to find patterns and speed the problem resolution process. Netscout already has a strong APM offerings, and it will be made stronger by the addition of the statistical performance analytics purchased from Quantiva. Enterprises should be careful however to evaluate the Quantiva technology and make sure that it fits the demand patterns, utilization patterns, and speed of change inherent in the underlying application system.

Bernd Harzog
CEO
APM Experts
bernd.harzog@apmexperts.com

Virtual Servers and Applications Performance Management

2005-01-19T10:55:00.000-05:00

One of the hottest trends in the management of servers these days is to virtualize them using either VMware or Microsoft Virtual Server. Replacing lightly used physical servers with virtual ones has many benefits including:

Many small dual processor servers can be consolidated into one server with more CPU’s. This reduces complexity, saves money on networking, power, etc., while also collapsing several underused resources into one appropriately used resource
Virtual servers can be more easily provisioned than can a new physical server
In some cases (VMware, VMotion, and a SAN) you can even move live virtual servers from one machine to the next
If you are paying someone (either an outsourcer or your IT department via chargeback) a per server fee to manage your servers, you can save quite a bit of money by moving from N smaller servers to fewer large servers.

However, moving to an environment of virtual servers brings with it a new set of challenges:

As is always the case with new and innovative platforms, the vendor of the platform (Microsoft, Sun, or in this case EMC/VMware) does not provide all of the management tools that you need to effectively use the platform in production.
The vendors of management tools typically lag behind the platform vendors in terms of full support for the platform (Citrix got to be a $750M company, and support for Citrix among the management vendors stills lags behind the support for J2EE and Windows/COM+/.Net.
The per server licensing models used by most management vendors do not work in the virtual server environment. If an enterprise puts 10 VM’s onto a four CPU server, that enterprise is going to be loath to pay $2000 in licensing fees for each VM, in place of the previous charge of $2000 or $4,000 for the license for one physical four CPU server.
There are no systems management or APM products available today that are structured to deal with properly monitoring resources across both the host OS (in the case of VMware ESX, a flavor of Unix), and several guest OS’s each of which (again the case of VMware) might be different from each other. For example of two processes running in two different VM’s contend for a resource in the host OS, and each experience high context switches as a result, there is no way to find out that it is those two processes and what the resource in contention is.
There are no tools for basic capacity planning. There is simply no good way to know how many virtual servers of a given type will fit onto a physical server of a given type. So, you cannot calculate the economic benefits of a consolidation project before attempting to implement at least a part of that project.

In summary, while virtualization of servers is a hot trend in IT right now, this trend cannot continue to make inroads into core production systems unless a set of APM vendors emerges that allows the IT staff to ensure the service level of the applications on the virtual servers. Since the APM industry is still struggling to deal with everything except custom developed J2EE applications (for which there are a number of effective solutions), it will likely be a while before this need is met. However, it would not be surprising to see a nimble APM startup jump on this opportunity, as a way to start gaining traction in that difficult to penetrate market for IT management solutions.

Bernd Harzog
CEO
Applications Performance Management Experts
www.apmexperts.com
bernd.harzog@apmexperts.com