Why Is Measuring Application Response Time via Scripts a Bad Thing?

When the web based applications emerged, enterprises realized that now that public users from far away places were going to use these applications, that existing script oriented QA tools for Win32 needed to evolve. Evolve they did, into products like Topaz from Mercury, and services from Keynote that measured the response time to a URL and then to various clicks and actions on the pages underneath that URL. The vast majority of the systems management and APM industry has now run off with this concept as the source of the end user response time data in their APM products. This has the following flaws :

• Applications change constantly, and therefore so must the scripts. It is hard to get the scripts working in the first place (in a meaningfully comprehensive way), and daunting to keep them up to date. The average Fortune 1000 enterprise has over 1000 custom developed applications. The budget and manpower simply do not exist to create and maintain the scripts for all of these ever evolving and numerous applications.
• It is very hard to script all of the (sometimes dumb) things that users do. There are numerous permutations of code paths through an application. Some are valid (intended by the designer) and some are not. You need to know when users (for whatever reason) take invalid paths through an application. It is impossible to script every possible sequence that a user might take through an application, and you want to know when a user takes in invalid one (might point to a training or design issue).
• N instances of a synthetic agent cannot simulate hundreds or thousands of users. Just like you cannot script all of the things that one user might do, you cannot simulate the interactions of your production user base as they concurrently hit the resources of the applications system with a few synthetic agents. Scripts are fine for making sure that the application works in QA, they simply underwhelm the problem of figuring when real users are having real problems.

Bernd Harzog
CEO
APMExperts.com
www.apmexperts.com
bernd.harzog@apmexperts.com

Wily Unveils Management 360 and new Browser Response Time Adaptor

Wily News & Events

This release highlights an important new feature of the Wily product set, the Browser Response Time Adaptor. The Browswer Response Time Adaptor measures the performance of every user request as it hits the web server. This allows Wily to be unique in that it avoids using scripts and synthetic agents. Avoiding scripts is desirable since scripts are difficult to write (make work) and maintain as the application changes. Furthermore, scripts run via synthetic agents by their very nature are measuring the response time for the synthetic user, and not all of the real users.

The ability to correlate degradations in end user response time with true root cause analysis inside of the J2EE application, while also pinpointing when the problem is in the back end database, makes Wily the first and only APM vendor that can provide a true applications level performance measure (of every user) with realistic actionable root cause information at the applications layer. The combination of comprehensive end user response time measurement along with true root cause analysis into the J2EE application layer is why APMExperts rates Introscope as "Best in Class" for J2EE Applications Performance Management Products.

Bernd Harzog
CEO
APMExperts.com
www.apmexperts.com
bernd.harzog@apmexperts.com

VIEO Delivers End-to-End Citrix Management Solution

VIEO Delivers End to End Citrix Management Solution

Real Applications Performance Management is defined (by APMExperts) as the ability to know when any user of the applications system is having a performance problem, and to provide this information along with meaningful and actionable root cause analysis to the people who can address the issue.

True APM of Citrix and Microsoft Terminal Services environments is notoriously difficult (read why in the Analysis section of APMExperts.com. There are really only two groups of vendors who have even ever tried to do APM in the Citrix world. The first are the infrastructure vendors who confuse resource utilization with applications performance (performance is bad because CPU or context switches are too high). The second use scripts to measure performance at the ICA client, and fall prey to all of the issues associated with scripts (again read about why scripts are bad in the Analysis section of APMExperts.com

In this release Vieo announces that they have added support for Citrix to their existing support for web based applications. Vieo combines an appliance that can act as a Level 2 switch with agents on the servers. Therefore Vieo has the potential to combine deep analysis of network packets (and the performance information that can be gained as a result) with more server intimate information gathered by the server agents. If and when Vieo demonstrates that it can add true root cause analysis to its offerings in the same manner is Wily Technology, and can demonstate that it does not need scripts to gather end user response time information, this could end up being a winning solution for enterprises with significant Citrix installations.

The Unmet Customer Need For Real APM

Enterprises (large and small) collectively spend $7B a year for software and hardware to manage the performance of their systems. Well here is the ugly little secret. Precious little of the spending for "APM" products and services has to do with applications at all. Based upon numerous conversations with large enterprise organizations that are struggling with this issue, APMExperts believes that a real APM solution must incorporate the following elements:

1. It must measure the performance at the presentation layer of the application system. So, if an application system has 50 web servers as the front end, performance measurement must occur on each web server while covering ever user and every transaction. Sampling user experience with scripted synthetic agents is not sufficient.
2. It must collect detailed performance and utilization data from every layer of the application system (presentation. business logic and database) and it must synthesize this data with the performance information collected at the presentation layer.
3. It must be able to use applications level information in combination with infrastructure information (however collected) to be able to tell Systems Administrators if the problem is in the underlying infrastructure (network, server, OS, middleware), or in the application itself.
4. If the problem is in the application itself, it must be able to pinpoint the root cause of a performance degradation with a sufficient level of detail so that an application owner can go back to either the vendor or the in-house developer with proof that it is an application problem, and enough data to get the resolution process started.
5. In addition to gathering performance information across the application system, it must be able to verify the integrity of specific transactions (this is the valid role for scripts).
   It must support at least one of the major web based applications architectures (Windows/COM+/.Net or J2EE).

At APMExperts we help enterprises identify the applications performance tools that they really need, and we help vendors meet the real need in the market. So, our focus is upon the unmet need for real applications performance management on the part of enterprises and the strategy development for vendors who desire to meet those needs.

Bernd Harzog
CEO APMExperts.com
www.apmexperts.com
bernd.harzog@apmexperts.com

Application Performance Management Industry Dynamics

The Application Performance Industry is rife with rapid change. The big vendors are not standing still, and the small ones are trying to be nimble and fill in cracks as they emerge. Here are the key vendor dynamics driving the industry:

• Microsoft has lost control of the applications architecture and tools industry. More important business applications are being built today to the J2EE applications server platforms from BEA and IBM, than to the Windows platform. However, Microsoft is determined to gain that market share back with .Net and the CLR environment for C# and VB.Net.
• Due to the fact that a large number of business critical applications are being written to J2EE, the near term nexus of innovation in Systems and Application Performance Management has shifted from Windows to the J2EE platform. Therefore as .Net gains traction, it will be temporariliy underserved relative to J2EE in terms of real applications performance management solutions.
• With MOM 2005 Microsoft is entering the infrastructure management business on Windows servers with a vengence, and will likely move upstream into APM in future releases.
• Agentless monitoring is rapidly gaining traction in the infrastructure monitoring space. This is collapsing price points for infrastructure monitoring, and is wrecking havoc with the business models of vendors like BMC, CA, IBM/Tivoli and NetIQ that rely on the per server tax to grow their businesses.
• Appliances are the order of the day. Vendors like Vieo are able to package an astonishing array of fuctionality into an easy to implement and support package by delivering a software solution in a pre-configured hardware box.
• While there is much talk, and much in the way of point solutions offering real value and progress, the notion of application infrastructure as a plug-and-play utility remains as elusive now as it was when the idea was first surfaced 10 years ago.
• The "aircraft battle group" approach to systems and appliations management is dead. Applications environments within and across enterprises are so complex that no one "framework" can deal with and keep up with the complexity.
• Since no one vendor can meet the needs for real APM solutions across the range of environments and customer requirements, small, creative, and nimble vendors will be needed to meet the needs for real APM solutions. APMExperts was formed to help vendors and customers address the requirements for real APM solutions.

For more unbiased information about Application Performance Management, please visit APMExperts.

Bernd Harzog
CEO
APMExperts.com
bernd.harzog@apmexperts.com

Meta Trends Driving the Need for Real Applications Performance Management

Real Applications Performance Management (of applications, not just their infrastructure) is a growing problem for enterprises, and a growing opportunity for vendors for the following really big reasons:

• Microsoft and the Java camp (Sun, IBM, BEA, Oracle) are in an arms race to improve programmer productivity. This is what .Net and the CLR for managed code vs. the J2EE applications servers is all about.
• The first order consequence of this arms race are that developers are increasingly able to deliver more business applications functionality with less effort.
• The second order consequence is that control of and responsibility for applications performance and reliability has been wrested from the hands of the developers who have been losing those skills at the hands of high level languages for years anyway.
• All of which results in the "blame game" when things go wrong with applications in production. The Developers blame IS infrastructure groups and the platforms that support the applications. The IS groups blame the Developers and the users. The vendors all proclaim innocence and lack of issues in their products. And, enormous amount of time and money is wasted trying to find the root cause of applications performance issues.
• And it is not going to get any better. The really important war is for control of the applications infrastructure. The major vendors (Microsoft, Sun, IBM, BEA) wage this war by making it easier for developers to develop new applications. not by making those applications scalable and reliable in production. So, enterprises are going to have to continue to fill in cracks created by the "elephant war" with creative solutions from innovative vendors.

For more unbiased information on Applications Performance Management, visit APMExperts.com

Bernd Harzog
CEO
APMExperts
bernd@apmexperts.com