Measuring What Matters

Enterprises (large and small) collectively spend $7B a year for software and hardware to manage the performance of their systems. Well here is the ugly little secret. Precious little of the spending for management products and services has to do with what really matters - ensuring that the users of these systems can do their jobs. These systems for the most part measure things that are important to the technical team supporting a part of the system (the network, the servers, the J2EE layer, the database, etc.). These systems for the most part do not measure what really matters - the experience of the user trying to do his or her job. Measuring what matters means measuring:

• The performance at the presentation layer of the application system. So, if an application system has 50 web servers as the front end, performance measurement must occur on each web server and cover every user and every transaction. Sampling user experience with scripted synthetic agents is not sufficient.
• Detailed performance and utilization data from every layer of the application system (presentation. Business logic and database) and it must synthesize this data with the performance information collected at the presentation layer.
• Combining applications level information with infrastructure information (however collected) to be able to tell Systems Administrators if the problem is in the underlying infrastructure (network, server, OS, middleware), or in the application itself.
• Pinpointing the root cause of a performance degradation with a sufficient level of detail so that an application owner can go back to either the vendor or the in-house developer with proof that it is an application problem, and enough data to get the resolution process started.
• In addition to gathering performance information across the application system, it must be able to verify the integrity of specific transactions (this is the valid role for scripts).
  It must support at least one of the major web based applications architectures (Windows/COM+/.Net or J2EE).

We help enterprises identify the end user experience tools that they really need, and we help vendors meet the real need in the market. So, our focus is upon the unmet need for real end user experience management on the part of enterprises and the strategy development for vendors who desire to meet those needs.