Application Performance Monitoring and Management – How It has Evolved and Where It is Headed

Application monitoring started off with a simple need - to help an administrator or manager of an application, monitor and manage its performance.  It was narrow in scope and limited in use - to monitor a single application and provide metrics useful for managing that application only. Monitoring tools were often provided by application vendors, but the growing and complex nature of IT environments necessitated the entry of third-party monitoring tools. These were more specialized, with the ability to centrally monitor several different applications. They helped administrators gain visibility across several different applications, understand where problems occurred, and helped to quickly resolve them.

Evolution: From Silos to IT-Centric Monitoring

Today, there is a proliferation of vendors who provide monitoring tools for each silo of IT specialization - including servers, network, operating systems, databases, CRM (customer relationship management) applications, ERP (enterprise resource planning) applications, and middleware.  This can be called an IT-centric approach to monitoring. 

However, there are several drawbacks to this approach. The proliferation of existing monitoring tools can create information overload as the environment gets complicated rather quickly, Very soon it becomes difficult to understand what issues are important and where to focus resources and energy, and organizations are therefore not able to make effective decisions based on the collected data.  Monitoring every piece of the infrastructure and every application also becomes expensive, especially for large IT organizations.  With IT organizations forced to reduce costs and do more with less, there is a constant pressure to deploy tools that can help them be more efficient. The introduction of newer technologies such as SOA, virtualization and cloud computing has only served to compound these challenges.

State of Application Performance Management: The Customer-Centric Approach

With the growing demand on stringent SLAs (service level agreements) and proliferation of composite applications, monitoring the end user response has started to draw more attention.  For example, many internet-based applications, such as online banking involve many different functions such as checking and money transfers, and all of these functions can involve many applications which reside on numerous types of infrastructure.  So, in order to manage these applications efficiently it becomes important to focus on the end user response.

However, monitoring the end user response introduces a new challenge for IT organizations: the need to reduce costs while meeting stringent SLAs. In fact, redundant infrastructure often hinders an organization from meeting service level requirements.  To go back to the online banking example, if a customer experiences a problem and calls the customer service call center for help, it will be very difficult for the call center to identify where the problem occurred. The many applications and infrastructure that are monitored can create a data overload. Very soon, this can lead to inefficiency and IT organizations may tend to forget the end goal of this exercise - ability to provide efficient service.  Also, the many monitors on non-critical applications and infrastructure can add to the IT cost overhead.

Thus, this new challenge has started a relatively new trend - the confluence of end user  (transaction) performance management with application/infrastructure management.  It is no longer sufficient for organizations to monitor only the infrastructure. In order to meet stringent SLAs, if there is a performance issue, IT organizations need the ability to quickly map that issue to the underlying application / infrastructure and quickly rectify it. In contrast to the IT-centric approach, this can be called the customer-centric approach.  This process starts with monitoring the composite application transactions end to end, and goes all the way down to the underlying applications and infrastructure behind it. By monitoring only the relevant infrastructure and applications, information overload and noise is reduced, and by integrating the end user response, IT organizations gain the ability to quickly identify and diagnose the performance issues. This approach can be taken one step further, through a business-oriented approach, which maps infrastructure performance to SLAs.  With the help of easy to create and use dashboards, businesses and IT can have a view of how they are performing against their SLAs and how the performance of the underlying infrastructure can have an impact on those SLAs.  This can help prioritize work in case of outages or SLA violations. With a constant pressure to reduce costs, the boundaries of the monitoring applications are getting blurred as they get more and more tightly integrated with other solutions. For example monitoring solutions can be integrated with service desk applications so that when an application fails, a ticket would automatically be created with accurate details.  Similarly, integration with other solutions such as Configuration and change management are on the rise.

Application Performance Management in the Future

Current economic conditions and tight IT budgets have made the advantages of the customer-centric approach very important. Consequently, this customer-centric approach may give rise to a new trend - tool consolidation. The more monitoring point products an IT organization has, the more difficult it becomes to adapt a customer-centric approach, and the more expensive it becomes to meet stringent SLAs due to complex integrations, training in multiple tools, and constant upgrades.  IT organizations with these characteristics may be better off with tools from fewer vendors and less integration.

For some IT organizations, the applications they monitor are so mission-critical that they need the ability to do proactive monitoring and avoid outages. A common practice is to provide a preset (static) threshold for infrastructure and application performance. If the threshold is exceeded, alerts and warnings are issued, and a lot of times, an impending outage can be avoided. There is also a new technology that is gaining popularity: predictive analytics. This includes predictive trending and dynamic thresholding, and allows the monitoring of resource consumption trends, anticipates future performance issues, and avoids or resolves problems more quickly. This can be used, for example, to predict application bottlenecks and create alerts for potential service threats.

By monitoring the most important customer-facing applications and tracking the performance of the infrastructure associated with that customer transaction, problems can be easily identified and solved.  As the customer-centric approach makes important data more accessible there will continue to be a trend towards predictive analytics.  This trend towards predictive analytics and the customer-centric focus will allow organizations to better manage their data, improving operational efficiency and reducing costs by eliminating the need for redundant infrastructure.