Abstract

The use of aggregate notifications can be a powerful tool to provide an optimized display of information to operators as well as a mechanism for generically linking events together. This feature allows the users to see and interact with the Aggregate Notifications while keeping the original notifications just a click away in the originating server. This article describes the function of Aggregate Notifications, how to produce them using existing adapters, and possible reasons for their use.

Overview

The Service Assurance Manager (SAM), and its close cousin the SMART Adapter Platform, provide a Manager of Manager (MOM) function by taking events from other EMC|SMARTS Domain Managers as well as represent generic events from just about any source and display those to an operator. One of the basic troubles with this approach is that without any substantial controls on the flow of events from those generic sources into SAM, it can quickly overwhelm the ability of the system to present critical and pertinent information to the operators.

One powerful and often overlooked feature of the SMART Adapter Platform and SAM that can be used to solve that problem is the notion of aggregate notifications. This topic can be a little confusing for people that have been working with Domain Managers for many years as we had also used the notion of Aggregate or Compound events, but while both types of aggregates provide similar functionality they are actually quite different in implementation. The aggregate notifications are, in essence, a logical combination of other existing notifications and in SAM and the SMART Adapter Platform provide the following inherent features:

• They inherit the state and severity information for the notifications they aggregate. This means that the highest severity of the aggregated notifications will be the severity of the aggregate notification itself. Similarly, the aggregate will not clear itself until the notifications underneath it have all cleared.
• Only the aggregate itself flows to higher servers in the hierarchy.
• SAM provides an inherent ability to double-click on an aggregate notification and display the aggregated notifications – even if they are in a different server.
• Administrators can provide a uniform event message regardless of source while still preserving the original details of the incoming event.

This provides a powerful way to collapse disparate events into a single consolidated view while still maintaining (although remotely) the details of the original event. To help illustrate some common uses of the aggregation function, here are a few examples:

Consolidation of identical messages from multiple sources

One of the most common cases of aggregate usage is to unify messages coming from multiple sources. A typical example would be that for Cisco devices identical messages could be received through either Syslog processing or SNMP Traps. Both messages are really equivalent, but you cannot necessarily know if either or both of them will arrive for a given condition. At the same time, presenting two messages with the same meaning to the operators can be a drain on productivity. The solution is to create an aggregate notification that will collapse the two into a single message regardless of how many may be received. The syntax from the trap_mgr.conf is below:

# <event-aggregate>  Defines the aggregates. 
#                    Syntax is as below.
#                    Multiple entries can be defined.
#                    {
#                     EventName:   <string>
#                     ElementName: <object-handle>
#                     EventText:   <string>
#                    }
  

As long as the EventName and ElementName match the other sources, the notifications will be automatically collapsed into a single aggregate.

Consolidation of all raw notifications for a device

Similar to the case above, there may be so many raw notifications that it dilutes the quality of the RCA apparent to the operators. As an administrator, you may want to deal with all raw notifications as a single event that accurately conveys the highest state of all of the aggregated alarms. This allows the operators to focus on higher severity issues but also see all of the external alarms for a system in a single glance. By using the ElementName of the system ($SYS$) and a general EventName (System_Aggregate) this can be easily accomplished.

There is also no reason you could not define multiple aggregates for a given event in order to consolidate that information in multiple ways.

The best part of all is that this functionality is largely configuration driven for many of the adapters that communicate with the SMART Adapter Platform. The syslog, SNMP Trap adapters and even the sm_ems command support this functionality. Please refer to the EMC Smarts Service Assurance Manager Adapter Platform User’s Guide for specific instructions on the use of aggregate notifications.

About the Author

Bill Kuhhirte is an engineer in the EMC Smarts group.

Discuss

Click here to discuss this article