User Tools

Site Tools


observer_custom_alerts

Custom Alerts

Overview

Custom alerts allow you to define your own alerts to occur when ServiceNow events meet certain conditions. This provides you a way to monitor events that are not covered by the built-in alerts already included with Observer.

On each priority alert tab (high, medium and low), you will see its corresponding “Custom Alerts” section at the bottom of the screen:

Press the “+ New” and a row will appear so you can create your new custom alert:

Fields

1) Active : If this alert is active in the system (checked is active, unchecked is disabled).

2) Name : The name of the custom alert which is also the name of the event that will cause this alert to occur. This is the name that is sent in the “name” field of messages sent from your ServiceNow instance to Observer.

For example, if your ServiceNow instance has events called “Perspectium_Monitor” and you've set it up inside the Perspectium app (under Perspectium > Observer > Event Subscription) to send messages to Observer, then you can create a custom alert with this name to alert when certain conditions are met as described below.

If you try to enter a name for a custom alert that already exists, you will be prompted to enter a different name.

3) Description : A description for this custom alert. This is not used in the system but provides a way for you to have more detail on what and why this custom alert is.

4) Threshold Operator : A dropdown list that shows operators such as “>”, “<”, “>=”, “==”, etc. This is used with the valued entered for threshold to determine a minimum, maximum or specific event value in order for the alert to occur. For example if you have “>” here and “25” in threshold, the condition for this alert with be an event that has a value of greater than 25.

5) Threshold : The minimum/maximum value that must be met and/or exceeded for this custom alert to be triggered. This value must be a number value and is used in conjunction with the threshold operator value to determine how the threshold behaves (minimum, maximum or equals to).

6) Velocity Operator : A dropdown list that shows operators such as “>”, “<”, “>=”, “==”, etc. This is used with the valued entered for velocity to determine the velocity direction. For example if you have “>=” here and “5” in velocity, the condition for this alert with be a velocity of greater than or equal to 5.

7) Velocity : When an event is received, the rate of change per minute that must be met from the last time this event occurred for this alert to happen. This is used in conjunction with the velocity operator and must be a number value. For example, if you have the velocity condition of “>=” and “5”, then the alert requires that the event be increasing at a rate of greater than or equal to 5 per minute from the last time this event occurred.

Threshold and velocity conditions require both a value and operator be entered in order to be added as a condition to the alert.

8) Analysis Window : The time window in minutes to analyze if an alert will occur. Generally when an event occurs, it is logged in Observer and then if the same event occurs again in the time window specified here, then an alert will be created. This value must be a number value. For example, if the analysis window is “12”, then an event must occur twice in a 12 minute time window for an alert to occur.

Creating a Custom Alert

A custom alert only requires one condition of the three (threshold, velocity and analysis window) be entered but alerts will then require all conditions to be met if more than one is entered. For example, if you enter both threshold and velocity conditions, then they both must be met for an alert to be created. You could also just have a threshold value to trigger on a single datapoint.

After you have finished entering all the values for you new custom alert, press the “Save” button to save it into the system. The alert will then be created and display in the custom alerts section.

The above custom alert will be triggered if we get datapoints corresponding to the name “Custom_Alert” whose value is greater than 25, increasing by 5, within a 12 minute window.

Make sure to match up the name of the custom alert exactly to the message you are generating out of ServiceNow. Perspectium is also currently working on supporting a wildcard format for this as well.

Modifying a Custom Alert

Custom alerts that are already in the system can be edited, tested and deleted. To edit a custom alert, click into the value you want to edit. You will then notice that the field becomes editable, allowing you to make changes and after you press enter or leave the field, the changes will automatically save into the system.

In v3.13.0, custom alerts can be tested and deleted after they are saved by clicking on the play and trash icons, respectively, located under the Actions field.

After clicking the Test button, a window will pop up, indicating that the custom alert is being tested with a sample message. If successful, the sample message sent will be displayed along with the triggered priority alert.

Examples

Modifying Built in Alerts

There is currently a High Priority Alert for “Available DB Connections Exhausted”. This keys off the message we are already sending out of ServiceNow. For a single instance you can have a couple variations of the following:

Name Description
Available DB connections(glide) : statsx DB connections for one of the nodes
Available DB connections(glide.ts) : statsx DB connections for text indexing
Available DB connections(glide.fastlock) : statsx Internal Mutex, does not affect user experience
Available DB connections(glide) : acme001 DB connections for this node, acme001
Available DB connections(glide) : acme002 DB connections for this node, acme002

If you want to modify the Alert to only trigger on a subset of these you can de-activate the original alert and create a series of custom alerts. The original alert does not use a velocity or analysis window, so when we create our custom alert we can do the same. The following example will create an alert when the DB connections for this node (dev1845001) goes under 5 or when the DB connections for the text indexing is 0:

You can look at the messages being sent out of ServiceNow to match up the datapoint's name.

observer_custom_alerts.txt · Last modified: 2017/04/19 12:33 by jackson