Experimenting with Remote Manager monitors (i.e. web hooks)

 

Introduction


This tutorial goes through the steps required to create a web hook that receives several types of information

  • It receives information about the status of all devices in your Remote Manager account

  • It receives alerts when devices in a particular group are offline for an extended period of time (5 minutes)

  • It receives any data points uploaded from all devices to the data stream metrics/wan/1/tx/bytes in order to keep track of the number of bytes sent out the wan.

You might want a solution like this so you can easily track your registered inventory and the state of it, including knowing when to debug a critical connectivity problem if a device loses connectivity.
 

Monitor Introduction


During both normal (uploading data from devices or device state changes) and abnormal (alarms triggered for device health) operation of Remote Manager resources, the platform generates events for a variety of scenarios.

Monitors effectively provide an event driven alternative to repeated queries with (possibly complex) conditions. Monitors also provide a way to receive events of many types from many devices. Fetching the data for all these devices and heterogeneous resource types may require a very large number of queries in order to retrieve all the data.
 

Push Monitors


A push monitor is an interface that a user can use to write device cloud applications that take advantage of event driven programming to watch for or "monitor" when these events occur. A push monitor involves a user’s HTTP server (or a client application that connects with TCP) to wait for events to be received. Push monitors can be "persistent", so that when the user’s server crashes or is unavailable, events that may have been missed by the monitor can be resent or replayed.

Performance is improved by using push monitors because the user doesn't have to poll for data and only those events which match the monitor "topic" will be delivered.

Create a push monitor using a post to the /ws/Monitor API.
 

Query Monitors


A polling or query monitor is an interface very similar to a push monitor. A query monitor will aggregate various types of monitor events into a single queue. Instead of the platform pushing the events to a target application or through an active TCP connection, the user can later query that queue in an efficient polling fashion, requesting only the “next” events from the queue, and easily skipping all events they’ve seen already.

Performance is improved by using query monitors because the caller doesn’t have call many APIs to get a variety of data from the system and the query monitor parameters allow continuing where the last call left off using a “polling cursor”. Query monitors can also reduce complexity because a server infrastructure doesn’t need to be maintained.

Create a query monitor using a post to the /ws/Monitor API, retrieve events from the monitor using the /ws/v1/monitors/history API.
 

Event types


There are a wide variety of events that occur and the events also indicate an operation (for example, create, update, delete of a resource)

  • devices event - When a device is added, removed or changes its properties (a modern version of DeviceCore)

  • alert_status events - Alerts are triggered, reset or acknowledged (a modern version of AlarmStatus)

  • DataPoint event - when a device uploads an individual data point

  • DataStream event - when operations on a data stream occur including changing the current value

  • DeviceCore event - When a device itself is created, updated or deleted

  • XbeeCore events - When an Xbee device is created, update or deleted

  • FileData and FileDataCore events - when files are manipulated on the platform

  • Job and JobResult events - when jobs on the system created, run, or are completed

  • AlarmStatus event - Alarms are triggered, reset or acknowledged

  • Others...






 



Alerting Introduction

In general, this tutorial isn’t about the alerting capabilities in the remote manager platform.

Suffice it to say that the remote manager platform has Alarms and Alerts. They are the same thing. The term “alarms” is used in many of the classic APIs and user interfaces, while “alerts” typically refer to the same feature in modern (/ws/v1) APIs and user interfaces. As of the writing of this tutorial, you can create alerts using the Remote Manager API explorer or the classic Remote Manager user interface. The modern user interface is being updated to fully support creating alerts.

In general, an alert definition is created to trigger on some condition. The alert fires when the condition occurs and stays in a fired state unless:

  • The alert is manually reset

  • The alert is configured to automatically reset upon some other condition

  • The alert is marked acknowledged by manual user action

The system may send notification emails for alerts, but that is not the topic for discussion in this tutorial.

Some of the types of alerts:

  • Device Offline - fires when a device is offline for a specific amount of time

  • Device Excessive Disconnects - Fires when the number of disconnects of a device reaches a threshold

  • DataPoint Condition - Fires when a data point is updated to a certain value or range

  • DataPoint On Change - Fires when the value of a data point changes from a previous value to a new value

  • Missing DataPoint - Fires when a data point isn’t updated in a timely fashion

  • Device Name On Change - Fires when the name of a device is changed

  • Others…


Alert UI - Classic


Until the modern Remote Manager user interface update for alerts is complete, you can see all of the available alert types and their properties by using the classic Remote Manager UI and navigating to Device Management → Alarms and using the Add alarm dialog. After adding an alarm in this way, you can call ws/v1/alerts/inventory to see the modern version of an alert that is represented by the alarm.
 

 

Receive Monitor Events

 

In our example, we’re going to use the Hookbin web tool at https://hookbin.com/ to capture and inspect monitor events sent from the Remote Manager platform.
 

Setup an Alert

 

We’re going to start by setting up an alert to watch for device offline events. When a device in our Remote Manager account group “Stores” disconnects, and stays disconnected for 5 minutes, we want an alert to fire so that we can be notified of the condition.

In the introduction to alerts section, we showed how to navigate to the classic user interface and create an alert. In this section, we’re going to use the API explorer to create this alert by POSTing a payload directly to the /ws/v1/alerts/inventory API.

The payload looks like this, and creates an alert named “Store Network offline” for any device in group “Stores”.

{
    "name":"Store Network offline",
    "type":"Device Offline",
    "description":"Detect if any store networks are offline",
    "priority":"high",
    "scope": {
        "type":"Group",
        "value":"Stores"
    },
    "fire": {
        "parameters": {
            "reconnectWindowDuration":"5"
        }
    }
}

In the following screen shot you can see we created an alert, with alert ID of 236592. We’ll use that in the monitor and identify an event sent with that alert ID later.

Setting up the Monitor

 

 

Receiver

 

Go to https://hookbin.com/ and create your own web hook. In the example that follows, I created a hook and it had a URL of https://hookb.in/lJNNXEmV3PsJBNooB0Vl.

You can test that hook using Postman or any stand alone tool that calls web APIs. Here is an example in cURL on the command line:


➜ curl -X POST -d '{"test":"42"}' https://hookb.in/lJNNXEmV3PsJBNooB0Vl
{"success":true}

 

Create the monitor

 

We create the monitor by posting a payload to the /ws/Monitor API. The monitor payload describes some required configuration features for the monitor.

For example, the payload we’ll use as follows:

<Monitor>
    <monTopic>devices,alert_status,DataPoint/*/metrics/wan/1/tx/bytes</monTopic>
    <monTransportType>http</monTransportType>
    <monTransportUrl>https://hookb.in/lJNNXEmV3PsJBNooB0Vl</monTransportUrl>
    <monTransportToken>username:password</monTransportToken>
    <monTransportHeaders>example-header: header-value</monTransportHeaders>
    <monTransportMethod>POST</monTransportMethod>
    <monFormatType>json</monFormatType>
    <monBatchSize>100</monBatchSize>
    <monBatchDuration>10</monBatchDuration>
    <monReplayOnConnect>true</monReplayOnConnect>
</Monitor>


Some of these parameters are obvious, others are not, I’ll briefly describe what they represent.

  • monTopic - Specific event topics from the monitor introduction section. Note that some events, like DataPoint take qualifies to get specific about what type of resource is being monitored.

  • monTransportType - We’re setting up an outgoing HTTP monitor, other options are “tcp” and “polling” for a query only monitor

  • monTransportToken - Is the basic auth required for this monitor

  • monTransportHeaders - Headers that can additionally be sent along with each request (useful for API keys or other types of authentication)

  • monTransportMethod - Post or put are used

  • monFormatType - What format should the payload be sent to the target in? Supported format types are json or xml

  • monBatchSize - Maximum of how many events should be batched together? Don’t recommend setting this to 1, your monitor may not be able to keep up in busy situations.

  • monBatchDuration - If the batch isn’t full, at what point should the data be sent anyway?

  • monReplayOnConnect - If your monitor is not up and can’t receive data, should data be kept so it can be re-sent? Note, all monitor data expires after 7 days. This is sometimes referred to as a “persistent monitor” because it ensures we persist all data that would have been destined to the monitor even if its down or there have been systemic problems with it.

    • NOTE: Any monitor that has monReplayOnConnect and is saving data, can have its history (events that were generated) retrieved using the ws/v1/monitors/history API until those events expires.


So, in the following screen shot you can see that we’ve created the monitor with ID 1356702. We’ll use that later in other API calls.
 


 

 

Watch the monitor


At this point, we can go back to hook bin and look at what events we might be receiving. We will immediately start to receive some events.

  • Any time the device status is updated (it connects, disconnects, is moved to a new group, gets a new name, etc, etc)

  • Any time it uploads a data point for stream metrics/wan/1/tx/bytes (my test device is set to upload every 5 minutes)

  • Any time any alert status changes

The first message we receive at hook is shown below.
 

Monitor Payload: initial


We get a Basic authorization and example-header HTTP header matching what we’ve setup in our monitor. In this post, we’ve received a single monitor event as indicated by the monitor-aggregate-count header.

The payload is as follows. This is an alert status event showing that the monitor was connected. The alert_status event has an id of 192163 which is a different alert than the one we created so we don’t have to pay attention to it.

Some fields of the payload:

  • Monitor payloads have a Document and Msg element where the Msg element is either an individual event or a list of events (in json, “Msg” is either an object or an array of objects)

  • topic - Depending on the event type, the topic represents routing/summary information about the payload.

  • group, operation, timestamp - More information about when and why the individual event was generated

  • alert_status - This element will be named depending on what the data type is. In payloads in this tutorial, you’ll see this element being a “device”, a “DataPoint” or the “alert_status” because those are the topics used when we created the monitor.

{
  "Document": {
    "Msg": {
      "topic": "57639/alert_status/192163/Monitor%3A1356700",
      "group": "*",
      "operation": "INSERTION",
      "timestamp": "2021-11-18T18:22:55.604Z",
      "alert_status": {
        "customer_id": 57639,
        "enabled": true,
        "id": 192163,
        "last_update": "2021-11-18T18:22:55.535Z",
        "maintenance_mode": "off",
        "notes": "Monitor connected",
        "severity": "high",
        "source": "Monitor:1356700",
        "status": "reset"
      }
    }
  }
}

The screen shot of the first event from hookbin is shown here:


 

Monitor Payload: device and DataPoint


When the device updates its metric data, the monitor receives another payload. Note in this payload, there are 2 events in the monitor payload, so the “monitor-aggregate-count” HTTP header is 2.

Also note that the “Msg” element in the payload is a list, containing each of the two types of events, a “DataPoint” and the “device” payload with the device being updated.

Of special interest in this payload:

  • The data point is being updated from a previous value of 39558 to 41358 bytes (The streamUnits field indicates bytes)

  • That this device being updated is in the “Stores” group, so that the handler of the event can take some different action depending on that information.
     


Monitor Payload: Device Offline alert_status

 

After disconnecting a device in the stores group, and waiting five minutes, the monitor receives another payload.

There are a couple of things going on in this payload. Since we have the monitor batch size set to 100 and the batch duration set to 10 seconds, we get to see both the device event and the alert at the same time.

Some interesting things that caused this payload:

  • You can see by the device “last_disconnect” field, that the device disconnected at 18:55.

  • You can see by the alert_status event timestamp field that the alert was fired at 19:00 (5 minutes later because of the alert configuration)

  • The device is only included in this payload because at 19:00, the alert firing caused the number of alerts associated with the device to be bumped from 1 to 2 (refer back to a prior device payload where it was 1). That change caused a device status to occur so the “device” event was included in this payload too.


 

Clean up

 


We should be sure to delete the monitor and the alert so that we don’t create noise in our account or run up usage.
 

Deleting the alert

 


Deleting the monitor

 


 




Further Reading:

Digi RM User Guide - Monitor - Monitor web service description and examples

Digi RM User Guide - v1/monitors - use v1/monitors API to retrieve saved push notifications

Have fun!!!

 

Last updated: Jan 27, 2022

Filed Under

Digi Remote Manager

Recently Viewed

No recently viewed articles

Did you find this article helpful?