Home > Blog > Thunderheads

Archive for the ‘Thunderheads’ Category

Thunderheads: March 2012

Supporting the Cloud
— Meet Michael Toenies —

Hello. My name is Michael Toenies, and I’m with the Technical Support team at Digi International®. I’ve been providing Technical Support to the IT industry for over 18 years, the past 13 of them as a member of the Digi® family. Since 2008, my main focus has been supporting the exciting iDigi® Device Cloud™ and Drop-In Networking product lines. I’m a huge fan of the iDigi Thunderheads and the audience that reads it, so I wanted to contribute an article that gives you some tools you can use to ensure your devices get iDigi connected and you too can be part of the Internet of ANYthing™.


Getting iDigi Connected
— A Troubleshooters Guide —

Meet the iDigi Servers
We won’t be able to get iDigi Connected if we don’t know what we’re connecting to. I’d like to help you get started by first of all discussing our iDigi Device Cloud server. The iDigi Device Cloud is our premier subscription-based cloud computing platform. It has been designed to meet the toughest standards of the device networking and management industry. At a network level, the server goes by several names:

my.idigi.com (login)
device.idigi.com (login)
energy.idigi.com (login)
app.idigi.com (login)

All of these resolve to the same nslookup address, which is something we’ll go a bit deeper into later. The iDigi Device Cloud also has the developer sandbox, so you can try out our state of the art iDigi API and the powerful iDigi Manager Pro™ device management interface free of charge for up to 5 devices. The sandbox has a name of its own:

developer.idigi.com (login)

Why talk about these network names? Well, we can’t really introduce you to our servers properly if you don’t know their names right? Actually, this will be an important piece of information we’ll talk about later. You will also need to know this if your device is in a disconnected state and you engage with Digi Technical Support.

Common Issues
Since my focus in Digi Tech Support is helping our customers get iDigi connected, at some time or another I’ll pretty much see any issue a customer using our equipment or services can run into. Today I’d like to discuss some of the more common issues at a high level, but also link you to some resources you can use as a shovel if you want to dig deeper and get your hands dirty fixing an iDigi connectivity issue yourself.

“Common issues” are really just indicators we have to tell us that something is preventing the device from communicating with the iDigi Device Cloud. For example, when using a ConnectPort® X2e for Smart Energy gateway or Digi’s ERT/Ethernet gateway with your Smartlee™ account you might see this:



While over on iDigi Manager Pro, you might see this:



Or even the dreaded:



Yes indeed, these error messages mean we have a problem. But it also means we are now aware that there is a problem. The next step is to find out what caused this dreadful state of iDigi non-connectedness and fix it!

Common Causes
If you have a cellular device and are seeing one of the errors above, the first thing to check would be that you’ve configured your gateway properly to connect to the cellular network. This can be done by pointing a web browser at the IP address your gateway is using on the Local Area Network (LAN) to check the cellular configuration using the Web User Interface (Web UI):



Judging by the red ink on that screen, I’d say this device needs to be provisioned, wouldn’t you? There’s a button right there on the screen which will help you provision the gateway for your chosen cellular provider. To make the configured settings on that page “live,” use the “Apply” button on the page once the changes have been made and you should be all set. Take note of the iDigi link under the Configuration menu on the left of the screen. This is where you configure which iDigi server you want to connect to. Do you remember their names?

Here at Digi we really want our products to be as plug-n-play as possible so you’ll always have a “WOW” experience with them. Unfortunately, the plug-n-play focus that will work for 98% of our customers also conceals the fact that certain key pieces of information are required in order for a Digi gateway to successfully get back to the iDigi Device Cloud. If those blanks aren’t filled in during a plug-n-play install, you’ll need to know which pieces of information are still missing, as well as how to configure those pieces manually. However, not all of our products have a Web UI. In a moment we’ll visit the Command Line Interface (CLI) of a Digi RF gateway and do our configuration and troubleshooting from there.

When you connect an Ethernet or Wi-Fi connected Digi gateway to your LAN, it first tries to acquire an IP address from a router with enabled DHCP server on your network. If successful at acquiring the IP address, it pulls in the default gateway and DNS server addresses at the same time if possible. The default gateway address is what tells the gateway where the exit door on your LAN is (i.e. how to get to the Internet of Things and the iDigi Device Cloud). The DNS address is where the name of the iDigi Cloud server is paired with an IP address. Think of DNS as a giant phone book, which knows the number (IP address) of the iDigi Device Cloud (developer.idigi.com, my.idigi.com, etc.) so that your device can phone home. If you’re unsure of what DNS server address to use, Google’s primary and secondary OpenDNS servers (8.8.8.8 and 8.8.8.4 respectively) know how to get back to the iDigi Device Cloud.

If you don’t have a router on your network or the DHCP server is turned off, you’ll have to configure the required information manually using the Command Line Interface (CLI). You can access the CLI via telnet or ssh session to the IP address of your product. If you don’t know the IP address, don’t worry, you can discover it using the Device Discovery Tool for Windows.

The CLI commands “set network ?” and “set mgmtconn ?” will show you the command options and syntax to configure a static IP address, default gateway and DNS server address manually, while the CLI commands “ping” and “set trace ?” are very useful for troubleshooting. For more info, we have a Knowledge Base article which has screenshots of the CLI commands and a step by step walkthrough to configure and troubleshoot an iDigi connection from the CLI.

The ConnectPort X2e for Smart Energy has neither a CLI nor Web UI enabled by default. However, if you push the little button on the corner of the faceplate, a temporary Web UI will be enabled which can then be used for network and iDigi configuration. After the button is pushed, the Device Discovery Tool for Windows can be used to find the ConnectPort X2e SE. If you select the “Open web interface” option, you’ll see the temporary Web UI, which looks like this:




The screen above is the homepage, which basically tells us what information the ConnectPort X2e SE learned when it was plugged into your LAN and how far the connection into the iDigi Device Cloud progressed. If it appears like the screenshot above, with the Network Connectivity Status green and all boxes checked, you’re most likely already connected.

Any unchecked boxes on the homepage indicate which piece(s) of required information are still missing from the configuration and will therefore need to be configured manually. To configure an IP address, default gateway or DNS server address on a ConnectPort X2e SE, click the Configuration ⇒ Network link of the ConnectPort X2e SE’s Web UI and access the Network Configuration screen:



“Server Address” is the very last field on the Network Configuration page. This is where knowing our iDigi server names comes in handy again. The device in the screenshot above is configured to talk to the iDigi Device Cloud (my.idigi.com). If, for example, you wanted to reconfigure this ConnectPort X2e SE to connect to the iDigi Developer Cloud you could change the Server Address field to developer.idigi.com, then press “Apply.” Lastly, when changes to the Network Configuration are made, click “Reboot.”

If you’ve made sure your Digi device has an IP address, default gateway and primary DNS server address, you should now be connected to the iDigi Device Cloud.

Firewall Concerns
If you’re still seeing a “disconnected” status at this point, you may need to talk to the Network Administrator (or whoever setup the LAN for you), because the last common cause of a failed connection into the iDigi Device Cloud is that your network firewall isn’t allowing the outbound connection to the server. To create a firewall rule which will allow this traffic, your Network Administrator would want to do a nslookup of the iDigi Device Cloud servername you’re trying to connect to (e.g. my.idigi.com), then allow outbound socket connections to that IP address on TCP ports 3197 and 3199 (the TCP ports iDigi talks on). You’ll also want them to open UDP port 123 (NTP time-server access) as gateways typically require access to an NTP server as well.

Resolution
I hope the preceding article helped get your device iDigi connected, or at least provided you with some valuable information should you ever need to troubleshoot your connection into the iDigi Device Cloud. Digi Technical Support stands ready around the clock to assist you should you need a helping hand or your technical questions answered.


On the Blog
— Projects, News & Notes —

Remotely Manage Digi Cellular Routers Using iDigi Manager Pro
— Corey Plender, iDigi Solutions Architect, shows how easy it is to use iDigi Manager Pro to manage cellular gateways and routers.

More >




XBee Resources — The Tymkrs (Toymaker Television) have focused on working with XBee® modules over the last month. They are constantly providing great XBee information that we’d love to share. More >




XBee Project Gallery — So many of you are using XBee modules to create amazing things, that we’ve created a place to feature your work. Musical shoes, digital dominoes, interactive sculptures and autonomous penguins can all be found at the largest collection of XBee projects on the Web. More >

You can read more about the iDigi platform,
Digi projects and technology at the iDigi blog
or follow us on Twitter, Facebook and YouTube.

      

Share Your Projects & Ideas — There are lots of great applications that use the iDigi Device Cloud and we want to hear about yours. Contact the team at thunderheads@digi.com to share us your ideas and stories.

 

<

Thunderheads: February 2012

Projects from the Mind of Mark

Mark Geller’s Arduino Project
Part Four

In my previous articles I showed you how to configure an Arduino project to communicate with a ConnectPort® X gateway, how to send serial commands via iDigi® web services, how to build a mobile web site that uses iDigi web services and how to build a native iOS application that uses iDigi web services. For this installment, we are going to look at how we can upgrade our existing Arduino sketch so that it reports which lights are on and how to write a custom iDigi® Dia driver in Python. If you haven’t had a chance to read the previous articles, please take a moment to check them out on the iDigi blog.

Parts List

Project Requirements

There were two shortcomings with the original project.

First, there wasn’t a way to see which LEDs on the Arduino project were on without looking at the hardware. It would be nice if we could send a command that would report back the state of the LEDs or if the current state of the LEDs was reported back whenever it changed.

Second, it would be nice if we could have a central place to track when the LEDs turn on or off.Here is a list of requirements for this update that will address these shortcomings.

  • Send the current status of all the LEDs if any of the LEDs turns on or off.
  • Create a simple command that will report the current state of all the LEDs.
  • Upload the serial data from the Arduino project to the iDigi Device Cloud.

From these requirements, it is clear that we need to update the Arduino sketch. And although the iDigi Dia XBee serial terminal driver already handles reading and writing serial data from XBee modules, we are going to want to create our own iDigi Dia device driver so we can precisely control how our project data is uploaded to the iDigi® Device Cloud™.

Updating the Arduino Sketch

Step 1

We are going to start by updating the switchCase2 example Arduino sketch. First we are going to add some variables to keep track of each LED’s state. Add the following code above the setup() method:

int firstLight = 0;
int secondLight = 0;
int thirdLight = 0;
int fourthLight = 0;
int fifthLight = 0;

Step 2

Now we need to update the switch statement in the loop() method so that the variables we added in step 1 are updated when the LEDs are turned off or on. We are also going to add a command that will report back the state of the LEDs without changing them. Here are the changes to the switch statement:

switch(inByte) {
  case 'a':
    digitalWrite(2, HIGH);
    firstLight = 1;
    break;
  case 'b':
    digitalWrite(3, HIGH);
    secondLight = 1;
    break;
  case 'c':
    digitalWrite(4, HIGH);
    thirdLight = 1;
    break;
  case 'd':
    digitalWrite(5, HIGH);
    fourthLight = 1;
    break;
  case 'e':
    digitalWrite(6, HIGH);
    fifthLight = 1;
    break;
  case 'r':
    break;
  default:
    // turn all the LEDs off;
    for (int thisPin = 2; thisPin < 7; thisPin++) {
      digitalWrite(thisPin, LOW);
    }
    firstLight = 0;
    secondLight = 0;
    thirdLight = 0;
    fourthLight = 0;
    fifthLight = 0;
}

Step 3

Now we need a way to report back the current state of each LED. All LEDs that are on should report a value of 1 while all the LEDs that are off should report a value of 0. We also want to add an end of line character so that any program reading this datacan see where each statement ends. To keep things simple, we are going to write ascii data to the serial port. Add this code below the switch statement:

Serial.print(firstLight);
Serial.print(secondLight);
Serial.print(thirdLight);
Serial.print(fourthLight);
Serial.print(fifthLight);
Serial.print(":");

Step 4: Testing

Now is a good time to test our Arduino sketch to make sure everything is working. Click the “Upload” button in the Arduino IDE to compile and upload your sketch to your Arduino board.

NOTE: Don’t forget to disconnect the transmit and receive wires from the XBee module on your Arduino board before you try to upload your sketch.Click the “Serial Monitor” button in the Arduino IDE to open up a serial terminal session. Enter the letter ‘a’ into the text box and click the send button. You should see 10000: printed in the terminal window and you should see the first LED turn on. Now we are going to verify that sending the letter ‘r’ will report back the state of all the LEDs without turning any on or off. Enter the letter ‘r’ into the text box, click the send button and verify that 10000: was printed in the terminal window.

Now we are going to try a more complicated command. Enter ‘fabc’ into the text box and click the send button. You should see 00000:10000:11000:11100: printed to the terminal window. As the Arduino sketch parses each letter in the serial command, it will write the current state to the serial port. We are going to want to account for this behavior in our custom iDigi Dia device driver and make sure we only report the final sequence to the iDigi Device Cloud.

We are done writing the Arduino sketch. Don’t forget to reconnect the transmit and receive wires from the XBee module to your Arduino board.

Creating an iDigi Dia Device Driver

We are going to use the iDigi Dia framework to handle sending data to the Arduino board. The iDigi Dia framework was built to make it easy to communicate and configure XBee devices and to take advantage of the iDigi Device Cloud. We are going to create a new device driver for our Arduino board based off the XBee serial terminal device driver.

Step 5: Create a new iDigi Dia project

Start the Digi ESP™ for Python development environment. If you don’t have Digi ESP installed, you can download it from the Digi web site. Select “File > New > iDigi Dia Project” from the menu to create a new project. Use the project wizard to create your project. Click the help icon in the lower left corner of the project wizard window to view the help documentation.

Step 6

Right click or command click on the “custom_devices” folder in your project and select “New > Python Module” to create a new module. I named my module “xbee_arduino”.

 

 

Step 7

Open the “xbee_serial_terminal.py” module. You can find it in the src/devices/xbee/xbee_devices/ package. Copy all the code from this file into your xbee_arduino file.

 

Step 8

We are going to store incoming serial data from the XBee module in a buffer before processing it. We are going to specify the maximum buffer size as a module constant. Add the following code just below the import and from statements at the top of the file:

 

MAX_RX_BUFFER = 512

 

 

Step 9

Change the class name from “XBeeSerialTerminal” to “XBeeArduino”.

Step 10

We need to create a local variable to hold our receive buffer. Add the following code in the __init__ method just below the self.__xbee_manager = None line:

        # the buffer that holds the serial data before parsing
        self.__rx_buffer = ""

 

 

Step 11

The XBee serial terminal driver has 2 data channels: read and write. We are going to change the name of the read channel to “lights” and give it a default value of “00000″. Change the first ChannelSourceDeviceProperty so that it matches the code below:

ChannelSourceDeviceProperty(name="lights", type=str,
    initial=Sample(timestamp=0, unit="", value="00000"),
    perms_mask=DPROP_PERM_GET, options=DPROP_OPT_AUTOTIMESTAMP),

 

 


Step 12

Our custom driver extends the base XBee serial device driver. The XBee serial device driver handles configuring and communicating with the XBee module, but it does require that you implement the following methods in your driver:

  • read_callback — This method is called when the XBee module receives a message.
  • apply_settings — This method is called during setup. It allows you to read and apply settings from your project yml file.
  • start — This method is called when the device driver starts. All the settings for the driver are available at this point so this is where you register your driver with the XBee device manager and run any initialization code, such as starting threads.
  • stop — This method is called when your device driver is stopped.

Luckily the code we copied from the XBee serial terminal driver implements all these methods for us. It even has basic implementations for these functions:

  • serial_read — This method is called when serial data is received.
  • serial_write — This method is called when a new value is set for the “write” channel on our driver. It tells the gateway to transmit the data in the “write” channel to the remote XBee device.

We need to make a small change to the “serial_read” method. Replace that method with the code below:

    def serial_read(self, buf):
        self.__tracer.debug("Read Data: %s", buf)
        # Update channel
        sample = self.__parse_data(buf)

        self.property_set("lights", Sample(0, value=sample, unit=""))

 

 

This code sends the serial data to a private method called __parse_data that is going to parse the serial data and return a value that we then write to the “lights” channel.

Step 13

We now need to add the method that will parse our serial data:

    def __parse_data(self, buf):
        #append data to the buffer
        #create a var to store the samples
        sample = ""
        self.__rx_buffer += buf
        # check for overflow
        if len(self.__rx_buffer) > MAX_RX_BUFFER:
            try:
                self.__rx_buffer = (
                        self.__rx_buffer[self.__rx_buffer.rindex(':') + 1:])
            except:
                #Throw away everything and only include the latest receive
                self.__rx_buffer = buf

        # advance through the buffer and read the good samples. The while loop will
        # only return the the last good sample since this Arduino will send the status
        # info for every byte it is sent
        while 1:
            eos = self.__rx_buffer.find(':')
            # only read the statements if they end in ':'
            if eos == -1:
                break
            # get a copy of the sentence
            sentence = self.__rx_buffer[:eos]
            #remove the sentence from the receive buffer
            self.__rx_buffer = self.__rx_buffer[eos+1:]
            #check to make sure sentence is the correct length
            if len(sentence) != 5:
                # sentence is too short, skip it
                continue
            sample = sentence

        return sample

 

 

This method does quite a bit of work. First it tries to append the message to the receive buffer. If the message is too large, it tries to find the last complete reading and throws away the rest of the buffer. It then cycles through the buffer and finds the last complete message (ends with “:”) that was 5 characters long and returns it.

Step 14

Now we need to update our dia.yml file to use our new driver. Double-click your dia.yml file to open it. Click the “Source” tab at the bottom of the window to switch from the Graphic editor to the source view.

Step 15

Add an entry for your Arduino device driver just below the xbee_device_manager entry:

  - name: arduino
    driver: custom_devices.xbee_arduino:XBeeArduino
    settings:
        xbee_device_manager: "xbee_device_manager"
        extended_address: "00:13:a2:00:40:76:35:2a!"

Remember to replace the extended address listed above with the extended address of your XBee module.

Step 16

Compile and upload your project to your gateway. I usually select my project in the project explorer, click the build button and manually upload my project and reboot my gateway, but you can click the “Play” button in the toolbar to automatically deploy your code to your gateway.

Step 17: Testing

Now we are going to use the iDigi Dia command line interface to make sure everything is working properly. Use your favorite telnet client and connect to port 4146 on your gateway. If you are using a Mac, just open up terminal and type “telnet (IP ADDRESS) 4146″ where (IP Address) is the local IP address of your gateway. You should see the following message:

Welcome to the iDigi Device Integration Application CLI.
=>>

 

 

First lets check to see if our channels are properly configured and are showing the correct default values. Type in “channel_dump” and hit enter to get a list of your device driver channels. If everything is working correctly, you should see a value of “00000″ for your lights channel.

Now lets send a command. Type the following command at the prompt and press enter to set the value of your write channel:

channel_set arduino.write abc

Now send a second channel_dump command. You should see that the value of your lights channel has changed to “11100″. If you got the correct value, then congratulations! You have just written and deployed a custom iDigi Dia device driver.


Summary

In this article we have seen how to create a custom iDigi Dia device driver to handle reading data from and sending data to remote serial devices. Because we used the iDigi Dia framework, we can use iDigi web services to send commands to the remote device and read any serial data we receive.


On the Blog

— Projects, News & Notes —

48 Hour Day
Art Hack Day brings together technologists, artists, innovators and programmers to produce a series of works, part tech and part art, for a one night only exhibition. You can check out all of the projects at Art Hack Day. More >

Extending the Internet of ANYthing™ — The Iridium satellite network now supports the iDigi Device Cloud, allowing Digi devices with an Iridium data transceiver to send and receive data via the iDigi Device Cloud over the Iridium network. This capability extends connectivity to the remote corners of the globe faster and easier than ever before. More >

You can read more about the iDigi platform,
Digi projects and technology at the iDigi blog
or follow us on Twitter, Facebook and YouTube.
      

Share Your Projects & Ideas — There are lots of great applications that use the iDigi Device Cloud and we want to hear about yours. Contact the team at thunderheads@digi.com to share us your ideas and stories.

Thunderheads: January 2012

Posted on: No Comments
Powell Project Log
— Graphing Dia Sensor Data —

Most iDigi® users build robust websites to access sensor data from the iDigi® Device Cloud™, either themselves using in-house development staff or utilizing the iDigi Applications Team. Others are just learning about the iDigi platform and M2M technology, or want to hack something up to graph data while that robust website is being developed. When I want to quickly build a website with graphs from Dia data I use RRDTool.

About the Author —

John Powell is an iDigi Solutions Architect. He has had a long career in various parts of the computing industry, both with manufacturers like Digi International®, as well as in large enterprise IT groups.

RRDTool is a free, open source and powerful tool that is awesome for time-series data like sensor graphing. It is both a database and a graph creation tool. Graphs can be simple, or very complex. Whether you know it or not, you have probably seen many graphs that were created using RRDTool as it has been a mainstay graphing tool for more than a decade. Check out the links:

RRDTool Gallery — This page shows some really cool graphs people have created using RRDTool.

In-home Monitoring — This is a website I created to monitor the temperature and electricity usage at my vacation home in the Rocky Mountains.

I highly recommend reviewing the online RRDTutorial to gain an understanding of how it works and the concepts involved.

In this article I am going to show you how to pull down your sensor data from a Dia application from the iDigi Device Cloud, parse the XML, update the RRDTool database and create a simple graph. I will be doing this in Linux, so you will need to be at least competent in Linux, with basic knowledge of how to setup a Linux server with a web server, basic shell scripting and basic HTML creation. You do NOT have to be a programmer (other than basic shell scripting skills).

Note: RRDTool does have a Windows build and Powershell has great XML facilities, so this could be easily accomplished in Windows too. If you are a Windows Systems Admin you can probably get a good idea of what to do by reading this article. I may do a follow up article on how to do this with Windows Powershell.

I am going to keep this simple and not make it a full tutorial on RRDTool, the iDigi Device Cloud or Linux. There is plenty of documentation, including some great tutorials on the RRDTool website, as well as a wealth of examples on the web that can be found using your favorite search engine. Hopefully this will give you a jumpstart with basic graphing and you can go from there.

To start, here are the tools we are going to use:

  • A Linux server — I use a Rackspace cloud server based on Ubuntu server
  • A web server — I use Apache, but feel free to use your preferred web server; the web server needs are basic
  • Cron to run the update script on a regular basis
  • Wget — I use this as a simple “web services client.” Basically we will use it to pull the XML data
  • XMLstarlet — A command line utility to inspect and parse data from an XML file
  • Basic GNU Linux tools like “date” and “sed”
  • RRDTool — This is both a database and graph creation tool

Note: I will be using BASH as my shell; you may need to adjust the syntax if you use a different shell.

Pulling the Data
Step one is to pull the data down from the iDigi Device Cloud. I am going to use wget to do this, as RESTful web services are really just URLs and wget is good at this. There are other tools like “curl” that will do the job as well.

Here is a simple wget command line (all on one line, remove the ”):

/usr/bin/wget --user=youruserid --password=yourpassword 
https://developer.idigi.com/ws/DiaChannelDataFull

This will return an XML file “./DiaChannelDataFull”. This XML file has all the latest sensor readings for all the devices associated with your iDigi developer account.

Parsing a Sensor Measurement from the XML File

Here we use a tool called XMLstarlet that should be available in the package repository for most Linux distributions. XMLstarlet is a simple command line tool to parse an XML file.

Tip: You can inspect the XML file structure using ‘xmlstarlet el DiaChannelDataFull’

Here is a simple xmlstarlet command line to pull the temperature from sensor0.temperature (all on one line, remove the ”s):

xmlstarlet sel -t -m 
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' 
and ddInstanceName='sensor0' and dcChannelName='temperature']" 
-v ../dcdFloatValue DiaChannelDataFull

You will need to correct the ConnectwareID to match your device. You can get this by inspecting the XML file with less or cat. This line should return the current temperature from sensor0.temperature (adjust as necessary for your situation).

These two lines can go in your bash script to pull out the temperature ($sensorvalue) and timestamp ($iso8601time) and assign them to variables for use later in the script:

sensorvalue=`xmlstarlet sel -t -m 
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' 
and ddInstanceName='sensor0' and dcChannelName='temperature']" 
-v ../ dcdFloatValue DiaChannelDataFull`

iso8601time=`xmlstarlet sel -t -m 
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' 
and ddInstanceName='sensor0' and dcChannelName='temperature']" 
-v ../ dcdUpdateTime DiaChannelDataFull`

Converting Timestamp for Use in RRDTool
iDigi returns timestamps in ISO 8601 format. For example: 2011-11-27T01:09:56Z. RRDTool wants timestamps in epoch time, which is the number of seconds since Midnight 1/1/1990 GMT. There was no easy way I could find to convert ISO time to epoch time. I came up with the slightly convoluted method of using sed to convert to a format the “date” command can use, then used date to convert to epoch time.

# Step 1 is to convert the ISO timestamp to something the Linux date command can read.
# The result is changing "2011-11-27T01:09:56Z" to "2011-11-27 01:09:56 +0000"
datecompatibletime=`echo $iso8601time | sed s/T/' '/ | sed s/Z/' +0000'/`

# The next line uses the date command to convert to epoch time. 
#The result is changing "2011-11-27 01:09:56 +0000" to "1322356196"
epochtime=`date +%s -d "$datecompatibletime"`

The result is that variable $epochtime contains the timestamp in Unix epoch time.

Creating a Database Using RRDTool
Now we have our temperature ($sensorvalue) and a timestamp compatible with RRDTool ($epochtime) so we are almost ready to update the rrd database with those values.

First, we need to create the database. This is a one-time step. You will likely want to refer to the tutorials and documentation on the RRDTool website for all the options here, but for now I am going to create a database that keeps 5 minute values for 1 day, 15 minute averages for 1 week and 1 hour averages for 1 year.

Here is a sample command line to create the database. Remember, this is a one-time operation, so it should not be part of your cron script!

rrdtool create sensor0.temperature.rrd --start "now - 8h" --step=300 
DS:tempC:GAUGE:1000:U:U 
RRA:AVERAGE:0.5:1:288 
RRA:AVERAGE:0.5:3:672 
RRA:AVERAGE:0.5:12:8760

This will create a file called sensor0.temperature.rrd. This is the RRD database. Note that the initial database is filled with null values, so it will never grow in size. Do not worry about it filling up your filesystem over time!

Updating the RRDTool Database with Sensor Values
Now that we have our database created, we are ready to update it. This line is very simple. It is simply updating the database we created with the latest values, using the variables we created earlier in our script:

rrdtool update sensor0.temperature.rrd $epochtime:$sensorvalue

Creating a Simple Graph
Now it’s time for the fun part, creating the graph. Again, we are going to keep it simple. Refer to the RRDTool doc and samples on the web for more complex graphing help.

Note: You will need to have a few updates in the database before the graph will be able to show anything useful.

This command line will create a graph in /var/www/ (adjust as necessary for your webserver’s document path) with the temperature over the last 2 days.

/usr/bin/rrdtool graph /var/www/graphs/temp_last2days.png 
-t "Temperature - Last 2 Days" 
-v "Degress Celsius"  -F --start -2d  --end now 
DEF:temp=sensor0.temperature.rrd:tempC:AVERAGE 
LINE2:temp#FF0000:"Temperature"

You can change the duration of the graph by changing “-start 2d” to something like “-start 12d” for 12 days.

You will likely want to wrap this up in HTML and place your graphs in a nice web page — I will leave that up to you. Here is some simple HTML to show the graph:

<p>Here is my cool new graph:</p>
<img src="/graphs/temp_last2days.png">
<p>I hope you like it!</p>

Creating Your Script to Run in Cron
I have given you a bunch of code snippets, now it is time to put it all together in a bash script. I put together this sample script that you can add to cron and run on a regular interval. You will likely need to modify paths for your system and change the sensor parameters to pull the data points you want.

#!/bin/bash

# Change these values to match your system and modify graph properties

deviceid="00000000-00000000-00409DFF-FF999999"
diainstance="sensor0"
diachannel="temperature"
idigicloudurl="https://developer.idigi.com"
idigiuserid="idigiusername"
idigipw="idigipassword"
graphfile="/var/www/graph/temp_last2days.png"
graphtitle="Temperature - Last 2 Days"
graphtype="LINE2"
graphlinecolor="#FF0000"
graphbgcolor="#FFFFFF"
graphfontcolor="#000000"
graphvaxis="Degrees Celsius"
graphunits="C"
graphsensorlabel="Temperature"
graphstart="-2d"
# Note:  rraname must be the same as the rra name you used when creating the RRD database!
rraname="tempC"
rrddatabase="/home/rrdtool/graph_temp/sensor0.temperature.rrd"
workdir="/home/rrdtool/graph_temp/"

# You should not need to adjust much below this line.
# Paths to executables might need to be adjusted to match your system.

cd $workdir

rm ./DiaChannelDataFull

/usr/bin/wget --user=$idigiuserid --password=$idigipw 
$idigicloudurl/ws/DiaChannelDataFull

# Parse the XML, grabbing the sensor value and the timestamp
sensorvalue=`xmlstarlet sel -t -m 
"/result/DiaChannelDataFull/id[devConnectwareId='$deviceid' 
and ddInstanceName='$diainstance' and dcChannelName='$diachannel']" 
-v ../dcdFloatValue DiaChannelDataFull`

iso8601time=`xmlstarlet sel -t -m 
"/result/DiaChannelDataFull/id[devConnectwareId='$deviceid' 
and ddInstanceName='$diainstance' and dcChannelName='$diachannel']" 
-v ../dcdUpdateTime DiaChannelDataFull`

# We will convert the timestamp from ISO format to Unix Epoch format in 2 steps.
# Step 1 is to convert the ISO timestamp to something the Linux date command can read.
# The result is changing "2011-11-27T01:09:56Z" to "2011-11-27 01:09:56 +0000"
datecompatibletime=`echo $iso8601time | sed s/T/' '/ | sed s/Z/' +0000'/`

# The next step uses the date command to convert to epoch time.
# The result is changing "2011-11-27 01:09:56 +0000" to "1322356196"
epochtime=`date +%s -d "$datecompatibletime"`

# Uncomment the following 2 lines to output some debug info to the console
# echo "Sensor value: $sensorvalue ; ISO8601 time: $iso8601time"
# echo "date compatible time: $datecompatibletime ; epochtime: $epochtime"

# Post the sensor value to the RRDTool database
/usr/bin/rrdtool update $rrddatabase $epochtime:$sensorvalue

# Set a variable with server time info for use in graph below
timedatestring=`date +%F %H\:%M\:%S %Z`

# Create the graph
/usr/bin/rrdtool graph $graphfile 
-c BACK$graphbgcolor -c FONT$graphfontcolor 
-t "$graphtitle" 
-v "$graphvaxis"  -F --start $graphstart  --end now 
COMMENT:" l" 
COMMENT:"-------------------------------------------l" 
COMMENT:"SENSOR         MIN    MAX    AVG    LAST   l" 
COMMENT:"-------------------------------------------l" 
DEF:value1=$rrddatabase:$rraname:AVERAGE 
$graphtype:value1$graphlinecolor:"$graphsensorlabel" 
VDEF:value1max=value1,MAXIMUM 
VDEF:value1min=value1,MINIMUM 
VDEF:value1avg=value1,AVERAGE 
VDEF:value1las=value1,LAST 
GPRINT:value1min:"%2.0lf $graphunits " 
GPRINT:value1max:"%2.0lf $graphunits " 
GPRINT:value1avg:"%2.0lf $graphunits " 
GPRINT:value1las:"%2.0lf $graphunits l" 
COMMENT:"Generated on: $timedatestringr"

Sample Graphs
The script above will create a graph something like this:



With a few quick changes, you can change the colors and graph type. Change these configuration variables:

graphtype="AREA"
graphlinecolor="#D2691E"
graphbgcolor="#000000"
graphfontcolor="#7FFF00"

and you will get a graph like this:



RRDTool has many graph options, I have just demonstrated a few simple ones. Read the RRDTool doc and examples for more ideas.

For a listing of color codes, use an Internet search engine and search for “html color chart”.

Summary
I hope you found this interesting and give it a try. RRDTool is an awesome tool for use with iDigi sensor data. Once you get familiar with RRDTool you can do some really cool things. Again, I highly recommend the RRDTool tutorial, then digging into the various graph options to make your graphs more functional and cool looking.

Have fun!


Meet Us in Nice
— WaveForum is Coming to France —

Digi is bringing the WaveForum Developers’ Conference to the heart of the Riviera. Join us in Nice, France for two days of expert education focused on the latest advances in wireless M2M device networking. Digi will offer many technical sessions covering a range of topics, including:

  • Setting up a mesh network
  • Protocol selection
  • Web/Smartphone applications
  • Using wireless WAN networks
  • AMI/AMR wireless solutions
  • iDigi® Device Cloud™ for M2M networks
Dates: February 14-15, 2012

Location: Radisson Blu Hotel, Nice, France
Cost:199.00

On the Blog
— Projects, News & Notes —

XIG Ego Ticker — Matt Richardson, host of Make: Live, a Masters student at ITP and a notable technophile, is using Digi’s XBee® Internet Gateway to track and display his online scores (Google results, YouTube followers, etc.) on a big LED display he calls the Ego Ticker. More >


CheerLights Project — This social networking project allows people from across the globe to synchronize their holiday lights. Find out how to make a CheerLights controller with Arduino, XBee® and Digi ConnectPort® X2. Learn more about the project by visiting the iDigi blog, the CheerLights site or Twitter.


International CES — There were a lot of innovative products and exciting technologies at this year’s International Consumer Electronics Show. Here’s a few of our favorites:

  • The Marvell chip and Smart Appliances Platform is making household items network-aware.
  • WiMM Lab’s new class of Android-based smart watches
  • The Escort Live speed-trap alert system won a 2012 Innovations Design and Engineering Award at CES.
  • BiKN is an iPhone geo-location app system that finds tags you attach to the stuff you care about (keys, wallet, etc.).

You can read more about the iDigi platform,
Digi projects and technology at the iDigi blog
or follow us on Twitter, Facebook and YouTube.

      

Share Your Projects & Ideas — There are lots of great applications that use the iDigi Device Cloud and we want to hear about yours. Contact the team at thunderheads@digi.com to share us your ideas and stories.

 

Thunderheads: December 2011

Posted on: No Comments
Season’s Greetings
— Happy Holidays from Digi International —

The iDigi team and everyone here at Digi International® would like to wish you, your family and friends a joyous holiday season and a prosperous new year.




On the Blog
— Projects, News & Notes —

12 in 2012 — Rob Faludi of Digi International and NYU was featured in “Episode 7: Rob Faludi of NYU – Objects Networked for Interaction” in the New York Observer’s series “12 to Watch in 2012.” The series profiles New Yorkers doing innovative things with technology and design. Rob discusses the Internet of Things, Digi®, data, design, engineering and what’s at their intersection. More >

More from Mr. Faludi!

In case you missed it, Rob participated in the short film Networked Society: On the Brink. Read more, watch the video or view the trailer at the iDigi blog.

— and —

Rob and iDigi Solutions Architect Corey Plender recently hosted Simplifying Device Networking. This webinar examines the challenges associated with managing device networks.


Exploratorium Exhibit — Digi sponsored an XBee® Wireless Networking workshop for the staff at San Francisco’s interactive science museum, the Exploratorium. Exhibit builders learned the basics of wireless networking, resulting in interactive doorbell projects with the promise of more advanced projects to come. More >

XBee Internet Gateway — This application, written for Digi’s ConnectPort® X gateways, gives any device the ability to connect to the Internet, providing devices with a simple and flexible pathway to any web service. More >



Share Your Projects & Ideas — There are lots of great applications that use the iDigi Device Cloud and we want to hear about yours. Contact the team at thunderheads@digi.com to share us your ideas and stories.


What’s Getting Written?
— iDigi Articles —

Health at Home — In partnership with Freescale, Digi launched the iDigi Telehealth Application Kit, a development kit facilitating the creation of cloud-connected medical devices.

Digi Makes Developing Wireless, Cloud-Connected Medical Devices Easy
by Sean Fenske, Wireless Design and Development

Freescale Home Health Hub Wants to Usher in the
Era of Connected Medical Devices

by Terrence O’Brien, Engadget

The kit will be available in January 2012. Visit www.digi.com/hhh for more information.

Keeping Up with the Cloud — iDigi Product Management director Cary Stronach spoke with TMCnet to explain the emerging importance of the cloud and how to take advantage of the capabilities it offers.

Digi Explains Embedded M2M Solutions Remote Device
Management in the ‘Cloud’

by Peter Bernstein, Embedded M2M Magazine/TMCnet

Designed for Ease — The iDigi Gateway Development Kit is designed to make it easy to setup a ZigBee network, upload a custom iDigi Device Integration Application and provide seamless connectivity to the iDigi Device Cloud.

Digi Launches iDigi Gateway Development Kit with 3G Cellular Connectivity
by Deepika Mala, Embedded M2M Magazine/TMCnet

First Prize — The iDigi® Device Cloud™ received first prize in the Industrial IT category of the “Palmarès Technologique 2011” awards. This award recognizes the most outstanding innovation of the year in the industrial IT sector.



Apply Yourself
— Looking for iDigi Support —

Help Wanted — The iDigi Applications Team has been keeping busy, which is why we’re looking for an iDigi Applications Customer Support Representative to help us:

Manage Accounts | Scope Projects | Maintain Customer Information
Prepare Reports | Provide iDigi Information

The team is looking for somebody with:

  • Experience working with engineering or software services
  • Excellent communication and presentation skills
  • Good organizational skills and the ability to work on multiple projects
  • A willingness to travel
  • A sense of humor

Visit the Digi Careers page to learn more about this and other opportunities.


See You Next Year
— Upcoming Events —



Arrow’s Machine-to-Machine (M2M) Seminar Series offers expert technical training and exposure to the latest M2M solutions and services. This event focuses on the future of M2M technology. Digi International and our wireless design division, Spectrum Design Solutions, will be at each event, demonstrating how to build complete end-to-end M2M solutions.

More Upcoming Events


Intersec 2012

January 15-17, 2012
Dubai International Convention & Exhibition Centre
Dubai, UAE

 

DistribuTECH 2012
January 24-26, 2012
San Antonio, TX
Booth 4629
Featured speakers include actor, author and economist Ben Stein

 

E-world energy & water
February 7-9, 2012
Messe Essen
Essen, Germany
Hall 2/Stand 327

 

WaveForum 2012 Nice

February 14 – 15, 2012
Nice, France

 

Embedded World 2012
February 28 – March 1, 2012
Nuremberg Messe
Nuremberg, Germany
Hall 1/Booth 432

 

Stay up-to-date with the all the latest news and information from Digi®.
Follow us on Twitter, Facebook and YouTube.

      

Thunderheads: November 2011

Posted on: No Comments
Knowledge is Powell
— Meet John Powell —

Hi. My name is John Powell. I am an iDigi Solutions Architect.

I have had a long career in various parts of the computing industry, both with manufacturers like Digi International®, as well as in large enterprise IT groups. Almost every job has involved managing large numbers of devices, such as physical servers, virtual servers, PCs, modem banks, etc.

The key to efficiently managing a large quantity of devices is a good set of tools. You generally need a server to monitor them, perhaps another server to manage firmware and configuration and almost certainly a toolkit of scripts to automate various management tasks. Additionally, you probably have a set of spreadsheets or some kind of database to maintain configuration data.

The iDigi® Device Cloud™ has addressed many of those needs in a single hosted solution. Your toolkit of scripts have been replaced by built-in functions featured in iDigi Manager Pro™, such as the ability to upgrade firmware or perform other maintenance functions on hundreds (or thousands) of devices with a few clicks of a mouse. There is little or no need to write Bash, Python, Perl or Powershell scripts to manage your devices if you are managing them with iDigi Manager Pro.

The next step is replacing those configuration spreadsheets. Our development team came up with two great features in iDigi v2.3 collectively called “Groups.” I will walk you through them, as well as give you some tips on how to use them, both through the iDigi Manager Pro web GUI and through the iDigi Web Services API.


Managing Many Devices
— New Features on iDigi v2.3 —

Device Tags
If you are familiar with the “Labels” feature in Gmail, you will immediately recognize iDigi device tags. Using this feature you can apply one or more “Tags” to each gateway. They are completely free-form, so they can be used in any way that fits your organizational needs. Here’s an example scenario:

Device1: projecta, staged, ChicagoDevice2: projecta, production, BerlinDevice3: projectb, production, ChicagoDevice4: projectb, decommissioned, BerlinDevice5: projectb, production, Berlin

Now you can group by project, status and/or location! You just type the tag name into the search box in the iDigi Manager Pro web interface, hit “Enter,” and only devices with that tag will appear in the device list. You can now select one or more (using ctrl-click) devices to perform actions on, such as firmware updates, push out Python programs, etc.

To add or remove device tags from a device, right click on the device and select “Edit Tags.”

 

Tip: You can multi-select more than one device to set a tag on multiple devices. Then enter your new tag in the add tag field and click the “Add Tag” button.

To remove a tag, just click on the adjacent “X” and click “Save” when done.

Tags can also be added using the iDigi Web Services API. Send an HTTP PUT formatted similar to this:

<DeviceCore><devConnectwareId>00000000-00000000-00000000-00000000</devConnectwareId><dpTags>,projecta, production, Chicago, </dpTags></DeviceCore>

 

You can also restrict queries or operations by specifying a tag in your iDigi Web Services API operation. For example, the DeviceCore GET operation below will only return device information for devices with the tag “production.”

 

http:⁄⁄developer.idigi.com/ws/DeviceCore/?condition=dpTags
like ‘%25production%25,’
 

Note: ”%25″ is the URL Encoded representation of a percent sign (%). The % sign is a wildcard character in our Java-style pattern matching scheme.

To show how this can be used in a real world scenario, do a compound query using tags and device state to select devices:

 

http:⁄⁄developer.idigi.com/ws/DeviceCore/.xls?condition=dpTags%20
like%20%27%25,production%25,%27%20and%20dpConnectionStatus=0
 

The query above will return an Excel spreadsheet with full device information for only those devices with the tag “production” that are currently disconnected! This same conditional structure can be used to apply operations, such as firmware updates, etc. to a select set of devices.

Pretty cool, eh?

UserMetaData Field
This free-form field allows you to enter a description of your gateway with any information you need. Many have used the SNMP Description field for this purpose. The SNMP Description has a fundamental disadvantage in that you need to have the gateway powered up to configure that field. UserMetaData is set and stored in the iDigi device database, so it can easily be set at the time it is added, even if the device has not been removed from the box.

An example might be “GW-35-15W basement next to telecom panel – Chicago.”

To set the UserMetaData for a gateway in the iDigi Manager Pro web interface, right click on the device, and select “Edit MetaData.”

Once you enter the information and click “Save” the text will appear in your device list with that gateway.

In addition to saving that information in a useful spot, you can now use that information to filter the devices shown on the devices list in the Web GUI. For example, type “GW-35-15W” in the search box and only devices with that string will appear on the list.

They can also be added using the iDigi Web Services API. Send an HTTP PUT formatted similar to this:

<DeviceCore><devConnectwareId>00000000-00000000-00000000-00000000</devConnectwareId><dpTags>,projecta, production, Chicago, </dpTags></DeviceCore>

 

You can also restrict queries or operations by specifying a string found within UserMetaData. For example, this DeviceCore GET operation will only return device information for devices with “GW-35-15W” in the UserMetaData field.

 

http:⁄⁄developer.idigi.com/ws/DeviceCore/?condition=dpUserMetaData
like ‘%25GW-35-15W%25,’
 

Summary

As you can see, these are two great new features to help you manage your devices in the iDigi ecosystem. Device tags provide a flexible and dynamic method to group your devices in a way that will fit how you manage and organize your network. UserMetaData provides a method to add descriptive information, even before you deploy the device. Together they help eliminate the need for separate configuration spreadsheets or databases, allow you to organize your devices, as well as perform queries and operation based on this information.

For more information, go to www.idigi.com.


What’s Up in the Clouds?

Digi® recently participated in the short film “Networked Society: On the Brink.” Visit the iDigi blog to watch the film or view the trailer. Look for commentary from:

  • Caterina Fake, the founder of Flickr
  • David Rowan, chief editor of Wired UK
  • Eric Wahlforss co-founder of Soundcloud
  • Rob Faludi talking about the Internet of Things


Looking Forward to the Year Ahead
— Digi Completes Fiscal Year 2011 —

Digi International® just completed another successful quarter and fiscal year, with 11.8% growth from the previous year. Highlights include:

  • Expanding Digi X-Grid™ Solutions to Itron ERT-enabled meters, used by many electric, gas and water utilities throughout the U.S.
  • Extending our relationship with Freescale Semiconductor with the launch of theConnectCore i.MX embedded module product family.

Our success is built upon a commitment to innovation and providing reliable, high-performing products and solutions. We’re proud of our achievements from the last year and are excited by the opportunities promised by the year to come.

 

Stay up-to-date with the all the latest news and information from Digi®.
Follow us on TwitterFacebook and YouTube.
      

Thunderheads: October 2011

Posted on: No Comments
What’s the Word?
— Meet Bill Word —

Hi. My name is Bill Word, a sales engineer with our carrier account team. My job is to provide pre-sales technical assistance with wireless WAN carriers (a.k.a. cellular data networks) in North America. I have been in this role for over six years, and have been with Digi® since 1992.

The iDigi Applications Team asked me to write a little article on how private WAN plans work in conjunction with the iDigi® Device Cloud™.

 


iDigi Device Cloud and Private APNs
— Private Wireless WAN Plans —

Traffic from remote Digi M2M gateways, like our ConnectPort® X products, to the iDigi Device Cloud normally routes over the Internet. While these connections can be secured using SSL, this can pose problems for devices using private wireless WAN plans that do not allow traffic over the Internet.
(more…)

Thunderheads: September 2011

Posted on: No Comments
What’s Your Story?
— Tell Us How You Use iDigi —

There are a lot of interesting projects and applications that utilize our wireless technology and the iDigi® Device Cloud™, such as:

  • E2 America’s automated building energy control system, used to reduce energy consumption and demand by optimizing HVAC efficiency. More >
  • Inteligistics’ Dynamic Smart Box™, an intelligent inventory tracking system that connects tens of thousands of shipping containers worldwide. More >
  • AddEnergie’s remote electric vehicle (EV) charging stations, which rely on Digi®technology to enable remote energy management. More >

Tell us your story
The iDigi Solutions Architecture team is looking for new and interesting applications. Share your ideas, projects, questions, etc. with us. You can contact the team atthunderheads@digi.com.


Projects from the Mind of Mark
Mark Geller’s Arduino Project
Part Three

In the first two parts of this project log, I showed you how to send serial commands to an Arduino serial project using XBee® modules, aConnectPort® X2 gateway and iDigi web services. For this installment, I’m going to walk you through creating a native iOS application to send serial commands to the same Arduino project using iDigi web services.

Mark Geller at work in Minnetonka, MN

 

Before You Get Started
For this project I will be using Xcode running on an Apple computer to create a native iOS application. The application shown below will run fine in Apple’s iOS Simulator that is included with Xcode. If you are interested in distributing iOS applications on Apple’s App Store, or if you want to run your iOS programs on more than the iOS Simulator, then you will need to sign up for Apple’s iOS developer program.
(more…)

Thunderheads: August 2011

Posted on: No Comments
Welcome to Thunderheads
— News & Knowledge from the iDigi Solutions Architecture Team —

Monthly iDigi Information
Thunderheads is a personal communiqué from the iDigi Solutions Architecture team. You’ve been included in this newsletter either because you are a user of the free iDigi® Developer Cloud or because you’ve expressed an interest to learn more about the iDigi Device Cloud™.


Mark’s Project Log
Internet-Enabled Arduino
Part Two
In part one of this project log, I showed you how to connect an Arduino serial project to an XBee® ZB moduleand how to create code for a ConnectPort® X gatewaythat would allow you to send ASCII serial commands over a ZigBee network. Now we are going to look at using theiDigi® Developer Cloud and iDigi web services to send serial commands to the Arduino project from a mobile web site. About the Author —
Mark Geller began his career at Digi as a web and Flash programmer. Over the years Mark’s focus has shifted towards the creation of iDigi applications and demos, bringing him to his current position as an iDigi Application Developer.

Getting Started
Before I could start sending serial commands, I first had to connect my device to the iDigi Developer Cloud. You can create your own iDigi Developer Cloud account for free. Visit www.idigi.com/getstarted to set up your account.
(more…)

Contact a Digi expert and get started today! CONTACT US

Desktop Site