DIA Event Uploader

From Digi Developer

(Redirected from Dia Event Uploader)
Jump to: navigation, search

Contents

Uploading Events or discrete data samples to DIA

Comparing iDigi_DB to iDigi_Upload

The stock iDigi upload presentation (named iDigi_DB) assumes you only want a sampling (or subset or snap-shot) of data samples handled by DIA. For example, if 20 tanks send new level reading to Dia once per minute, this amounts to 1200 samples per hour. Yet the Device Cloud host may not desire any more than 20 samples per hour - just one per tank per hour. Therefore the stock iDigi_DB presentation is configured with an interval setting (for example: 3600 seconds or 1 hour), then all configured channels are sampled and uploaded to Device Cloud once per hour.

This design does not support uploading real time events. For example, if a door sensor is monitored, then Device Cloud will only see the door status on the hour (the sample interval). Device Cloud will not record that the door was opened three times during the hour.

iDigi_Upload is a custom adaptation of the original idigi_db.py file, modified to save a copy of every new channel sample for upload. These are saved in a cache and only cleared after successful upload.

Description iDigi_DB iDigi_Upload
Are discrete events uploaded to Device Cloud? No - only the I/O status at the sample interval Yes - every new sample is uploaded
If upload fails, are samples saved and retried? No - new samples will be taken at the next sample interval Yes - copies are made of every sample and only deleted after successful upload (unless cache grows beyond configured limit)
If a channel doesn't change, is it uploaded every interval? Yes - all channels are sampled at each sample interval No - copies are made only when sample change
Is the upload size limited per interval? Yes - at most one sample of every channels is sent No - every changed sample is uploaded
Is memory usage limited? Yes - existing samples are used to create upload No - a duplicate copy of every changed sample is held until upload

New settings

iDigi_Upload supports the original iDigi_DB settings, plus adds two new settings:

cache_size

Is the maximum number of bytes of flash to use buffering for upload. It defaults as 0, which disables this drastic changes of behavior.

  • When cache_size = 0, idigi_upload holds all data in memory only. Restarting the gateway will discard your data.
  • When cache_size is set > 0, then idigi_upload queues a deep-copy of every new sample first in memory, then saves this to flash approximately once a minute. You will see files such as 'data_cache.txt' appear and disappear from your Python directory. These are cleared only after upload or when the caches fills, then in FIFO design the oldest data is lost. NOTE that the design keeps a backup when deleting the cache, so the actual size of flash required will be twice the configured amount.
clean_minute_interval

Is an alternative to the base interval setting. It accepts only minutes in the set (0, 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, 30, 60)! It causes upload ON THE HOUR, then minute intervals in a predictable pattern. So setting 10 cause uploads at 00:00:00, 00:10:00, 00:20:00 and so on.

  • Setting clean_minute_interval to zero (0) causes the base interval setting to be used.
  • Setting clean_minute_interval to non-zero over-rides the interval setting.

Download Info

Python Code

This version requires Dia 1.4.14 or newer. The files includes are:

  • src\common\helpers\sleep_aids.py
  • src\samples\annotated_sample.py
  • src\presentations\idigi_upload\__init__.py
  • src\presentations\idigi_upload\idigi_upload.py
  • src\presentations\idigi_upload\string_file.py
Personal tools
Wiki Editing