TCP to Zigbee port binding
Introduction
This page is to document a demonstration application that operates on a Digi Gateway product to tunnel data between a TCP network and a XBee network.
WARNING This script is deprecated! for an up to date script, use xbee_transport.py
Requirements
- A Python enabled Digi Gateway
- A XBee device associated with the Gateway (Preferably with a method to monitor traffic sent to it)
- CLI access to the Digi product
Overview
The application works to tunnel data from the TCP network and the XBee network by defining a TCP port XBee address pair. The TCP port is bound to the Gateway any any client connection will have its data forwarded to the XBee address specified. Correspondingly, the data received by the Gateway from that XBee address will be forwarded to the TCP client connection.
To help define the TCP port and XBee addresses is a helper script called 'table_generator.py'. After uploading the script to the Gateway, run it. This performs a discovery on the XBee network, identifies the unique addresses, and outputs a TCP port to XBee address mapping to a configuration file. By default the TCP port starts at port 4000, and increments by one per additional XBee address found. The output file is placed into the WEB/Python directory named as 'bind_table.py'. To edit it, download it from the Gateway and manually change the parts desired.
The main application 'tcp_zig_binding.py' takes one parameter optionally. The parameter represents the amount of time to wait before sending a message received from the XBee network to the corresponding TCP port. The usefulness of this is to potentially send fewer messages over the TCP network, or to group related messages into a single TCP packet. The parameter represents the amount of time in seconds, for example, .2 represents 200 ms delay. By default the application has a delay of .3 ms. Use a value of 0 to disable.
Code
This is the code from the main application 'tcp_zig_binding.py'.
"""
This script is intended to act as a data tunneling application between the TCP network and zigbee network.
It assigns multiple TCP sockets to listen on specific ports on the gateway device, this is defined in
bind_table.py, which can be hand made or generated. Each of the TCP sockets corresponds to a specific Zigbee
address on the zigbee network. This allows an application to send data to port 4000 and always have it be in
turn sent to the same zigbee address.
There are several limitations and liabilities. First the Series 2 radios have a maximum payload size of 72
bytes of data, with no internal ability to understand fragmented data. The Series 1 radios have a maximum payload
of 100 bytes, again with no internal ability to understand fragmented data. While this application can perform
some packet reassmebly on this end, the zigbee endpoint would not be able to do so, and whatever device or
application attached to said zigbee device will have to reassemble the data as needed.
"""
import socket
import select
import bind_table
import sys
import zigbee
import errno
import struct
import traceback
from time import clock
tcp_conn_dict = {}
tcp_port_dict = {}
tcp_data_dict = {} ##Data associated with tcp port
zig_port_dict = bind_table.node_list ##Where we obtain the zigbee to address lookup
zig_data_queue = []
listen_list = []
client_list = []
MAX_ZIGBEE_PACKET_SIZE = 100
MAX_TCP_PACKET_SIZE = 8192
segment = 0
end_point = 0
profile_id = 0
cluster_id = 0
"""
This value should be changed to suite the data tunneling needs. Factors such as the remote device's baud rate,
packetization time, and size of data all come into play, test and retest as needed.
"""
TCP_BUFFERING_TIME = .300 #300 milliseconds is out timeout period
###################################################################################################
# Parse Args
###################################################################################################
#We only have one argument, the TCP_BUFFERING_TIME
try:
if len(sys.argv) > 1:
TCP_BUFFERING_TIME = float(sys.argv[1])
except Exception, e:
print get_exception_info()
print "TCP_BUFFERING_TIME requires a floating number, it's default is .3 (300 ms)"
sys.exit(0)
###################################################################################################
# Detect our radio type and assign address information accordingly
###################################################################################################
"""
Due to the differences between series 1 and series 2 radios, we much be able to determine which type of radio
is currently onboard our gateway, and set the end_point, profile_id, and cluster_id appropriately. To do this,
we use the ddo_get_param(local_radio, 'HV'), which is the AT command to retrieve the hardware version.
Once we have the hardware version, we have to unpack the binary string into a tuple, and take the first
(and only) element from that tuple, and assign it to hw_version. We test that number returned, if it's
over 6400, it must be a Series 2 radio, under 6400, series 1.
"""
try:
hw_version = zigbee.ddo_get_param(None, "HV")
hw_version = struct.unpack("=H", hw_version)[0]
except Exception, e:
hw_version = None
print get_exception_info()
print "Failed to retrieve hardware version from local radio"
print "Assuming it's a series 1 device"
if hw_version != None:
if hw_version > 6400: #If the hardware version is greater then 6400, it must be a series 2 radio
print "Detected Series 2 radio in gateway, configuring zigbee socket appropriately"
end_point = 0xE8 #232
profile_id = 0xC105
cluster_id = 0x11 #Out the UART
MAX_ZIGBEE_PACKET_SIZE = 72 #Packet sizes are at maximum 72 bytes
else:
print "Detected Series 1 radio in gateway, configuring zigbee socket appropriately"
###################################################################################################
# cleanUpConn()
###################################################################################################
def cleanUpConn(conn):
try:
client_list.remove(conn) #We remove it from our listening list
sock = tcp_conn_dict[conn] #We locate the associated listener
tcp_conn_dict[sock] = None #We set the listener's lookup to None
tcp_data_dict[sock] = [] #We reset the data queue
del tcp_conn_dict[conn] #We delete the key value pair for the reverse lookup
conn.close() #We close the instance
conn = None #We set the instance to None
except Exception, e:
print get_exception_info()
def _format_exception_info(max_tb_level=20):
e_type, e_value, e_traceback = sys.exc_info()
e_name = e_type.__name__
e_args = []
try:
for item in e_value.args:
e_args.append(str(item))
except AttributeError:
pass
try:
for item in e_value.message:
e_args.append(str(item))
except AttributeError:
pass
tb_string = traceback.format_tb(e_traceback, max_tb_level)
return (str(e_name), e_args, str(tb_string))
def get_exception_info():
"""Formats the last raised exception, and returns a string that
contains the name of the exception, any info associated with
the exception, and a call trace for the exception."""
e_name, e_args, e_tb = _format_exception_info()
e_string = "".join(["Exception ", e_name, ":\n",
"".join(e_args), "\n",
"Traceback:\n", "".join(e_tb)])
return e_string
###################################################################################################
# Init
###################################################################################################
"""
We loop through the current zigbee address to tcp port dictionary, create a socket per entry, bind
the socket to the specified tcp port, and map the various dictionaries to provide fast lookups
"""
##Note we are doing keys(), so we only generate the list once at the start, otherwise
##It will change due to our assignment of port -> address lookups
for addr, port in zig_port_dict.items():
##Create socket, bind to localhost:port, set it to listen and append it to the listening socket list
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(("", port))
sock.listen(1)
listen_list.append(sock)
tcp_conn_dict[sock] = None ##Init the listener to connection dictionary
tcp_port_dict[sock] = port ##Assign listener to port lookup
tcp_port_dict[port] = sock ##Assign port to listener lookup
tcp_data_dict[sock] = [] ##Assign listener to tcp_data_queue lookup
zig_port_dict[port] = addr ##Assign port to zigbee_address lookup
##The lookup between address and port already exists by this point
#end for key in zig_port_dict
## Creating the zigbee socket, binding it to a local endpoint, set it to not block
zig_sock = socket.socket(socket.AF_ZIGBEE, socket.SOCK_DGRAM, socket.ZBS_PROT_TRANSPORT)
zig_sock.bind(('', end_point, profile_id, cluster_id))
zig_sock.setblocking(0)
###################################################################################################
# Mainloop
###################################################################################################
print "starting mainloop"
while True:
"""
We listen for reads on the listening sockets for new connections the client sockets for new data
from the tcp side and the zigbee socket for incoming zigbee data
We listen for write ability on the client sockets so we can write the received and queued zigbee
data on them, and we listen for write ability on the zigbee socket so we can write the received
and queued tcp data on it.
"""
rl, wl, el = select.select(listen_list + client_list + [zig_sock], #Reading sockets
client_list + [zig_sock], #Writing sockets
[],
.5) #Not used, (error lists, see man select)
###################################################################################################
# Handle zigbee reading
###################################################################################################
if zig_sock in rl:
data, addr = zig_sock.recvfrom(MAX_ZIGBEE_PACKET_SIZE)
print "Zigbee read: %d bytes from address " %(len(data)), addr
if len(data) != 0: #if we've received data
try:
port = zig_port_dict[addr[0]] #We find the matching tcp port
sock = tcp_port_dict[port] #we find the socket matching our port
if len(tcp_data_dict[sock]) != 0: #If the queue isn't empty
(old_data, packet_time) = tcp_data_dict[sock][-1] #Copy the last packet off
if clock() - packet_time >= TCP_BUFFERING_TIME: #Check the time of the last packet
#If the difference in time is greater then our buffer time
#We should just append the data to the queue
tcp_data_dict[sock].append((data, clock()))
else:
#The difference in time ISN'T greater then our buffer time
#We should append our new data to the old data
lump_data = old_data + data
tcp_data_dict[sock][-1] = (lump_data, packet_time)
# We have to be careful of how we append the data, and to assign the older
# time back into the tuple. Otherwise we may continue to receive data
# append it out of order, and never send it anyways because the buffer time
# may keep reseting
else:
tcp_data_dict[sock].append((data, clock()))
# We can have a valid key error here, in this case, if we receive a zigbee packet from an
# Address that is not in the bind table, we will throw a key error. An alternative is
# Receiving a packet from the local zigbee radio, which will have the 0000 address, something
# That is not normally in the bind table if automatically generated.
except KeyError, e:
pass
except Exception, e:
print "We received an unknown error, ", e
print "Please report this!"
sys.exit(0)
#end if zig_sock in rl
###################################################################################################
# Handle zigbee writing
###################################################################################################
if zig_sock in wl:
if len(zig_data_queue) != 0:
(addr, data) = zig_data_queue[0]
if len(data) < MAX_ZIGBEE_PACKET_SIZE:
segment = len(data)
else:
segment = MAX_ZIGBEE_PACKET_SIZE
try:
sent_data = zig_sock.sendto(data[:segment], 0, (addr, end_point, profile_id, cluster_id))
data = data[sent_data:]
if len(data) == 0: ##If empty, pop element, if not, return modified element to position in queue
zig_data_queue.pop(0)
else:
zig_data_queue[0] = (addr, data)
except socket.error, e: #If its a socket error
if e.args[0] == errno.EAGAIN: #If its a blocking error
pass #Do nothing, come again next select call
else:
print get_exception_info() #Not a blocking error, print and exit
sys.exit(0)
except Exception, e: #Unknown error, print and exit
print get_exception_info()
sys.exit(0)
#end if zig_sock in wl
###################################################################################################
# Handle listener sockets
###################################################################################################
for sock in listen_list:
if sock in rl: #if it is receiving
conn, addr = sock.accept() # accept conn
print "New connection from: ", addr
conn.setblocking(0) # No blocking on connection
tcp_conn_dict[sock] = conn # assign sock to conn lookup
tcp_conn_dict[conn] = sock # assign conn to sock lookup
tcp_data_dict[sock] = [] # Clean any queued data for new connection
client_list.append(conn) # append conn to client_list for select statement
#endfor sock in listen_sock
###################################################################################################
# Handle read client sockets
###################################################################################################
for conn in client_list:
if conn in rl:
try:
data = conn.recv(MAX_TCP_PACKET_SIZE) #Receive our data
print "Receiving %d bytes from tcp socket" %len(data)
except socket.error, e: #We have a socket exception
if (e.args[0] == errno.EAGAIN): #If it's a blocking related exception
continue #come back again next select call
else: #If it's not a block related exception
print get_exception_info() #Clean it up
cleanUpConn(conn)
continue
except Exception, e: #An unknown exception that's not socket related
print get_exception_info() #clean it up
cleanUpConn(conn)
continue
if len(data) == 0: #Did we receive a clean break?
cleanUpConn(conn) #Clean it up
continue
else:
sock = tcp_conn_dict[conn] #Find the associated socket with connection
port = tcp_port_dict[sock] #Find the associated port with socket
addr = zig_port_dict[port] #Find the associated zigbee address with port
zig_data_queue.append((addr, data))
#endfor conn in client_sock
###################################################################################################
# Handle write client sockets
###################################################################################################
for conn in client_list:
if conn in wl: ##We can write
sock = tcp_conn_dict[conn] ##Find the associated socket with connection
if len(tcp_data_dict[sock]) != 0: ##Find the associated data list with socket, check if length of list is 0
(data, packet_time) = tcp_data_dict[sock][0]
if clock() - packet_time >= TCP_BUFFERING_TIME:
#This packet has exceeded the time buffering, it is ready to send!
try:
if len(data) < MAX_TCP_PACKET_SIZE:
segment = len(data)
else:
segment = MAX_TCP_PACKET_SIZE
sent_data = conn.send(data[:segment])
print "TCP: Sent %d bytes of %d total" %(sent_data, len(data))
data = data[sent_data:]
#If all data was sent, pop element out of queue, else restore modified element to queue
if len(data) == 0:
tcp_data_dict[sock].pop(0)
else:
tcp_data_dict[sock][0] = (data, packet_time)
except socket.error, e:
if (e.args[0] == errno.EAGAIN):
continue
else:
print get_exception_info()
cleanUpConn(conn)
continue
except Exception, e:
print get_exception_info()
cleanUpConn(conn)
continue
#end for conn in client_list
#end main loop
PDF
