How to store a large data from serial port without loosing any data?

Question

I have a GPRS module for reading data from a web page. I am receiving the data through a serial port, but I cannot store all data at once. When I have a loop for reading and storing the data, I loose some of the data between iterations.
I'm wondering why this is being happened and what is the best way for storing the data?

Thank you

Answer 1

Ideally I'd like more info on the serial data? Is it a particular protocol or a custom stream of data? What baud rate? Any hardware hand shaking? etc.

But in general:

1. Buffering is important so have you set your serial receive buffers big enough? You need a buffer big enough to hold all the data that might arrive between your attempts to read it.

2. If your program is implemented as a single task you might consider splitting it up into multiple tasks either using the costates functionality of Dunamic C or using the uCos II real time operating system that comes with the Dynamic libraries.

3. Use serial ports E or F if your data rate is high as these have an inbuilt 4 byte receive buffer which helps to reduce dropping of data (ports A to D have a single byte receive buffer).

Regards,
Peter

Answer 2

It's HTTP, at 115200 bps with no handshaking at the moment. I also use port B for communication. I guess to fix my problem I need to use (XON/XOFF) software handshaking to control my receiving data, but I'm not sure how to do it.

The data I need to receive is about a few thousands of characters. I'm using a char array to store the read data, but when I make my buffer to be quite large, it still cannot store more than 128 data.

Answer 3

Are you using PPP to connect to the GPRS module or getting a raw data feed from the module? I.e. is the module being used as a proper network connection or as an autonomous system which is simply providing a serial data stream to the Rabbit system?

In the first instance the Dynamic C TCP/IP stack should manage the connection and throttle things/retry etc.

In the second instance, 115200 bps is a lot to ask without handshaking. I'd be inclined to add handshaking and also drop the baud rate if possible.

Regards,
Peter