Performance issue: recieve real-time encoder data via TCP

Hello again,

I have the following setup:

A winch moves a small rectangular object up and down.
Small server is connected to a winch. It receives encoder data and sends it via TCP.
The packets are ASCII with delimiter ‘-’ … (so its: 01594-01592-01590-)
I tested with netcat and the data is realtime, in sync with motion and there are no lags.

Now I have an ofApp which does almost the same thing as netcat for now. It connects to remote server via TCP, reads encoder data and renders values on the screen. And here I have a huge lag. The winch have already stopped and for almost 20 seconds I still recover the packets.

The code is very simple for now (and probably that’s why it doesn’t work)

ofxTCPClient clientEnc;

void ofApp::setup() {
clientEnc.setup(“”, 5000, false);

void ofApp::update() {
encData = clientEnc.recieve();

void ofApp::draw() {
ofDrawBitmapString(encData, 400, 500);

I suspect this is related to the frequency at which update() and draw() are called.
I would imagine that I need to use a different thread and read my TCP/encoder data from there.
Am I thinking right, or there is something else ?


Or… maybe need to use recieveRawBytes() instead of recieve()…

Just a quick guess, since I’ve used ofxTCPServer a while back, but have you used:

“The delimiter in the client and the server have to be the same” as line 19 of examples/communication/networkTcpServerExample/src/ofApp.cpp states

Yes, it is the same. However maybe I was too quick to ask for help. After reading previous posts related to TCP, it turned out that recieve() function is not very fast. So now I use recieveRawBytes() trying to grab as much as possible at once and it looks like timing is good. At least there is no visible delay. I check how it actually tracks the moving object tomorrow.

In case the delimiter is the problem (or to find out if it is), you can try:

    string data = clientEnc.recieveRaw();
    cout << data << endl;
    while(data.length() > 0){ // We read something, so let's see if there's more
        data = clientEnc.recieveRaw(); // If not: data.length() will return 0 and break the while loop
        cout << data << endl;

Which is based on this, from

//no terminating string you will need to be sure
//you are receiving all the data by using a loop
std::string receiveRaw();

Hello @Jildert

No, delimiter is not a problem at all. It works fine splitting where it should. It’s just painfully slow, but probably due to high speed at which data arrives and only 60 cycles/second at which update/draw being called. I’m going to move it all to a separate thread.

recieveRaw gives strange result, so I abandoned it. The highest performance I get in current configuration (calling from update()) is using receiveRawBytes…

ps. or maybe even using UDP…

Alright, good to know.
And a for a continues (high speed) data-stream UDP seems like a good candidate.
I’m curious what will do the trick eventually, so good luck!

My suggestion:
a) Use a separate thread to read the TCP stream.
Whenever this thread, because it has detected a delimiter, knows the new position, write this position into a global variable.

b) The value assignment of the new position into the global variable must be protected. With a lock.

c) In the main thread, more precisely in the update function, activate the lock as long as the global variable is read.

This causes:

  1. The TCP stream is read with maximum speed.
  2. The update function sees the newest value. But not all values.
  3. The lock is expensive for the operating system, but otherwise there is the danger that a half written value is already read in an inconsistent manner.

(To avoid the lock the global variable could be atomic.)

it has been a while but I really liked ZMQ for this. It was fast enough to send H264 data from a RPi to a Mac to create a pseudo-streamer

Here is my modified fork

not working here on Windows:
Assertion failed: rc == 0, file f:\openframeworks\addons\ofxzmq\libs\zmq\include\zmq.hpp, line 419

yeah windows may need some tweaking - I only used on Mac/RPi