I am using a pose estimation model from tensorflow.js that takes a camera stream and estimates my ‘pose’ then draws a pose skeleton on top of the camera feed in a browser. What I want to do is take the coordinates of each keypoint and stream them over to my OF app for further processing
PoseNet: https://github.com/tensorflow/tfjs-models/tree/master/posenet Try a Demo of it here.
Im sadly not too familiar with web development and all the different communication protocols. At first I thought I could try and make the browser a virtual serial device and send the data over serial, but didnt look like that was going to work.
I thought maybe I could use OSC and send the data over UDP but It looks like there is something significant that blocks browsers from sending/receiving UDP packets?
Im looking into websockets right now and the approach looks quite complex. I just wanted to know if there were any reliable AND low-latency communication options that I could use before I decided to dive into websockets?
Thanks for any help!
As far as I know, Websockets is the fastest when dealing with a browser, it’s not instant but in most cases it’s good enough. Have a look at https://github.com/robotconscience/ofxLibwebsockets it has a lot of examples and is pretty straight forward to integrate in your OF project, you would setup your OF app as a Websocket server.
Good luck !
usually websockets work quite well and i think it is the way to go.
websockets are very well supported across the major browsers, and nodejs. they are easy to use in the web. it might be a bit harder to get them running in c++. ofxLibWebsocket has not been updated for a while and might not run out of the box.
if you are more looking for a pub/sub solution, dont mind running a communication server, then i can recommend mqtt. it runs on top of websocket or tcp and you need a mqtt broker.
also check out Chris’ addons, e.g.
there is also an electron app which stream postnet data via osc:
hope that helps