Sending data to Flash

The only way I know that Flash can communicate to the outside world is via a Java XML socket connection.

I’ve used FLOSC before (Flash open sound control) but I’d like to avoid that if possible.

The data I have to send is very simple, 3 integers.

One way might be to emulate a keyboard press event from OFW, and flash listens to for that keyboard event. Is there anything in c++ like this?

The other is to use a bit piece of software that listens for midi notes and turns them to keyboard presses. Although then that is the same as using Flosc, expect the Flash side is a lot simpler.

hi chris,

I’d like to hear about the problems with flosc – we’ve just had the osc addon made and it works very well for c++ -> other app communication.

I’m not sure about keyboard control. I’ve used “DeviceIoControl” to turn on and off LEDs, but I’m looking for a lowlevel, windowsy way to press keys from within code –

I do think the easiest way to do this is to use the socket options in flash, and to setup a call and reponse type operation flash’s xmlSocket (which is a fnacy name for tcpSocket)

ie, flash sends a request and the c++ app sends data.

I’m a little hazy, it was a while since I did it – but I remember there were only two caveats, as I remember with flash:

a) the flash data needed to be null terminated (1 extra byte at the end) which is not how data is normally sent.

b) the flash data needed to be in base64, which was ok for just sending unsigned chars (0-255) - didn’t have to worry, but for larger values required a little packing / depacking on the other end. (I guess as3 has non base64 net)

you might bypass both those issues if you can use this:
http://livedocs.adobe.com/flex/2/langre-…-ocket.html

but besides that, it worked great on a couple of projects.

I will try to take a look for the code I have and see if I can’t post an example with the ofTcpSocket addon that could be helpful – I think I have something simple from when I was testing…

HTH -

take care!
zach

Hey Chris,

The new ofTCPServer works wonders with flash - I just got it running with Laser Tag and now laser tag can send the x y coords to flash using the xml socket onData.

You can try the code here:
http://svn.openframeworks.cc/browser/li-…-AndServer-
For flash just change one thing in ofTCPClient.cpp

in the function bool ofTCPClient::send(string message){

change -> message += “[ofTCPEnd]”;
to -> message += “[ofTCPEnd]\0”; //this is needed for flash

Let me know if that works for you - otherwise I’ll post the updated code shortly.
Theo

Hi Zach & Theo

Flosc works great, but I want to avoid it for making an easy installation. The less software needed for communication the better.

I was hoping there was a way of sending messages direct via tcp and it looks like you’ve got that covered now. Thanks! Will test it out in a bit.

Hi all :wink:

i’ve been adding the necessary addons for OF and get to work the ofTCP communication system. My goal is to work with TCPserver to be able to communicate with a Flash application.

I’ve been able to communicate OF with FLASH though ofTCPServer but now i face anothe problem …

i’m doing a silhouette tracking and then i pass the points of the the contour of the captured blob to Flash via TCPServer. Flash then represents it and make some “cool flash stuff” with it.

i’m facing the problem that seems to me like a buffer overrun in Flash reception , if i pass from OF to Flash a contour with more then 240-290 points, the Flash application starts to freeze and get really slow down …

i’ve been trying from OF to make the buffer as big as possible as well as trying to change the TCP message size but i get no gain …

does anybody know something about sending arround 800 (x,y) points to Flash every 25 frames ? is there a way to define the Reception buffer on the flash side ? or where could be the bottle neck ?¿

any idea much appreciated …

e*

Hey Eloi,

I am glad you have got OF talking to flash nicely.
I have done some work in flash and I have also hit seemingly arbitrary bottlenecks. Like drawing lots of points and suddenly getting massive slow downs after a certain number.

Do you have the slow down from just sending the data or only when you try and draw?

Can you send smaller amounts of data every 4 frames?

It might end up just being the limitations of flash - you could try and do some clever stuff like somehow compressing the data in someway?

But 800 points every 25 frames seems to be quite a lot especially for flash.

Can you simplify the blob contour by resampling?

Sorry if these questions sound obvious maybe something is useful : )

hola Theo :wink: even its a bit slow for high data transfers … i’m starting to become crazy because of this slow downs … :twisted:

answering your questions ::

//? Do you have the slow down from just sending the data or only when you try and draw?

sending data and trying to print it out on Flash is enough to slow it down.

//? Can you send smaller amounts of data every 4 frames?

well basically yes … i’ve made a test bank and if i reduce the frameRate (or the numer of points) on the OF side i get better results on Flash receptions … so for my experiments i’m quite clear that flash is blocking because of excessive data. BUT !! … some people told me that Flash has no clear “limits” on data reception and they assure me it could work fine with arround 1000 points !! ?¿ …

//? It might end up just being the limitations of flash - you could try and do some clever stuff like somehow compressing the data in someway?

do you know how to “know” if there could be a buffer size limitation on Flash Side ? I’ve tried to look on the net and didn’t find much info about what’s inside Flash Libs … ?¿

compressing ? … mmm i need the points so the only thing i’ve thought i can compress or reduce is the number of points on blob, so resampling ?

i talked with Chris who is in Hangar in Barcelona and she also point me to an OpenCV function to resample the point array but i could’nt find it … Do you know which OpenCV function could let me resample the points in a smart way ?? (i can reduce points by just skipping some every x, but i’m looking for a nicer way to keep morphology better)

And my questions …

? let’s say … if i need 500 points on contour, that could be aproximately … arround 3Kbytes of data … so which configuration of TCP_MaxLength_Message and TCP_SendBufferSize could be the best ?? could it be affecting the communication to have some of these values on a bad range for my needs ?? Does it matter if these TCPBuffer and MessageLength are diferent on Flash side ??

? could it be a problem of the DLLs or LIBs used ? are there any other libs that could make this work ? or could i be using a wrong lib or DLL for or any other misconfigurations of my Windows ?

which object are u using to receive on Flash side ? I’m using the XMLSocket() is that the only object Flash that could be used to receive from TCP ¿?

Thanks for your attention Theo :wink:

e*

hi eloi,

this might help:
http://smallvoid.com/article/windows-tc-…-tings.html

also I have heard that in actionscript 3 there is a UDP class, that allows for much faster communication then TCP, and which I recommend for realtime communication. TCP has higher overhead.

this is code that I have somewhere for resampling in opencv based on DP (douglas pecker) algorithm, could be helpful and I’ve actually used it in an app that communicates with flash for the same reason.

  
  
// ------------------------- get the douglas pecker approximation  
CvSeq *result = 0;  
// Approximates polygonal curves with desired precision  
result = cvApproxPoly(a_contour, sizeof(CvContour), storage,   
CV_POLY_APPROX_DP, cvContourPerimeter(a_contour)*0.004, 0);  
//printf("contour %p %d\n", a_contour, result->total);  
_blobs[i].nPtsSimple = MIN( result->total,CVCF_MAX_LENGTH_APPROX_CONTOURS);  
  
//useful for adjusting the resolution  
if (result->total > CVCF_MAX_LENGTH_APPROX_CONTOURS){  
	printf("over %i \n", result->total);  
}  
for(int j = 0; j < result->total; j++){  
	CvPoint * pt = (CvPoint*)cvGetSeqElem(result, j );  
	float npx = (float)pt->x;  
	float npy = (float)pt->y;  
	if (j < CVCF_MAX_LENGTH_APPROX_CONTOURS){  
		_blobs[i].ptsSimple[j].set(npx, npy, 0);  
	}  
}  
  

I just ripped it out of my code, let me know if you have questions. with the cvApproxPoly call you can adjust the precision of the resampler.

hope that helps -
zach

hola ALL :wink:

thanks zach for your code and words .

i’ve been trying to boost up the TCP with no luck.
flash is still getting slow when i throw more then 250 points at 10 fps …

:_(

keep on investigating … if its possible to make it faster or if Flash is not able to do so. A friend told me that he has seen Flash running 30 fps getting Blob points from processing with arround 1000 points … so there’s a way to do so … but i don’t find it …

e*

e*

if you can posted a simplified version which plays a movie (along with the flash code), I’d be happy to take a look at the problem.

take care,
zach

Hey Eloi,

How are you sending the data to flash?
as a series of tcp send for each point or as one very large string containing all the data?

eg pseudo code:

  
  
for(int i = 0; i < 1000; i++){  
  tcpServer.sendData("0.3444/0.7234");  
}  
  
//or   
  
string str = "";  
  
for(int i = 0; i < 1000; i++){  
 str+= xData[i]+"/"+yData[i];  
}  
  
tcpServer.sendData(str);  
  
  

When I said about compressing the data you could do something where if you are sending the data as floats in a string instead of doing 0.34563422 you do 0.346 so you are reducing the amount of data you sending by half. Does that make sense?

But maybe you are sending it as int instead.

I think there is also a lot you can do on flashes end to optimize the data handling.
Can you try sending the data to flash and not tracing it out just to see if the trace is slowing things down?

hi theo, thnkx again for your questions :wink:

the flash part has been done by someone and fixed the comunications roules :

sending data to flash

very large string containing all the points in the contour in one string.
data is in the ‘int’ format .

"s"pointX"e"pointY…for N points on the contour…(byte)/0

  
      
    // send contour over TCP   
    ////////////////////////////  
    string s = "";  
    for(int i=0; i<contourFinder.blobs[0].nPts;i=i+mySTEPS){  
            s += "<s" ;  
            s += ofToString(int(contourFinder.blobs[0].pts[i].x));  
            s += "e" ;  
            s += ofToString(int(contourFinder.blobs[0].pts[i].y));}  
    s = s + (char)0 ;  
    for(int i=0; i<TCP.getNumClients(); i++){  
                       TCP.send(i,s);}  

tomorrow i’'ll have a meeting with the flash code so we’ll make test with diferent TCP configurations on both ends. i’ll let u know our final tests.

i think the most reasonable is begining to make smaller packets, but i’m not sure if that’s could be a solution, or even if necessary as i think that the socket should make data chunking by itself on his layer …

on my code i’ve added this line tryin to make it all run faster but not sure if it’s really doing something, has anybody experienced with this settings like SendBufferSize on ofTCPServer ?? i’ve tried to make it much bigger and nothing changed on the way flash paints …

  
//--------------------------	  
bool ofTCPServer::setup(int _port){  
	if( !TCPServer.Create() ){  
		if(verbose)printf("ofTCPServer: create() failed\n");  
		return false;  
	}  
    TCPServer.SetSendBufferSize(65536);  
    TCPServer.SetReceiveBufferSize(65536);  
	  
	if( !TCPServer.Bind(_port) ){  
		if(verbose)printf("ofTCPServer: bind(port = %i) failed\n", _port);  
		return false;  
	}  
	  
	connected = true;  
	port	  = _port;  
	  
	startThread(true, false);  
	return true;  
}  
  

[/code]

ah - a couple things to note / try:

a) if you send the string representation of the float, it’s alot more info then the float itself, for example:

“3.91214341” = 10 bytes, while the float is only 4 bytes.

please correct me if I am wrong (because I’ve only done this flash - C++ stuff once), as I remember, the best way to do it was to base64 encode the bytes of the float, because flash has a silly system of using a “/0” character to represent the last byte (so you can’t send “raw” data , you will need to convert it to something like text, thus base64). Then on the flash end, you decode the data. I dont’ know how well that would work with variable sized packets, but I typically send the same amount of data per frame (and pad with zeros if there are less points then the max) so for me it worked well. I have done this before for float arrays c++ -> flash. I hear that in actionscript 3, you can avoid the xml socket (and thus the /0 to end the data) so that it’s easier to send raw data, and over udp which is even faster.

anyway, base64 = save you alot of space in the packet.

b) I haven’t looked at your code yet, but it occurs to me that you should employ a model of call and response for sending data - ie, flash says to c++, “I am ready for data” and c++ says, “ok, here’s the data”. If not, you may be operating a time scale that is inappropriate, like flash is processing the data way to slow for c++, and then backlog of unprocessed packets gets to be too much. This is a big cause of things like latency (for example, receiving serial data that get further and further away from being the current data). I strongly recommend call and response for most models of communication, such as network or serial, because devices operate at different frequencies and this keeps everyone in sink.

c) you should print the packet size when it gets slow so you know how big it is bytes. I am curious.

c) you should try c++ -> c++ to see if this is a flash thing, or if you are just maxing a packet size that might be too massive for the tcp transmision and thus it could be solved by altering tcp settings, etc.

hope that helps!

best,
zach

Hi Zach & all :wink: I also like the idea of adding 0’s at the end to keep an stable packet size !!

b// you’re right again… now i’m not doing any “protocol” communication, just every frame in OF send the contour points list.
So i get the idea of making a simple “call/response” implementation to suit needs of flash reception.

c// i’m gonna do so and let u know :wink:
in my laptop running both applications, it get’s stuck when TCPManager send size is between 750-850 …

d// i’ve tried to send the same data to an ofTCPClient on the same computer and it was getting strangely slow ALL together … the send and the receive applications where slowing down a lot … I didn’t test it much, but perceived that something was making a bottle neck.

hey cool -

when you say slow, do you mean slow as in slow frame rate, or slow as in there is a “lag” or delay between the camera and flash. if there is a lag, then the call and response model will work immensely with helping fix this. if OF is running at 60 fps and flash is running at 25fps, you will build up unread packets and there is just now way to keep them in sync or in time. call and response is the way to do go with two apps communicating that have different frame rates.

keep us posted and take care!

take care!!
zach

hola zach !

slow i mean “slow frameRate” . No lag exists between OF and FLASH display.

so that means that to make OF go “slower” what i do is to add a small delay (sleep(x)) … so then i can reach a 25-30 fps in OF and that’s “eatable” by Flash, They both work aprox. at same speed .

i still have to add the call-response protocol, but as i said now is working fine on that new hardware :wink:

e*

hi zach and all :wink:

e*

wow this sounds awesome. I have been working on this. I got the connection from OF to Flash via OF TCP Server and Flash XML socket. I get a connection true but a bit lost after that. Anyhelp ?

Thanks,
T

hi vanderlin …

well basically what we’ve done is :

  • setting up a ofTCPServer in the OF side (open IP and PORT)
  • in the flash side open an XMLsocket object with the IP and port specified.
  • all the messages in order to be read by flash need to be finished with a (byte)0 <— old style end of string !!

what we’ve changed from the ofTCPServer and ofTCPClient code is the [EndofFile] … take a look at the ofTCP code and look for [ofTCPEnd] in the .cpp files.

we’ve replaced the [ofTCPEnd] by (char)’\0’ in order to make all the strings sent through the TCP connection ended by the /0…

hope it helps … let me know if u success, i can pass u some code of the OF side if needed.

e*

It’s a good thing to make a simple TCP monitor on the flash end so you can always read what’s comming from OF, and where u can write down some chars from Flash to OF to test the protocole.

Huh that is strange.

The current ofTCPServer uses
‘[ofTCPEnd]\0’ so it can work with both the client and also flash.

Well I guess if you are trying to conserve number of bytes this makes sense - but for a more complete class I would use ‘[ofTCPEnd]\0’ (without the quotes).

Then every message that arrives in flash has a [ofTCPEnd] which you can strip off.

Here is the code I use to get laser tag data into flash - i am sending just an x and y coordinate using a ‘!’ character to separate the x and y coordinates.

WARNING Actionscript code :slight_smile:

  
System.security.allowDomain("*")   
var mySock = new XMLSocket();   
mySock.onData = function(src){   
    //data coming in is formatted   
    //x!y[ofTCPEnd]\0   
    //eg: 0.453!0.223[ofTCPEnd]\0   
    //trace("DATA IS " + src);   
    var strArr = src.split("!");   
    var strX   = Number(strArr[0]);   
    var strArr = strArr[1].split("[ofTCPEnd]");   
    var strY   = Number(strArr[0]);   
    var x = strX*640;   
    var y = strY*480;   
    //now we can draw with x and y   
}   
mySock.onClose = function(){   
    //trace("connection closed");   
}   
if( !mySock.connect("10.10.10.130", 10420) ){   
    //trace ("connect: Connection failed!");   
}