ofxLibwebsockets

Hey all,

Just starting to put the finishing touches on a new OF wrapper for libwebsockets, ofxLibwebsockets.

https://github.com/labatrockwell/ofxLibwebsockets

It’s based on the excellent work by paul reimer (https://github.com/paulreimer/ofxWebUI) and jason vancleave (https://github.com/jvcleave/ofxLibWebSockets).

The main difference is that I’ve added support for both WebSocket clients and servers, allowing for the following cases:

* OF app as WebSocket server, allowing anyone who can speak WebSockets (javascript, python, etc) talk to it
* OF app as a client connecting to a OF WebSocket server
* OF app as a client of a remote or local WebSocket server, so you can send packets from OF to servers anywhere

There are definitely some bugs in the library, and my examples right now don’t really do much… But I’d love for people to check it out and let me know what you think! Feel free to post here or add issues on git.

Hey,

I tried to use ofxLibwebsockets under Linux and got it compiled, but it’s crashing. I documented my steps: http://frauzufall.de/en/2012/ubuntu-12-10-and-libwebsockets/

I installed the original libwebsockets version as well as your clone. The segmentation fault in the end was the same.

Thank you for all the work, perhaps this can be of any use. I would be very happy for hints regarding openFrameworks and WebSockets under Linux.

Deborah

Hey Deborah,

Sorry you’re having issues. Your blog post is great, will factor in the changes you noted.
Unfortunately, I’m on OS X, so I can’t seem to reproduce the same error. Do both the
libwebsockets test-server and test-client work for you? I’m trying to figure out the
segfault, but it’s pretty weird. In the server example, can you comment out line 7

  
//bConnected = server.setup( 9093 );  

and uncomment lines 14-19:

  
ofxLibwebsockets::ServerOptions options = ofxLibwebsockets::defaultServerOptions();  
    options.port = 9092;  
    options.protocol = "of-protocol";  
    bConnected = server.setup( options );  

Then, in the web folder (bin/data/web), edit js/main.js and add the prototcol to the connect event:

  
if (BrowserDetect.browser == "Firefox") {  
		socket = new MozWebSocket(get_appropriate_ws_url(), "of-protocol");  
	} else {  
		socket = new WebSocket(get_appropriate_ws_url(), "of-protocol");  
	}  

Let me know if that works; it’s not as flexible, as you will need to always use that protocol name when you connect. But, if that works maybe I can track down the problem lines in the library.

Best,

Brett

Hey Brett,

these libwebsocket-examples both work together:

  
libwebsockets-test-server  
libwebsockets-test-client localhost --port 7681  

I edited the lines but the result is the same. The following are the values of the objects my debugger shows when the segmentation fault happens. This looks a bit messy but perhaps there is some valuable data?

  
	buf	<unavailable synchronous data>	  
	context		libwebsocket_context  
		canonical_hostname	@0x10c0f50	char [1024]  
		count_protocols	1	int  
		extensions	 @0xf40020	libwebsocket_extension  
		fd_hashtable	@0x10b4440	libwebsocket_fd_hashtable [32]  
		fd_random	14	int  
		fds	@0x10baa40	pollfd [3201]  
		fds_count	4	int  
		http_proxy_address	@0x10c0e50	char [256]  
		http_proxy_port	0	unsigned int  
		last_timeout_check_s	1350402178	long unsigned int  
		listen_port	9092	int  
		options	0	unsigned int  
		protocols	 @0x109d2e0	libwebsocket_protocols  
			broadcast_socket_port	50083	int  
			broadcast_socket_user_fd	0	int  
			callback	<optimized out typedef> @0x431e7e	callback_function  
			name	"of-protocol"	char *  
			owning_server	 @0x10b4440	libwebsocket_context  
			per_session_data_size	80	size_t  
			protocol_index	0	int  
	len	<optimized out typedef>	size_t  
	n	<optimized out typedef>	size_t  
	wsi		libwebsocket  
		active_extensions	@0x7fffec0008c8	libwebsocket_extension *[10]  
		active_extensions_user	@0x7fffec000918	void *[10]  
		all_zero_nonce	0 '\0'	char  
		c_address	0x0	char *  
		c_callback	0x0	callback_function *  
		c_host	0x0	char *  
		c_origin	0x0	char *  
		c_path	0x0	char *  
		c_port	0	int  
		c_protocol	0x0	char *  
		candidate_children_list	0x0	libwebsocket *  
		close_reason	LWS_CLOSE_STATUS_NOSTATUS (0)	enum lws_close_status  
		count_active_extensions	0	int  
		current_alloc_len	256	int  
		extension_data_pending	0 '\0'	char  
		extension_handles	0x0	libwebsocket *  
		final	0 '\0'	unsigned char  
		frame_mask_04	@0x7fffec001bde	unsigned char [20]  
		frame_mask_index	0 '\0'	unsigned char  
		frame_masking_nonce_04	@0x7fffec001bda	unsigned char [4]  
		ietf_spec_revision	0	int  
		initial_handshake_hash_base64	@0x7fffec001c19	char [30]  
		key_b64	@0x7fffec001b30	char [150]  
		lws_rx_parse_state	LWS_RXPS_NEW (0)	enum lws_rx_parse_state  
		masking_key_04	@0x7fffec001bc6	unsigned char [20]  
		mode	LWS_CONNMODE_WS_SERVING (0)	enum connection_mode  
		name_buffer	@0x7fffec000970	char [64]  
		name_buffer_pos	2	int  
		opcode	0 '\0'	unsigned char  
		parser_state	WSI_PARSING_COMPLETE (22)	enum lws_token_indexes  
		pending_timeout	NO_PENDING_TIMEOUT (0)	enum pending_timeout  
		pending_timeout_limit	0	long unsigned int  
		pings_vs_pongs	0	int  
		protocol	 @0x109d2e0	libwebsocket_protocols  
			broadcast_socket_port	50083	int  
			broadcast_socket_user_fd	0	int  
			callback	<optimized out typedef> @0x431e7e	callback_function  
			name	"of-protocol"	char *  
			owning_server	 @0x10b4440	libwebsocket_context  
				canonical_hostname	@0x10c0f50	char [1024]  
				count_protocols	1	int  
				extensions	 @0xf40020	libwebsocket_extension  
				fd_hashtable	@0x10b4440	libwebsocket_fd_hashtable [32]  
				fd_random	14	int  
				fds	@0x10baa40	pollfd [3201]  
				fds_count	4	int  
				http_proxy_address	@0x10c0e50	char [256]  
				http_proxy_port	0	unsigned int  
				last_timeout_check_s	1350402178	long unsigned int  
				listen_port	9092	int  
				options	0	unsigned int  
				protocols	 @0x109d2e0	libwebsocket_protocols  
			per_session_data_size	80	size_t  
			protocol_index	0	int  
		protocol_index_for_broadcast_proxy	0	int  
		rx_frame_type	LWS_WRITE_TEXT (0)	enum libwebsocket_write_protocol  
		rx_packet_length	0	size_t  
		rx_user_buffer	@0x7fffec000ae4	char [4115]  
		rx_user_buffer_head	0	int  
		sock	17	int  
		state	WSI_STATE_HTTP (0)	enum lws_connection_states  
		this_frame_masked	0 '\0'	char  
		user_space	0x0	void *  
		utf8_token	@0x7fffec0009c0	lws_tokens [18]  
		xor_mask	0	unsigned char (*)(struct libwebsocket *, unsigned char)  
  

I am now using UDP messages from openFrameworks to the server and WebSockets from the server to the client, that’s working alright.

But since I guess there is much effort in your work and since it would be pretty cool to be able to use WebSockets with Linux someday, I would be pleased to test new code.

Hey Deborah,

Good to hear the libwebsockets examples work, means the fault is in my code.

Looking into the segfault. I don’t know a ton about linux, but will let you know if/when I figure something out! There is a fork on Github that is linux compliant, but it looks like it’s mostly just the same changes you made! https://github.com/emmanuelgeoffray/ofxLibwebsockets

Let me know if you have any more insights! We’ve been using this on OSX and Windows with really consistent performance… Is there anything you can think of that would make it different on Linux?

Thanks!

Brett

Hey Deborah, quick update!
Can you try the “windows-fix” branch on our git repo? https://github.com/labatrockwell/ofxLibwebsockets/tree/windows-fix

I was getting a segfault on the same line under windows, turned out
that we were just out of sync with the newest libwebsockets. Let me know
if this fixes it! You’ll have to install from the latest libwebsockets branch
(not our fork).

Hope that works!

Brett

Hey Brett,

yes, it’s working, thank you!

EDIT: It’s partially working. The Server example is fine, but I am still not able to connect with echo.websocket.org.

  
setgid: Operation not permitted  
libwebsocket init success  
client connection success  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
ERROR writing to socket  
lws_client_interpret_server_handshake: NULL c_protocol  
lws_client_interpret_server_handshake WSI_TOKEN_PROTOCOL is null  
on open  
Segmentation fault (core dumped)  
  

The segmentation fault appears at line 690 in libwebsockets/lib/libwebsockets.c.

Regards, Deborah

Hey Deborah, great to hear! I’ve merged most of the changes into the main branch; can you check and make sure you’re up-to-date?

It looks like the client is trying to send before it’s connected, but then crashes once it actually is connected and can send. Did you connect with or without SSL? (bool connected = client.connect( “echo.websocket.org” ); is without,
bool connected = client.connect( “echo.websocket.org”, true ); is with)

Just checking this out today. Great work on the library. I noticed you have code in the JS for handling binary Blob transfers directly, rather than looping over pixels in an arraybuffer. Have you tried sending these with oF? Displaying each pixel from a large arraybuffer is pretty slow. I assume I need to do some jpeg/png encoding… but I’m unsure exactly how to send it through this lib.

Glad you’re enjoying the addon!

You’re right, the way we’re doing the binary transfer in the examples is among the least efficient ways to do this. We’ve played around with sending images with proper headers, but don’t have a simple example. The best way to go about it to use something like libturbojpeg to encode a jpeg in OF and send that via the binary socket. Then, you can receive that in the frontend and dump it right into an img tag, I believe through the experimental URL.createObjectURL on the binary Blob data: https://developer.mozilla.org/en-US/docs/Web/API/URL.createObjectURL

Here’s an addon for TurboJpeg: https://github.com/armadillu/ofxTurboJpeg

Hope that helps! Let me know if you get it working, would love to include that as an advanced example.

Started working on this. MUCH faster with blobs: https://github.com/zebradog/ofxLibwebsockets/ (needs a modified version of ofxTurboJpeg: https://github.com/zebradog/ofxTurboJpeg). Was able to get 1024x768 stream at 30fps with minimal latency. Unfortunately, it appears Chrome has trouble clearing the received websocket message blob from memory at that rate… so memory leaks extremely quickly.

I’ll be playing around with base64 encoding it with oF and sending that, or perhaps even encoding a mjpeg stream and polling: http://stackoverflow.com/questions/13500558/motion-jpeg-in-html5-canvas

Would be interesting to benchmark the different solutions.

Fantastic! That’s super exciting, would love to add these examples to the core. Keep us posted as you continue to experiment and toss us a PR when you think things are good to share!

What happens with the Chrome leak? Does it just run out of memory and crash?

Hey,
I was running into the same problem with the browser crashing after about 7 minutes at 30fps. I finally ran across a function on the Blob “revokeObjectURL” which you can call after the data has been loaded into an image tag (on the image onLoad callback or whatever). Once I started using that, I never had a memory leak. I could run the video streamer for 2+hours with no issues.

oh, and base64-encoding I believe triples the amount of data, so as opposed to JPG where you are compressing your data and sending a smaller packet, you may pretty quickly run into bandwidth issues with base64 encoding. If you still do it, benchmarks would be cool though.

hello,i am trying your addon in vs2012. i have followed your instructions to add the lines in c+±>general and linker->general and linker->input. but it failed to build.
it shows below:
Error 1 error C2039: ‘clock_t’ : is not a member of ‘global namespace'' c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 18 1 example_client_hello_world Error 2 error C2873: 'clock_t' : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 18 1 example_client_hello_world Error 3 error C2039: 'asctime' : is not a member of 'global namespace’’ c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world
Error 4 error C2873: ‘asctime’ : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world
Error 5 error C2039: ‘clock’ : is not a member of ‘global namespace'' c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world Error 6 error C2873: 'clock' : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world Error 7 error C2039: 'ctime' : is not a member of 'global namespace’’ c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world
Error 8 error C2873: ‘ctime’ : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 20 1 example_client_hello_world
Error 9 error C2039: ‘difftime’ : is not a member of ‘global namespace'' c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world Error 10 error C2873: 'difftime' : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world Error 11 error C2039: 'gmtime' : is not a member of 'global namespace’’ c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world
Error 12 error C2873: ‘gmtime’ : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world
Error 13 error C2039: ‘localtime’ : is not a member of ‘global namespace'' c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world Error 14 error C2873: 'localtime' : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 21 1 example_client_hello_world Error 15 error C2039: 'mktime' : is not a member of 'global namespace’’ c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world
Error 16 error C2873: ‘mktime’ : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world
Error 17 error C2039: ‘strftime’ : is not a member of ‘global namespace'' c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world Error 18 error C2873: 'strftime' : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world Error 19 error C2039: 'time' : is not a member of 'global namespace’’ c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world
Error 20 error C2873: ‘time’ : symbol cannot be used in a using-declaration c:\program files (x86)\microsoft visual studio 11.0\vc\include\ctime 22 1 example_client_hello_world