How to OF + WebRTC

#1

Anybody have any idea where to get started with implementing a WebRTC video client in OF (for MacOS)?

I’m building an OF application that runs on a mac mini that needs to do some low latency video chatting via WebRTC.

There seems to be a lot of support for Android and iOS, see https://www.twilio.com .

Anyone have experience doing this? this post: https://forum.openframeworks.cc/t/of-used-to-create-an-interactive-christmas-miracle/13927/3 describes an integrated WebRTC approach by @globacore but has no description of how it was done…

#2

this: https://github.com/arturoc/ofxGstRTP can do video & audio in sync over rtp, do nat transversal using stun… which are the protocols that webrtc uses internally but it won’t be directly compatible with webrtc so if you need to communicate with other webrtc client it won’t work but if you just need to send video/audio between 2 OF apps it’ll do

#3

interesting, thanks Arturo! We’re actually already using ofxGstreamer to playback RTSP streams…

the requirement for this one is to actually integrate with a server that’s managing/hosting/archiving (and potentially providing the signaling needed to setup a webRTC connection.) I am definitely considering a “roll your own” approach using gstreamer to decode WebRTC (if it’s even possible), but this seems it might be a massive rabbit hole.

#4

Just a heads-up that Gstreamer 1.14+ now has the webrtc component. I did some work on ofxGStreamer to get it compiling w/ macOS via the brew tap installed GStreamer, or via manual builds of gstreamer components.

Fork: https://github.com/autr/ofxGStreamer
Notes: https://github.com/autr/ofxGStreamer/blob/master/Notes.md

It should be possible with gstreamer 1.14+ to get webrtcsink pipeline via ofGSTUtils. When compiling the samples, the final pipeline looks like:

# via https://gitlab.freedesktop.org/gstreamer/gst-examples

"webrtcbin bundle-policy=max-bundle name=sendrecv  stun-server=stun://stun.l.google.com:19302 videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! queue ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! sendrecv. audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! sendrecv. "
3 Likes