How to OF + WebRTC

beginners
#1

Anybody have any idea where to get started with implementing a WebRTC video client in OF (for MacOS)?

I’m building an OF application that runs on a mac mini that needs to do some low latency video chatting via WebRTC.

There seems to be a lot of support for Android and iOS, see https://www.twilio.com .

Anyone have experience doing this? this post: https://forum.openframeworks.cc/t/of-used-to-create-an-interactive-christmas-miracle/13927/3 describes an integrated WebRTC approach by @globacore but has no description of how it was done…

#2

this: https://github.com/arturoc/ofxGstRTP can do video & audio in sync over rtp, do nat transversal using stun… which are the protocols that webrtc uses internally but it won’t be directly compatible with webrtc so if you need to communicate with other webrtc client it won’t work but if you just need to send video/audio between 2 OF apps it’ll do

#3

interesting, thanks Arturo! We’re actually already using ofxGstreamer to playback RTSP streams…

the requirement for this one is to actually integrate with a server that’s managing/hosting/archiving (and potentially providing the signaling needed to setup a webRTC connection.) I am definitely considering a “roll your own” approach using gstreamer to decode WebRTC (if it’s even possible), but this seems it might be a massive rabbit hole.

#4

Just a heads-up that Gstreamer 1.14+ now has the webrtc component. I did some work on ofxGStreamer to get it compiling w/ macOS via the brew tap installed GStreamer, or via manual builds of gstreamer components.

Fork: https://github.com/autr/ofxGStreamer
Notes: https://github.com/autr/ofxGStreamer/blob/master/Notes.md

It should be possible with gstreamer 1.14+ to get webrtcsink pipeline via ofGSTUtils. When compiling the samples, the final pipeline looks like:

# via https://gitlab.freedesktop.org/gstreamer/gst-examples

"webrtcbin bundle-policy=max-bundle name=sendrecv  stun-server=stun://stun.l.google.com:19302 videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! queue ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! sendrecv. audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! sendrecv. "
4 Likes
#5

Hi ! Great topic !
For an artistic project, I need to make an OF app that can send and receive webRTC audio/video to/from other devices (using a web app)…
I’m trying your solution but nothing works when I test examples. Both, Arturo and autr “ofxGStreamer”.
Are there out to date ?
Or the problem come from my setup (Mac OS X 10.15.7 / Xcode 11.6) ?
Or maybe I do not correctly install or make something ?

With Arturo addon and ofxGstRTP examples, the error is :

‘file_wrapper.h’ file not found

In audio_processing_impl.cc

With autr addon and his ofxGStreamer examples, the error is :

d: file not found: @rpath/lib/libgstnet-1.0.0.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)