OpenFrameworks as a Unity plugin

I want to use OpenFrameworks to get information from a webcam or a Kinect-like device, process it in OpenCV and send it to the Unity3D game engine. I have all of the parts working:

  • I can process the image data in OpenFrameworks.
  • I can send placeholder data as a Unity plugin from C++ to C#.
  • I compiled my OpenFrameworks app as a shared library.

However, when I try to load my OpenFrameworks app to Unity as a plugin, the Unity app segfaults.

I’m on Ubuntu Linux 64bit.

Any call to cout crashes Unity. I set the application to OF_LOG_SILENT, but there might be more. I was sure that OpenFrameworks would work fine as a plugin, but it turns out it seems I’m wrong about that. I’ve seen a few opinions about this on this forum, but can anyone weight in on whether I should persist with OF or turn to something else?

1 Like

Although lacking actual code makes difficult to investigate your problem, I guess the solution is to link the Unity app to C++ standard library as cout is causing problems. I remember I couldn’t use I/O functions when I tried linking C++ and D.

Btw, doesn’t switching to OSC fulfill your needs? (note that UnityOSC does not support OSC bundles so you need to tweak ofxOsc)

My codebase it too big to post here, and any part of it might cause a problem. For now, because we’re pressed by time, I will be going the OSC route. Then maybe try using OpenCV and OpenNI without OpenFrameworks. (Which is a shame because I’m really extremely impressed by OF.)

I had to recompile Poco with -fPIC to get my library to compile as a shared object. I think that might have broken things. In hindsight, it’s very likely that my problem is too nebulous for someone else to diagnose. Now that I’m thinking about it, I could make a dummy C++ app that calls my OF library to see where it crashes. If I did that, I’d have access to a debugger and it would not be as opaque as with Unity, which really is of no help in this case.

What I was really wondering about is if people out there had successfully gotten OF to work as a library and if so why, what were the advantages and what were the drawbacks. As much as I have grown to love OpenFrameworks, it simply does not look like it’s made to build libraries.

Hey I have an example of openFrameworks compiling to a Unity plugin working here from April 2013

You can even draw in the unity world with openFrameworks drawing commands – but there was some issues with the cameras.

Thanks, @obviousjim. I actually found that example yesterday, but it seems to be very Mac-specific. Cross-platform with a strong emphasis on Linux is a high priority for us. And I only want the data to be sent to Unity; I don’t want to draw from OF. I’m just sending the data from an OpenCV optical flow calculation. I’ll look at your example again with fresher eyes, though. I was probably to tired yesterday to understand anything.

As a massive fan of both, if I had some more time, I’d try and help out. Maybe in spring? Jim, I’d love to see at least mac support. Drawing 2d in Unity is like devishly hard. It would be nice to have this as an additional creative option. Any plans to do more with it?

I was wondering if anyone has investigated this further? My main objective is using OF as a communicator with hardware. I’ve tries OSC but would prefer something more plugin-like.