I’m currently trying to code an augmented reality mobile app.
is it possible to have a three.js rendered scene with a transparent background on top of the device “environment” camera stream below?
the goal is to use opencv with openFrameworks to recognize monument facades from the camera then show some three.js objects on top.
You could try to compile first your OF project with emscripten (http://openframeworks.cc/setup/emscripten/), you will obtain an html page with a link to your OF project translated to js.
running opencv on the browser on mobile is proabbly going to be to much. i don’t think there’s also an easy way to comunicate between OF and three.js but i guess you could create an application that has 2 views in the main activity and one of them is an html viewer then send some kind of message from one view to the other from the java side but it’s kind of convoluted and it’s surely easier to just use OF for everything
I dont know about IOS, but I’ve had good luck with a transparent webkit overlay on straight osx, using the approach from @admsyn:
maybe the code / approach is a good place to look?
thank you anyway
yes i was talking about android, it should be easier to do in ios
To be more clear:
I want to code in c++, no obj-c nor java.
I want to reuse 90% of my code for each platform.
For maximum performance, c++ for camera input, opencv to get the corners matching, then show some three.js scene with camera locked to the device using the values from the gyroscope using c++.
androidOpenCVFaceExample is a good start, I can build upon it…
I guess/hope I can make it
i’m almost sure there’s no way to do this in c++ alone in android, at least not by having a browser view you’ll always need some java code to interface between the 2 parts of the application. another possibility woudl be to wrap some browser library like CEF in OF and run the js app from there but it seems pretty overkill for a mobile app
You may need to wrap SpiderMonkey in an ofxaddon, and use three.js on top of SM.
ps. This repo has an Android build of SM.