Render HTML Browser (with Flash) to opegl Texture?

matthias · August 22, 2009, 10:37am

Hello all,

I want to use a Flash gui inside an openframeworks application, and came up with the idea to use an offscreen htmlbroswer and render its contents to an opengl texture. Connection for interaction with the flash movie might be through flosc or maybe via the externalinterface of flash (?)

I want to use Flash, because it is much easier for me to integrate complicated layouts, but i need openframeworks, becuase the projection for the interface is a halfsphere, so we need to “stretch” the texture on a bezier-surface. (view screenshot)

I found several resources on the net:

a opengl browser based on llmozlib
http://www.ubrowser.com/

awesomium, based on google chrome
http://princeofcode.com/blog/?p=28

of maybe a flashplugin renderer like flashinapp that only works within cocoa?
http://mac.eltima.com/cocoa-framework.html

There is also gnash, a gnu flash renderer (?)
http://www.gnu.org/software/gnash/

And a commercial solution, scaleform gfx
http://www.scaleform.com/

Does anyone have experience with one of the listed solutions and managed to integrate with of?

drwoohoo · August 23, 2009, 6:54pm

/ *
it’s not OSC, but here’s an example using flex builder <-> OF communicating via a TCP socket that might be helpful. i use it as an alternative sometimes to the assume simpleGuiToo when i need to tap into ‘components’ that are unsupported in sg2 and when i need my control panel to be removed from the actual piece that is projected.

http://glue69.ning.com/page/flashflex-builder-1

* /

woohoo!
drew

mikecreighton · August 23, 2009, 8:19pm

I think that you’ll find Scaleform GFx to be the most functional for what you want to do. Additionally, it’s all hardware accelerated as well, so everything is incredibly fluid. However, it’s obviously marketed toward game development studios as an easy way to create UIs.

I’m not sure what the price tag is for a license, but I know it’s not cheap. I personally haven’t used it with openFrameworks, but I think it’s pretty easy to integrate. I’ve really only been on the Flash side of working with GFx.

You’ll have to develop everything with a subset of Flash 8’s ActionScript 2 language API. Overall, it’s incredibly capable and makes building any sort of 2D user interface ridiculously easy.

memo · August 25, 2009, 7:19am

If you’re on mac, I"d also recommend Quartz Composer. Setting up the UI is as simple as just declaring the property and publishing it, then you just tie it to an OSC Sender and you’re laughing! (the composition itself doesn’t even need to do anything visually, other than send all the params as OSC).
http://vimeo.com/1569676

matthias · August 25, 2009, 8:08am

Thanks for your answers so far…
My main Problem to be solved is not the connection between flash and of, i think there will be various ways to do so.

My first and (i.m.o) biggest problem is:
How do i get the interactive Flash visuals into an opengl Texture that i can bind to a bezier surface.

I thought a “webbrowserrenderer” might do so, or maybe there is a way to directly use the flash plugin, maybe there are other ways, anyone has experience?

I need to make a decision, what framewoks to use for a quite large project, and if flash doesnt work, we will have to build the ui with openframeworks itself, thats the alternative. But i think that will be much more of an effort, since we would have to build all gui elements by ourselves, for instance a textfield with multiple textweights and so on.

Thank you far all suggestions.

sth · August 25, 2009, 11:18am

what platform?

I’ve used zFlash (see http://vjforums.com/showthread.php?t=19743 for more info) to embed flash content in OpenSceneGraph on Windows (should be possible with OpenFrameWorks too) and a cocoa webview with a loaded html which embeds flash-content on OS X. Both versions did work quite well, but I didn’t work out sending events to the flash-plugin. I was only interested in playback, performance was ok,but not as good as in a browser…

There are even some more solutions: http://f-in-box.com/ or http://mac.eltima.com/cocoa-framework.html (which mac/pc, costs money)

HTH,
Stephan

croby · October 8, 2009, 2:20am

Anyone have any luck with this? I’m also working on a few projects that could use an embedded web browser. Awesomium looks promising, but I can’t get the libraries to link in the last compiling stage. i haven’t been able to get llmozlib (and llmozlib2 for that matter) working (compiling).

Has anyone else dealt with this? There must be some way to embed a web browser inside of an opengl environment.

jefftimesten · January 13, 2011, 9:01pm

Here is an example of how to embed a WebKit window using Awesomium – OSX only

http://4u.jeffcrouse.info/of/Awesomium.zip

There are several disadvantages

Free for non-commercial, but then goed up to $300 for small company
No linux

Otherwise it is pretty, ahem, awesome.

edit:
PS: I am working on getting Berkelium working now, which has neither of those disadvantages

jvcleave · January 14, 2011, 8:09am

Thanks Jeff - never heard of this - pretty amazing (Flash works too, and well!)

liquid · January 15, 2011, 5:34am

yes,the Awesomium works really well with of, and pretty straightforward.

jefftimesten · January 17, 2011, 12:04am

I just got Berkelium working. It seems to be a little slower, and the code isn’t as pretty, but it’s free (BSD licensed), and there is a linux version. It supports Flash.

Enjoy:

http://4u.jeffcrouse.info/of/Berkelium.zip

It says this in the code, but if you are copying this project, make sure to take a look at the “Run Script” build phase. Several things have to be copied into the executable bundle.

This is only the Mac version, but you could get the binaries for other platforms here:

https://github.com/sirikata/berkelium/downloads

jvcleave · January 17, 2011, 5:18am

Thanks again Jeff. I was able to Build and Run the Berkelium version but one strange thing is that I get 2-3 app icons (instances?) in the Dock. Is this normal?

jefftimesten · January 17, 2011, 4:21pm

Wow – I hadn’t noticed that. When you quit out of the apps, some of the delegate methods get called:

*** onCrashedPlugin Flash Player
*** onCrashedPlugin Flash Player
*** onCrashedPlugin Flash Player
*** onCrashed

So it seems the first app is the oF one, the second one drives the browser, and the third is the Flash plugin. After quitting the 2nd one, navigation no longer works.

You can fix that by adding “Application is Agent” to the openFrameworks-info.plist, as-described-in-under-“Issue-#2”-in-this-thread. That same thread says that you have to call TransformProcessType on startup to ensure that the application gets its dock icon back, but I haven’t looked into that yet.

jefftimesten · January 17, 2011, 9:43pm

I made this into a real add-on, added a listener class, and fixed some other things. Here is a new version:

http://github.com/jefftimesten/ofxBerkelium

I am still having some trouble with passing keyboard events, so if anyone has has any ideas and wants me to add them to the GitHub project, let me know.

theo · January 18, 2011, 4:26pm

wowza - that is epic!

hal · February 8, 2011, 12:35am

Hi,

First off, great job on the Berkilium Jeff.

I have 2 questions:

1.I have run Berkilium and Awesomium and neither include a scroll function.
Is there a reason that this can’t be added? It is probably a bit beyond me to add it and I don’t want to try if people already know of reasons that are going to make it really difficult.

In both cases I am having trouble with some embedded videos. youtube and vimeo are fine but things like bbc iplayer (http://www.bbc.co.uk/iplayer/) won’t run. Is this due to flash issues, is there a possibe fix?

any help would be much appreciated.

thanks for the raps!
hal

chemabc · April 19, 2011, 3:11pm

Hi,

I have been playing with Berkelium and the berkelium addon using VisualStudio 2008 y 2010. I have been searching for berkelium.a to use it in codeblocks (I prefer open-source) but I couldn’t find it. With berkelium.lib for VS does not work.
My B plan is try to compile from source but I already tried twice 2 weeks ago and after 1 hour compilation it gave me lot of errors. So my A plan , is to ask you if someone please has already compiled berkelium for CodeBlocks with MinGw and has the required library (“berkelium.a” presumably or “libberkelium.a”) to compile and use the dll.

Thanks for your help!
And thanks to jefftimesten for the addon!!

akorchuk · May 8, 2011, 9:44pm

Hi.

I am trying to compile ofxBerkelium example in VS2010 but receive an link error “unresolved external symbol”:

1>------ Build started: Project: openframeworksLib, Configuration: Debug Win32 ------
1>Build started 09.05.2011 0:11:53.
1>InitializeBuildStatus:
1> Creating “obj\Debug\openframeworksLib.unsuccessfulbuild” because “AlwaysCreate” was specified.
1>ClCompile:
1> All outputs are up-to-date.
1>Lib:
1> All outputs are up-to-date.
1> openframeworksLib.vcxproj -> D:\programing\of_preRelease_v0062_vs2010_FAT\libs\openFrameworksCompiled\project\vs2010…\lib\vs2010\openframeworksLibDebug.lib
1>FinalizeBuildStatus:
1> Deleting file “obj\Debug\openframeworksLib.unsuccessfulbuild”.
1> Touching “obj\Debug\openframeworksLib.lastbuildstate”.
1>
1>Build succeeded.
1>
1>Time Elapsed 00:00:00.14
2>------ Build started: Project: emptyExample, Configuration: Debug Win32 ------
2>Build started 09.05.2011 0:11:53.
2>InitializeBuildStatus:
2> Touching “obj\Debug\emptyExample.unsuccessfulbuild”.
2>ClCompile:
2> All outputs are up-to-date.
2>ManifestResourceCompile:
2> All outputs are up-to-date.
2>main.obj : warning LNK4075: ignoring ‘/EDITANDCONTINUE’ due to ‘/INCREMENTAL:NO’ specification
2>ofxBerkelium.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: virtual __thiscall Berkelium::WindowDelegate::~WindowDelegate(void)” (__imp_??1WindowDelegate@Berkelium@@UAE@XZ) referenced in function __unwindfunclet$??0ofxBerkelium@@QAE@II_N@Z$0
2>ofxBerkelium.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: void __thiscall Berkelium::Window::setDelegate(class Berkelium::WindowDelegate *)” (__imp_?setDelegate@Window@Berkelium@@QAEXPAVWindowDelegate@2@@Z) referenced in function “public: __thiscall ofxBerkelium::ofxBerkelium(unsigned int,unsigned int,bool)” (??0ofxBerkelium@@QAE@II_N@Z)
2>ofxBerkelium.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: static class Berkelium::Window * __cdecl Berkelium::Window::create(class Berkelium::Context const *)” (__imp_?create@Window@Berkelium@@SAPAV12@PBVContext@2@@Z) referenced in function “public: __thiscall ofxBerkelium::ofxBerkelium(unsigned int,unsigned int,bool)” (??0ofxBerkelium@@QAE@II_N@Z)
2>ofxBerkelium.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: static class Berkelium::Context * __cdecl Berkelium::Context::create(void)” (__imp_?create@Context@Berkelium@@SAPAV12@XZ) referenced in function “public: __thiscall ofxBerkelium::ofxBerkelium(unsigned int,unsigned int,bool)” (??0ofxBerkelium@@QAE@II_N@Z)
2>ofxBerkelium.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: __thiscall Berkelium::WindowDelegate::WindowDelegate(void)” (__imp_??0WindowDelegate@Berkelium@@QAE@XZ) referenced in function “public: __thiscall ofxBerkelium::ofxBerkelium(unsigned int,unsigned int,bool)” (??0ofxBerkelium@@QAE@II_N@Z)
2>ofxBerkelium.obj : error LNK2001: unresolved external symbol “public: virtual void __thiscall Berkelium::WindowDelegate::freeLastScriptAlert(struct Berkelium::WeakString<wchar_t>)” (?freeLastScriptAlert@WindowDelegate@Berkelium@@UAEXU?$WeakString@_W@2@@Z)
2>ofxBerkelium.obj : error LNK2001: unresolved external symbol “public: virtual void __thiscall Berkelium::WindowDelegate::onWidgetDestroyed(class Berkelium::Window *,class Berkelium::Widget *)” (?onWidgetDestroyed@WindowDelegate@Berkelium@@UAEXPAVWindow@2@PAVWidget@2@@Z)
2>ofxBerkelium.obj : error LNK2001: unresolved external symbol “public: virtual void __thiscall Berkelium::WindowDelegate::onWidgetPaint(class Berkelium::Window *,class Berkelium::Widget *,unsigned char const *,struct Berkelium::Rect const &,unsigned int,struct Berkelium::Rect const *,int,int,struct Berkelium::Rect const &)” (?onWidgetPaint@WindowDelegate@Berkelium@@UAEXPAVWindow@2@PAVWidget@2@PBEABURect@2@IPBU52@HH3@Z)
2>ofxBerkelium.obj : error LNK2001: unresolved external symbol “public: virtual void __thiscall Berkelium::WindowDelegate::onCursorUpdated(class Berkelium::Window *,class Berkelium::Cursor const &)” (?onCursorUpdated@WindowDelegate@Berkelium@@UAEXPAVWindow@2@ABVCursor@2@@Z)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) void __cdecl Berkelium::init(struct Berkelium::WeakString<wchar_t>)” (__imp_?init@Berkelium@@YAXU?$WeakString@_W@1@@Z) referenced in function “public: virtual void __thiscall testApp::setup(void)” (?setup@testApp@@UAEXXZ)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: bool __thiscall Berkelium::Window::navigateTo(char const *,unsigned int)” (__imp_?navigateTo@Window@Berkelium@@QAE_NPBDI@Z) referenced in function “public: virtual void __thiscall testApp::update(void)” (?update@testApp@@UAEXXZ)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) void __cdecl Berkelium::update(void)” (__imp_?update@Berkelium@@YAXXZ) referenced in function “public: virtual void __thiscall testApp::update(void)” (?update@testApp@@UAEXXZ)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: __thiscall Berkelium::Script::Variant::~Variant(void)” (__imp_??1Variant@Script@Berkelium@@QAE@XZ) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: __thiscall Berkelium::Script::Variant::Variant(void)” (__imp_??0Variant@Script@Berkelium@@QAE@XZ) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) public: __thiscall Berkelium::Script::Variant::Variant(class Berkelium::Script::Variant const &)” (__imp_??0Variant@Script@Berkelium@@QAE@ABV012@@Z) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) void __cdecl Berkelium::Script::toJSON_free(struct Berkelium::WeakString<wchar_t>)” (__imp_?toJSON_free@Script@Berkelium@@YAXU?$WeakString@_W@2@@Z) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: struct Berkelium::WeakString<wchar_t> __thiscall Berkelium::Script::Variant::toString(void)const " (__imp_?toString@Variant@Script@Berkelium@@QBE?AU?$WeakString@_W@3@XZ) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: enum Berkelium::Script::Variant::Type __thiscall Berkelium::Script::Variant::type(void)const " (__imp_?type@Variant@Script@Berkelium@@QBE?AW4Type@123@XZ) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>testApp.obj : error LNK2019: unresolved external symbol “__declspec(dllimport) struct Berkelium::WeakString<wchar_t> __cdecl Berkelium::Script::toJSON(class Berkelium::Script::Variant const &)” (__imp_?toJSON@Script@Berkelium@@YA?AU?$WeakString@_W@2@ABVVariant@12@@Z) referenced in function “public: virtual void __thiscall testApp::onJavascriptCallback(void *,class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> >,class Berkelium::Script::Variant *,unsigned int)” (?onJavascriptCallback@testApp@@UAEXPAXV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@3@PAVVariant@Script@Berkelium@@I@Z)
2>bin\emptyExample_debug.exe : fatal error LNK1120: 19 unresolved externals
2>
2>Build FAILED.
2>
2>Time Elapsed 00:00:01.54
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

I understand that ofxBerkelium is missing library for VS2010 so I’ve tried with berkelium.lib from https://github.com/downloads/sirikata/berkelium/berkelium-win32-beta-2011-04-29.7z and from https://github.com/downloads/sirikata/berkelium/berkelium-win32-2010-10-31.7z but result was the same.
My knowledge of VS2010 is not good so maybe I’m doing something wrong?
Link to my project: http://www.ex.ua/view-storage/177396299471

I’m using of_preRelease_v0062_vs2010_FAT

P.S. on MacOS example compiles, but i need it on Win.

ascorbin · May 9, 2011, 6:28pm

WOW! really great! a fully functionable browser in openframeworks! haha! and a flash player!

greetings ascorbin

would be nice to have it also for windows…

ascorbin · May 23, 2011, 3:40pm

hi

@KOV i’ve seen a VS2008-project for berkelium in UNITY here ->https://github.com/jdierckx/UnityBerkeliumPlugin . Maybe this could help a bit. I dont know.

greetings ascorbin