Other video libraries (e.g. VideoWrapper or libdc 1394)

hi openframeworks girls and guys
i’m involved in multi-touch research and am using the Open Source software Touchlib from the nuigroup to track fingers and send their coordinates to my Flash applications.

the nuigroup and especially one of the developers who is also active in this forum cerupcat (aka Seth Sandler) has got a new tracking application which is called tBeta, which is based on OF and is really cool.

the point is that OF only supports cameras that are Quicktime enabled. under windows that should mean that they have a DirectShow driver and under Mac OS X that means that they have to show up as Quicktime video input devices.

now i’ve got a problem that i own a great Firewire camera called Pointgrey Firefly MV. great means that the camera has a really good near-infrared performance and that the proprietary driver is really fast and causes few lag. but its proprietary driver is only running with software that does support this driver called FlyCapture under Windows. although under Mac OS X the camera shows up in the Quicktime video device lists it does yield any image i.e. the standard Firewire connection seems to be a bit buggy from the camera.

the thing is: this type of camera is used really often in computer vision projects and in multi-touch setups especially because of the great near-infrared performance. thus, the FlyCapture driver is supported by some Open Source projects.

under Windows a good Open Source project for the FlyCapture driver is the VideoWrapper project and under Mac OS X (and Linux?) it is libdc1394.

i know that those video libraries are working with the Firefly MV because VideoWrapper is integrated into the Touchlib and libdc1394 is integrated into the Mac OS X Open Source multi-touch tracking software Touché. both softwares are working with the Firefly.

basically, i know that not each camera and camera type can be supported by OF and i cannot demand that it does. however, i just wanted to say that if you’re planning to enhance camera compatibility beyond purely Quicktime enabled devices it would be SO COOL if you could consider integrating VideoWrapper and/or libdc 1394 support into OF.

i’ve posted a similar post in the tbeta-thread-of-the-nuigroup’s-forum.

I agree IIDC support would be cool (and maybe even GigE Vision). Also, there is quite some effort already going on to implement a videoGrabber for IIDC cameras (like the firefly)

http://forum.openframeworks.cc/t/true-iidc-under-osx/439/0

Since there has been a lot of talk about implementing Firefly, I was wondering if it would be easy to add in Firefly’s SDK directly into ofVideograbber? Or for that matter, any video’s sdk? Is it an easy thing to do?

one option is to mirror to extend the basic functionality of ofVideoGrabber, but implement all the different sdks, etc. I am worried about making the mother of all grabber classes, since there might be some functionality and difference that make functions irrelevant, etc. (like controlling camera parameters and modes, which might make sense with IIDC cams, but not unicap cams, etc)

I recently made a videoGrabber class based on the seeSaw quicktime code, to get better dv camera quality. I mirrored all the functionality of ofVideoGrabber, so using my code was a simple as changing

  
  
ofVideoGrabber grabber;  
  

to

  
  
ofxSeeSawVideoGrabber grabber;  
  

I like the idea of different grabber options (the more the better) as opposed to one huge swiss army knife of a class, which is hard to read and harder to maintain.

that’s just my two cents – we are happy to support different cameras, and I think the firewire work that’s been going on it quite valuable and should be a useful addon soon.

take care!
zach

thanks for the advice zach!

U think you guys could whip up a quick example of how to integrate a different camera sdk for us to follow? And stuff like how to get the pixels into OF. I am thinking of using VideoWrapper actually.

@zach

I like the idea of different grabber options (the more the better) as opposed to one huge swiss army knife of a class, which is hard to read and harder to maintain.

well, i agree with you in that matter. as long as you do not have to implement functionality twice or just copy and paste existing functionality in the mirrored class and then the original class changes and makes your mirrored class obsolete :frowning:

however,i’m of the opinion that you’re suggestion is the best choice for now. i stick with Progen and ask you to give us a bit of an example how another video library could be embedded in a mirrored class.

cheers,
johannes

OK so it doesn’t have _exactly_ the same interface (although it probably will in the future), I created a DV addon for Linux which pretty much replicates the regular video grabber if you need an example. You don’t need to open the codeblocks project…you can just dig around in the code and follow the header files:
http://forum.openframeworks.cc/t/dv-capture-for-linux-of-addon/1233/0

I like the idea of a separate libdc1394 video grabber addon though. I looked at damien douxchamps’ site and found this info:

video1394 capture uses Direct Memory Access (DMA) to pipe camera data directly from the interface card to the computer memory, without copy or processor load (the data does not go through the processor at all.) This allows the maximal framerate to be reached in an efficient way. Video1394 is, in general, the capture mode that you should use.

libraw1394 uses libraw1394 to transfer images from the camera to a user-accessible buffer. This is not efficient because it requires copying data between internal and accessible buffers. This seriously reduces the maximum framerate and increases the processor load. In fact, due to other technical reasons the maximum framerate using libraw1394 is 50% of the framerate used by the camera

From what I can tell, the linux Unicap library uses libraw1394…so using video1394 could result in some speed benefits, not only for linux but possibly for all platforms.

From what I can tell, the linux Unicap library uses libraw1394…so using video1394 could result in some speed benefits, not only for linux but possibly for all platforms.

The last version of ofUCUtils already support dma, I changed the code to use system buffers that according to the unicap documentation should procide dma for the devices that support it:

http://unicap-imaging.org/tutorial/capture.htm

Also, I’ve just discovered there’s a new version of opencv: 1.1.0, apart from some new interesting algorithms, they’ve redone all the capture module, now it’s really similar to the way it works in oF and according to the changelog it supports:

  • HighGUI:
    * [Windows, 32bit] Added support for videoInput library.
    Hence, cvcam is [almost] not needed anymore
    * [Windows, 32bit] FFMPEG can now be used for video decoding/encoding
    via ffopencv*.dll
    * [Linux] Added unicap support
    * Improved internal video capturing and video encoding APIs

And in the code of highgui there’s some other interesting modules like cvcap_dc1394. cvcap_gstreamer or cvcap_xine. All of the modules inherit from a common cvcap. From this it should be really easy to do an oF addon that supports all this, also all of them return IplImage, but getting the pixels from that is really easy and it can be useful for opencv applications to have a grabber that directly returns ofCVImage.

Just an update to this thread, I’ve posted a (hopefully) definitive IIDC solution here:
http://forum.openframeworks.cc/t/ofxvideograbber—libdc1394-grabber-with-plugin-sdks/2487/0