Infrared tracking using Flea3 PointGrey camera

Hi, I’m going to do some experiment on infrared tracking on stage for live performances.

I read a lot of forum posts about this topic, there is enough around here to start some experiments. Thank you to this amazing community for all the advices.
I’d like to write here my feedback about my future experiments, to help someone else that maybe will meet my same problems.

My requirements are:

  • There will be visible light on the stage during tracking
  • Good tracking image quality and good performances
  • Track a 6x4m wall area on the stage
  • IR Camera and Projectors can’t be more close than 6m from the stage

I’m thinking about kind of setup:
Stage lights during tracking
Stage lights + Dichroic IR Block Visible pass filter.

IR Flooding
http://www.supercircuits.com/Infrared-Illuminators/IR145

IR Camera
Camera - I’m thinking about Flea3 PointGrey camera USB3, because it has a good resolution (1280 x 1024) and very good FPS performances (150FPS). I read a bunch of post saying that Flea2 Firewire has good performances with OF, but a couple of examples (here and here) in witch Flea3 has some problems.
IR Pass filter - PointGrey monochrome cameras haven’t the IR Block filter, so you have nothing to remove from the camera, but you have to add a IR Pass filter. I’ll try a Lee87 or 87c filter.
Lenses - One of these, according to the final distance from the camera to the tracking area.

My questions are:

  • Does anyone used Flea3 USB3 camera without problems on OSX?
  • Can anyone suggest me the right kind of lens considering a 6x4m wall area on the stage and a distance of the camera of at least 6m?

Other important OF posts about this topic:

Hey @LukeNeo -

Check out this if you already haven’t:
https://code.google.com/p/ofxvideograbberptgrey/

I have a Flea3/USB3 at work and have checked this out on with a MacBook Pro. The example app compiled and all was well, but I never made anything else or used it for very long to get any sense of reliability. Cool little cameras nonetheless.

good luck!

I am also currently researching cameras for IR tracking on OSX and the Flea3 looks like a great option.
@LukeNeo, glad to hear that it works on OSX.
Has anyone had long term testing with the Flea3 or other usb3 point grey cameras?
Anyone have any recommendations regarding extending a USB 3 connection?

Also been looking into Ximea Usb3 cameras that have OSX support.
http://www.ximea.com/support/wiki/apis/XIMEA_OSX_Software_Package

Hey, I can confirm the Flea3’s run well over long term. We are using 3 in a Science museum, which are still running 8+ hours a day since December. Couple of quirks…

  • they run very hot (almost untouchable), but Point Grey support said this is ok.
  • make sure you don’t have any other USB devices plugged in to the same bus, as the Flea’s require a lot of bandwidth.
  • must use an active usb3 extension (i think we got ours from pt grey).
  • a variable focus lens (2.8-8mm) will make your life easier.

I was on windows and didn’t use the ofxvideograbberptgrey addon, so can’t confirm the above for osx. But I think @danoli3 and @julapy had one of these running for an event on mac recently.

@nickhubben Thank you very much for your feedback! You are the first to confirm that Flea3 definitely works without any problem on OSX!
@NickHardeman I think that an active usb3 extension from ptGrey will be fine, like trentbrooks suggested. I asked Ximea support for MQ013MG-ON model, they said that MQ013RG-E2 model also might be interesting for IR-tracking. Their BW cameras have filter glasses in their optical path, but these glasses do not cut NIR wavelengths (See chapter 3.4.1, page 18).
@trentbrooks: I agree that variable focus lens will make life easier!

I’d like to share with you guys some considerations about what lens to use based on a reference: the famous Klaus Obermaier (Apparition). In this show dancers can move free in an area of about 4.8 x 2.4 x 3.5 m, based on the following screenshot and on the fact that a dancer reach the frontline in 7 steps starting from the bottomline-wall:

Bearing in mind Flea3 ptGrey camera sensor, I choose Fujinon YV28x28SA-2 HD Vari-Focal Lens to test some distances (it may require a C-CS mout adapter for this kind of lens). Here you can see 2.8mm and 8mm configurations:

I put the cameras as close as possible to fill the tracking area, to loose less camera resolution.
With 2.8mm the distance from the wall is about 6m, and from the frontline is about 2.5m. This is the view from the camera:

With 8mm the distance from the wall is about 10m, and from the frontline is about 6.5m. This is the view from the camera:

Since Projector and IR-camera have to be as close as possible, the 8mm approach is far more appropriate to use. In the 2.5mm approach the projector is too close to the front line, and maybe the infrared mask will not fit correctly the dancer silhouette if he goes from frontline to the bottomline.

What do you think about it? Is this the right approach to simulate and verify what lens are the best?

Oof - OK, I need to back up on my initial statement. I wanted to be sure of my memory so I hooked up the Flea3 and took a closer look at the addon. NO GO. The addon wraps libdc1394 for mac, which is obviously a Firewire lib.

A solution, as always, is to use Bootcamp if you need to use a Mac, as it should use the FlyCapture SDK instead.

I’ll be taking another look and will see what I can dig up for any other folks using a USB3 Point Grey camera with OSX, as I now really want this to work.

Sorry for the false information, will do my best to rectify it.

best, Nick

@nickhubben thanks for the clarification.
@LukeNeo. I contacted Allied Vision about IR tracking. The GigE cameras work with OSX 10.4.x - 10.8.x. via the legacy SDK: http://www.alliedvisiontec.com/us/products/legacy.html

Since they are GigE, the cables can run long distances and connecting a couple of cameras to a single computer shouldn’t be an issue on a standard GigE network.

They can have external power or power over ethernet.

The cameras they suggested:
Manta G-095: http://www.alliedvisiontec.com/emea/products/cameras/gigabit-ethernet/manta/g-095bc.html
Manta G145 - NIR: http://www.alliedvisiontec.com/emea/products/cameras/gigabit-ethernet/manta/g-145b-nir.html
Manta G-223 - NIR: http://www.alliedvisiontec.com/emea/products/cameras/gigabit-ethernet/manta/g-223b-nir.html

1 Like

So far this is one of the better posts I’ve found on the subject, which leads me to think if you want to use a Flea3 on OSX, you’d be better off just using Windows… :wink:

http://sourceforge.net/p/libdc1394/mailman/message/32063018/

Good luck folks, I’ll be paying attention.

@nickhubben Thank you for your advice. So, it looks like that to be sure Flea Point Grey camera works, either one have to buy Flea2 firewirecamera and use ofxvideograbberptgrey or use Flea3 USB3 camera on Windows. Like @trentbrooks said, because of Flea3 USB3 drivers are for Windows, it will work without any problems in OF.
@NickHardeman Thank you for your suggestion about Allied Vision cameras, but since no one use them with OF since now… would you trust to buy them?

Hi @LukeNeo,

Maybe you can use Flea3 USB3 cam on OSX with the ofxLibdc addon. At least in my environment, it works.


(Forked from Kyle’s ofxLibdc and slightly modified.)

@LukeNeo - I contacted Allied Vision and they confirmed that the cameras work on OSX 10.8, but I have not been able to get confirmation from the OF/OSX community. Not sure if Allied Vision has tested multiple cameras hooked up to a single OSX computer, but they said in theory that multiple cameras over a standard GigE network should function properly.
@wtshm - Thank you for the info! Great to get a confirmation about USB 3.0 cameras on OSX. Can you tell us more about your environment? By any chance have you tested more than one camera connected to a single computer?

I was searching for a gig E vision library for OSX for a long time. There is a version here

http://www.1stvision.com/AVT-gige-camera-software-downloads.html

The only other implementation I could find is used inside Halcon.

http://www.halcon.com/

I did see that even when you download the trial you have an option to download their pre compiled gig e vision libs for OSX as well as a sample of code. I dont have a Gig e vision camera to try and get it working and it is not really legal but I wanted to test anyway.

@wtshm Yes, like @NickHardeman said, It would be nice if you can tell us more about your environment, OSX / OF / Xcode versions, if you tested more cameras together, etc.
If we can confirm a working environment for Flea 3 USB cam it will be great for the OF/OSX community!

According to IEEE1394/USB cameras list from damien douxchamps, Pt Grey cameras seems to be the only ones compliants with IIDC specs. I agree it would be great to have more details from @wtshm about his experience with flea3.

Concerning infrared,
Does anyone have a sample video of a flea3 used to capture near IR only ? (in theater conditions, with no sunlight).
I wonder if it’s bright enough, or if a sensor with enhanced NIR sensitivity is really required
What I am afraid of is to have a too noisy picture with a flea3 compared to a NIR-enhanced camera. Do you have any experience about this ?

Hi,

Sorry for late reply…
I tested with the environment below.

  • MacBook Pro Retina Mid 2014
  • OSX 10.9.4
  • oF v0.8.3
  • Xcode 5.1.1

Below is information about USB devices in my Mac when connecting Flea3.

And the capture from Flea3. It may not be reliable because the camera lens is so poor quality.

I’ve been using an Allied Vision Prosilica GC 750 for a year now in OF. It’s great!
It has the same MT9V022 sensor as my old Flea2 FW cam (and the cheaper Firefly MV for that matter)
This sensor is a little more sensitive to NIR than most sensors, but it’s a lot cheaper than official NIR sensors.
The fact the camera is gigabit gives back the only advantage old analogue cameras still had: Cheap wiring over long distances.

I’ve been using my own prosilica addon for a year now, and it’s running smooth. When i find the time I’ll finish it and make it opensource, but when you want to test it feel free to pm me.

As it comes to IR lights I would recommend going high power led:
I’m quite happy with these kong jings

hey moostrik

it would be super if you could share your pro silica code with us.
i am trying to get a AVT mako working on OS X 10.9 with OF 0.8.4

i have done this in the past on an older system but that setup does not work anymore.

thanks.

I’m updating the addon now with code from my latest project that included a mako G125C (color) camera.
Can you PM me your email so i can invite you to the repository?

I am using a PtGrey Flea3 USB3 (FL3-U3-88S2C), trying to get this working with ofxLibdc and oF_0.8.4… without much luck.

I managed to get it running (beginners luck) in black and white but after trying to add the colour without backup-ing the working Xcode project I can’t seem to get it running again not even in black and white. I’m struggling since than stoping with a ...Member function 'getPixelsRef' not viable: 'this' argument has type 'const ofImage' (aka 'const ofImage_<unsigned char>'), ... error message. This is using the openframeworks.cc (oF_0.8.4_release) download and latest ofxLibdc.

The project works with above mentioned oF version and an older ofxLibdc but the camera can’t be seen, Xcode shouting [ error ] No cameras found. after compiling OK and running the project.

Correct me if I’m wrong: the older version of ofxLibdc is to old for the camera model? … which lead me back in trying to solve the function '...getPixelsRef' not viable... error message.

Thank you

Hi,

I’m trying to configure and grab images from a PT-Grey FLEA3 USB3 Camera (FL3-U3-88S2C-C) in Opernframeworks (v0.8.4_osx_release) with ofxVideoGrabberPtgrey but I reach this semantic issue error (www.i59.tinypic.com/10cuozp.jpg)

src/testApp.h:26:26: error: field type ‘ofxVideoGrabberPtgrey’ is an abstract class
ofxVideoGrabberPtgrey vidGrabber;

Anyone have an idea how resolve this?

Thanks!