ofFiducialFinder

Everyone should know ReacTivision by now, its a great piece of software that tracks fiducial markers, dtouch markers and finger presses, and send them out over OSC (tuio format) to whatever software you need.
http://reactable.iua.upf.edu/?software

I wanted a way of being able to do this directly within openframeworks, rather than running their software. This is because:

  • if you only have one camera input, you cant share it across two pieces of software
  • the software does thresholding & camera undistortion, something you might want to do within ofw
  • surely using a local udp connection is a little slower than being able to access it directly?

So i’ve spent a day or so going through the reactivision software and their specific classes to create ofFiducialFinder, an addon for openframeworks that lets you directly track their objects by passing in a grayscale image (once you threshold it). You can also load in your own tree file, should you want to create your own fiducial patterns, like Karsten did for a Processing project recently.

To get a list of markers, you simply say:

fidtracker.findFiducials( grayimage.getPixels() );

Here is a first video

http://www.chrisoshea.org/storage/vid/p-…-=340&h=260

I need to tidy it up a little, improve the drawing output & check with them I have included the right credits as its all pulled from their code. I will be uploading it tomorrow or Sat.

Great!
This will be super useful.

Are you currently getting all the information like rotation, speed, etc or just position?

Theo

Man I love the OF community we may take over the world.

Hi Chris

any news on this?

Cheers

Nick

Hi Nick

Sorry I’ve been super busy as I am now away for the next week. I’ll make sure to finish it the week I’m back and to upload it by 4th April.

thats cool - just checking I hadn’t missed it somewhere

Hi Chris

any joy with this or are there still issues?

Cheers

Nick

Hello,

I’ve been trying to figure this one out. When I pass the iplimage imageData pointer I get nothing. it expects an const unsigned char* so I tried using

  
(const unsigned char*)img->imageData  

just to see if it worked and ofcorse I got nothing.
What is the status of this ofFiducialTrack addon?

Thanks,
Ding

Sorry for anyone who has been waiting for this, have been super busy.

Will post it tomorrow (Wednesday).

Thanks so much for working on this - I’ll test it out as well if you post code.

Hey chrisoshea,

I don’t mean to be inpatient, but I was just wondering what is the word with this addon. Sorry if I sound like I am rushing you, I also have a life and have very little time to do this kind of stuff. I was just working on something similar when I hit a stumbling block and I found this thread. So I am looking forward to see how you have solved what I was trying to figure out. Anyway thanks and looking forward to it.

ding

OK here is my untested code, will test tomorrow! Must likely VERY BUGGY!!!
Any suggestions or help will be appreciated.

UPDATE: tested it works!

fiducial definition-----------------

  
  
#ifndef OFFIDUCIAL_H  
#define OFFIDUCIAL_H  
  
#include "ofMain.h"  
  
class ofFiducial {  
	  
	public:  
		int fidId; // id (as in treeidmap)  
		float fidX;  // x   
		float fidY; // y  
		float fidAngle; // phi (radiant)  
      short top,bottom,left,right; //fiducial coordinates I THINK!  
	  
	ofFiducial() {  
		int fidId = NULL;  
		float fidX = NULL;  
		float fidY = NULL;  
		float fidAngle = NULL;  
      top = bottom = left = right = NULL;  
	}  
	  
	//this is a test until I can figure out how to get the fiducial dimentions  
	void draw(float _x = -1, float _y = -1) {  
		ofNoFill();  
        ofSetColor(0xffffff);  
		ofCircle( _x, _y, 10);//just a test  
	}  
};  
  
#endif  
  
  

fid finder header---------------------

  
  
  
#ifndef OFFIDUCIALFINDER_H  
#define OFFIDUCIALFINDER_H  
  
#include "ofFiducial.h"  
#include "ofMain.h"  
#include "ofAddons.h"  
#include "ofCvGrayscaleImage.h"  
  
#include <map>  
#include <vector>  
#include <string>  
  
#include "fidtrackX.h"  
#include "segment.h"  
  
class ofFiducialFinder {  
	  
	public:  
	  
		//constructor & destructor  
		ofFiducialFinder();  
		~ofFiducialFinder();  
	  
		//vector to store fiducials  
		vector<ofFiducial>  _fiducials;  
	  
		//width and height of gray image  
		int m_width, m_height;  
		//is the tree initialized  
		bool initialized;  
		  
		//segmenter  
		Segmenter segmenter;  
		//tree ID map  
		TreeIdMap treeidmap;  
	  
		//fiducial tracker  
		FidtrackerX fidtrackerx;  
		//max number of fiducials  
		FiducialX fiducials[ 1024 ];  
	  
		//initialize the tree where the fiducial data is  
		void initTree( const char *file_name );  
	  
		//find fiducials in gray image  
		void findFiducials( ofCvGrayscaleImage& input );  
		  
		//deinitialize segmenter  
		void deinitSegmenter();  
};  
  
#endif  
  
  

fid finder cpp-------------------------------------

  
  
  
  
#include "ofFiducialFinder.h"  
  
//constructor  
ofFiducialFinder::ofFiducialFinder() {  
	  
	initialized = false;  
	m_width = 0;  
	m_height = 0;  
  
	/*terminate_treeidmap() knows that this is kind of uninitialized */  
	memset(&fidtrackerx, 0, sizeof(fidtrackerx));  
	memset(&treeidmap,   0, sizeof(treeidmap));  
	//initialize trees data  
	initTree("all.trees");	  
	  
}  
  
  
ofFiducialFinder::~ofFiducialFinder() {  
	  
	deinitSegmenter();  
}  
  
  
void ofFiducialFinder::deinitSegmenter() {  
	  
	if (initialized){  
		terminate_segmenter(&segmenter);  
	}  
	initialized=false;  
}  
  
  
void ofFiducialFinder::findFiducials( ofCvGrayscaleImage& input ) {  
	  
	if(input.width!=m_width || input.height!=m_height) deinitSegmenter();  
	  
	m_width = input.width;  
	m_height = input.height;  
	  
	const unsigned char* pixels = input.getPixels();  
	  
	if(!initialized){  
		initialize_segmenter( &segmenter, m_width, m_height, treeidmap.max_adjacencies );  
		initialized=true;  
	}  
	  
	step_segmenter( &segmenter, pixels );  
	int count = find_fiducialsX( fiducials, MAX_FIDUCIAL_COUNT,    
								&fidtrackerx ,   
								&segmenter,   
								m_width, m_height);  
	  
	_fiducials.clear();  
	  
	int i;  
	for(i=0;i< count;i++) {  
		if(fiducials[i].id!=INVALID_FIDUCIAL_ID){  
			_fiducials.push_back( ofFiducial() );  
			_fiducials[i].fidId		= fiducials[i].id;         // id (as in treeidmap)  
			_fiducials[i].fidX		= fiducials[i].x;  // x (normalized)  
			_fiducials[i].fidY		= fiducials[i].y; // y (normalized)  
			_fiducials[i].fidAngle	= fiducials[i].angle;      // phi (radiant)  
		}  
	}  
	  
}  
  
  
void ofFiducialFinder::initTree( const char *file_name ) {  
	  
	terminate_fidtrackerX(&fidtrackerx);  
	terminate_treeidmap  (&treeidmap);  
	deinitSegmenter();  
	  
	initialize_treeidmap_from_file( &treeidmap, file_name );  
	initialize_fidtrackerX( &fidtrackerx, &treeidmap, NULL);  
	if(treeidmap.max_adjacencies<=0) printf("could not load TreeIdMap from '%s'", file_name);  
}  
  

ding

Tested and it works. Just make sure that you do this before passing the image to it.

  
  
void testApp::update(){  
	  
	ofBackground(100,100,100);  
	vidGrabber.grabFrame();  
	  
	if (vidGrabber.isFrameNew()){  
		  
		colorImg.setFromPixels(vidGrabber.getPixels(), 320,240);  
		grayImage = colorImg;  
		  
		if (bLearnBakground == true){  
			grayBg = grayImage;  
			bLearnBakground = false;  
		}  
		  
		// take the abs value of the difference between background and incoming and then threshold:  
		grayDiff.absDiff( grayBg, grayImage );  
		grayDiff.threshold(threshold);  
		fidfinder.findFiducials( grayDiff );  
	}  
	  
}  

Will clean up the sample app and post it tomorow. But if you cant wait here is the sample app header------------

  
  
#ifndef _TEST_APP  
#define _TEST_APP  
  
  
#include "ofMain.h"  
#include "ofAddons.h"  
#include "ofFiducialFinder.h"  
#include "ofCvMain.h"  
  
class testApp : public ofSimpleApp{  
	  
	public:  
	  
		ofVideoGrabber 		vidGrabber;  
		ofCvGrayscaleImage 	grayImage;  
		ofCvGrayscaleImage 	grayBg;  
		ofCvGrayscaleImage	grayDiff;  
		ofCvColorImage		colorImg;  
	  
		ofFiducialFinder	fidfinder;  
	  
		int 				threshold;  
		bool				bLearnBakground;  
	  
		  
		void setup();  
		void update();  
		void draw();  
		  
		void keyPressed(int key);  
		void keyReleased(int key);  
		void mouseMoved(int x, int y );  
		void mouseDragged(int x, int y, int button);  
		void mousePressed(int x, int y, int button);  
		void mouseReleased();  
		  
		  
};  
  
#endif  
  
  

and the cpp-------------

  
  
void testApp::setup(){	   
  
	vidGrabber.setVerbose(true);  
	vidGrabber.initGrabber(320,240);  
	colorImg.allocate(320,240);  
	grayImage.allocate(320,240);  
	grayBg.allocate(320,240);  
	grayDiff.allocate(320,240);  
	  
	threshold = 80;  
	bLearnBakground = true;  
  
        //going to make a copy file build phase to the target and add all.trees to it  
	//fidfinder.initTree("../Resources/data/all.trees");  
  
}  
  
  

the update is above and here is the draw-------

  
  
void testApp::draw(){  
	  
	colorImg.draw(20,20);  
	grayImage.draw(360,20);  
	grayBg.draw(20,280);  
	grayDiff.draw(360,280);  
	  
	for(int f=0; f<fidfinder._fiducials.size() ;f++) {  
		float id = fidfinder._fiducials[f].fidId;  
		float _x = fidfinder._fiducials[f].fidX;  
		float _y = fidfinder._fiducials[f].fidY;  
		cout << "fiducial" << id << "found at (" << _x << ","<< _y << ")" << endl;  
	}  
	  
}  
  
  

ding

If anyone has suggestions on how to make this better or clean it up let me know. I still haven’t figured out how to draw a box around the fiducials.

BTW: you need to download the reactivision code and use these files from it:

default_trees.h
fidtrackX.h
fidtrackX.c EDIT: make .cpp
floatpoint.h
segment.c EDIT: make .cpp
segment.h
treeidmap.c EDIT: make .cpp
treeidmap.h
all.trees

ding

Here it is with example. Sorry but I didnt have time to clean up the xcode project so it is not included, just the sample app source and the addon source code. It will be up for 7 days sorry about that I just dont have a server service.

http://download.yousendit.com/E7A41C5763C79153

ding

Hey ding… thanks for this I look forward to testing it out :slight_smile:

I went ahead and saved it to a permanant space here:

http://ofx.nuigroup.com/ofFiducialFinder.zip

Cool. If anyone can make it better it would be awesome. Right now it only plots the center point and fiducial number but I saw in chrisoshea video on the first post that he was able to track the bounding box. I think that functionality would be cool. The ofFiducial can be better but it’s a good working start. Also the tracker doesnt work unless the imagedata comprises of either 0’s or 255’s (just the nature of the tracker) I would like to implement a check to see if the image is good or not and pass a warning on what to do if it is not good.

BTW thanks for the hosting. Much appreciated.

[edit] just looked at the video again and chris was just drawing the same size box no matter how close/far or perspective.

ding

It works great after some tweaking. Thanks a lot.

http://www.vimeo.com/1124339

Sorry about my lack of input & posting stuff. Everytime I go to do any coding on it, something comes up. I’m mega busy on projects at the moment, so no time to dedicate to fiducials. It looks like you’ve got everything working though. Perhaps a little tidying up like putting the draw function within the addon class, and also drawing angle & acceleration indicators would be nice.

By the way, you should check out (Processing guru) Karstens work in creating his own Fiducials. Last year he paired two fiducials together to create more identifiers than the free ones that reactivision give you, then he made his own, and just recently did this:

http://www.vimeo.com/1110950

I would really like it if you can it better chris. What I got going was just the mere basics. I am still a beginner/part-time programmer. I am sure someone with experience could take a couple of minutes and hook it up nice. And if you can post a link on how to make your own fiducials and tree file that would be awesome.

[EDIT] like compensating for ghost frames in which fiducials momentarily disappear.

ding