ofxAssimpModelLoader + ofxARToolKitPlus lighting problem

Hi.

I’m trying to combine examples from both addons and have some success (in OF062). But have some troubles with GL Lights - after applying transformations from AR tracker my models draws without lighting.
here is my code:

  
void testApp::draw(){  
	ofSetColor(0xffffff);  
	grabber.draw(pos_X, pos_Y,w,h);  
	//-------------------  
	glEnable (GL_DEPTH_TEST);  
    ofPushMatrix();  
	ofTranslate(model.getPosition().x+100, model.getPosition().y+200, 0);  
	ofRotate(-mouseX, 0, 1, 0);  
	ofRotate(180, 1, 0, 0);  
	ofTranslate(-model.getPosition().x, -model.getPosition().y, 0);  
	model.drawFaces();    //-------- with lights  
	ofPopMatrix();  
	glDisable (GL_DEPTH_TEST);  
	//-------------------  
	  
	  
	gui.draw();  
	  
    //this is where we use the calculated matrix from ARToolkitPlus to put  
	//in our graphics at the location and orientation of the marker.  
	//- the matrix has the 0,0 point as the center of the marker.   
	  
	int numDetected = tracker->getNumDetectedMarkers();  
	//cout<<"numberofmarkers:"<<numDetected<<endl;  
	  
	glViewport(pos_X, pos_Y, w, h );  
	glMatrixMode( GL_PROJECTION );  
	glLoadMatrixf(tracker->getProjectionMatrix());  
	  
	for(int i=0; i<numDetected; i++)  
	{  
		  
		ARToolKitPlus::ARMarkerInfo marker = tracker->getDetectedMarker(i);  
		  
		float m34[ 3 ][ 4 ];  
		float c[ 2 ] = { 0.0f, 0.0f };  
		float w = 40.0f;  
		  
		  
		tracker->rppGetTransMat( &marker, c, w, m34 );  
		  
		float m[ 16 ]; //this is some crazy matrix transformative stuff. I think initially it came out of one of the arToolkit functions.... but i got it from here: [http://chihara.naist.jp/people/STAFF/imura/computer/OpenGL/artp/disp-content](http://chihara.naist.jp/people/STAFF/imura/computer/OpenGL/artp/disp-content)  
		for ( int i = 0; i < 3; i++ ) {  
			for ( int j = 0; j < 4; j++ ) {  
				m[ j * 4 + i ] = m34[ i ][ j ];  
			}  
		}  
		for ( int j = 0; j < 3; j++ ) {  
			m[ j * 4 + 3 ] = 0.0f;  
		}  
		m[ 3 * 4 + 3 ] = 1.0f;  
		  
		  
		glMatrixMode( GL_MODELVIEW );  
		glLoadMatrixf( m );  
		  
		glPushMatrix();  
		glEnable (GL_DEPTH_TEST);  
		glPushMatrix();  
		glTranslatef(tr_X, tr_Y, tr_Z);  
		model.drawFaces();  // - without lights  
		glPopMatrix();  
		glDisable(GL_DEPTH_TEST);  
		glPopMatrix();  
	}  
	  
}  

result is in attached image.

Hi!

Allow me to confirm this as an issue.

It’s in ARTK+, somewhere with

  
  
 artk.applyProjectionMatrix(ofGetWidth(),ofGetHeight());  
...  
 artk.applyModelMatrix(i);	  
  

Running this in 007. The Assimp models look great when positioned manually, but something about those two lines makes the jaggies you see in the attached image.

Any ideas?

Thanks kindly,

C

Similar problem here.

Hi,

Did you managed to solve this jaggies?

I’m combined ArtoolkitPlusExample+3DModelLoaderExample (3ds)+ofx3DUtils (addon) and found out the following:

  • the model works fine standalone, renders properly and with correct lights without artoolkit code applied;

  • once i apply the artk.applyProjectionMatrix() the jaggies appear and the lights get reversed.

Any idea?

Thanks.

could it have something to do with the near/far clipping planes causing z fighting?

if you can get the fov from artoolkit, you can try setting the projection matrix manually with gluPerspective(), which will allow you to set the near and far clipping planes.

Hi Kyle,

i’m a little newbie in arToolkitPlus and openGL on of. Can you be more specific ?

Thanks,
b0t

hey behem0t – sure.

the function that sets up the camera’s intrinsics (i.e., everything apart from the position of the camera) is called gluPerspective() http://www.opengl.org/sdk/docs/man/xhtml/gluPerspective.xml this takes the fov, aspect ratio, and near and far clipping planes. if the near/far clipping planes are too far apart, then you’ll get something like this:

one solution is to change the units of the world space – for example, changing from mm to cm. i’m not sure how you do this with artoolkit. but if you don’t have control over that, you can try setting up the projection matrix manually by calling gluPerspective() on your own.

you can extract the fov from the projective matrix, but i haven’t done this before. it’s probably better to ask artoolkit for the fov.

one more idea is to just disable the drawing of the back faces by using culling:

  
  
glEnable(GL_CULL_FACE);  
glCullFace(GL_BACK);  
  

i think this will solve your case, behem0t – but not 5cameron’s case.

actually – try that first, before you draw your model. if that doesn’t fix the problem, then it’s something other than

I’ve tried that GL_CULL_FACE and it didn’t work.

Maybe i haven’t explained properly before:

  • if i load a 3dModel (3ds), turn the camera on, and just manipulate de model, with mouse rotation and scale it works fine. Something like this: (see good-withoutAR.jpg file for examples at various scales):
  
  
  
//--------------------------------------------------------------  
void testApp::setup(){  
	  
	aFocus = true;  
	aExposure = false;  
	aWhiteBalance = false;  
	  
	thumbScaleFactor = 4;   
	modelScaleFactor = 0.3; //0.30;  
	modelAngleX  = 90;   
	modelAngleY	 = 0;   
	modelAngleZ  = 0;   
	  
	  
	  
	camW = ofGetWidth();  
	camH = ofGetHeight();  
	cameraControl = [[UVCCameraControl alloc] initWithVendorID:0x046d productID:0x0821];  
	[cameraControl setAutoExposure:AUTO_EXPOSURE_ON];  
	[cameraControl setAutoWhiteBalance:AUTO_WHITEB_ON];  
  
	vidGrabber.setVerbose(true);  
	vidGrabber.setDeviceID(deviceID);  
	vidGrabber.initGrabber(camW,camH);  
	  
	colorImage.allocate(camW, camH);  
	grayImage.allocate(camW, camH);  
	grayThres.allocate(camW, camH);  
	  
	[cameraControl setExposure:exposureValue];  
	[cameraControl setWhiteBalance:whiteBalanceValue];  
	  
	// This uses the default camera calibration and marker file  
	artk.setup(camW, camH);  
	threshold = 85;  
	artk.setThreshold(threshold);  
	  
	ofBackground(250,250,250);  
	ofSetVerticalSync(true);  
	glEnable(GL_DEPTH_TEST); //lights look weird if depth test is not enabled  
	centerX = ofGetWidth()/2;  
	centerY = ofGetHeight()/2;  
	centerZ = 0;  
	  
	rotX = 0;  
	rotY = 0;  
	bSmoothLight = true;  
	  
	//reflexions!!  
	ofxMaterialSpecular(120, 120, 120); //how much specular light will be reflect by the surface  
	ofxMaterialShininess(128); //how concentrated the reflexion will be (between 0 and 128  
	  
	//each light will emit white reflexions  
	light1.specular(255,255,255);  
	//    camera.setOrigin(OF_ORIGIN);  
	//	camera.position(centerX, centerY, 5000); //initialize the camera at a far position from the sphere  
	d3DSmodel.loadModel("models/PART2_BLACK.3DS", modelScaleFactor);  
	  
    //you can create as many rotations as you want  
    //choose which axis you want it to effect  
    //you can update these rotations later on  
	d3DSmodel.setRotation(0, -90, 1, 0, 0);  
    d3DSmodel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
}  
  
//--------------------------------------------------------------  
void testApp::update(){  
  
	vidGrabber.grabFrame();  
	bool bNewFrame = vidGrabber.isFrameNew();  
	  
	if(bNewFrame) {  
		  
		colorImage.setFromPixels(vidGrabber.getPixels(), camW, camH);  
		  
		// convert our camera image to grayscale  
		grayImage = colorImage;  
		// apply a threshold so we can see what is going on  
		grayThres = grayImage;  
		grayThres.threshold(threshold);  
		  
		// Pass in the new image pixels to artk  
		artk.update(grayImage.getPixels());  
	}  
	rotX += 1;  
	rotY += 2;  
	  
	//light1  
	float L1DirectionX = 1;  
	float L1DirectionY = 0;  
	float L1DirectionZ = 0;  
	  
	light1.directionalLight(255, 255, 255, L1DirectionX, L1DirectionY, L1DirectionZ);  
	ofxLightsOff();  
	  
	  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
	  
		// Main image  
		(doDrawBlackAndWhite)?  grayImage.draw(0,0) : colorImage.draw(0, 0);  
	  
		drawFrameRate();  
		ofEnableAlphaBlending();  
		printf ("numDetected:(%i)\n",numDetected);  
		glPushMatrix();  
		glEnable (GL_DEPTH_TEST);    
		glPushMatrix();   
		ofxLightsOn(); //turn lights on  
		ofxSetSmoothLight(true);	  
		d3DSmodel.setRotation(1, modelAngleX, 1, 0, 0);  
		d3DSmodel.setRotation(0, modelAngleY, 0, 1, 0);  
		d3DSmodel.setRotation(2, modelAngleZ, 0, 0, 1);  
		d3DSmodel.setPosition(0,0,50);  
		//////////////////////////  
		/// JUST FOR TESTING   
		//////////////////////////  
		//tumble according to mouse  
		glTranslatef(ofGetWidth()/2,ofGetHeight()/2,0);  
		glRotatef(-mouseY,1,0,0);  
		glRotatef(mouseX,0,1,0);  
		glTranslatef(-ofGetWidth()/2,-ofGetHeight()/2,0);  
		d3DSmodel.setScale(3.0,3.0,3.0);  
		//////////////////////////  
		//////////////////////////  
		d3DSmodel.draw();  
		ofxLightsOff(); //turn lights off to draw text  
		glPopMatrix();  
		glDisable (GL_DEPTH_TEST);   
		glPopMatrix();  
}  
  

( AR Toolkit just being initialized, not applied in draw).

  • In the other way around, if i turn on artoolkit and start drawing using the matrix, it behaves differently. Very close to the camera it renders almost good, but when you start to move away it starts degrading progressively (closeRange-withAR.jpg, badRenderWithAR.jpg) :
  
  
//--------------------------------------------------------------  
void testApp::draw(){  
	  
	// Main image  
	(doDrawBlackAndWhite)?  grayImage.draw(0,0) : colorImage.draw(0, 0);  
	  
	drawFrameRate();  
	// Draws the arToolkit marker recognition camera view   
	if(doDrawCamera){  
		ofxCvGrayscaleImage mom;  
		mom.allocate(camW/thumbScaleFactor,camH/thumbScaleFactor);  
		mom.scaleIntoMe(grayThres, CV_INTER_LINEAR);  
		mom.draw(camW-(camW/thumbScaleFactor), 0);  
		ofPushMatrix();  
		ofSetColor(0,0,0);  
		ofFill();  
		ofRect(0,0, 400, 140);  
		ofSetColor(255,255,255);  
		ofDrawBitmapString(ofToString(artk.getNumDetectedMarkers()) + " marker(s) found", 10, 20);  
		ofDrawBitmapString("Threshold: " + ofToString(threshold), 10, 40);  
		ofDrawBitmapString("Use the Up/Down keys to adjust the threshold", 10, 60);  
		  
		//		string aExposureSuffix = (aExposure)? ofToString(""): ofToString(exposureValue) ;   
		string aExposureSuffix = (aExposure)? "": "("+ofToString(exposureValue)+")";   
		string aWhiteBSuffix   = (aWhiteBalance)? "": "("+ofToString(whiteBalanceValue)+")";   
		ofDrawBitmapString("autoFocus [1]: " + ofToString(aFocus), 10, 80);  
		ofDrawBitmapString("autoExposure [2]: " + ofToString(aExposure) + aExposureSuffix, 10, 100);  
		ofDrawBitmapString("autoWhiteBalance [3]: " + ofToString(aWhiteBalance) + aWhiteBSuffix, 10, 120);  
		ofNoFill();  
		ofPopMatrix();  
		  
		//  
		//	// ARTK draw  
		//	// An easy was to see what is going on  
		//	// Draws the marker location and id number  
		artk.draw(camW-(camW/thumbScaleFactor), 0,camW/thumbScaleFactor,camH/thumbScaleFactor);  
	}  
	//	// ARTK 3D stuff  
	//	// This is another way of drawing objects aligned with the marker  
	//	// First apply the projection matrix once  
 	artk.applyProjectionMatrix();  
	//	// Find out how many markers have been detected  
	int numDetected = artk.getNumDetectedMarkers();  
	ofEnableAlphaBlending();  
	if (numDetected>0) {  
		// Set the matrix to the perspective of this marker  
		// The origin is in the middle of the marker	  
		artk.applyModelMatrix(0);		  
		printf ("numDetected:(%i)\n",numDetected);  
		glPushMatrix();  
		glEnable (GL_DEPTH_TEST);    
		glPushMatrix();   
		ofxLightsOn(); //turn lights on  
		ofxSetSmoothLight(true);	  
		//		glEnable(GL_CULL_FACE);    
		//		glCullFace(GL_BACK);    
		d3DSmodel.setRotation(1, modelAngleX, 1, 0, 0);  
		d3DSmodel.setRotation(0, modelAngleY, 0, 1, 0);  
		d3DSmodel.setRotation(2, modelAngleZ, 0, 0, 1);  
		d3DSmodel.setPosition(0,0,50);  
		d3DSmodel.draw();  
		//		glDisable(GL_CULL_FACE);  
		ofxLightsOff(); //turn lights off to draw text  
		glPopMatrix();  
		glDisable (GL_DEPTH_TEST);   
		glPopMatrix();  
	}  
}  
  

If i turn the lights off, the model renders fine. It’s just a black text 3D model.

humm just guessing, but could it be a texturing/UV map problem?

I’m asking this because:

  1. you said that without lights (=you’re basically watching an untextured model) everything looks fine
  2. some time ago, dealing with a similar project (OF+ARToolKit+3ds) I saw similar “stair-like” artifacts; after some head scratching, the 3D artist re-textured the models using power-of-2 textures and everything looked fine.

even if the GL_CULL_FACE didn’t work, i’m still pretty convinced it’s a z-fighting issue. if i had experience with artoolkit i would give you more advice, but i really think the key is just called gluPerspective() on your own, with the fov from ART and some better clipping planes. also: “it looks better near the camera” is a class z-fighting symptom.