ARtoolkit on iPhone

Just wondering if anyone was experimenting with getting ARtoolkit into the iPhone release. It looks like the people who make ARtoolkit have put something together. Haven’t really explored the level of difficulty in porting it to OF but it would be an exciting addon to have.

at this point its impossible to get any kind of video data from the iphone camera unless you have a jailbroken phone. I havent really looked into what AR has done, but I imagine it’s with that kind of situation. I suppose you could always take a photo and look for AR codes in a static situation, but that sounds like less fun haha

most of us really done any work with jailbroken iphones yet, but it sounds like it would be pretty freaking cool if you got it working

There is a way to grab the camera by hacking at UIImagePickerController (http://blog.airsource.co.uk/index.php/2-…-ontroller/), a few apps in the store somehow either customize this view or take rapid, sequential shots of it.

I am not sure the exact technique to get the actual image data, nobody seems to be sharing and I’ve not spent any time poking at it. Mainly because Apple seems to be rejecting any application that touches this view (even simple ones that just customize the view to have a different label). This is frustrating because some applications got in recently and even prominently featured in the store as staff picks, etc., which were obviously customizing and cheating the view (ReplayCam, QuadCamera, JitterPic). Seems to stem that they are now testing app submissions with OS 3 and the aforementioned apps break (so, they changed something in the view between OS 2 - 3).

For now - you can hack it, but there is a good chance any AR/camera-as-input application will just get bounced.

[quote author=“alinear”]There is a way to grab the camera by hacking at UIImagePickerController (http://blog.airsource.co.uk/index.php/2-…-ontroller/), a few apps in the store somehow either customize this view or take rapid, sequential shots of it.

I am not sure the exact technique to get the actual image data, nobody seems to be sharing and I’ve not spent any time poking at it. Mainly because Apple seems to be rejecting any application that touches this view (even simple ones that just customize the view to have a different label). This is frustrating because some applications got in recently and even prominently featured in the store as staff picks, etc., which were obviously customizing and cheating the view (ReplayCam, QuadCamera, JitterPic). Seems to stem that they are now testing app submissions with OS 3 and the aforementioned apps break (so, they changed something in the view between OS 2 - 3).

For now - you can hack it, but there is a good chance any AR/camera-as-input application will just get bounced.[/quote]

You can use uigetscreenimage to get the current image displaying on the phone, thats how the bar scanner app works. As for arkit , its great, yes it would be nice to be able to pull some data directly from the video but you can’t have everything can you.

https://devforums.apple.com/message/149553

it looks like UIGetScreenImage is a totally fine way to deal with this issue. Apple gave the go ahead. Also now on 3.1 you are able to customize the UIImagePickerController with custom overlay views.

more work clearly needs to be done, but this is good news.

http://cmgresearch.blogspot.com/2010/01-…-to-01.html

so, some more information is revealed.

Leaving aside the issue of getting the camera data of the iPhone

I worked to setup an xCode project for iPhone with ARToolKit and a static image instead of the live video

I have no compilation errors however there is no marker detection

in testApp.h

  
  
//-----Augmented Reality------  
arTKPMultiTracker arTracker;  
bool useBCH;  
ofImage	arLiveVideo;  
  

in testApp.c

  
  
void testApp::setup(){  
//------ Augmented Reality-------------  
	arLiveVideo.loadImage("images/arLiveVideo.jpg");  
	arLiveVideo.setImageType(OF_IMAGE_GRAYSCALE);  
        int width   = 320;  
	int height  = 480;  
	useBCH = false; //true;  
        arTracker.setup(width,height,useBCH);  
//------ Augmented Reality-------------  
}  
  
  
//--------------------------------------------------------------  
void testApp::update()  
{  
	//------ Augmented Reality-------------  
	unsigned char * pixels = arLiveVideo.getPixels();  
	arTracker.update(pixels);  
	cout<<"Markers Found: "<<arTracker.numOfMarkersDetected<<"\n";  
	//------ Augmented Reality-------------  
	  
}  
  
  

I have done the same code for Mac 10.4 and it works great

There is another working augmented reality project that made the integration with iphone…
In this project a wrapper of NyARToolkit was done for Objective-C

http://www.morethantechnical.com/2009/07/01/augmented-reality-on-the-iphone-using-nyartoolkit-w-code/

is it necessary to follow this approach and make an Objective-C wrapper for the working ARToolkit addon for OF?

here is the link to the xCode project of_v0.06
http://www.5elementos.com.mx/alecsGarza/graphics.zip

I’m trying to compile the Nyartoolkit in iphone, but I’m having problems, it compiles everything, but the console appears the following error:

"Unknown class NyartoolkitCrossCompile int the Interface Builder "

Someone knows how can I fix it?

tks
Juliano

Does anyone have any examples running on the iPhone? I tried to run with NyArtoolKit and also with the OfxArtoolkit, but both return errors…

tks

I’m quite interested in this as well. I did have some working code for the iPhone and I’ll dig it up. My problem was getting an OpenGL overlay over the top that DIDN’T get picked up with the UiGetScreenImage. Essentially taking screenshots will take a screenshot of the camera AND the 3D stuff you are overlaying.

I’ll check this out again when I have more time.

I just got an email about this the last week, with the offer of early access to the library. I spoke to Mark Billinghurst, one of the guys at ARToolworks about extending this invite to the OF community and he gave the green light. If you’d like access to this with a view to writing an add-on you can email mark [dot] billinghurst [at] artoolworks [dot] com.