ARToolKit 5 Add-On

Hi everyone,
The Natural Feature Tracking in ARToolKit has made radical performance improvement over the last month.
daqri who aquired ARToolKit made all the code open source under LGPL and they put a lot of work in to it.

I don’t know how to write the code to get ARToolkit 5 running as an addon but maybe its a good moment to look in to it again? The last time somebody looked in to ARToolKit for OF was like 4 years ago.

Here is a link: http://artoolkit.org

3 Likes

This is a really good idea. I don’t really have the experience to pull this off by myself but I would be more than happy to contribute towards this. In fact, I’d really like to do this.

I was going to use the library a while ago but went in favour of Arturo’s ofxAruco addon due to difficulties with the former.

Maaaaaaybe I could do it by myself but I feel like there’s a realistic possibility it wouldn’t get completed… Let me know if you (or anyone) would like my time.

1 Like

I will see what I can do. We have a student that can help us building the add-on but it will take very long for us.

@julapy what do you think about ARToolKit? You have the most experience with AR and OF building the vuforia add-on.
And @fishkingsin what is your opinion? You have been working on some ARToolKit Plus add-on when the NFT was a commercial product.

With ARToolKit it seems like that we can create tracking datasets directly from a jpg image without logging in to a proprietary and complicated online platform. This means a project compiled now, will not become unusable in 4 years. There is no commercial license needed to prevent water marks. Unlike the Vuforia License Agreement, the ARToolKit License and community supports open source on every aspect. To me it seems like a much better fit for the Open Framework culture then vuforia.

1 Like

I have started a new repository for ARToolKit 5. For now I just added the ARToolKit 5 library for iOS and Android in a github repository and started to ask around within our openhybrid forum, friends and students for interested developers to join. Lets see where we get.

This here are the other two repositories from 2 and 4 years ago:


Any other work that should be considered to look in to?

Cool! Are you thinking of implementing a similar API to the last two addon’s blueprints or fit around the new lib? I’ll be the first to admit I don’t have much (if any) experience with API design.

I’m going to go and play around with the library and keep an eye on goings on. I’ll talk to some people at my university and see if anyone might be interested in helping out as well.

Cheers!

I think the best would actually be to follow @julapy’s ofxQCAR API.

Hi everyone,

I’m reaching out to you from the ARToolKit-Team. If there is anything I can support you with, just contact me.
I’m more than happy to assist.

Kind regards
Thorsten Bux

2 Likes

Hey,

This is super interesting indeed. I don’t have much time on my hand but I am happy to test things out if needed.
Also did you see this repo: https://github.com/naus3a/ofxArtool5 Seems like a similar attempt.

Hi @Thorsten_Bux if you guys want to put some energy in to this addon, maybe it would be best if you host the open framework addon in the official ARToolKit github repository?

@Meach, this is super cool, I think if the offical ARToolKit folks could host the addon, then all of us would have a common ground. Over the last 4 years there are 3 different add-on that showed up, each doing their thing.

@julapy did an amazing job with the vuforia addon. We have the best OF/AR API concept in his work (https://github.com/julapy/ofxQCAR). Using this API would make it easy for @julapy to join efforts and for everyone else who uses ofxQCAR easy to switch back and forth.

Btw, I gave a try to naus3a repo and had it working without much troubles on OS X.

I think it could also be a good idea to have this repo concentrated as I get lost as well when it comes to AR addon for OF :smiley:

Hi @va_li we can host it in the ARToolKit repository if you like. I think best would be to copy your repository https://github.com/openhybrid/ofxARToolKit9 over to us. Do you agree?

How do you propose to move forward from there?

1 Like

Hi @Thorsten_Bux, yes please copy, I will then delete the openhybrid/ARToolKit repository and fork yours.

I will then point everyone that I am talking with to the repository on your github and maybe we can mobilize the makers of



https://github.com/naus3a/ofxArtool5

to join forces in one addon?

I belive it is essential that the API is similar to https://github.com/julapy/ofxQCAR because this is the go to AR addon for Open Frameworks at the moment. If we keep a similarity it makes it easy for projects to switch. What I like about ofxQCAR is that it is not just the bare minimum, but it actually provides a good amount of control by keeping simplicity.

1 Like

Do we have a repo set up yet? I just stumbled upon this for my new AR project idea.

@va_li I thanks for starting this thread. Looking at that topic from 2008 when AR was super fresh, it’s a bit sad that it seems now the hype had winded down, so much that we don’t even have a good updated addon for doing this anymore. (ofxQCAR is great but sadly, it’s mobile only.)

I would love to help contribute anyway I can. Just let me know.

ps. on the other note, does anyone know if there is any AR solution that works on raspberry pi?

Hi everyone, and sorry if I’m a little late to the party :slightly_smiling:

My addon was written to satisfy specific project needs, so I basically wrapped only the functionalities I needed.
What is really nice in the new ARToolkit is the NFT tracking part: for a while I had my own recipe of OpenCV feature finding + some matrix lerping to smooth the tracking, but their “out of the box” tracker running in a separate thread is surely convenient.

I agree that a generic, clean API is the way to go: I started to sketch something in that direction with this abstract class, but I was focusing on my project needs, so the design could be greatly improved.

One important thing to keep in mind: in ARToolkit5 the various trackers (pattern, NFT, stereo) are not handled exactly in the same way, so creating a common API for all of them is not going to be a trivial wrapping of the original library.
Some things are more or less the same (i.e.: look at my update() functions), but the setup is different for every tracker and the classic marker is designed differently from the NFT one. In my original addon I did not want to create too many wrappers to hide these differences: maybe we can talk about it and decide together what would be the best approach.

1 Like

Hi naus3a,
first of all thank you for the most complete ARToolKit 5 Add-On that is out there right now.
I like how simple it is to use.

I would see 5 points that would be good to add in the API:

  1. Different Camera/App orientation modes
  2. Camera Focus Control
  3. Adding datasets from different file locations
  4. Handling more then one Marker in the same video image (up to 5 should be no problem) and more then one Dataset to look for (ARToolKit supports up to 500 NFT Datasets at the same time).
  5. Access to the Projection Matrix and each ModelViewMatrix

We could implement these with the following API:

setup

  • Set Orientation of the Video
    ARToolKit.setOrientation(<orientation>);
    OFX_ARTOOLKIT_ORIENTATION_DEFAULT
    OFX_ARTOOLKIT_ORIENTATION_180
    OFX_ARTOOLKIT_ORIENTATION_90_LEFT
    OFX_ARTOOLKIT_ORIENTATION_90_RIGHT

  • Set Camera Focus Mode
    ARToolKit.autoFocusOn(); // (continuous autofocus)
    ARToolKit.autoPoiFocusOn(); // (point-of-interest) (since ARToolKit version 5.2.1)
    ARToolKit.autoFocusOff();

  • Add path where to look for NFT datasets
    ARToolKit.addMarkerDataPath("file");

  • Set maximum number of Markers to be searched in the camer stream
    ARToolKit.setMaxNumOfMarkers(<amount>);

  • Setup
    ARToolKit.setup();

update

  • update tracker
    ARToolKit.update();

draw

  • get projection matrix
    ARToolKit.getProjectionMatrix();

  • for the amount of found Markers
    -> get the ID and Model View Matrix for each marker
    -> 3D transform for each found marker.

         ARToolKit.beginAR();
         for (int i = 0; i < ARToolKit.numberOfMarkersFound; i++) {
            ofMatrix4x4 xxx = ARToolKit.getMarker(i).modelViewMatrix;
            ofMatrix4x4 yyy = ARToolKit.getMarker(i).markerName);
            ARToolKit.loadMatrix(i); // change the 3D transformation for each Marker
            ofDrawBox(0, 0, 0, 50, 50, 50);
          }
          ARToolKit.endAR();
    
  • interface for debuging
    ARToolKit.drawDebug();

extra functions

  • Add a new NFT dataset while the app is running.
    ARToolKit.addExtraTarget("file");

If we add one more setup function for the bar code tracker and then use ARToolKit.getMarker(i).markerName for the NFT and the bar code tracker in the same time. Would that work? How much else is the Bar Code tracker different from the NFT tracker?

For the stereo tracker,
If we abstract a lot in to the setup could we do something like:

  • Define the source camera or cameras for the tracker
    ARToolKit.camera(CameraA,CameraB); //Camera sources

  • Define the tracking mode via setup
    ARToolKit.setup(); //default mono NFT and Bar Code
    or
    ARToolKit.setup(stereo);

  • Calibrate the Stereo Cameras
    ARToolKit.calibrateStereo();

Some other interesting thing will be HMD Stereo screens:

  • Draw Left and Right Screen
    ARToolKit.drawLeft(x,y,w,h);
    ARToolKit.drawRight(x,y,w,h);

  • Handle Left and Right 3D Transforms

       ARToolKit.beginARLeft();
          for (int i = 0; i < ARToolKit.numberOfMarkersFound; i++) {
             ofMatrix4x4 xxx = ARToolKit.getMarker(i).modelViewMatrixLeft;
             ofMatrix4x4 yyy = ARToolKit.getMarker(i).markerName);
             ARToolKit.loadMatrixLeft(i); // change the 3D transformation for each Marker
             ofDrawBox(0, 0, 0, 50, 50, 50); 
          }
       ARToolKit.endARLeft();
    
      ARToolKit.beginARRight();
          for (int i = 0; i < ARToolKit.numberOfMarkersFound; i++) {
             ofMatrix4x4 xxx = ARToolKit.getMarker(i).modelViewMatrixRight;
             ofMatrix4x4 yyy = ARToolKit.getMarker(i).markerName);
             ARToolKit.loadMatrixRight(i); // change the 3D transformation for each Marker
             ofDrawBox(0, 0, 0, 50, 50, 50);
           }
      ARToolKit.endARRight();

I already implemented some of this stuff (custom paths, debug view, matrices…) in my local version: I’ll push as soon as I have 10 minutes.

For the rest, here’s my 2 cents:

Camera orientation/ autofocus:
I think we should handle this externally through a set of helper functions: in order to stay flexible and oF-friendly, I’m handling pixels in the generic form of ofBaseHasPixels, so they could come from any kind of camera, video, random_piece_of software_able-to_generate_pixels. If we make helper functions (or classes?) we can leave people free to choose their own camera (i.e. UVC and Libdc handle autofocus differently) without having to load the tracker with heavy and ugly switch cycles.

Multimarker:
For the old square patterns it should be easy.
For NFT I kind of remember I read multimarker is still non supported (I probably spotted also an hardcoded 1 somewhere in the source). We should try to switch NFT marker “live” and see what happens.

I spoke with Wally Young from the ARToolKit Team the other day and he said that the NFT can do multi marker but due a bug in the moment its off. It will be enabled again in the next version.

cool!

Hello everyone,
Thanks to @Thorsten_Bux the ARToolKit OF Addon has now a solid home in the official ARToolKit github repository. You can find it here: https://github.com/artoolkit/ofxARToolKit

This is the result from the conversations in this topic here so that we could focus all the different ARToolKit OF Add-On ambitions in one spot.

@naus3a right now @Thorsten_Bux forked the incomplete add on foundation that I had started to uploaded to the openhybrid.org repository. But your work is the most complete one. Do you want to contribute it to the common goal and just overwrite what I had published?

Roy (Media Lab Community) is working to make your code run on iOS. If you want, we could commit the iOS code after you committed your code so that the authors timeline is well represented on github.

@naus3a, @fishkingsin and @julapy what do you think?

3 Likes