OpenNI Skeleton Tracking

im not sure if line 28 is correct for initializing the vector. If it is then it looks fine.

  
modelTrans(tempNode);  

no, ofNode doesn’t include easyCam. Its the other way round, easycam inherits from ofNode.
I was discussing some time ago with elliotwoods about having an interactive ofNode, yet we were having trouble about inherintace stuff. I don’t know if he solved it, he might. take a look at his github as he has some really nice 3d related addons and stuff.

best

yeah it comes up with a list of errors

  
C:\Users\Ciaran\Desktop\of_v0.7.4_win_cb_release\apps\myApps\kinect_project\src\testApp.cpp||In member function 'virtual void testApp::setup()':|  
C:\Users\Ciaran\Desktop\of_v0.7.4_win_cb_release\apps\myApps\kinect_project\src\testApp.cpp|27|error: 'class ofNode' has no member named 'setRotation'|  
C:\Users\Ciaran\Desktop\of_v0.7.4_win_cb_release\apps\myApps\kinect_project\src\testApp.cpp|28|error: no match for call to '(std::vector<ofNode>) (ofNode&)'|  
C:\Users\Ciaran\Desktop\of_v0.7.4_win_cb_release\apps\myApps\kinect_project\src\testApp.cpp|31|error: 'class ofNode' has no member named 'setRotation'|  
C:\Users\Ciaran\Desktop\of_v0.7.4_win_cb_release\apps\myApps\kinect_project\src\testApp.cpp|32|error: 'class ofNode' has no member named 'setRotation'|  
||=== Build finished: 4 errors, 0 warnings (0 minutes, 2 seconds) ===|  
  

is there a way to use my original code as shown here

  
void testApp::setup() {  
    pMouseX = 0;  
 pMouseY = 0;  
 qMouseX = 0;  
 qMouseY = 0;  
 rMouseX = 0;  
 rMouseY = 0;  
 sMouseX = 0;  
 sMouseY = 0;  
  
	ofSetVerticalSync(true);  
  
    //some model / light stuff  
    glEnable (GL_DEPTH_TEST);  
    glShadeModel (GL_SMOOTH);  
  
    /* initialize lighting */  
    glLightfv (GL_LIGHT0, GL_POSITION, lightOnePosition);  
    glLightfv (GL_LIGHT0, GL_DIFFUSE, lightOneColor);  
    glEnable (GL_LIGHT0);  
    glLightfv (GL_LIGHT1, GL_POSITION, lightTwoPosition);  
    glLightfv (GL_LIGHT1, GL_DIFFUSE, lightTwoColor);  
    glEnable (GL_LIGHT1);  
    glEnable (GL_LIGHTING);  
    glColorMaterial (GL_FRONT_AND_BACK, GL_DIFFUSE);  
    glEnable (GL_COLOR_MATERIAL);  
  
    //load the squirrel model - the 3ds and the texture file need to be in the same folder  
  tableModel.loadModel("actualtable.3ds", 20);  
    teaModel.loadModel("cupatea.3ds", 20);  
    teapotModel.loadModel("teapot.3ds",20);  
    plateModel.loadModel("plate.3ds",20);  
    spoonModel.loadModel("spoon.3ds",20);  
  
    //you can create as many rotations as you want  
    //choose which axis you want it to effect  
    //you can update these rotations later on  
    tableModel.setScale(0.3, 0.2, 0.2);  
    teaModel.setScale(0.04,0.04,0.04);  
    teapotModel.setScale(0.15,0.15,0.15);  
    spoonModel.setScale(0.006,0.006,0.006);  
    plateModel.setScale(0.02,0.02,0.02);  
    teaModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
    teapotModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
   tableModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
    plateModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
    spoonModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);  
  
  
  
  
    ofSetLogLevel(OF_LOG_NOTICE);  
  
    numDevices = openNIDevices[0].getNumDevices();  
  
    for (int deviceID = 0; deviceID < numDevices; deviceID++){  
        //openNIDevices[deviceID].setLogLevel(OF_LOG_VERBOSE); // ofxOpenNI defaults to ofLogLevel, but you can force to any level  
        openNIDevices[deviceID].setup();  
        openNIDevices[deviceID].addDepthGenerator();  
        openNIDevices[deviceID].addImageGenerator();  
        openNIDevices[deviceID].addUserGenerator();  
        openNIDevices[deviceID].setRegister(true);  
        openNIDevices[deviceID].setMirror(true);  
		openNIDevices[deviceID].start();  
    }  
  
    // NB: Only one device can have a user generator at a time - this is a known bug in NITE due to a singleton issue  
    // so it's safe to assume that the fist device to ask (ie., deviceID == 0) will have the user generator...  
  
    openNIDevices[0].setMaxNumUsers(1); // defualt is 4  
    ofAddListener(openNIDevices[0].userEvent, this, &testApp::userEvent);  
  
    ofxOpenNIUser user;  
    user.setUseMaskTexture(true);  
    user.setUsePointCloud(true);  
    user.setPointCloudDrawSize(2); // this is the size of the glPoint that will be drawn for the point cloud  
    user.setPointCloudResolution(2); // this is the step size between points for the cloud -> eg., this sets it to every second point  
    openNIDevices[0].setBaseUserClass(user); // this becomes the base class on which tracked users are created  
                                             // allows you to set all tracked user properties to the same type easily  
                                             // and allows you to create your own user class that inherits from ofxOpenNIUser  
  
    // if you want to get fine grain control over each possible tracked user for some reason you can iterate  
    // through users like I'm doing below. Please not the use of nID = 1 AND nID <= openNIDevices[0].getMaxNumUsers()  
    // as what you're doing here is retrieving a user that is being stored in a std::map using it's XnUserID as the key  
    // that means it's not a 0 based vector, but instead starts at 1 and goes upto, and includes maxNumUsers...  
//    for (XnUserID nID = 1; nID <= openNIDevices[0].getMaxNumUsers(); nID++){  
//        ofxOpenNIUser & user = openNIDevices[0].getUser(nID);  
//        user.setUseMaskTexture(true);  
//        user.setUsePointCloud(true);  
//        //user.setUseAutoCalibration(false); // defualts to true; set to false to force pose detection  
//        //user.setLimbDetectionConfidence(0.9f); // defaults 0.3f  
//        user.setPointCloudDrawSize(2);  
//        user.setPointCloudResolution(1);  
//    }  
  
}  
  
//--------------------------------------------------------------  
void testApp::update(){  
    ofBackground(0, 0, 0);  
    pMouseX = mouseX;  
    pMouseY = mouseY;  
    qMouseX = mouseX;  
    qMouseY = mouseY;  
    rMouseX = mouseX;  
    rMouseY = mouseY;  
    sMouseX = mouseX;  
    sMouseY = mouseY;  
  
  
    for (int deviceID = 0; deviceID < numDevices; deviceID++){  
        openNIDevices[deviceID].update();  
    }  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
	ofSetColor(255, 255, 255);  
ofImage myImage; //allocate space for variable  
myImage.loadImage("commercial-kitchen.jpg"); //allocate space in ram, decode jpg, load pixels.  
myImage.draw(0,0,1360,768);  
    ofPushMatrix();  
  
    for (int deviceID = 0; deviceID < numDevices; deviceID++){  
  
        // debug draw does the equicalent of the commented methods below  
   openNIDevices[deviceID].drawDepth(0, 0, 320, 260);  
     openNIDevices[deviceID].drawImage(1074,0, 320, 260);  
    openNIDevices[deviceID].drawSkeletons(0, 0, 320, 260);  
  
    }  
  
    // do some drawing of user clouds and masks  
    ofPushMatrix();  
    ofEnableBlendMode(OF_BLENDMODE_ALPHA);  
    int numUsers = openNIDevices[0].getNumTrackedUsers();  
    for (int nID = 0; nID < numUsers; nID++){  
        ofxOpenNIUser & user = openNIDevices[0].getTrackedUser(nID);  
        user.drawMask();  
        ofPushMatrix();  
        ofTranslate(320, 240, -1000);  
        user.drawPointCloud();  
        ofPopMatrix();  
    }  
    ofDisableBlendMode();  
    ofPopMatrix();  
  
  
  
  
  
    glTranslatef(ofGetWidth()/3,ofGetHeight()/1.25,0);  
        //tumble according to mouse  
        glRotatef(-105,1,0,0);  
        glRotatef(180,0,1,0);  
        glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);  
  
  
       tableModel.draw();  
         glPopMatrix();  
  
         glPushMatrix();  
  
        glTranslatef(ofGetWidth()/2,ofGetHeight()/1.15,0);  
       glRotatef(-qMouseY,1,0,0);  
        glRotatef(qMouseX,0,1,0);  
      teaModel.setRotation(0, 90, 1, 0, 0);  
    teaModel.setRotation(1, 270, 0, 0, 1);  
        glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.15,0);  
        teaModel.draw();  
         glPopMatrix();  
         glPushMatrix();  
        glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);  
        glRotatef(-pMouseY,1,0,0);  
        glRotatef(pMouseX,0,1,0);  
      teapotModel.setRotation(0, 90, 1, 0, 0);  
    teapotModel.setRotation(1, 270, 0, 0, 1);  
        glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);  
        teapotModel.draw();  
         glPopMatrix();  
         glPushMatrix();  
        glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);  
        glRotatef(-rMouseY,1,0,0);  
        glRotatef(rMouseX,0,1,0);  
      plateModel.setRotation(0, 90, 1, 0, 0);  
    plateModel.setRotation(1, 270, 0, 0, 1);  
        glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);  
        plateModel.draw();  
         glPopMatrix();  
         glPushMatrix();  
        glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);  
        //tumble according to mouse  
        glRotatef(-sMouseY,1,0,0);  
        glRotatef(sMouseX,0,1,0);  
      spoonModel.setRotation(0, 90, 1, 0, 0);  
    spoonModel.setRotation(1, 270, 0, 0, 1);  
        glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);  
        spoonModel.draw();  
         glPopMatrix();  
    glPopMatrix();  
  
}  

to interact with the models that I have created ?

for the modelTrans vector initialize like this

  
  
  for(int i = 0; i<models.size(); i++){// here I'm considering that models and modelTrans are the same size    
ofNode  tempNode;    
modelTrans.push_back(tempNode);    
  
modelTrans.back().setRotation(0, 90, 1, 0, 0);    
modelTrans.back().setRotation(1, 270, 0, 0, 1);    
}    
  

to use the code yo posted, first, replace the part that you posted before.
then to make the models interact with the openni data you first need to get the openNI data that you want to use. I guess that hands. use the code below to get hands.

  
  
ofVec3f l, r;  
for(int i = 0; i < openNIDevice.getNumTrackedUsers(); i++){  
            ofxOpenNIUser & usr = openNIDevice.getTrackedUser(i);          
            ofxOpenNIJoint left = usr.getJoint(JOINT_LEFT_HAND);  
            ofxOpenNIJoint right = usr.getJoint(JOINT_RIGHT_HAND);  
            if (left.isFound()) {  
                l.set(left.getProjectivePosition());//this will give the hand position in the projected image.  
                //l.set(left.getWorldPosition());//this will give you the position in 3dspace  
            }  
            if (right.isFound()) {  
                r.set(right.getProjectivePosition());  
            }  
  

if you use the hands projective position you can interpret that data just like it was a mouse.
once you have hands you must have some sort of way to select the object you want to interact with. Maybe using a push gesture. Also check to point selection example to sort out whether the hand is over the object.
to interact with the object use the ofnode method, like rotate, translate or whaterver you like.

best!

the modelTrans still isnt initializing , says about the setRotation again,

is the hand projective position for this code

  
    void testApp::setup() {    
        pMouseX = 0;    
     pMouseY = 0;    
     qMouseX = 0;    
     qMouseY = 0;    
     rMouseX = 0;    
     rMouseY = 0;    
     sMouseX = 0;    
     sMouseY = 0;    
        
        ofSetVerticalSync(true);    
        
        //some model / light stuff    
        glEnable (GL_DEPTH_TEST);    
        glShadeModel (GL_SMOOTH);    
        
        /* initialize lighting */    
        glLightfv (GL_LIGHT0, GL_POSITION, lightOnePosition);    
        glLightfv (GL_LIGHT0, GL_DIFFUSE, lightOneColor);    
        glEnable (GL_LIGHT0);    
        glLightfv (GL_LIGHT1, GL_POSITION, lightTwoPosition);    
        glLightfv (GL_LIGHT1, GL_DIFFUSE, lightTwoColor);    
        glEnable (GL_LIGHT1);    
        glEnable (GL_LIGHTING);    
        glColorMaterial (GL_FRONT_AND_BACK, GL_DIFFUSE);    
        glEnable (GL_COLOR_MATERIAL);    
        
        //load the squirrel model - the 3ds and the texture file need to be in the same folder    
      tableModel.loadModel("actualtable.3ds", 20);    
        teaModel.loadModel("cupatea.3ds", 20);    
        teapotModel.loadModel("teapot.3ds",20);    
        plateModel.loadModel("plate.3ds",20);    
        spoonModel.loadModel("spoon.3ds",20);    
        
        //you can create as many rotations as you want    
        //choose which axis you want it to effect    
        //you can update these rotations later on    
        tableModel.setScale(0.3, 0.2, 0.2);    
        teaModel.setScale(0.04,0.04,0.04);    
        teapotModel.setScale(0.15,0.15,0.15);    
        spoonModel.setScale(0.006,0.006,0.006);    
        plateModel.setScale(0.02,0.02,0.02);    
        teaModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);    
        teapotModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);    
       tableModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);    
        plateModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);    
        spoonModel.setPosition(ofGetWidth()/2, ofGetHeight()/2, 0);    
        
        
        
        
        ofSetLogLevel(OF_LOG_NOTICE);    
        
        numDevices = openNIDevices[0].getNumDevices();    
        
        for (int deviceID = 0; deviceID < numDevices; deviceID++){    
            //openNIDevices[deviceID].setLogLevel(OF_LOG_VERBOSE); // ofxOpenNI defaults to ofLogLevel, but you can force to any level    
            openNIDevices[deviceID].setup();    
            openNIDevices[deviceID].addDepthGenerator();    
            openNIDevices[deviceID].addImageGenerator();    
            openNIDevices[deviceID].addUserGenerator();    
            openNIDevices[deviceID].setRegister(true);    
            openNIDevices[deviceID].setMirror(true);    
            openNIDevices[deviceID].start();    
        }    
        
        // NB: Only one device can have a user generator at a time - this is a known bug in NITE due to a singleton issue    
        // so it's safe to assume that the fist device to ask (ie., deviceID == 0) will have the user generator...    
        
        openNIDevices[0].setMaxNumUsers(1); // defualt is 4    
        ofAddListener(openNIDevices[0].userEvent, this, &testApp::userEvent);    
        
        ofxOpenNIUser user;    
        user.setUseMaskTexture(true);    
        user.setUsePointCloud(true);    
        user.setPointCloudDrawSize(2); // this is the size of the glPoint that will be drawn for the point cloud    
        user.setPointCloudResolution(2); // this is the step size between points for the cloud -> eg., this sets it to every second point    
        openNIDevices[0].setBaseUserClass(user); // this becomes the base class on which tracked users are created    
                                                 // allows you to set all tracked user properties to the same type easily    
                                                 // and allows you to create your own user class that inherits from ofxOpenNIUser    
        
        // if you want to get fine grain control over each possible tracked user for some reason you can iterate    
        // through users like I'm doing below. Please not the use of nID = 1 AND nID <= openNIDevices[0].getMaxNumUsers()    
        // as what you're doing here is retrieving a user that is being stored in a std::map using it's XnUserID as the key    
        // that means it's not a 0 based vector, but instead starts at 1 and goes upto, and includes maxNumUsers...    
    //    for (XnUserID nID = 1; nID <= openNIDevices[0].getMaxNumUsers(); nID++){    
    //        ofxOpenNIUser & user = openNIDevices[0].getUser(nID);    
    //        user.setUseMaskTexture(true);    
    //        user.setUsePointCloud(true);    
    //        //user.setUseAutoCalibration(false); // defualts to true; set to false to force pose detection    
    //        //user.setLimbDetectionConfidence(0.9f); // defaults 0.3f    
    //        user.setPointCloudDrawSize(2);    
    //        user.setPointCloudResolution(1);    
    //    }    
        
    }    
        
    //--------------------------------------------------------------    
    void testApp::update(){    
        ofBackground(0, 0, 0);    
        pMouseX = mouseX;    
        pMouseY = mouseY;    
        qMouseX = mouseX;    
        qMouseY = mouseY;    
        rMouseX = mouseX;    
        rMouseY = mouseY;    
        sMouseX = mouseX;    
        sMouseY = mouseY;    
        
        
        for (int deviceID = 0; deviceID < numDevices; deviceID++){    
            openNIDevices[deviceID].update();    
        }    
    }    
        
    //--------------------------------------------------------------    
    void testApp::draw(){    
        ofSetColor(255, 255, 255);    
    ofImage myImage; //allocate space for variable    
    myImage.loadImage("commercial-kitchen.jpg"); //allocate space in ram, decode jpg, load pixels.    
    myImage.draw(0,0,1360,768);    
        ofPushMatrix();    
        
        for (int deviceID = 0; deviceID < numDevices; deviceID++){    
        
            // debug draw does the equicalent of the commented methods below    
       openNIDevices[deviceID].drawDepth(0, 0, 320, 260);    
         openNIDevices[deviceID].drawImage(1074,0, 320, 260);    
        openNIDevices[deviceID].drawSkeletons(0, 0, 320, 260);    
        
        }    
        
        // do some drawing of user clouds and masks    
        ofPushMatrix();    
        ofEnableBlendMode(OF_BLENDMODE_ALPHA);    
        int numUsers = openNIDevices[0].getNumTrackedUsers();    
        for (int nID = 0; nID < numUsers; nID++){    
            ofxOpenNIUser & user = openNIDevices[0].getTrackedUser(nID);    
            user.drawMask();    
            ofPushMatrix();    
            ofTranslate(320, 240, -1000);    
            user.drawPointCloud();    
            ofPopMatrix();    
        }    
        ofDisableBlendMode();    
        ofPopMatrix();    
        
        
        
        
        
        glTranslatef(ofGetWidth()/3,ofGetHeight()/1.25,0);    
            //tumble according to mouse    
            glRotatef(-105,1,0,0);    
            glRotatef(180,0,1,0);    
            glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);    
        
        
           tableModel.draw();    
             glPopMatrix();    
        
             glPushMatrix();    
        
            glTranslatef(ofGetWidth()/2,ofGetHeight()/1.15,0);    
           glRotatef(-qMouseY,1,0,0);    
            glRotatef(qMouseX,0,1,0);    
          teaModel.setRotation(0, 90, 1, 0, 0);    
        teaModel.setRotation(1, 270, 0, 0, 1);    
            glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.15,0);    
            teaModel.draw();    
             glPopMatrix();    
             glPushMatrix();    
            glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);    
            glRotatef(-pMouseY,1,0,0);    
            glRotatef(pMouseX,0,1,0);    
          teapotModel.setRotation(0, 90, 1, 0, 0);    
        teapotModel.setRotation(1, 270, 0, 0, 1);    
            glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);    
            teapotModel.draw();    
             glPopMatrix();    
             glPushMatrix();    
            glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);    
            glRotatef(-rMouseY,1,0,0);    
            glRotatef(rMouseX,0,1,0);    
          plateModel.setRotation(0, 90, 1, 0, 0);    
        plateModel.setRotation(1, 270, 0, 0, 1);    
            glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);    
            plateModel.draw();    
             glPopMatrix();    
             glPushMatrix();    
            glTranslatef(ofGetWidth()/2,ofGetHeight()/1.25,0);    
            //tumble according to mouse    
            glRotatef(-sMouseY,1,0,0);    
            glRotatef(sMouseX,0,1,0);    
          spoonModel.setRotation(0, 90, 1, 0, 0);    
        spoonModel.setRotation(1, 270, 0, 0, 1);    
            glTranslatef(-ofGetWidth()/2,-ofGetHeight()/1.25,0);    
            spoonModel.draw();    
             glPopMatrix();    
        glPopMatrix();    
        
    }    

take it this is for the update field

  
    ofVec3f l, r;    
    for(int i = 0; i < openNIDevice.getNumTrackedUsers(); i++){    
                ofxOpenNIUser & usr = openNIDevice.getTrackedUser(i);            
                ofxOpenNIJoint left = usr.getJoint(JOINT_LEFT_HAND);    
                ofxOpenNIJoint right = usr.getJoint(JOINT_RIGHT_HAND);    
                if (left.isFound()) {    
                    l.set(left.getProjectivePosition());//this will give the hand position in the projected image.    
                    //l.set(left.getWorldPosition());//this will give you the position in 3dspace    
                }    
                if (right.isFound()) {    
                    r.set(right.getProjectivePosition());    
                }    

having problems with the ProjectivePosition.
would I be right in thinking that ProjectivePosition.x = mouseX or am I along the complete wrong lines ?

theprojective position is the position of the hand “in the image”, thus it’s values are in pixels.
if you get the projective position and draw the kinect image and a dot in the hand projective position you should see the dot over the hand.

You can assume to some extent that the projective position is comparable to the mouse position.

as for the modelTrans vector, the setRotation method doesnt exist, it is either setOrientation or rotate.
this should work:
for(int i = 0; i<models.size(); i++){// here I’m considering that models and modelTrans are the same size
ofNode tempNode;
modelTrans.push_back(tempNode);

modelTrans.back().rotate(90, 1, 0, 0);
modelTrans.back().rotate(270, 0, 0, 1);
}

Hey, this seems like the most logical place to post - although I can always move it to a separate thread if needs be:

I’m currently trying to do some single user skeleton tracking across multiple kinects, covering a distance of 8-9m. I have managed to correctly install openNI 1.5.7 and NiTE 1.5.2.21 etc and run examples that get either multiple kinects running with no skeleton tracking or a single kinect with skeleton tracking - with a calibration pose. However I can’t get kinects to track the same skeleton across kinects…

My initial plan was to pass the calibration of a the initial tracked skeleton from the first kinect to the following kinects, with the logic being: if a kinect can ‘remember’ the tracking data for up to 15 seconds without the user on camera if this was sent (at the right time/refreshed on a timer) to the 2nd and 3rd kinects they would know what to look for, and would track the skeleton between the different kinects without having the user stop and pose again/recalibrate. However I have been unable to figure out how to do this.

Can anyone help?

And/or I read in this thread:

roymacdonaldAug '12
try the experimental branch
https://github.com/gameoverhack/ofxOpenNI/tree/experimental5

it has a different implementation and it’s supposed to be v2.0
there you have no pose skeleton tracking.

don’t be afraid of the word “experimental” it works really well. I’ve been using it for months now.

cheers!

I didn’t need to recalibrate to track skeleton between kinects it would negate the need to pass the tracking data between kinects and would also solve my problem! But I can’t seem to get openNI2 working…

Any help would greatly appreciated.