ofxOpenNI - removing and adding UserGenerators failing

Hi All,

I’m attempting to make an app that allows for the recording of .oni files, and playback with skeleton tracking.

The example provided by gameover is excellent:

https://github.com/gameoverhack/ofxOpenNI/blob/master/examples/opeNI-SimpleExamples/src-ONIRecording-Simple/testApp.cpp

But it doesn’t allow for skeleton tracking in the player, only the recorder.

If I change the testApp::keyPressed(int key) function to the following:

  
void testApp::keyPressed(int key){  
  
    switch (key) {  
        case ' ':  
            {  
                string newFilename = ofGetTimestampString();  
                newFilename = newFilename+".oni";  
                  
                if(!openNIRecorder.isRecording()){  
                    openNIRecorder.startRecording(ofToDataPath(newFilename));  
                }else{  
                    openNIRecorder.stopRecording();  
                }  
                break;  
            }  
        case 'l':  
            {  
                openNIRecorder.removeUserGenerator();  
                  
                ofFileDialogResult results = ofSystemLoadDialog();  
                  
                openNIPlayer.setupFromONI(results.filePath);  
                  
                openNIPlayer.addUserGenerator();  
                break;  
            }  
    }  
      
}  

Then I get the following output to the console (after recording a new .oni file with skeleton tracking successfully activated onscreen)

  
[ofxOpenNIDevice[0]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[0]:notice] Init context...  
[ofxOpenNIDevice[0]:verbose] Context initilizedstatus:OK  
[ofxOpenNIDevice[0]:notice] openni driver version: 1.5.4.0  
[ofxOpenNIDevice[0]:notice] Adding licence...  
[ofxOpenNIDevice[0]:verbose] Adding licence: PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=status:OK  
[ofxOpenNIDevice[0]:notice] Init device...  
[ofxOpenNIDevice[0]:verbose] Enumerate devicesstatus:OK  
[ofxOpenNIDevice[0]:notice] Found1devices connected  
[ofxOpenNIDevice[0]:verbose] Creating production tree for device 0status:OK  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_DEPTH  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_DEPTH generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Setting Depth1 resolution: 640 x 480 at 30fpsstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_DEPTH generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Allocating depth  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_IMAGE  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_IMAGE generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Setting Image1 resolution: 640 x 480 at 30fpsstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_IMAGE generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Allocating image  
[ofxOpenNIDevice[0]:verbose] Register viewpoint depth to RGBstatus:OK  
[ofxOpenNIDevice[0]:verbose] Set mirror depth ONstatus:OK  
[ofxOpenNIDevice[0]:verbose] Set mirror image ONstatus:OK  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_USER  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_USER generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_USER generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Allocating users  
[ofxOpenNIDevice[0]:notice] User generator DOES NOT require pose for calibration  
[ofxOpenNIDevice[0]:verbose] Set skeleton profilestatus:OK  
[ofxOpenNIDevice[0]:notice] Starting ofxOpenNI with threading  
[ofxOpenNIDevice[1]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[1]:notice] Init context...  
[ofxOpenNIDevice[1]:verbose] Context initilizedstatus:OK  
[ofxOpenNIDevice[1]:notice] openni driver version: 1.5.4.0  
[ofxOpenNIDevice[1]:notice] Adding licence...  
[ofxOpenNIDevice[1]:verbose] Adding licence: PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=status:OK  
[ofxOpenNIDevice[1]:notice] Init device...  
[ofxOpenNIDevice[1]:verbose] Enumerate devicesstatus:OK  
[ofxOpenNIDevice[1]:notice] Found1devices connected  
[ofxOpenNIDevice[1]:verbose] Creating production tree for device 1status:Failed to set USB interface!  
[ofxOpenNIDevice[1]:warning] REAL Device could not be initialized - you can still use an ONI  
[ofxOpenNIDevice[1]:notice] Starting ofxOpenNI with threading  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_RECORDER  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_RECORDER generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_RECORDER generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Set recording output file: ../../../data/2013-03-12-23-01-37-672.onistatus:OK  
[ofxOpenNIDevice[0]:notice] Starting ONI recording...  
[ofxOpenNIDevice[0]:verbose] Adding device generator to recordingstatus:OK  
[ofxOpenNIDevice[0]:verbose] Adding depth generator to recordingstatus:OK  
[ofxOpenNIDevice[0]:verbose] Adding image generator to recordingstatus:OK  
[ofxOpenNIDevice[0]:verbose] (CB) New User1  
[ofxOpenNIDevice[0]:notice] Calibration requested for user1  
[ofxOpenNIDevice[0]:notice] Create tracked user1  
[ofxOpenNIDevice[0]:verbose] (CB) Calibration start for user1  
[ofxOpenNIDevice[0]:verbose] (CB) Calibration end for user...1  
[ofxOpenNIDevice[0]:verbose] ...success1  
[ofxOpenNIDevice[0]:notice] Start tracking user1  
[ofxOpenNIDevice[0]:notice] Skeletonfoundfor user1  
[ofxOpenNIDevice[0]:verbose] Force stopping user tracking1  
[ofxOpenNIDevice[0]:notice] Skeleton tracking stopped for user1  
[ofxOpenNIDevice[0]:notice] Stop tracking user1  
[ofxOpenNIDevice[0]:verbose] Mark for deleting user1  
[ofxOpenNIDevice[0]:verbose] Deleting user1  
[ofxOpenNIDevice[0]:notice] Stopping ONI recording...  
[ofxOpenNIDevice[0]:verbose] Removing device generator from recordingstatus:OK  
[ofxOpenNIDevice[0]:verbose] Removing depth generator from recordingstatus:OK  
[ofxOpenNIDevice[0]:verbose] Removing image generator from recordingstatus:OK  
[ofxOpenNIDevice[0]:notice] Removing generator typeXN_NODE_TYPE_USER  
[ofxOpenNIDevice[0]:verbose] Stopping XN_NODE_TYPE_USER generatorstatus:OK  
[ofxOpenNIDevice[1]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[1]:notice] Starting ONI player:/Users/joel/Documents/Projects/HellicarAndLewis/liseNorwayMovement/oF/of_v0.7.4_osx_release/apps/newSync/Recorder/bin/data/2013-03-12-23-01-37-672.oni  
[ofxOpenNIDevice[1]:verbose] Loading ONI: /Users/joel/Documents/Projects/HellicarAndLewis/liseNorwayMovement/oF/of_v0.7.4_osx_release/apps/newSync/Recorder/bin/data/2013-03-12-23-01-37-672.onistatus:OK  
[ofxOpenNIDevice[1]:verbose] Creating depth generator from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Allocating depth  
[ofxOpenNIDevice[1]:verbose] Creating player from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Creating device from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Creating image generator from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Allocating image  
[ofxOpenNIDevice[1]:notice] Starting ofxOpenNI with threading  
[ofxOpenNIDevice[1]:warning] Currently it is only possible to have a user generator on one device in a single process!!  

I.E. it states that the user generator has been removed from ofxOpenNIDevice[0], but can’t then be added to ofxOpenNIDevice[1]. Has anyone had success with the removal and then adding of user generators?

Cheers,

Joel

Hi Joel,
the thing is that openni only allows to have one user generator per process. the warning that you’re getting is because there must be some other user generator registered to an instance of ofxOpenNI. So first make sure you remove all the user generators before adding a new one. also, if I remember correctly, you should first add the generators and then calling the player. I’m almost sure that I had to do that once to make it work.
I’ll search my computer for some useful code.

best regards

Hey Rory,

Thanks for the response. The only way I have found to get Skeleton tracking working with recorded .oni files is not to have another ofxOpenNI instance of any kind in the program. Even if I create another instance of ofxOpenNI without adding a UserGenerator, the second instance cannot have a UserGenerator of any kind:

Code:

testApp::setup

  
  
openNIRecorder.setup();  
openNIRecorder.addDepthGenerator();  
openNIRecorder.addImageGenerator();  
openNIRecorder.setRegister(true);  
openNIRecorder.setMirror(true);  
//    openNIRecorder.addUserGenerator(); //NO USER GENERATOR!  
//    openNIRecorder.setMaxNumUsers(1);  
openNIRecorder.start();  
  
openNIPlayer.setup();  
//    openNIPlayer.addDepthGenerator();  
//    openNIPlayer.addImageGenerator();  
//    openNIPlayer.setRegister(true);  
//    openNIPlayer.setMirror(true);  
//    openNIPlayer.addUserGenerator();  
//    openNIPlayer.setMaxNumUsers(1);  
openNIPlayer.start();  
  

testApp::keyPressed

  
  
case 'l':  
    {  
        openNIPlayer.addDepthGenerator();  
        openNIPlayer.addImageGenerator();  
        openNIPlayer.setRegister(true);  
        openNIPlayer.setMirror(true);  
        openNIPlayer.addUserGenerator();  
        openNIPlayer.setMaxNumUsers(1); //1 for this one....  
          
        ofFileDialogResult results = ofSystemLoadDialog();  
          
        openNIPlayer.setupFromONI(results.filePath);  
  
        break;  
    }  
  

Console output:

  
  
[ofxOpenNIDevice[0]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[0]:notice] Init context...  
[ofxOpenNIDevice[0]:verbose] Context initilizedstatus:OK  
[ofxOpenNIDevice[0]:notice] openni driver version: 1.5.4.0  
[ofxOpenNIDevice[0]:notice] Adding licence...  
[ofxOpenNIDevice[0]:verbose] Adding licence: PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=status:OK  
[ofxOpenNIDevice[0]:notice] Init device...  
[ofxOpenNIDevice[0]:verbose] Enumerate devicesstatus:OK  
[ofxOpenNIDevice[0]:notice] Found1devices connected  
[ofxOpenNIDevice[0]:verbose] Creating production tree for device 0status:OK  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_DEPTH  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_DEPTH generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Setting Depth1 resolution: 640 x 480 at 30fpsstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_DEPTH generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Allocating depth  
[ofxOpenNIDevice[0]:notice] Adding generator typeXN_NODE_TYPE_IMAGE  
[ofxOpenNIDevice[0]:verbose] Creating XN_NODE_TYPE_IMAGE generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Setting Image1 resolution: 640 x 480 at 30fpsstatus:OK  
[ofxOpenNIDevice[0]:verbose] Starting XN_NODE_TYPE_IMAGE generatorstatus:OK  
[ofxOpenNIDevice[0]:verbose] Allocating image  
[ofxOpenNIDevice[0]:verbose] Register viewpoint depth to RGBstatus:OK  
[ofxOpenNIDevice[0]:verbose] Set mirror depth ONstatus:OK  
[ofxOpenNIDevice[0]:verbose] Set mirror image ONstatus:OK  
[ofxOpenNIDevice[0]:notice] Starting ofxOpenNI with threading  
[ofxOpenNIDevice[1]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[1]:notice] Init context...  
[ofxOpenNIDevice[1]:verbose] Context initilizedstatus:OK  
[ofxOpenNIDevice[1]:notice] openni driver version: 1.5.4.0  
[ofxOpenNIDevice[1]:notice] Adding licence...  
[ofxOpenNIDevice[1]:verbose] Adding licence: PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=status:OK  
[ofxOpenNIDevice[1]:notice] Init device...  
[ofxOpenNIDevice[1]:verbose] Enumerate devicesstatus:OK  
[ofxOpenNIDevice[1]:notice] Found1devices connected  
[ofxOpenNIDevice[1]:verbose] Creating production tree for device 1status:Failed to set USB interface!  
[ofxOpenNIDevice[1]:warning] REAL Device could not be initialized - you can still use an ONI  
[ofxOpenNIDevice[1]:notice] Starting ofxOpenNI with threading  
[ofxOpenNIDevice[1]:notice] Adding generator typeXN_NODE_TYPE_DEPTH  
[ofxOpenNIDevice[1]:verbose] Creating XN_NODE_TYPE_DEPTH generatorstatus:Failed to set USB interface!  
[ofxOpenNIDevice[1]:error] setGeneratorResolution() called on invalid generator!  
[ofxOpenNIDevice[1]:notice] Adding generator typeXN_NODE_TYPE_IMAGE  
[ofxOpenNIDevice[1]:verbose] Creating XN_NODE_TYPE_IMAGE generatorstatus:Can't create any node of the requested type!  
[ofxOpenNIDevice[1]:error] setGeneratorResolution() called on invalid generator!  
[ofxOpenNIDevice[1]:verbose] Depth generator is not on  
[ofxOpenNIDevice[1]:warning] Currently it is only possible to have a user generator on one device in a single process!!  
[ofxOpenNIDevice[1]:warning] Using a NASTY hack to silence SIGNAL errors on exit - read the comments at line ~1712 of ofxOpenNI.cpp  
[ofxOpenNIDevice[1]:notice] Starting ONI player:/Users/joel/Documents/Projects/HellicarAndLewis/liseNorwayMovement/oF/of_v0.7.4_osx_release/apps/newSync/Recorder/bin/data/2013-03-13-10-15-11-098.oni  
[ofxOpenNIDevice[1]:verbose] Loading ONI: /Users/joel/Documents/Projects/HellicarAndLewis/liseNorwayMovement/oF/of_v0.7.4_osx_release/apps/newSync/Recorder/bin/data/2013-03-13-10-15-11-098.onistatus:OK  
[ofxOpenNIDevice[1]:verbose] Creating depth generator from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Allocating depth  
[ofxOpenNIDevice[1]:verbose] Creating player from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Creating device from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Creating image generator from ONI/XML  
[ofxOpenNIDevice[1]:verbose] Allocating image  
[ofxOpenNIDevice[1]:notice] Starting ofxOpenNI with threading  
[verbose] OF app is being terminated!  
ofxOpenNIDevice[0]: stop called  
ofxOpenNIDevice[0]: trying to lock  
ofxOpenNIDevice[0]: trying to stop thread  
ofxOpenNIDevice[0]: releasing all nodes  
ofxOpenNIDevice[0]: releasing depth generator  
ofxOpenNIDevice[0]: releasing image generator  
ofxOpenNIDevice[0]: releasing device  
ofxOpenNIDevice[0]: releasing context  
ofxOpenNIDevice[0]: releasing depth texture & pixels  
ofxOpenNIDevice[0]: releasing image texture & pixels  
ofxOpenNIDevice[0]: full stopped  
ofxOpenNIDevice[1]: stop called  
ofxOpenNIDevice[1]: trying to lock  
ofxOpenNIDevice[1]: trying to stop thread  
ofxOpenNIDevice[1]: releasing all nodes  
ofxOpenNIDevice[1]: releasing player  
ofxOpenNIDevice[1]: releasing depth generator  
ofxOpenNIDevice[1]: releasing image generator  
ofxOpenNIDevice[1]: releasing device  
ofxOpenNIDevice[1]: releasing context  
ofxOpenNIDevice[1]: releasing depth texture & pixels  
ofxOpenNIDevice[1]: releasing image texture & pixels  
ofxOpenNIDevice[1]: full stopped  
ofxOpenNIDevice[1]: destructor called  
ofxOpenNIDevice[1]: stop called  
ofxOpenNIDevice[0]: destructor called  
ofxOpenNIDevice[0]: stop called  
  

It would seem that even if you don’t have a UserGenerator on the initial instance of ofxOpenNI, then you still can’t add one to a second instance of ofxOpenNI. Is this to do with there not being a second physical device? Some of the console output would seem to point to this, but it does state that .oni files are still loadable. I have completely working separate applications, but I would love to be able to play back files and record in the same application.

If I have a single instance of ofxOpenNI how do I safely halt playback to switch it into Record mode or vice versa?

I’m going to try experimenting with that now. Onwards!

Cheers,

Joel

If I just have one instance of ofxOpenNI then I can sucessfully start live, record a new .oni file, and play it back. However, once playback is started, how do I stop playback, and switch into record mode? If I call stop() on the instance, and just setup again, then I crash out when attempting to start recording. Do I have to wait for the thread?

  
  
//--------------------------------------------------------------  
void testApp::keyPressed(int key){  
  
    switch (key) {  
        case ' ':  
            {  
                string newFilename = ofGetTimestampString();  
                newFilename = newFilename+".oni";  
                  
                if(openNIRecorder.isPlaying()){  
                    //lets stop it  
                    openNIRecorder.stop();  
                    //then start it again  
                    openNIRecorder.setup();  
                    openNIRecorder.addDepthGenerator();  
                    openNIRecorder.addImageGenerator();  
                    openNIRecorder.setRegister(true);  
                    openNIRecorder.setMirror(true);  
                    openNIRecorder.addUserGenerator();  
                    openNIRecorder.setMaxNumUsers(1);  
                    openNIRecorder.start();  
                }  
                  
                  
                if(!openNIRecorder.isRecording()){  
                    openNIRecorder.startRecording(ofToDataPath(newFilename));  
                }else{  
                    openNIRecorder.stopRecording();  
                }  
                break;  
            }  
        case 'l':  
            {  
                ofFileDialogResult results = ofSystemLoadDialog();  
                  
                openNIRecorder.setupFromONI(results.filePath);  
                  
                openNIRecorder.addDepthGenerator();  
                openNIRecorder.addImageGenerator();  
                openNIRecorder.setRegister(true);  
                openNIRecorder.setMirror(true);  
                openNIRecorder.addUserGenerator();  
                openNIRecorder.setMaxNumUsers(1); //1 for this one....  
  
                break;  
            }  
    }  
      
}  
  

Hi Joel,

I had this problem on a project last year for the Southbank which demoed all the OpenNI generators. Just had a quick looking to see how I fixed it.

Before loading a file to playback I called a reset like so:

  
  
void Controller::resetOpenNI() {  
    if (openNI) {  
        openNI->waitForThread(true);  
        delete openNI;  
    }  
    openNI = new ofxOpenNI();  
    openNI->setup();  
}  
  

Also when adding an removing user generators I was sure to pause OpenNI, for example (I had different “Screens” which demoed a different generator):

  
  
void GestureScreen::enable() {  
    controller->openNI->setPaused(true);  
    controller->openNI->addImageGenerator();  
    //... adding more stuff  
    controller->openNI->setPaused(false);  
    ofAddListener(controller->openNI->gestureEvent, this, &GestureScreen::gestureEvent);  
}  
  
  
void GestureScreen::disable() {  
    controller->openNI->setPaused(true);  
    ofRemoveListener(controller->openNI->gestureEvent, this, &GestureScreen::gestureEvent);  
    controller->openNI->setPaused(false);  
}  
  

I can’t quite remember why I done the pausing, but I am sure it help cleanly add/remove the generators.

Hope that is of some help! The result was an app which could record, playback and hot swap generators without reopening.
Ross

@JGL,
you can’t have more than one user generator per process (application). If I remember correctly ofxOpenNi implementation will allocate by default the usergenerator and attach it to the first instance of ofxOpenNI.
the easiest way to record and play is to use just a single instance, just like you did.
Yet there’s no need to add the generators for switching between playback and record. I would stop and pause the ofxOpenNI instance before switching from record to playback.
I dont have a kinect at hand right now but in the afternoon I can check it and paste the code for it.
best