ofxOpenNI Development

@gameover thanks. could you explain the difference between ONI_CYCLIC and ONI_STREAMING?

ONI_CYCLIC - records a user-settable number of seconds to a cyclic memory buffer;
ONI_STREAMING - records straight to hard drive;

There’s a note at the top of ofxOpenNIRecorder.cpp too:

// Rcorder memory consumption (when using cyclic recorder mode – from NiBackRecorder readme)
// the recorder stores frames in memory in a cyclic buffer that
// is dependent on the streams, their resolutions and the amount of time requested.

// Memory consumption
// ------------------
// 1 second, depth VGA: 30*2*640*480 = 18432000 bytes = 17.6MB
// 1 second, depth QVGA: 30*2*320*240 = 4608000 bytes = 4.4MB
// 1 second, image VGA: 30*3*640*480 = 27648000 bytes = 26.4MB
// 1 second, image QVGA: 30*3*320*240 = 6912000 bytes = 6.6MB

// This means:
// 2 minutes, depth QVGA, image QVGA: (4.4MB+6.6MB)*120 = 1.3GB
// 2 minutes, depth VGA, image VGA: (17.6MB+26.4MB)*120 = 5.2GB

// So be warned, the memory consumption can get quite high when
// recording high resolutions and/or long periods in cyclic mode.

// On the other hand streaming straight to disk doesn’t chew memory (just your disk;-)

I implemented both as a test, and indeed the ONI recorder is over complicated as back when I wrote it openNI required this method…it’s much simpler now.

I recommend just using the STREAMING method…

…and the experimental (simpler) version will be ready soon.

BOTH. it limits the limbs being tracked but also only those limbs need to be in the image to get the user trakced.

Hey,

I am trying the ofxopenNI from gameover, but I have strange problem with the UserandCloud example. It compiles and links without problems but after running for a couple of seconds it receives a BAD_ACCES signal. Is onyone familiar with is problem? Below is the error I get in the console:

Program received signal: “EXC_BAD_ACCESS”.
sharedlibrary apply-load-rules all
warning: .o file “/Developer/usr/lib/gcc/i686-apple-darwin10/4.0.1/libgcc.a(_eprintf.o)” more recent than executable timestamp in “/Developer/of_preRelease_v007_osx/apps/openNI/opeNI-SimpleExamples/bin/openNISample007Debug.app/Contents/Frameworks/GLUT.framework/Versions/A/GLUT”
warning: Could not open OSO file /Developer/usr/lib/gcc/i686-apple-darwin10/4.0.1/libgcc.a(_eprintf.o) to scan for pubtypes for objfile /Developer/of_preRelease_v007_osx/apps/openNI/opeNI-SimpleExamples/bin/openNISample007Debug.app/Contents/Frameworks/GLUT.framework/Versions/A/GLUT
warning: Could not find object file “/Users/theo/Documents/CODE/__OPENFRAMEWORKS/gitOF/openFrameworks/apps/devApps/macDragDropExample/buildGlutFrameworkHackedWindowLevel10.4/libForeground.a(macx_foreground.o)” - no debug information available for “/Users/mcast/Code/GLUT-ToPost/macx_foreground.m”.

thanks!
A3-Sh

@A3-Sh
I’ve run into that same EXC_BAD_ACCESS,

[quote=“A3-Sh, post:118, topic:7403”]
warning: Could not find object file "/Users/theo/Documents/CODE/__OPENFRAMEWORKS/gitOF…
[/quote] we must blame theo for this. (just jokin’)

The truth is that I can’t exactly remember how I fixed it.
At least I know that at the end I did a complete reinstall of OF, openNI, nite and kinect drivers and problem gone.
try to run again the app and when it crashes press the debug button on xcode and try to figure out which function triggers the problem.

Regards

Other than the “EXC_BAD_ACCESS” I think that console output is just telling you that GDB can’t get hold of some debug symbols from libraries that were presumably compiled without them…they don’t tell me or you what is causing the crash. It’s more useful to look at the stack trace or even to see what line Xcode highlights on crash.

For me it crashed on line 79 of the testApp.cpp:

  
ofxOpenNIUser & user = openNIDevices[0].getTrackedUser(nID);  

If you haven’t pressed the stop button you can hover over variables and see there current values at the time of the crash OR (a trick I just got shown yesterday) is to click down next to (gdb) in the trace and type ‘print somevariablename’ or print *this and you can look at variable values. Typing print numUsers gave me some ridiculous number like 42789 - which told me that currentTrackedUserIDs (which is what keeps track of the number and IDs of users) was not getting init’d on startup.

There was also a problem with the iteration of number of users (because I changed the method) in testApp - although this wouldn’t cause a crash it was doing it wrong.

Both problems are update and pushed to the repo…hopefully it fixes the problem :slight_smile:

Let me know.

i know this is the ofxOpenNI thread but since in here are people that worked with the kinect i wanted to ask if you can help with my project:

http://forum.openframeworks.cc/t/touch-table-with-kinect/8819/0

@gameover, The last changes you pushed to the repo fixed the problem! :slight_smile: Thanks!!

Since I am going to record some test streams, because I can work in my test location onyl for a few hours, I wanted to make sure that my recordings are setup right.

Does it make a difference depth resolution wise, when i threshold the recordings to the area i work in (3 to 4 meters away from the camera)? i can remeber that there were some comments in the ofxKinect source that it makes a difference when the raw depth values are converted to the depth images. Is it the same for ofxOpenNI?

Since toggleRegisterViewport() does not affect the playContext can I assume that the streams are recorded with registration when activated?

I don’t think setting depth thresholds will matter to the recording, but perhaps do a test to make sure.

And, yes streams are recorded with registration and cannot be toggled when you’re playing them back.

In case it’s of use I’ve just added ONI recording to the experimental branch and a simple example demonstrating how it works.

I’m going to be using the recording feature a lot over the next three days for one of my projects so will update it if/when it needs during that time.

Hi @gameover, you wouldn’t be able to post a quick example of how to retrieve the raw depth data from the current experimental branch of ofxOpenNI. The new structuring is a huge leap forward, so much easier to work with, but I’m having real trouble with this part.

Once you could do something like:

  
        unsigned short* rawpix = _niDepthGen.getRawDepthPixels();  
                  
        glBegin(GL_POINTS);  
        for(int y=0; y < KINECT_HEIGHT; y += step) {  
              for(int x=0; x < KINECT_WIDTH; x += step) {  
                        float d = rawpix[y * WIDTH_CAM + x];  
                        glVertex3f(x, y, d / depthScale);  
              }  
       }  
        glEnd();   

But now, if I try:

  
        ofShortPixels& pixels = _openNI.getDepthRawPixels();  
        XnDepthPixel* rawPix = pixels.getPixels();  
        glBegin(GL_POINTS);  
        for(int y=0; y < KINECT_HEIGHT; y += step) {  
              for(int x=0; x < KINECT_WIDTH; x += step) {  
                        float d = rawpix[y * WIDTH_CAM + x];  
                        glVertex3f(x, y, d / depthScale);  
              }  
       }  
       glEnd();   

… I get thread errors when it calls getDepthRawPixels().

Please feel free to point and laugh if I’m doing something stupid here…

@zenbullets: sorry I’d never used raw depth pixels before so I didn’t make it work right. It’s fixed by last commit to the repo…your second code example now works :slight_smile:

hi,

@gameover I tested your example project in visual studio 2010 sp1 on win7x64 on an i5 m560 laptop with an NV 420m graphics chip. i am getting lots of “ERROR Could not open file mapping object (2).” in the console while running. First I thought that this was also the reason the project running with just 15 fps, while the codeblocks compiled projects runs at 60 fps without errors. After another compile and run of the codeblocks projects the visual studio project jumped to 60 fps too. the errors are still printed to the console, one every second. i can reproduce the fps jump everytime I run the codblocks project. the next run in visual studio solution is fast, the second run it is slow again. Any Ideas?

thanks

Hey Chris can you tell me a couple of things:

* which branch of ofxOpenNI;
* which version and branch of of007;
* and which version of openNI, NITE and SensorKinect

you are using?

@gameover
I am at the latest master branch of OF and ofxOpenNI.
i used the following openni installers:
openni-win32-1.5.2.23-dev
nite-win32-1.5.2.21-dev

i cannot find the installer i used for sensor kinect so my guess is i used SensorKinect091-Bin-Win32-v5.1.0.25.msi from avin2’s github. the kinect camera device driver says version 3.1.2.0

Can you try (in this order):

  1. Master branch of ofxOpenNI with develop branch of oF007

and then

  1. Experimental branch of ofxOpenNI with develop branch of oF007

The Master branch of ofxOpenNI is not really supported anymore - all my time is spent on the Experimental branch. It’s way better and way simpler. I haven’t tried it on Windows, but it ‘should’ work…would be good to know and I have more incentive to fix it if it’s not :wink:

@gameover ok will try that now.
i am really confused by the frame rates. sometimes they start with 30 fps and drop then to 10. sometimes they start with 80 and slowly cycle between 70 and 30, without starting the codeblocks project first. at first i thought it is connected with the powermanangement setting of the nividia driver but it is random. performance with the intel graphics is horrible too compared to codeblocks.

still this error is printed all the time in the console. do you have an idea where it comes from?

thanks.

so i switched branches and pulled the latest code with

  
git checkout develop  
git pull  

on my OF repository witch made no difference performance wise.

i switched to the experimental branch of your repository.
the visual studio solution was not adapted to your code refactoring. i deleted the old and added the new files of the src fold of ofxOpenNI. i put the source files of ONIrecordingSimple into the src folder of the example project folder.
when I try to compile the solution I get the following error:

  
  
1>------ Build started: Project: openNISample007, Configuration: Release Win32 ------  
1>Build started 20.02.2012 14:10:35.  
1>InitializeBuildStatus:  
1>  Touching "obj\Release\openNISample007.unsuccessfulbuild".  
1>ClCompile:  
1>  ofxOpenNI.cpp  
1>..\..\..\libs\openFrameworks\graphics\ofPixels.h(152): warning C4003: not enough actual parameters for macro 'max'  
1>..\..\..\libs\openFrameworks\graphics\ofPixels.h(153): warning C4003: not enough actual parameters for macro 'max'  
1>c:\openframeworks\addons\ofxopenni\src\ofxOpenNITypes.h(689): warning C4003: not enough actual parameters for macro 'max'  
1>c:\openframeworks\addons\ofxopenni\src\ofxOpenNITypes.h(689): error C2059: syntax error : ')'  
1>c:\openframeworks\addons\ofxopenni\src\ofxOpenNITypes.h(689): error C2334: unexpected token(s) preceding ':'; skipping apparent function body  
1>c:\openframeworks\addons\ofxopenni\src\ofxOpenNITypes.h(691): error C2143: syntax error : missing ')' before '}'  

When looking at the point of error I have no idea why the compiler does not like that line of code. what about you?

Hi Chris

Couldn’t spend too long on this today, but have fixed cross-compilation problems on windows, updated the project files for VS2010 and Codeblocks as well as the examples.

The “ERROR Could not open file mapping object (2)” was being spat out by the openNI log for me, until I removed user, hand and gesture nodes from the openniconfig.xml file (I assume you were using XML to setup?)…it won’t do that now.

Performance is pretty choppy on my windows box (and it is not lacking for processing power)…I’ve fiddled around a bit with how it updates the context, but haven’t really found the reason as yet.

Anyway, see if this makes any difference to performance. Be aware that since this version is threaded the reported app frames per second don’t reflect the kinect frames per second. I found that without threading performance was pretty average, which I find surprising (15 - 25fps)…I don’t have this problem on Mac. Could be stable drivers are better on Windows, could be my method makes the whole thing choke, or could be PC’s don’t play nice with openNI. Let’s hope it’s one of the first two :wink:

When I get a chance I’ll try a straight implementation of depth.image capture with no fancy threading, masks, textures etc and try to get to the bottom of it.

In the meantime I hope this works for you.

Hi @ all

I had the same “ERROR Could not open file mapping object (2)” & tried gameovers experimental branch of ofxOpenNI. The error disappeared as well with develop branch & master branch of oF007.
But for now I would better like to stay on the master branch of ofxOpenNI with my project… So I followed the error with my debugger. It always appears after the update() function of my ofxOpenNIContext instance - in detail after running line 5934 in XnCppWrapper.h … But I’ve no idea how to solve that.

Besides I have similar performance trouble regarding the frame rate after starting my compiled application. Sometimes it starts with less than 20fps. After restarting (once or twice or more) it works fine up to 50/60/70fps, but I didn’t had to compile it again. I’m not sure - did you mean this, Christoph?

@gameover: Thanks a lot for your work!

Btw, I work on Windows7 & VisualStudio2010