Record depth image using kinect

advanced
1

hey guys,

I need to record a video maybe a minute long for my final year project using the kinect so that i can test my code. However i am having trouble finding a solution to record a depth image using the kinect. I saw that ofxKinect addon has the ofxKinectRecorder, but i have no idea how i can use it :S. I need to record a short video using the depth image of the kinect into a .mov file. Any idea how that can be done? Apologies, but i am pretty bad at programming and have trouble understanding how things work still. So far i have written only the code below that loads the depth video and rbg video.

Any help would be much appreciated,
Dean.

  
#include "ofMain.h"    
  
#include "ofxOpenCv.h"    
#include "cv.h"    
#include "ofxKinect.h"  
#include "ofxKinectRecorder.h"  
  
  
class testApp : public ofBaseApp{    
      
public:    
      
    void setup();    
    void update();    
    void draw();    
      
    void keyPressed  (int key);  
      
    ofxKinect               kinect;  
      
    ofxKinectRecorder       record;  
      
    //ofVideoGrabber        vidGrabber;  
      
    ofxCvColorImage         colorImg;  
    ofxCvGrayscaleImage     grayImg;  
  
    int tiltAngle;  
    int width, height;  
};  


  
#include "testApp.h"    
  
  
//--------------------------------------------------------------    
void testApp::setup()  
{      
    //kinect.setRegistration(true);  
      
    width = 320;  
    height = 240;  
      
    tiltAngle = 10;  
      
    kinect.init();  
    kinect.open();  
    kinect.setCameraTiltAngle(tiltAngle);  
      
    colorImg.allocate(kinect.width, kinect.height);  
    grayImg.allocate(kinect.width, kinect.height);  
}  
  
//--------------------------------------------------------------    
void testApp::update()  
{  
    kinect.update();  
      
    if (kinect.isFrameNew())  
    {  
        colorImg.setFromPixels(kinect.getPixels(), 640, 480);  
        grayImg.setFromPixels(kinect.getDepthPixels(), 640, 480);  
    }  
}  
  
//--------------------------------------------------------------    
void testApp::draw()  
{  
    colorImg.draw(20, 20, width, height);  
    grayImg.draw(width+40, 20, width, height);  
      
    //record.newFrame(kinect.getPixels(), kinect.getRawDepthPixels()); // ???  
    //record.init("images/depth.mov"); /// ???  
}  
  
  
//--------------------------------------------------------------    
void testApp::keyPressed  (int key)  
{  
    switch (key)   
    {  
        case OF_KEY_UP:  
            tiltAngle++;  
            if (tiltAngle > 30)   
                tiltAngle = 30;  
            kinect.setCameraTiltAngle(tiltAngle);  
            break;  
              
        case OF_KEY_DOWN:  
            tiltAngle--;  
            if (tiltAngle < -30)  
                tiltAngle = -30;  
            kinect.setCameraTiltAngle(tiltAngle);  
            break;  
              
        case 'f':  
            ofToggleFullscreen();  
    }  
}
2

Well… if you are in a hurry and don’t have much time at the moment you can always use FRAPS. :wink:
Render the video onto a full screen openF application at the resolution you want and record. http://www.fraps.com/
If I remember correctly, fraps records only full screen so there are other applications for recording windows or regions of the screen onto video files. Search for one but, if you value your computer, don’t download the newest HyperCam!!! never!!!.. NOOO!!! It’s free but installs toolbars and stuff even if you set it not to.
Anyways, after recording, just crop your video to the resolution you need (avoid resizing), export to mov and you can code for a “fake kinect” using this video.
I used this some time ago. The only problem you might notice is that compression artifacts become much more visible when you analize video on a difference pixel per pixel basis. Also you WILL see dct macroblocks (don’t know what this is? forget I mentioned it). :smiley:

There are many other ways to do this as code of course. ofxOpenNI has a recorder that uses ONI files and it seems to be used in the examples too. I have no idea how good it works.

3

I was thinking of using a screen record software to do my video, and i actually downloaded a youtube video, not a very good one, just so i can test and i think the problem with recording a regular file like that does not allow me to play the video when i use the function kinect.getDepthPixels(). It works fine on kinect.getPixels(), but didn’t work on the get depth. I will have a look at the OpenNI example, hopefully i can find a file that can be played back and allows to get the depth.

Thanks for the tip,
Dean.

4

Well, the kinect.getDepthPixels() returns something very similar to a monochrome video frame (8 bit). The ofxKinect addon has to make the distinction between the two. In fact they all do, some of them even allow you to look at the other streams, audio, RAWDepth (11 bit), etc.
When you play it from a video file you are using a different set of functions and ofVideoPlayer has no getDepthPixels but it will still play the video that would eventually be sent over for blob analysis. Check the opencvExample if you wish to continue trying this method.

I think you have a misconception about depth. It will mostly look like a grayscale gradient that is generated using depth values. You can even shift the gradient, make it a color rainbow… but ultimately it will be analyzed as blocks of memory with int or short values (that can be seen as color or depth).
If you have that depth video you can analize per pixel and that value is related to the depth of that pixel in the RGB video.

1 Like
5

You can use:

https://github.com/HellicarAndLewis/Sync

To record and playback kinect video depth images - you don’t need the drawing code, but the recording code should be fine.

J

2 Likes
6

@JGL
Nice!
Can I ask (slightly off topic) do you have any videos of Sync working with a dance routine? Like a vimeo or something. I’m very interested in this.

7

Thank you for the help, i will have a look at it now :slight_smile:

8

Nothing recorded as yet I am afraid, we will be returning to Norway soon.

J

9

@dean.spiridonov,
if you have trouble programming maybe the easiest way is to save the gray image from the code you posted into to create an image sequence. The you can simply open the image sequence in Quicktime Pro7 (there’s “Open image sequence…” in the File menu) and save it as a .mov in which ever codec you like.
Just have to add the following to the code you’ve got now.
to setup()

  
frame = 0;// init frame value to zero  
bSaveSequence = false;// this one will allow you to save just when you want to and not all the time that the app is running.

to update() AFTER what’s there right now.

  
  
if(bSaveSequence){  
grayImg.saveImage(ofToString(frame)+".jpg"); // change the ".jpg" for ".png" if you want a png sequence.  
frame++;  
}

and in keyPressed INSIDE the switch brackets:

  
  
case ' ': // this is the space bar. you can change it to any key you want to.  
bSaveSequence ^=true;//this will toogle from true to false each time it's called.

Finaly declare bSaveSequence and frame in the .h file.

  
  
bool bSaveSequence;  
int frame;

That’s all. Now run your app and press the spacebar to start and stop recording.

You could also add this to draw so that you can know if it is recording or not.
add AFTER what you have in draw()

  
  
if(bSaveSequence){  
ofPushStyle();  
ofSetColor(255,0,0);  
ofCircle(50,50, 10); this will draw a red circle in the upper left corner whenever you're recording.  
ofPopStyle();  
}

Let me know how it goes.

2 Likes