Trouble with Blackmagic Capture card and 10-bit images

advanced
#1

Hello. For some time I’ve been working of a project with a Blackmagic DeckLink Mini Recorder 4K and I can’t solve a problem. I need to save individual frames from the video stream, which I am doing with a combination of ofxBlackMagic by @fresla and ofxImageSequenceRecorder by @atduskgreg. It works fine for 8 bit images, but the capture card can provide 10 bit RGB images and that’s where the problem is. I have tried to rewrite all the methods replacing ofPixels for ofxShortPixels, but when I try to save the images the console tells me that the pixels are not allocated. This is fixed whenever I switch back to the previous methods with ofPixels. The video feed draws with no problem. Any help will be greatly appreciated. Thanks!

Maybe @arturo or @edapx know or could help me with a hint on where to look?

#2

10bits is not directly supported by ofPixels since it can’t be hold directly by a native type. you would need to get the data in it’s raw format without using ofPixels and then save the image by doing direct calls to freeimage or similar

#3

Thank you @arturo ! So ofShortPixels wouldn’t work either? Hypothetically if I manage to get the data without using ofPixels, how could I draw it on the screen? I guess getting an ofTexture from 10 bits would be just as difficult, right?

#4

ofShortPixels is 16bits so even if you are getting the frames they are surely in the wrong format with part of the memory in the pixels containing the actual info but with extra space not being used and wrong with and height distributio of the pixels

Also 10bits doesn’t mean much without knowing the format, is it 10bits yuv? 10bits rgb? without knowing the exact format you are receiving from the card and how it’s laid out in memory it’s impossible to know how to show it or store it

#5

Also just switching the container won’t work. You will need to check how the Blackmagic SDK deals with the data and then try to get it into a format you can see or save. Saving images is one thing but it is pretty difficult to view 10 bit images on a 10 bit display on a computer at the moment anyway.

#6

Thank you @arturo and @fresla. I broke my program but now I’m back on track and I think I have some information. Inside ofxBlackmagic (camera) there is an object called DeckLinkController (controller); inside this object here is another object called VideoFrame (rgbaframe), whose class inherits from IDeckLinkVideoFrame, which I suppose is the frame as it comes out of the capture card. This class has a method called getPixelFormat(), which could be of help to know what container to use, but when I try to call it like this

camera.controller.rgbaframe->IDeckLinkVideoFrame.GetPixelFormat();

I get this error

/home/antonio/dev/of_v0.11.0_linux64gcc6_release/apps/myApps/OpenCelluloid/src/ofApp.cpp:37: error: invalid use of ‘IDeckLinkVideoFrame::IDeckLinkVideoFrame’
             cam.controller.rgbaFrame->IDeckLinkVideoFrame.GetPixelFormat();
                                       ^~~~~~~~~~~~~~~~~~~

My guess is that by managing to grab the IDeckLinkVideoFrame directly I could get access to the 10 bit data. On the other hand, the recoder that I implemented uses ofSaveImage as a threaded function. ofSaveImage works with ofPixels, ofShortPixels and ofFloatPixels, but this containers wouldn’t work with the 10 bit images, is that correct?

Thanks again!

#7

I don’t have an overview of what you are trying to do, but it may be worth looking at just running ffmpeg from an ofSystem() call in your app.I know that you can compile ffmpeg with full decklink support and this will let you record 10 bit movies or image sequences and you can control it from within your OF app.

#8

That is a good idea, but wouldn’t that mean that when ffmpeg is running I cannot access the video stream and draw it on the screen?

#9

@fresla I am trying to capture 10 bit frames from the capture card and using the serial port as a trigger. I started digging through the examples of the SDK and there is one called CaptureStills, which I think has the solution for my problem. I am trying to build it with the addon but some things are not linking properly. Like the object of your addon DeckLinkController is dependent on another one called IDeckLinkAttributes, but that is no longer present in the newest DeckLinkAPI.h. I don’t know why it’s not there anymore but I think it means is deprecated, right?

#10

You can also build ffmpeg with syphon. But you will not see 10 bit very easily unless you use the Blackmagic hardware output.

#11

@fresla I think I’m going to go one step behind. I have read the sdk manual and I have seen your addon over and over and I think it’s way over my league to do this at this point. What I want to do is to be able to capture individual frames from the capture card with a trigger from the serial port. I found a way by integrating a very simple version of ofxImageSequenceRecorder to get pngs. They are 8 bit but they are a good beginning. However I a, running with some memory issues, I will have to read more about threaded functions. Do you think you could give it a look an tell me if it’s totally off what I am doing?

#12

@fresla there is also an example in the SDK called capture stills, which basically does what I am looking for but with a timer interval. I managed to organize the sdk and the files of the example in a layout for an eventual addon, but the example is a command line application and I need to draw the image on the screen and I guess it’s going to take time before I manage to do all those things. But it seems that that would be the ideal way to grab 10 bit pngs of a capture card. Anyway, I don’t know if you have time to check this, thanks a million for your add, it has helped me a lot. The final project is an open system to digitize analog film for artists (“http://www.opencelluloid.com/”)