Raspberry Pi camera board + OF


#1

Hi there,

i’m testing the RPi camera board with OF, i’ve found this addon :
https://github.com/jvcleave/ofxRPiCameraVideoGrabber

and it works quite well, but i have some kind of vertical sync problem, when moving ‘fast’, the image seems like cut in two, even with vertical sync turned on in my main app, or in the addon code.

it uses the OMX library to access the cam, which is a different approach than the raspistill program :
https://github.com/raspberrypi/userland/blob/master/host-applications/linux/apps/raspicam/RaspiStill.c

raspistill uses MMAL components.

on top of that the raspistill code seems a lot faster than the ofx addon.

does anyone know another way/addon to use the RPi cam board with OF ?

cheers,

guillaume


videoGrabberExample on Raspberry Pi 3 with v2.1 camera
#2

Hi. I’m also wondering whats up with this add-on - as I’m using the distributed IMG of OF 08 and the camera example, and my board randomly dies with various errors, does not respond to key presses with any sort of sane latency, and generally seems very error prone.

Is there a stable version of the camera / OF distro for RPI that isn’t seemingly flakey?

Is there particular mem-split or other tweaks / nuances to get it running well?

Thanks.


#3

So the MMAL stuff appears faster because it is not using textures but rendering directly to the screen. There seems to be no texture option via MMAL. OpenMax allows you to render directly to a texture or directly to the screen. I implemented the texture stuff first but recently have got the direct-to-screen stuff as an option as well.

It’s quite messy at this point (I was also working on a remote app to control the camera via OSC and adding a separate render option had me re-architect a bunch of stuff) but feel free to take a look at the develop branch of the repo if those features interest you.


#4

So, I am now running:

Model B Raspberry Pi
Latest Raspian
Latest RPI firmware
Running on a 1000mA 5V power supply:

OpenFrameworks main repo, latest master
Latest ofxRPICameraVideoGrabber master via https://github.com/jvcleave/ofxRPiCameraVideoGrabber (@jvcleave - should I be running your dev branch?)

And I am getting the following graphic hiccups in rendering:

Any insight would be awesome. Im going to try your dev branch now to see if its a difference.


#5

OK, so I did not notice at first, the vertical blanking being disabled. Your ‘develop’ branch is insanely better, which is awesome. Im going to leave the camera running to see if I get any crashes, but this is far far superior. Thanks for your work on this.


#6

Good to hear the develop branch is better. it needs a lot of cleanup still as I was mostly trying out features I didn’t know if possible (recording, etc).

It looks like my develop branch is currently set to be using the non-textured renderer which is why you may be seeing less glitchiness. The ofxOMXPlayer in textured mode works much better at 720p (1080p non-textured is ok) so I was suspecting you may be seeing the same with the camera.


#7

Briefly, are those video filters non shader based, but still hardware accelerated?


videoGrabberExample on Raspberry Pi 3 with v2.1 camera
#8

They are OpenMax filters built into the hardware actually - yeah - most of them have very little effect on framerate

btw - you will want to set

doRecording = false;

in the NonTextureEngine class (will be modular later)


#9

Interesting - I’m surprised those are built into the hardware. Amusing. Should look up more about OpenMax.

Im going to try to run some shader based tests later today - I ran the board over night, no issues at all. Thanks again, this is exciting.


#10

So running a simple passthrough shader seriously kills perf, as well as seemingly corrupts the OpenGL command stream so that the Info Text doesn’t render.

I’m assuming this is a GL State ‘leak’ in the texture submission process for the camera. Ill look into it, but I’m more surprised about performance, dropping down to what seems to be 20 or so FPS for just a sample and an assign.

What has been your experience with performance on the RPI?


#11

you are likely using the shader in draw() - you should be able to do this but there is a outstanding bug on the RPi/Programmable renderer

The solution I found to work the most reliably is to use an FBO/Shader combo inside of update.

I’ll try a shader today and see what my framerate is


#12

Hi all. I’ve been using this add-on as well recently (thanks for your work, Jason!) and I’ve been seeing some of the initial odd behaviors vade was originally experiencing, including the camera dying and the app not responding to keypresses quickly or at all. I’m wondering if it’s my code though because the nonTextureApp seems to perform a bit better.

I’m running oF 0.8.0 with the the add-on develop branch, using the nonTextureApp code as a base. I’ve got the camera size at 320x240 with omxCameraSettings.isUsingTexture set to true and I’m doing all my OpenCV processing and drawing stuff in testApp::draw(), including getting the camera pixel data by drawing into an fbo and using readToPixels(). I’m curious if any of those seems like red flags/wrong approaches to anybody who has gone through the same.

Does anyone think using the git/master branch of openFrameworks would help? Running the cam at 720/1080p?

Thanks for any advice/help/ideas!


#13

Hello everybody,

mattfelsen, for me the same thing happens with this addon - whole rpi just crashes and I have to restart it by plugging out power supply cable. This crash occurs when I’m running it in x-server(desktop), but if I start this program in terminal (without x server) it runs pretty good, except one thing - sometimes (randomly) whole screen goes down for 1-2 seconds and after that it works normally another minute or few minutes. I’m using rpi with : 1Ghz and 256/256Mb memory split.


#14

I was able to use a workaround for my project, though I imagine it won’t help for many projects. I was working on a photo booth, so I only needed to capture a still from a video stream. I moved all the code that draws the camera texture to an FBO, gets the pixels, CV background subtraction & differencing, and the rest of the image procesing into a code block that only happens after you press the “shutter” button instead of every update/draw pass and that seems to have solved my problem. Seems like it should’ve been a no-brainer, actually.

So, not sure if that helps you/others in the same situation but it seems like I was just overloading something with oF/the rPi.


#15

What is your monitor type/resolution when this happens? I have had this happen before but it is usually when I have my monitor at 1080p and HDMI boost active.

I use a cheap Element HDMI TV and been playing with my /boot/config.txt options - I have yet to see it happen when it is at 720p

Here are some of the settings I use

# uncomment to force a specific HDMI mode (this will force VGA)
hdmi_group=1
#720p
hdmi_mode=4
#1080p
#hdmi_mode=16

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
config_hdmi_boost=4

#16

Thanks for help.
I’ve tried two different displays with the same 1080p, in the past ir was the same situation with both of them, but now, when I changed configuration to yours, it looks better.
But when I use the same program in desktop it still crashes down whole raspberry (when moving mouse really fast) with 1080p (hdmi_mode=16), but it crashes just program if I use this program in desktop with 720p (hdmi_mode=4). Maby someone knows what happens in desktop? Is it memory overlap or something similar, maby pointer points somewhere outside the memory boundary or something else?

Thank you very much.


#17

does this addon work with the pi 2 and of 0.9.0 ?

i attached the pi camera board as per package.
downloaded the addon and then get an error message after calling make run.

here is the full terminal print out:

something about
error: could not convert ‘ofGLProgrammableRenderer::TYPE’ from ‘const string {aka const std::basic_string<char>}’ to ‘std::shared_ptr<ofBaseRenderer>’ ofSetCurrentRenderer(ofGLProgrammableRenderer::TYPE);

thanks,
stephan


#18

I have used this code.

#include "ofMain.h"
#include "testApp.h"

int main( ){

ofGLESWindowSettings settings;
settings.width = 1024;
settings.height = 768;
settings.setGLESVersion(2);
ofCreateWindow(settings);

ofRunApp( new testApp());

}

#19

I think I had the error as well. I got mine to work. Off the top of my head make sure you have the correct gcc version on your pi.


#20

@jeonghopark
i tried your version for main.h and also just commenting out ofSetCurrentRenderer(ofGLProgrammableRenderer::TYPE);

both compiled without errors.

but make run did not work and printed out this:

@Joshua_deLorimier_TR
i don’t know how to check or change the gcc for the pi2. :frowning: