Threaded video grabber

advanced
#1

Hello,

I am having trouble to make a threaded video grabber class. In the threaded function I update the camera grabber and save it into an ofImage.

Right now I don’t an image at all when I call the draw function.

Here is how my class look like, if anyone has an idea why it doesn’t work.

#pragma once

#include "ofMain.h"
#include <atomic>

class videoThread : public ofThread
{
public:

	//--------------------------------------------------------------
	~videoThread()
	{
		stop();
		waitForThread(false);
		_cam.close();
	}


	//--------------------------------------------------------------
	void setup() {
		_cam.setDeviceID(0);
		_cam.setDesiredFrameRate(60);
		_cam.setup(640, 480);

		_img.allocate(640, 480, OF_IMAGE_COLOR);

		// Start the thread
		start();
	}


	//--------------------------------------------------------------
	void draw() {
		ofSetHexColor(0xffffff);
		_img.draw(0, 0);
	}


private:
	ofVideoGrabber _cam;

	// Threaded functions
	//--------------------------------------------------------------
	void start()
	{
		startThread();
	}

	//--------------------------------------------------------------
	void stop()
	{
		std::unique_lock<std::mutex> lock(mutex);
		stopThread();
		condition.notify_all();
	}

	//--------------------------------------------------------------
	void threadedFunction()
	{
		while (isThreadRunning())
		{
			_cam.update();
			if (_cam.isFrameNew()) {
				
				if(lock()) {
					_img.setFromPixels(_cam.getPixels());
					unlock();
				}
			}
		}
	}

protected:
	ofImage _img;
	std::condition_variable condition;
};
#2

I think that this do not work because you can not draw inside a thread, make the _img public or create a get method to return the image. Create an instance of your threaded video grabber in the main of class and draw there…

#3

Thanks for your answer.
I am calling the draw function only in the main thread, I am aware that you cannot draw inside a thread directly.

I will try to return the _img to the main thread and draw it in there.

#4

I might be wrong but I don’t think you can update OpenGL across multiple threads.

ofImage::setFromPixels will update the texture ( for drawing ) inside of the ofImage so that won’t work in a separate thread.

The correct way would be to have a threaded ofPixels and a non threaded ofImage.

Set the pixels in the thread ( lock / unlock ) and then update the ofImage in an update function called in the main thread.

like:

void videoThread::update(){
  lock();
    _img.setFromPixels(_threadedPixels);
  unlock(); 
}

Then the OpenGL texture update will work correctly.

Hope that helps!
Theo

#5

You might also want to take a look at the ofThreadChannel class, which significantly simplifies accessing resources across threads. In your case you’ll send your ofPixels to the main thread, where you can update your ofImage

As @theo mentioned, you can’t make OpenGL calls outside of the main thread. I’m not sure it is necessary, but you also might want to tell the grabber not to use textures (which you are not interested in anyway, since you’re sending ofPixels to the main thread). You can do this by calling:

_cam.setup(640, 480, false);

c.

#6

Not sure what you are trying to accomplish putting the video grabber in a thread but it doesn’t make sense as it is. The grabber already runs internally in it’s own thread so the only thing you would be doing is making extra copies of the pixels which would be slower than not using a thread at all.

The only thing that ofVideoGrabber::update does is mostly uploading the new pixels to the texture so moving that onto a thread won’t make any difference.

As others have said you can’t update the texture from a thread, at least not using the same context, so you won’t get any advantage there either

#7

Hey guys,

Thanks for the answers. I looked again at the thread example and ended up doing the solution Theo mentioned and it is now working.

@arturo, thank you for this explanation. Reason I put the video grabber in a thread is that I want to do some more processing later on (thresholding, AR detection, …) and thought that putting all the video-related parts in a thread will not be blocking my main thread and will be cleaner.

But if I understand what you mean correctly, I should rather put the video grabber in the main thread (not to have its thread redundant, right?), send the pixels to my thread where I do all the post processing I need and then return the information/image after it’s being processed.
Would this be a better approach?

#8

yes, that’s it. you want to put in a thread the cpu intensive tasks, the video graber update is not, so it won’t only not give you any advantage but the way you are doing it, you are copying the pixels in the main thread which will slow things down. the videograbber is already doing that, in a much more efficient manner.