Mapping ofCamera's scene to a Webcam's image

I’m working on a project where I want to:

  1. capture a video stream from a webcam,
  2. render it on the screen, and
  3. render objects drawn with oF onto the webcam image

To do this, I figure that I should create a virtual camera with ofCamera and place it in the same position / orientation as the webcam I’m using in real life, and then set the field of view identical.

It’s almost identical to what you’d do for projection mapping.

That’s pretty easy – I do something like:

	mCamera.setPosition(cameraPosition);
	mCamera.setOrientation(cameraRotation);
	mCamera.setFov(90);

And then I created all of my objects with a cmToPixels() function to keep everything at the right scale.

Then I went into my webcam’s setting software and set zoom to ‘1x’ and FOV to 90 (I’m using a Logitech Brio).

The webcam should be capturing a 1080p image, and my openFrameworks window is opened with a 1080p resolution.

Unfortunately, the draw objects and the webcam image don’t line up, even through the virtual and real cameras have the same field of view. I’d expect objects to line up, given that all of my virtual objects were drawn with a real-life scale factor, and the camera’s field of view is identical to the webcam’s.

Anyone have any thoughts on what I should do? Maybe there’s a setting in ofCamera I haven’t found that’ll force the perspective right, or maybe there’s something that needs to match the webcam’s instrinsics other than the FOV…

[Edit] a screencap of what it looks like right now:

Hi, maybe if you share an image of the misalignment someone has a suggestion.

Sure thing – just posted an example.

The screencap shows the input from the webcam overlayed at something like 10% opacity; I requested that the video feed be returned at 1080p, but ofVideoGrabber gives me several log messages that have me a bit concerned:

SETUP: Setting up device 0
SETUP: Logitech BRIO
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 640 by 480
SETUP: trying requested format RGB24 @ 1920 by 1080
SETUP: trying format RGB24 @ 1920 by 1080
SETUP: trying format RGB32 @ 1920 by 1080
SETUP: trying format RGB555 @ 1920 by 1080
SETUP: trying format RGB565 @ 1920 by 1080
SETUP: trying format YUY2 @ 1920 by 1080
SETUP: Capture callback set
SETUP: Device is setup and ready to capture.

Could you verify (cout) the actual camera resolution after it is setup? 1080/480 = 2.25, and that’s more or less the size different between the lines and the hand… Maybe just coincidence.

ofVideoGrabber::getWidth() and getHeight() give me 1920 and 1080 – and the image looks exactly the same as it does in the Logitech Webcam viewer.

If that’s how big it is in the webcam viewer, then it seems like one of the settings in ofCamera isn’t quite right or updated properly…

Could it be that the webcam fov is horizontal fov? The ofCamera one is vertical fov (as usual in opengl) perhaps try: https://themetalmuncher.github.io/fov-calc/

That’s an interesting thought! I hadn’t thought that FOV was different for vertical / horizontal. Calculating the vertical FOV from a horizontal FOV definitely provides better results:

Now I should just double-check that other variables don’t need calibrating…