How to make e-con camera work with Openframworks

I just bought a brand new e-con Camera(USB 3.0 Cameras). It works fine with the application e-camview they provided. But when I try to use it in Open frameworks with ofVideoGrabber, It did not work. I am new to OF, Can anyone give me some hints on how to make it work? Can it work with openframework? I am working on Windows 10.

yes you can work with econ camera. I have worked with it. I have worked with a SEECAM 24CUG. What error you are getting?

I am also having problems with the e-con Cameras, I have the See3CAM_CU55M - 5MP Monochrome USB NIR Camera. When i run the ofVideoGrabber I get an error that the camera can not be initialised.

Any ideas?

SETUP: Setting up device 1
SETUP: See3CAM_CU135M_H03R1
SETUP: Couldn't find preview pin using SmartTee
SETUP: Default Format is set to 1920 by 1080
SETUP: trying requested format RGB24 @ 1920 by 1080
SETUP: trying format RGB24 @ 1920 by 1080
SETUP: trying format RGB32 @ 1920 by 1080
SETUP: trying format RGB555 @ 1920 by 1080
SETUP: trying format RGB565 @ 1920 by 1080
SETUP: trying format YUY2 @ 1920 by 1080
SETUP: trying format YVYU @ 1920 by 1080
SETUP: trying format YUYV @ 1920 by 1080
SETUP: trying format IYUV @ 1920 by 1080
SETUP: trying format UYVY @ 1920 by 1080
SETUP: trying format YV12 @ 1920 by 1080
SETUP: trying format YVU9 @ 1920 by 1080
SETUP: trying format Y411 @ 1920 by 1080
SETUP: trying format Y41P @ 1920 by 1080
SETUP: trying format Y211 @ 1920 by 1080
SETUP: trying format AYUV @ 1920 by 1080
SETUP: trying format Y800 @ 1920 by 1080
SETUP: trying format Y8 @ 1920 by 1080
SETUP: Capture callback set

Almost off topic : do you like it? does it have low latency? which model are you using?

@tomLen - I don’t see any errors in that printout, its basically trying a bunch of different formats before settling on Y8 ( Monochrome ).

Are there any other messages that indicate an error.
Is the video feed just not coming in?

You might want to see what the native format is ( it might not be 1920 x 1080 ).
The data sheets require registration, but if you try one of the native formats you might have better luck.

Hope that helps!
Theo

@theo Thank you for your response, I hope this helps, and thank you much in advance for the help

Based on the datasheet the camera can support Y8 and Y16 color space with the following resolutions, I am also using windows 10 64 bits:
1280 x 720, 1920 x 1080, 3840 x 2160, 4200 x 3120.

I tried OF_PIXELS_YV12, OF_PIXELS_MONO, and basically other different pixel formats.

But it seems to me that ofDirectShowGrabber only supports RGB pixel format, any suggestions on how to change to pixel format to Y8 or Y12 on windows?

vidGrabber.setDeviceID(1);
vidGrabber.setDesiredFrameRate(30);
vidGrabber.setPixelFormat(OF_PIXELS_YV12);// OF_PIXELS_MONO); //OF_PIXELS_YV12
vidGrabber.setVerbose(true);
vidGrabber.setup(w,h);

The app creashes in this line:


[notice ] ---
[notice ] ---
[warning] ofDirectShowGrabber: setPixelFormat(): requested pixel format not supported
[notice ] ofDirectShowGrabber: initGrabber(): choosing 1

@tomLen - that is weird. You shouldn’t have to set the pixel format, as videoInput converts whatever is the closest matching format to RGB as part of the setup.

Does it work if you set it to 1280x720? ( try not setting frame rate or pixel format either ).

If you are able to you could you try the example in my videoInput repo?

It is in:
videoInputSrcAndDemos/VS-videoInputDemoWithSrc/

If you just set the resolution and device ID it will probably crash, but it will show you where in setupDevice it is crashing.

@theo

Thanks, I got your videoInput running, and the camera is working after I click on the “Step out” debug option several times. I notice your code runs on win32 and I was running the example in 64 bits version, so I switched the videoGrabber example from win64 to win32 and that did the trick. Any ideas why is not working for 64 bits?

on the 64 bits version, I get this error. However, on the 32 bits the pins do connect successfully.

SETUP: trying format Y8 @ 1920 by 1080
SETUP: Capture callback set
ERROR: Could not connect pins - RenderStream()

SETUP: Disconnecting device 1
SETUP: freeing Grabber Callback
SETUP: freeing Renderer
SETUP: freeing Capture Source
SETUP: freeing Grabber Filter
SETUP: freeing Grabber
SETUP: freeing Control
SETUP: freeing Media Type
SETUP: removing filter NullRenderer...
SETUP: filter removed NullRenderer
SETUP: removing filter Sample Grabber...
SETUP: filter removed Sample Grabber
SETUP: removing filter See3CAM_CU135M_H03R1...
SETUP: filter removed See3CAM_CU135M_H03R1
SETUP: freeing Capture Graph
SETUP: freeing Main Graph
SETUP: Device 1 disconnected and freed

Many thanks!

Thanks @tomLen - that is super weird, but maybe there is some legacy pipeline that is enabled in DirectShow in 32bit that isn’t in 64bit?

I did see something relating to the RenderStream here:

If you are able to try CAPTURE_MODE instead with 64bit that might do the trick?
If so we might want to find a way to build that into the library.

Thanks!
Theo

Thanks @theo

I rebuilt the library to x64 and change the line to CAPTURE_MODE and recompile openframeworks with the new change and unfortunately it did not work. Are there other parts of the videoInput library that uses legacy 32 bits libraries or calls?
Thanks for the help
Thomas

@tomLen thanks for trying that.
If you run the videoInput demo ( from source ) as x64 does it show you which line in videoInput.cpp it is crashing on?

I was looking through the issues on Github and this one suggested maybe the size might need to account for pixel type.

Does changing this from the * 3 to * 1 ( For Y8 being a single channel format ) do anything to help the issue?

If so I think I’ll need to a proper accounting of the bytes per pixel instead of having it hardcoded to 3.

Thanks!
Theo

Actually @tomLen could you try rebuilding the x64 lib but with changing this line:

from:

	//buffer size
	VD->pAmMediaType->lSampleSize = attemptWidth*attemptHeight*3;

to:

	//buffer size
	VD->pAmMediaType->lSampleSize = 0;				//these are optional, so we do this so we don't try and guess the sample size
	VD->pAmMediaType->bFixedSizeSamples = false;	//these are optional, so we do this so we don't try and guess the sample size

I think this should allow capture from any format without crashing. :crossed_fingers:

@theo Thank you for the follow-up.

I tried both code changes and the library still breaks in

hr = VD->pCaptureGraph->RenderStream(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video, VD->pVideoInputFilter, VD->pGrabberF, VD->pDestFilter);

I did dig a bit further and found your library on OpenCV with some e-con camera includes, In specific and extra GUID is added for the Y16 data format . So I updated the videoInput src to include the Y16 part of the code,

I also specified the image size to be w * h *2

and memcpy,

I used the memcpy code because processPixels was crashing here, I suspect because of the input image size from the grabber.

The RenderSteam error disappeared and I can see the camera LED turn (which did not happen before in 64 bits) and the pixel shift on the contrast of the image.

The main videograpbber complaining about the pixel size setFromPixels from vidGrabber.update() Which I imagine ofx is trying to process the image byte size of wh2, Do you have an idea what would be the equivalent in openframeworks for that PIXEL_FORMAT,

Many thanks for all the help!
Thomas

Thanks @tomLen !

I think at this point we should probably use the videoInput from OpenCV as it seems to support non RGB output format as well as more devices.

The problem is that ofDirectShowGrabber ( OF’s wrapper which ofVideoGrabber uses ) assumes RGB only so that’s why the pixels are all messed up.

If you instead use videoInput directly and then pass the pixels to an OF texture you should be good.

Btw Y8 is a bit easier to work with as its just a 8bit / 1 byte mono image ( eg OF_PIXELS_GRAYSCALE ), whereas Y16 is using 2 bytes per pixel which is what some of the openCV code they added to videoInput accounts for.

eg: opencv/cap_dshow.cpp at 2273af0166f5c4eb825802a187501874d20256f8 · opencv/opencv · GitHub

I’ll see if I can pull in the OpenCv changes to videoInput to my repo so they can work standalone from OpenCV.

Apologies this has been such a headache!
Theo

@theo Thank you so much for the help!

I finally got it to work. for some reason this e-con camera is only accpeting the Y16 format

used this bit transformation that with mentioned to convert to from Y16 to RGB with this modification:

	for (int i = 0; i < (width * height); i++) {
		*dst_y = (uint8_t)(*((uint16_t*)src_y)/4);
		dst_y++;
		*dst_y = (uint8_t)(*((uint16_t*)src_y)/4);
		dst_y++;
		*dst_y = (uint8_t)(*((uint16_t*)src_y)/4);
		dst_y++;
		src_y += 2;
	}

For the dst pixel size I passed new pixel array with dimensions w *h *3 to the update function in the ofDirectshowGrabber class . Becaue before it was using w*h*2 size to create the pixel array.

unsigned char*  pixelsd = new unsigned char[1920*1080*3]; //setup

VI.getPixels(device, pixelsd,  true, true);
pixels.setFromPixels(pixelsd, width, height, OF_IMAGE_COLOR);

Thomas

2 Likes