Is it possible to capture any part of the screen, including the desktop or other application windows and get it into an openframeworks application?
I’ve done this before in windows using WxWindows and in Linux with X11 code (see http://code.google.com/p/binarymilleniu-…-hexModules ), is there an openframeworks specific way?
I think each way of doing this will be OS specific – I posted a mac / osx way to accomplish this here:
http://forum.openframeworks.cc/t/grabbing-from-the-screen-osx/2946/0
though I’m not sure how efficient it is (there are a few ways to do it it seemed like, and I didn’t experiment too deeply).
as for windows, I’m sure this is possible also – I haven’t tried, but will take a look when I have some free time.
take care,
zach
anton aka vade has an awesome Quartz Composer plugin (mac osx) to capture any part of the screen and retrieve it as an opengl texture. Everything stays on gpu so super fast. It’s opensource so maybe you can dig out the relevant bits. I think to make it cross platform shouldn’t be too complicated as it’s mostly opengl, but I may be wrong.
[quote author=“memo”]anton aka vade has an awesome Quartz Composer plugin (mac osx) to capture any part of the screen and retrieve it as an opengl texture. Everything stays on gpu so super fast. It’s opensource so maybe you can dig out the relevant bits. I think to make it cross platform shouldn’t be too complicated as it’s mostly opengl, but I may be wrong.
http://002.vade.info/?page-id=4[/quote]
That program has this function:
static GLuint _CreateWindowTexture(CGLContextObj internalContext, CGLContextObj mainDrawingContext, NSInteger width, NSInteger height, NSInteger originX, NSInteger originY,NSRect bounds)
{
CGLContextObj cgl_ctx = internalContext;
// Thread lock OpenGL Context
CGLLockContext(cgl_ctx);
GLuint mTextureName;
GLenum theError = GL_NO_ERROR;
// set up our texture storage for copying
glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
// Create and configure the texture - rectangluar coords
glGenTextures(1, &mTextureName);
glBindTexture(GL_TEXTURE_RECTANGLE_ARB, mTextureName);
glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
// define our texture - we're allowd to supply a null pointer since we are letting the GPU handle texture storage.
glTexImage2D(GL_TEXTURE_RECTANGLE_ARB,0,GL_RGBA, width,height,0,GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, NULL);
// read from the front buffer
glReadBuffer(GL_FRONT);
// glBindTexture(GL_TEXTURE_RECTANGLE_ARB, mTextureName);
// copy contents of a portion of the buffer to our texture
glCopyTexSubImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, 0, 0, originX, originY, width, height);
// fin
glFlush();
// Thread unlock
CGLUnlockContext(cgl_ctx);
//Check for OpenGL errors
theError = glGetError();
if(theError) {
NSLog(@"v002ScreenCapture: OpenGL texture creation failed (error 0x%04X)", theError);
CGLUnlockContext(cgl_ctx); // Thread unlock
return 0;
}
return mTextureName;
}
I’m not sure about the context parts but the straight opengl looks portable. I’ll have to experiment.
I think screen capture is a good generic capability- though it can be slow and inefficient, sometimes it’s the only way to get output from one program into another that doesn’t require external hardware. Maybe eventually there will be an official loadScreenData() that can grab from anywhere on the screen.
just had a go at this and i’m getting a grey texture but no errors
(it’s definitely filling with a grey texture, as it draws my test texture until it starts grabbing, at which point it’ll go grey)
i’m using the following code:
void testApp::screenGrab(int x, int y, ofTexture &tex)
{
int w, h;
w = tex.getWidth();
h = tex.getHeight();
CGLContextObj cgl_ctx = [theOpenGLContext CGLContextObj];
// Thread lock OpenGL Context
CGLLockContext(cgl_ctx);
GLenum theError = GL_NO_ERROR;
// Create and configure the texture - rectangluar coords
tex.bind();
// define our texture - we're allowd to supply a null pointer since we are letting the GPU handle texture storage.
glTexImage2D(GL_TEXTURE_RECTANGLE_ARB,0,GL_RGBA, w,h,0,GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, NULL);
// read from the front buffer
glReadBuffer(GL_FRONT);
// copy contents of a portion of the buffer to our texture
glCopyTexSubImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, 0, 0, x, y, w, h);
// fin
glFlush();
// Thread unlock
// CGLUnlockContext(cgl_ctx);
//Check for OpenGL errors
if (glGetError())
ofLog(OF_LOG_ERROR, "screenGrab: OpenGL didn't like it!");
}
as you can see, i commented out the thread lock. which could be the reason i’m getting this (none) result.
my other thought is that there might be an issue with ‘GL_FRONT’. Maybe we’re living in a different GL world to the rest of OSX here…?
elliot
ok, so it kind of works, but i can only get the front buffer of my oF application captured (hence the grey, i.e. repeating the background)
hey elliot,
did you ever get this working correctly? i’m looking to write an application that can real-time capture the user’s screen for an interaction. using opengl buffer would be ideal like this thread is talking about, so curious to know if you got this working.
I’ve been looking into this as well. Isn’t it supposed to be that openGL can’t read data from parts it doesn’t have a context for. I thought the trick was to setup an invisible/transparent fullscreen window and grab data from that using opengl. At least I’ve heard people do that before. However with modern compositing window managers this is ‘impossible’ since the desktop is being composited and every window gets it’s own pixel buffer. I’ve had success getting the desktop as a texture using Xlib with Shm in linux. However it involves coping from system memory to video memory which is heavy. I’m now looking into doing this purely in the GPU but I’m doubting it will be successful. You shouldn’t be able to access memory parts you don’t own. I’m not sure if the GPU has that much protection though.
syphon?
https://github.com/andreacremaschi/Syphon-virtual-screen
virtual desktop
https://github.com/mkernel/EWProxyFramebuffer
Syphon recorder
All OSX only unfortunately 
I don’t suppose anyone has this functionality working under windows?
I’ve seen it done in other apps so it should be possible I’m just not sure if it requires d3d instead of openGL
Hi,
This C++ program take screenshots and save them as .bmp files. (It require FreeImage.dll in the bin directory to run)
It capture the whole desktop or a named window, using Windows GDI (probably not the faster way to do it).
I think it’s possible to do the same in OF, and re-use the pixels instead of saving them to the disk.
Hi,
I would be interested in checking out the program you linked too (link is now dead). I’ve managed to get screen capture of a specific window working with GDI, however I’m getting mixed results with different hardware.
Hi,
screenshots.zip (23.8 KB)
It doesn’t work with my new harware, I only see the icons in the capture. Strange. But it reminds me that I faced the same problem in another topic recently, and I managed to make it work (even if I don’t really understand): Recursive Screen Recording Solution for desktop feedback loop!