Processing the screen picture on the fly. Is it possible?

hi,
I am playing with the idea of processing the screen image of a PC in order to add some warping, color adjustment, effects etcetera. Basically I’d like to get the whole screen as input (with whatever happens to be on that at the moment) and process it before it gets drawn.

Of course it is not possible to just grab the screen and work on that, as the output would have to be displayed on the same place and this would lead to an infinite recursion. Interesting may be but definitely not what I need :sweat_smile:

Is it possible to get the original screen, modify it and then draw the modified image? Some screensavers apparently do something like that when they add some effects to the desktop picture.

The target environment for this ideally would be Windows.

Thank you for helping
Davide

You can try OBS Studio to capture the display and send to a NDI video stream.
Then you can receive the stream into OF with the addons, to process and etc.

That could work, but it would suffer the same problem of direct screen grabbing. The modified image would be drawn on screen an then grabbed again, giving an infinite loop.

The trick would be able to intercepting the desktop picture before it gets drawn, modify it and present the altered version to the user. I do not even know if this is possible in principle, since it could rise some security concerns, but (at least in the past) I remember some screensavers doing some magic on the desktop so may be…

cheers
Davide

old school screensavers that manipulated the desktop took a snapshop of the desktop then started drawing that (manipulated) to screen. To manipulate a live desktop - as you describe - AFAIK you would need to intercept the system buffer before the desktop is drawn, manipulate it and then re-insert the data into the buffer.
either running OBS & NDi as @moebiussurfing suggests or grabbing a still and manipulating may be the only (without much pain) way -

Yes after a lot of searching I believe this is not possible, at least not without some deep knowledge of Windows internals that I do not have. If it were possible it could open a lot of awful security concerns, so may be it is better this way.

A workaround could be capturing the main monitor and drawing the manipulated screen on a second one… or may be if it is possible to trick Windows into believing there is a second (virtual) monitor do vice-versa, keeping the real one for the altered picture.