I have lots of rectangles randomly located in 3d space. I want each of the rectangles can be clicked and it will move to the center of the screen. I know that ofxMSAInteractiveObject or ofWorldToScreen may be useful but I’m not sure whether they can do the hit test in 3d space.
ofWorldToScreen will give you the screen coordinates of a point in space, so you’ll probably want to use it on all your rectangles: this way, for every rectangle, you’ll be able to find 4 points on your screen.
At this point you’ll just have to check if your mouse pointer is between your points; if your rectangles are billboarded, something like:
will probably be good enough; otherwise, if your rectangles are a warped, just google on how to find out if a point is inside a polygon.
Last consideration: if your rectangles can overlap one each other, you’ll probably want to index them depending on their z position and start your mouse tests with closest rectangle.