What’s the resolution and length of your videos?
For your first needs (multiple videos, fast switching, different speeds, seeking, …) You will most likely find the best performance with hardware accelerated video engines. These keep the load on CPU and HDD low and do the hard decompression work on the GPU. Such engines include ofxHapPlayer and my own addon ofxHPVPlayer. However, these addons don’t provide pixel access since this would require reading the un/decompressed pixels back from the GPU and…defeating the whole purpose.
For lengthy videos in HD+, you’re probably best off using platform specific video frameworks (AVFoundation on Mac). For shorter lengths, and smaller sizes, you might use image sequences that are easy to decode (e.g. JPEG, or even uncompressed if you have the HDD size) to buffer frames in RAM. This way you have full access to the pixels, although RAM will be the limiting factor here.