If you want to perform low-level operations, such as retrieving frames and managing them, then Xuggler would be the best choice, because the APIs are focused on this low level. It works on ffmpeg, so it can handle all types of video encodings.
Do not use JMF for anything old, outdated and bad - GStreamer is good, but the API gives you more options to play videos, rather than manipulate frames.
source share