Real-time streaming video streaming using overlay

Question

I play with different options for the video stream for a raspberry camera. Currently, the best solution that has the lowest latency is to transfer data from raspividto ncand then tomplayer

On raspberries:

/opt/vc/bin/raspivid -t 0 -hf -vf -w 640 -h 480 --nopreview -o - | nc -l 5000

On the client machine ( -fps 60trick to skip buffering)

nc $RASP_IP 5000 | mplayer -nosound -framedrop -x 640 -y 480 -fps 60 -demuxer +h264es -cache 1024 -

This job is really nice with almost zero latency.

Now I would like to overlay some dynamic data on the video. What would be the best way to achieve this?

I have seen solutions such as editing raspividand adding opencv directly, but this will not work in my case, since the display should be on a different machine than the one that is connected to the camera.

A technology (language / library) does not matter much if it cannot work on * nix (.NET is not an option)