What is derived from the OpenCV Dense function of the optical stream (Farneback)? How can this be used to build an optical stream map in Python?

Question

I am trying to use the Opencv optical stream function output with a dense optical stream to draw a portion of a quiver of motion vectors, but could not find what the function actually produces. Here is the code:

import cv2 import numpy as np cap = cv2.VideoCapture('GOPR1745.avi') ret, frame1 = cap.read() prvs = cv2.cvtColor(frame1,cv2.COLOR_BGR2GRAY) hsv = np.zeros_like(frame1) hsv[...,1] = 255 count=0 while(1): ret, frame2 = cap.read() next = cv2.cvtColor(frame2,cv2.COLOR_BGR2GRAY) flow = cv2.calcOpticalFlowFarneback(prvs,next,None, 0.5, 3, 15, 3, 10, 1.2, 0) mag, ang = cv2.cartToPolar(flow[...,0], flow[...,1]) hsv[...,0] = ang*180/np.pi/2 hsv[...,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX) rgb = cv2.cvtColor(hsv,cv2.COLOR_HSV2BGR) if count==10: count=0 print "flow",flow cv2.imshow('frame2',rgb) count=count+1 k = cv2.waitKey(30) & 0xff if k == 27: break elif k == ord('s'): prvs = next cap.release() cv2.destroyAllWindows() 

This is actually the same code as in the OpenCv Dense Optical Flow tutorial. I get the following output from the print function:

 flow [[[ 0.00000000e+00 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00] ..., [ 0.00000000e+00 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00] [ 0.00000000e+00 0.00000000e+00]] ..., [[ -3.54891084e-14 -1.38642463e-14] [ -2.58058853e-14 -1.54020863e-14] [ -5.56561768e-14 -1.88019359e-14] ..., [ -7.59403916e-15 1.16633225e-13] [ 7.22156371e-14 -1.61951507e-13] [ -4.30715618e-15 -4.39530987e-14]] [[ -3.54891084e-14 -1.38642463e-14] [ -2.58058853e-14 -1.54020863e-14] [ -5.56561768e-14 -1.88019359e-14] ..., [ -7.59403916e-15 1.16633225e-13] [ 7.22156371e-14 -1.61951507e-13] [ -4.30715618e-15 -4.39530987e-14]] 

I would like to know what exactly these values ​​represent? Original X, Y coordinates? The final coordinates of X, Y? Is the distance moved?

I plan to try to find the start and end coordinates to make a quiver using the code from the following page: https://www.getdatajoy.com/examples/python-plots/vector-fields This is because python has no function, about which I know, which displays an optical flow map for you.

Thank you in advance!

Answer 1

You were almost there. Let's first look at calcOpticalFlowFarneback Documentation , which says:

flow is a calculated image of a stream that has the same size as prev and type CV_32FC2 .

What you actually get is a matrix that is the same size as your input frame.
Each element in this flow matrix represents a point that represents the offset of this pixel from the prev frame. This means that you get a point with x and y values ​​(in units of pixels), which gives you delta x and delta y from the last frame.