Optical Flow Vehicle Identification

Question

I implemented an optical stream to track vehicles on the road, and it turned out to be very slow.

My code uses the following functions:

• cvGoodFeaturesToTrack
• cvFindCornerSubPix
• cvCalcOpticalFlowPyrLK

How to make this tracking fast and efficient?

My code is:

#include "highgui.h" #include "cv.h" #include "cxcore.h" #include <iostream> using namespace std; const int MAX_CORNERS = 500; int main() { CvCapture* capture=cvCreateFileCapture("E:\cam1.avi"); IplImage* img_A;// = cvLoadImage("image0.png", CV_LOAD_IMAGE_GRAYSCALE); IplImage* img_B;// = cvLoadImage("image1.png", CV_LOAD_IMAGE_GRAYSCALE); img_A=cvQueryFrame(capture); IplImage* imgA = cvCreateImage( cvGetSize(img_A), 8, 1 ); IplImage* imgB = cvCreateImage( cvGetSize(img_A), 8, 1 ); cvNamedWindow( "ImageA", CV_WINDOW_AUTOSIZE ); cvNamedWindow( "ImageB", CV_WINDOW_AUTOSIZE ); cvNamedWindow( "LKpyr_OpticalFlow", CV_WINDOW_AUTOSIZE ); while(1) { int couter=0; for(int k=0;k<20;k++) { img_B=cvQueryFrame(capture); } //cvCvtColor(imgA,imgA,CV_BGR2GRAY); //cvCvtColor(imgB,imgB,CV_BGR2GRAY); // Load two images and allocate other structures /*IplImage* imgA = cvLoadImage("image0.png", CV_LOAD_IMAGE_GRAYSCALE); IplImage* imgB = cvLoadImage("image1.png", CV_LOAD_IMAGE_GRAYSCALE);*/ CvSize img_sz = cvGetSize( img_A ); int win_size = 10; IplImage* imgC = cvCreateImage( cvGetSize(img_A), 8, 1 ); cvZero(imgC); // Get the features for tracking IplImage* eig_image = cvCreateImage( img_sz, IPL_DEPTH_32F, 1 ); IplImage* tmp_image = cvCreateImage( img_sz, IPL_DEPTH_32F, 1 ); int corner_count = MAX_CORNERS; CvPoint2D32f* cornersA = new CvPoint2D32f[ MAX_CORNERS ]; cvCvtColor(img_A,imgA,CV_BGR2GRAY); cvCvtColor(img_B,imgB,CV_BGR2GRAY); cvGoodFeaturesToTrack( imgA, eig_image, tmp_image, cornersA, &corner_count ,0.05, 5.0, 0, 3, 0, 0.04 ); cvFindCornerSubPix( imgA, cornersA, corner_count, cvSize( win_size, win_size ) ,cvSize( -1, -1 ), cvTermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.03 ) ); // Call Lucas Kanade algorithm char features_found[ MAX_CORNERS ]; float feature_errors[ MAX_CORNERS ]; CvSize pyr_sz = cvSize( imgA->width+8, imgB->height/3 ); IplImage* pyrA = cvCreateImage( pyr_sz, IPL_DEPTH_32F, 1 ); IplImage* pyrB = cvCreateImage( pyr_sz, IPL_DEPTH_32F, 1 ); CvPoint2D32f* cornersB = new CvPoint2D32f[ MAX_CORNERS ]; /*int jk=0; for(int i=0;i<imgA->width;i+=10) { for(int j=0;j<imgA->height;j+=10) { cornersA[jk].x=i; cornersA[jk].y=j; ++jk; } } */ cvCalcOpticalFlowPyrLK( imgA, imgB, pyrA, pyrB, cornersA, cornersB, corner_count, cvSize( win_size, win_size ), 5, features_found, feature_errors, cvTermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.3 ), 0 ); // Make an image of the results for( int i=0; i < corner_count; i++ ) { if( features_found[i]==0|| feature_errors[i]>550 ) { //printf("Error is %f/n",feature_errors[i]); continue; } //printf("Got it/n"); CvPoint p0 = cvPoint( cvRound( cornersA[i].x ), cvRound( cornersA[i].y ) ); CvPoint p1 = cvPoint( cvRound( cornersB[i].x ), cvRound( cornersB[i].y ) ); cvLine( imgC, p0, p1, CV_RGB(255,0,0), 2 ); cout<<p0.x<<" "<<p0.y<<endl; } cvShowImage( "LKpyr_OpticalFlow", imgC ); cvShowImage( "ImageA", imgA ); cvShowImage( "ImageB", imgB ); //cvCopyImage(imgB,imgA); delete[] cornersA; delete[] cornersB; cvWaitKey(33); } return 0; } 

Answer 1

I might go over this line a bit, but I suggest you check out OpenTLD . OpenTLD (aka Predator) is one of the most efficient tracking algorithms. Zdenek Kalal has implemented OpenTLD in MATLAB. George Nebehai made a very efficient C++ OpenCV port of OpenTLD .

It is very easy to install and track, really effective.

OpenTLD uses the Median Flow Tracker to track and implement the PN learning algorithm. In this YouTube video, Zdenek Kalal shows the use of OpenTLD.

If you just want to implement Median Flow Tracker, follow this link https://github.com/gnebehay/OpenTLD/tree/master/src/mftracker

If you want to use it in Python, I made Median Flow Tracker and also made Python Port OpenTLD . But the python port is not very efficient.

Answer 2

First of all, in order to track a car, you need to somehow detect it (for example, using color segmentation / background subtraction). When a car is detected, you must track it (track some points on it) using cvCalcOpticalFlowPyrLK . I did not find the code that is responsible for detecting the car.

Look this one and this one . Your idea should be the same.

Also your code is a bit wrong. For example, why do you call cvGoodFeaturesToTrack in the main loop? You have to call it once - before the loop to find good tracking functions. But it will also detect non-cars.

Take a look at the default OpenCV example: OpenCV / samples / cpp / lkdemo.cpp .