How to match MSER and HOG in Matlab

Question

I wanted to know if there is a full implementation of MSER and HOG image matching in Matlab. I am currently using VLFeat , but have found difficulty in performing image matching. Any help?

Btw, I tried the code below in a VLFeat -Matlab environment, but unfortunately the mapping could not be done.

%Matlab code % pfx = fullfile(vl_root,'figures','demo') ; randn('state',0) ; rand('state',0) ; figure(1) ; clf ; Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ; Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ; Ia = uint8(rgb2gray(Ia)) ; Ib = uint8(rgb2gray(Ib)) ; [ra,fa] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ; [rb,fb] = vl_mser(I,'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ; [matches, scores] = vl_ubcmatch(fa, fb); figure(1) ; clf ; imagesc(cat(2, Ia, Ib)); axis image off ; vl_demo_print('mser_match_1', 1); figure(2) ; clf ; imagesc(cat(2, Ia, Ib)); xa = ra(1, matches(1,:)); xb = rb(1, matches(2,:)) + size(Ia,2); ya = ra(2, matches(1,:)); yb = rb(2,matches(2,:)); hold on ; h = line([xa ; xb], [ya ; yb]); set(h, 'linewidth', 1, 'color', 'b'); vl_plotframe(ra(:,matches(1,:))); rb(1,:) = fb(1,:) + size(Ia,2); vl_plotframe(rb(:,mathces(2,:))); axis image off ; vl_demo_print('mser_match_2', 1); %%%%%% 

Answer 1

There are a couple of problems. Firstly, the code has several errors and does not work as is. I pasted my working version below.

More importantly, you are trying to use the SIFT function matching function to match MSER ellipsoids. This will not work because SIFT gives a very large dimensional vector object based on local gradients of the image, and the MSER detector just gives you a limited ellipsoid.

VLFeat does not seem to include the MSER mapping function, so you may have to write your own. Take a look at the original MSER paper to see how they fit:

“Reliable, broad-based stereo from the most resilient extreme areas,” Matas et al. 2002

 % Read the input images Ia = imread(fullfile(vl_root,'data','roofs1.jpg')) ; Ib = imread(fullfile(vl_root,'data','roofs2.jpg')) ; % Convert to grayscale Ia = uint8(rgb2gray(Ia)) ; Ib = uint8(rgb2gray(Ib)) ; % Find MSERs [ra,fa] = vl_mser(Ia, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ; [rb,fb] = vl_mser(Ib, 'MinDiversity',0.7,'MaxVariation',0.2,'Delta',10) ; % Match MSERs [matches, scores] = vl_ubcmatch(fa, fb); % Display the original input images figure(1); clf; imagesc(cat(2, Ia, Ib)); axis image off; colormap gray; % Display a second copy with the matches overlaid figure(2) ; clf ; imagesc(cat(2, Ia, Ib)); axis image off; colormap gray; xa = fa(1, matches(1,:)); ya = fa(2, matches(1,:)); xb = fb(1, matches(2,:)) + size(Ia,2); yb = fb(2, matches(2,:)); hold on ; h = line([xa ; xb], [ya ; yb]); set(h, 'linewidth', 1, 'color', 'y'); 

Answer 2

I don't know how, but MSER compatibility works in Matlab itself.

Code below

 file1 = 'roofs1.jpg'; file2 = 'roofs2.jpg'; I1 = imread(file1); I2 = imread(file2); I1 = rgb2gray(I1); I2 = rgb2gray(I2); % %Find the SURF features. % points1 = detectSURFFeatures(I1); % points2 = detectSURFFeatures(I2); points1 = detectMSERFeatures(I1); points2 = detectMSERFeatures(I2); %Extract the features. [f1, vpts1] = extractFeatures(I1, points1); [f2, vpts2] = extractFeatures(I2, points2); %Retrieve the locations of matched points. The SURF featurevectors are already normalized. indexPairs = matchFeatures(f1, f2, 'Prenormalized', true) ; matched_pts1 = vpts1(indexPairs(:, 1)); matched_pts2 = vpts2(indexPairs(:, 2)); figure; showMatchedFeatures(I1,I2,matched_pts1,matched_pts2,'montage'); legend('matched points 1','matched points 2'); 

gives the following image