Getting heart rate from a simple video: code inside

Question

I'm trying to find the heart rate from a video of my skin. To do this, I take the cropped rectangle of pixels from my video clip and averaging the red (or green) component in all these pixels (then, of course, seeing how this average change changes from frame to frame).

I take the fast Fourier transform of the vector (the average color value for each section of the cropped frame) to see which frequencies are most noticeable. I hope to see a person’s resting heart rate, ~ 1 Hz, as very noticeable.

As a test, I took a video only about the wall or other objects that should not change periodically. I used a tripod and three different cameras of different brands. Each of them has the same, if not identical, peak background frequencies, in particular, at frequencies of 1 Hz, 2 Hz, 5 Hz, and 10 Hz. I shot under natural light and fluorescent, and this is still happening.

My ultimate goal is to distinguish, with this analysis, living skin from unvascularized skin. Therefore, the understanding of why I get these peak frequencies for inanimate objects is VITAL.

Can any of you run this code on your own videos and explain if I'm just an idiot?

Camera shot:

Kodak playsport

1920x1080 30 frames per second (60i) imports as mp4

Canon Vixia HF200 1440x1080 30 frames per second (60i) 12 Mbps transfer speed imports as .mts, which I will encode to mp4

The code is based on:

http://www.ignaciomellado.es/blog/Measuring-heart-rate-with-a-smartphone-camera#video

clear all close all clc %% pick data file name to be analyzed, set directory it is found in dataDir = './data'; vidname = ['Filename.MP4']; %% define path to file and pull out video inFile = fullfile(dataDir,vidname); video = VideoReader(inFile); %% make 1D array with length equal to number of frames (time) brightness = zeros(1, video.NumberOfFrames); video_framerate = round( video.FrameRate); % note some places in the code must use integer value for framerate, others we directly use the unrounded frame rate %% set region of interest for what you want to get average brightness of frame = read(video, 1); imshow(frame) rect = getrect; close all xmin_pt = round(rect(1)); ymin_pt = round(rect(2)); section_width = round(rect(3)); section_height = round(rect(4)); %% select component of video (red green or blue) component_selection = 1; % pick red , green, or blue %% make 1D array of ROI averages for i = 1:video.NumberOfFrames, frame = read(video, i); section_of_interest = frame(ymin_pt:ymin_pt+section_height,xmin_pt:xmin_pt+section_width,:); colorPlane = section_of_interest(:, :, component_selection); brightness(i) = sum(sum(colorPlane)) / (size(frame, 1) * size(frame, 2)); end %% Filter out non-physiological frequencies BPM_L = 40; % Heart rate lower limit [bpm] BPM_H = 600; % Heart rate higher limit [bpm] This is currently set high to investigate the signal % Butterworth frequencies must be in [0, 1], where 1 corresponds to half the sampling rate [b, a] = butter(2, [((BPM_L / 60) / video_framerate * 2), ((BPM_H / 60) / video_framerate * 2)]); filtBrightness = filter(b, a, brightness); %% Trim the video to exlude the time where the camera is stabilizing FILTER_STABILIZATION_TIME = 3; % [seconds] filtBrightness = filtBrightness((video_framerate * FILTER_STABILIZATION_TIME + 1):size(filtBrightness, 2)); %% Do FFT on filtered/trimmed signal fftMagnitude = abs(fft(filtBrightness)); %% Plot results figure(1) subplot(3,1,1) plot([1:length(brightness)]/video.FrameRate,brightness) xlabel('Time (seconds)') ylabel('Color intensity') title('original signal') subplot(3,1,2) plot([1:length(filtBrightness)]/video.FrameRate,filtBrightness) xlabel('Time (seconds)') ylabel('Color intensity') title('after butterworth filter and trim') freq_dimension = ((1:round(length(filtBrightness)))-1)*(video_framerate/length(filtBrightness)); subplot(3,1,3) plot(freq_dimension,fftMagnitude) axis([0,15,-inf,inf]) xlabel('Frequency (Hz)') ylabel('|Y(f)|') title('Fft of filtered signal') 

Answer 1

Your goal is to "find the heart rate from the video of my skin" and "... to distinguish, using this analysis, living skin from unvascularized skin", but your approach is "take a video ... crop" ... medium red (or green ) component .. see how it changes. "

I think there is a problem with the package. Mean means the prevailing majority signal. Human vision is a (amazing) visual data processor, and it usually takes many decades of hard work to get the machine to come even in the small order of what they do. If you cannot look with your own eyes at 90% of the people around you and measure their heart rate, then the averaging method may not be the same as walking.

There is an infinite amount of statistics that you can view to see how the distribution changes over time, and only one of them means. If you look only at what you may lack information.

How would I do this:

• I would work with differences between frames.
• Most video codecs have an uneven time interval. I would get these values ​​to track changes by actual (not estimated) time. Your current (allegedly false) signals may come from CODEC as well as hardware.
• I would make the film in such a way that every new frame is the difference between it and the previous frame. I would look at it 20 times and see if my (amazing) human visualization system can see anything that is clearly related to heart rate or obviously NOT.
• For "NOT" remove them from processing, by mask, filter or the like. For relatives, find a way to make them an area of ​​interest by removing everything else, a histogram of pixel truncation intensity, or other image enhancements (blur, sharpness, repeated color, edge, rotation, so the largest signal along the updated pixel axes is then processed.)

AFTER the human vision apparatus can receive a good signal, I would work to get the math / computer signal with clear knowledge that the human apparatus is superior.

If I were tracking points / functions that moved over time, I would look at sub-pixel metrology methods. I did some of this decades ago ( thesis ), and although I think the content may be appropriate, it may also be different to merit the first reading before copying and running the code.

Good luck.