How do you know which word is most represented in a paragraph? (Matlab)

Question

I have a huge paragraph and you want to know which word is most in it. Can someone point me in the right direction? Any examples and clarifications would be helpful. Thanks!

Answer 1

Here is a simple solution, it should be pretty fast.

example_paragraph = 'This is an example corpus. Is is a verb?'; words = regexp(example_paragraph, ' ', 'split'); vocabulary = unique(words); n = length(vocabulary); counts = zeros(n, 1); for i=1:n counts(i) = sum(strcmpi(words, vocabulary{i})); end [frequency_of_the_most_frequent_word, idx] = max(counts); most_frequent_word = vocabulary{idx}; 

You can also check the answers here to get the most common word from an array of words.

Answer 2

Here is a very MATLAB-y way to do this. I tried to clearly define the variables. Play with each line and examine the results to understand how it works. Workhorse Functions: unique and hist

 % First produce a cell array of words to be analyzed paragraph_cleaned_up_whitespace = regexprep(paragraph, '\s', ' '); paragraph_cleaned_up = regexprep(paragraph_cleaned_up_whitespace, '[^a-zA-Z0-9 ]', ''); words = regexpi(paragraph_cleaned_up, '\s+', 'split'); [unique_words, i, j] = unique(words); frequency_count = hist(j, 1:max(j)); [~, sorted_locations] = sort(frequency_count); sorted_locations = fliplr(sorted_locations); words_sorted_by_frequency = unique_words(sorted_locations).'; frequency_of_those_words = frequency_count(sorted_locations).';