One Against All Regression

Question

I looked at an example from Andrew Ng's Machine Learning course, which I found at https://github.com/jcgillespie/Coursera-Machine-Learning/tree/master/ex3 . This example deals with logistic regression and one-all-all classification. I doubt this function:

function [all_theta] = oneVsAll(X, y, num_labels, lambda) %ONEVSALL trains multiple logistic regression classifiers and returns all %the classifiers in a matrix all_theta, where the i-th row of all_theta %corresponds to the classifier for label i % [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels % logisitc regression classifiers and returns each of these classifiers % in a matrix all_theta, where the i-th row of all_theta corresponds % to the classifier for label i % Some useful variables m = size(X, 1); n = size(X, 2); % You need to return the following variables correctly all_theta = zeros(num_labels, n + 1); % Add ones to the X data matrix X = [ones(m, 1) X]; % ====================== YOUR CODE HERE ====================== % Instructions: You should complete the following code to train num_labels % logistic regression classifiers with regularization % parameter lambda. % % Hint: theta(:) will return a column vector. % % Hint: You can use y == c to obtain a vector of 1 and 0 that tell use % whether the ground truth is true/false for this class. % % Note: For this assignment, we recommend using fmincg to optimize the cost % function. It is okay to use a for-loop (for c = 1:num_labels) to % loop over the different classes. % % fmincg works similarly to fminunc, but is more efficient when we % are dealing with large number of parameters. % % Example Code for fmincg: % % % Set Initial theta % initial_theta = zeros(n + 1, 1); % % % Set options for fminunc % options = optimset('GradObj', 'on', 'MaxIter', 50); % % % Run fmincg to obtain the optimal theta % % This function will return theta and the cost % [theta] = ... % fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ... % initial_theta, options); % initial_theta = zeros(n + 1, 1); options = optimset('GradObj', 'on', 'MaxIter', 50); for i = 1:num_labels c = i * ones(size(y)); fprintf('valores') [theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options); all_theta(i,:) = theta; end % ========================================================================= end 

I know that lrCostFunction takes as parameters: theta, X, y and lambda, but I can’t understand where the value of t comes from in the code I published above; especially in this part:

 [theta] = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options); 

any help?

Answer 1

fmincg takes the descriptor of the target function as the first argument, which in this case is the lrCostFunction descriptor.

If you go inside fmincg.m you will find the following lines:

 argstr = ['feval(f, X']; % compose string used to call function %---Code will not enter the following loop---% for i = 1:(nargin - 3) %this will go from 1 to 0, thus the loop is skipped argstr = [argstr, ',P', int2str(i)]; end % following will be executed argstr = [argstr, ')']; 

At the end of the above code snippet, the result will be,

 argstr=feval(f,X'); 

If you are a little ahead, you will see

 [f1 df1] = eval(argstr); % get function value and gradient 

Therefore, the function descriptor f will be launched with the argument X' . Therefore, t=X' , which also makes sense. The initial theta will converge to give you the final parameter vector for logistic regression.

Answer 2

You can really replace.

 for i=1 : num_labels [theta]= fmincg (@(t)(lrCostFunction(t, X, (y == i), lambda)),initial_theta, options); all_theta(i,:)=theta; 

Answer 3

try it

 for i = 1:num_labels, [all_theta(i,:)] = fmincg (@(t)(lrCostFunction(t, X, (y == i), lambda)), initial_theta, options); end; 

you also don't need to initialize all_theta at the beginning