Custom Jacobian template in MATLAB lsqnonlin ignored

Question

I am using the MATLAB function lsqnonlin and I am trying to install a custom Jacboian template using the JacobPattern option. I prefer to use the trust-region-reflective algorithm, and output from lsqnonlin indicates that it really was the algorithm used by the solver (required to use the JacobPattern option).

The problem I find is that if my JacobPattern too sparse (for example, a few lines from the Jacobian 500x500), it is ignored by the solver, and the full Jacobian is calculated instead.

This behavior is not documented; can anyone shed some light on this? I would like to get the solver to use my JacobPattern no matter how absurdly thin it is or how fine the gradient is found with it.

---

Update:

I did a few more experiments, and it seems that the Jacobian is revised only if the Jacobian template has all zero lines. Any number of zero-zero columns is fine if there is at least one '1' in each row. Although this helps to avoid the problem, the question remains: why does the solver require that each dependent variable have a corresponding gradient? In any case, I would expect that ignoring the user parameter would be at least worthy of warning ...

Answer 1

My guess is this:

If you look at what the Jacobian actually means, you will see that all zero strings mean that the corresponding function (part of a specific vector function) is independent of any variable. Thus, it is completely pointless to add it to the optimization.

As for the targeted transfer of the wrong Jacobian to the algorithm, why do you want to do this?