It completely depends on the case. In the example you gave a good solution, you could use zeros , because the edges are weighted, and for many purposes, the edge of weight 0 is equivalent to no edge. This is true if you do things like thread / cut algorithms.
Usually, choosing between NaN and Inf, I would go with NaN . Inf has some properties that you might not like as an “invalid” marker:
Inf*(-1) = -Inf
Inf+(-Inf) = NaN
Inf > 10 = True
etc...
A pure solution might be to support another logical matrix that has True, where the connection is valid, and False otherwise. It takes up a little memory, but if your matrix is not huge, I think that the readability of the code that it gives you is worth it.