How to build ROC curve using Tensorflow and scikit-learn?

Question

I am trying to build a ROC curve from a modified version of the CIFAR-10 example represented by a tensor flow. This is now for 2 classes instead of 10.

The network output is called logits and takes the form:

[[- 2.57313061 2.57966399] [0.04221377 -0.04033273] [-1.42880082] 1.43337202] [-2.7692945 2.78173304] [-2.48195744 2.49331546] [2.0941515 -2.10268974] [-3.51670194 3.53267646] [2.757766]

First, what are these logits? The last layer on the net is the "softmax linear" form of WX + b.

The model is able to calculate accuracy by calling

top_k_op = tf.nn.in_top_k(logits, labels, 1) 

Then, as soon as the chart is initialized:

 predictions = sess.run([top_k_op]) predictions_int = np.array(predictions).astype(int) true_count += np.sum(predictions) ... precision = true_count / total_sample_count 

This works great.

But now, how can I build a ROC curve from this?

I'm trying to use the sklearn.metrics.roc_curve () function ( http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html#sklearn.metrics.roc_curve ), but I don't know what for use as parameter "y_score".

Any help would be appreciated!

Answer 1

'y_score' here should be an array corresponding to the probability of each sample to be classified as positive (if positive was marked as 1 in your y_true array)

Actually, if your network uses Softmax as the last layer, then the model should output the probability of each category for this instance. But the data given here does not match this format. And I checked the example code: https://github.com/tensorflow/tensorflow/blob/r0.10/tensorflow/models/image/cifar10/cifar10.py seems to use the softmax_linear layer, I know little about this example, but, I think you should process the result using the Logistic Function to turn it into a probability.

Then just pass it along with your true label "y_true" to the scikit-learn function:

 y_score = np.array(output)[:,1] roc_curve(y_true, y_score) 

Answer 2

 import tensorflow as tf tp = [] # the true positive rate list fp = [] # the false positive rate list total = len(fp) writer = tf.train.SummaryWriter("/tmp/tensorboard_roc") for idx in range(total): summt = tf.Summary() summt.value.add(tag="roc", simple_value = tp[idx]) writer.add_summary (summt, tp[idx] * 100) #act as global_step writer.flush () 

then run the tensogram:

 tensorboard --logdir=/tmp/tensorboard_roc 

tensorboard_roc

for more information and code, you can visit my blog: http://blog.csdn.net/mao_feng/article/details/54731098