Convolutional Neural Networks and 3D Images

Question

I am interested in applying CNN to 3D images (i.e. medical data). Does TensorFlow already include this functionality?

Answer 1

TensorFlow now supports 3D convolution and 3D merging in the main branch.

You can use them with 5D tensors as input with the form: [batch_size, depth, height, width, channels] .

Answer 2

No, current implementations are made for 2D images (features like nn.conv2d). They support multiple channels (for example, RGB), and you can express 3D images as a multi-channel 2D image (each z-slice is a channel), but this is not always ideal. In addition, to use such approaches, you need a significant amount of image data, which is usually difficult to find in the medical field.

Update: both TensorFlow and Theano (subsequently Keras, Lasagne, etc.) now support 3D operations, as described above. It is important to note that 3D operations are much more computational and memory intensive than a similar 2D operation.

Answer 3

The implementation of TensorFlow for 3D convolutional neural networks was provided by the following open source projects:

Lip reading - cross-visual audio-visual recognition using three-dimensional convolutional neural networks

Using 3D convolutional neural networks to test speakers

Answer 4

If you want to use CNN with 3D images, then an alternative is to use Caffe PR . You will need to convert your data to HDF5 format.