Keras VGG extraction options

Question

I downloaded CNN's pre-trained VGG face and successfully executed it. I want to extract a hypercolumn mean from layers 3 and 8. I followed the section on extracting hypercolumns from here . However, since the get_output function did not work, I had to make a few changes:

Import

import matplotlib.pyplot as plt import theano from scipy import misc import scipy as sp from PIL import Image import PIL.ImageOps from keras.models import Sequential from keras.layers.core import Flatten, Dense, Dropout from keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D from keras.optimizers import SGD import numpy as np from keras import backend as K 

The main function:

 #after necessary processing of input to get im layers_extract = [3, 8] hc = extract_hypercolumn(model, layers_extract, im) ave = np.average(hc.transpose(1, 2, 0), axis=2) print(ave.shape) plt.imshow(ave) plt.show() 

Get function functions: (I followed this )

 def get_features(model, layer, X_batch): get_features = K.function([model.layers[0].input, K.learning_phase()], [model.layers[layer].output,]) features = get_features([X_batch,0]) return features 

Hypercolumn removal:

 def extract_hypercolumn(model, layer_indexes, instance): layers = [K.function([model.layers[0].input],[model.layers[li].output])([instance])[0] for li in layer_indexes] feature_maps = get_features(model,layers,instance) hypercolumns = [] for convmap in feature_maps: for fmap in convmap[0]: upscaled = sp.misc.imresize(fmap, size=(224, 224),mode="F", interp='bilinear') hypercolumns.append(upscaled) return np.asarray(hypercolumns) 

However, when I run the code, I get the following error:

 get_features = K.function([model.layers[0].input, K.learning_phase()], [model.layers[layer].output,]) TypeError: list indices must be integers, not list 

How can i fix this?

Note:

In the hypercolumn extraction function, when I use feature_maps = get_features(model,1,instance) or any integer instead of 1, it works fine. But I want to extract the average from layers 3 through 8.

Answer 1

I was very confused:

• After layers = [K.function([model.layers[0].input],[model.layers[li].output])([instance])[0] for li in layer_indexes] , the layers are a list of extracted functions .
• And then you send this list to feature_maps = get_features(model,layers,instance) .
• In def get_features(model, layer, X_batch): second parameter, namely layer , is used for indexing in model.layers[layer].output .

What would you like:

• feature_maps = get_features(model, layer_indexes ,instance) : passing level indices, not extracted functions.
• get_features = K.function([model.layers[0].input, K.learning_phase()], [ model.layers [l] .output for l in the layer ]) : the list cannot be used to index the list.

However, the function abstraction function is badly written. I suggest you rewrite everything, not mix codes.

Answer 2

I rewrote your function for a single-channel input image (W x H x 1). Maybe it will be helpful.

 def extract_hypercolumn(model, layer_indexes, instance): test_image = instance outputs = [layer.output for layer in model.layers] # all layer outputs comp_graph = [K.function([model.input]+ [K.learning_phase()], [output]) for output in outputs] # evaluation functions feature_maps = [] for layerIdx in layer_indexes: feature_maps.append(layer_outputs_list[layerIdx][0][0]) hypercolumns = [] for idx, convmap in enumerate(feature_maps): # vv = np.asarray(convmap) # print(vv.shape) vv = np.asarray(convmap) print('shape of feature map at layer ', layer_indexes[idx], ' is: ', vv.shape) for i in range(vv.shape[-1]): fmap = vv[:,:,i] upscaled = sp.misc.imresize(fmap, size=(img_width, img_height), mode="F", interp='bilinear') hypercolumns.append(upscaled) # hypc = np.asarray(hypercolumns) # print('shape of hypercolumns ', hypc.shape) return np.asarray(hypercolumns)