How to add and remove new layers in keras after loading weights?

Question

I am trying to do a learning transfer; for this I want to remove the last two layers of the neural network and add two more layers. This is sample code that also produces the same error.

from keras.models import Sequential from keras.layers import Input,Flatten from keras.layers.convolutional import Convolution2D, MaxPooling2D from keras.layers.core import Dropout, Activation from keras.layers.pooling import GlobalAveragePooling2D from keras.models import Model in_img = Input(shape=(3, 32, 32)) x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img) x = Activation('relu', name='relu_conv1')(x) x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x) x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x) x = Activation('relu', name='relu_conv2')(x) x = GlobalAveragePooling2D()(x) o = Activation('softmax', name='loss')(x) model = Model(input=in_img, output=[o]) model.compile(loss="categorical_crossentropy", optimizer="adam") #model.load_weights('model_weights.h5', by_name=True) model.summary() model.layers.pop() model.layers.pop() model.summary() model.add(MaxPooling2D()) model.add(Activation('sigmoid', name='loss')) 

I removed the layer using pop() , but when I tried to add its output to this error

AttributeError: the 'Model' object does not have the 'add' attribute

I know that the most likely cause of the error is the misuse of model.add() . what other syntax should i use?

EDIT:

I tried to remove / add layers to keras, but could not add it after loading external weights.

 from keras.models import Sequential from keras.layers import Input,Flatten from keras.layers.convolutional import Convolution2D, MaxPooling2D from keras.layers.core import Dropout, Activation from keras.layers.pooling import GlobalAveragePooling2D from keras.models import Model in_img = Input(shape=(3, 32, 32)) def gen_model(): in_img = Input(shape=(3, 32, 32)) x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img) x = Activation('relu', name='relu_conv1')(x) x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x) x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x) x = Activation('relu', name='relu_conv2')(x) x = GlobalAveragePooling2D()(x) o = Activation('softmax', name='loss')(x) model = Model(input=in_img, output=[o]) return model #parent model model=gen_model() model.compile(loss="categorical_crossentropy", optimizer="adam") model.summary() #saving model weights model.save('model_weights.h5') #loading weights to second model model2=gen_model() model2.compile(loss="categorical_crossentropy", optimizer="adam") model2.load_weights('model_weights.h5', by_name=True) model2.layers.pop() model2.layers.pop() model2.summary() #editing layers in the second model and saving as third model x = MaxPooling2D()(model2.layers[-1].output) o = Activation('sigmoid', name='loss')(x) model3 = Model(input=in_img, output=[o]) 

shows this error

 RuntimeError: Graph disconnected: cannot obtain value for tensor input_4 at layer "input_4". The following previous layers were accessed without issue: [] 

Answer 1

You can take the output latest model and create a new model. The lower layers remain the same.

 model.summary() model.layers.pop() model.layers.pop() model.summary() x = MaxPooling2D()(model.layers[-1].output) o = Activation('sigmoid', name='loss')(x) model2 = Model(input=in_img, output=[o]) model2.summary() 

Check How to use models from keras.applications to transfer learn?

Update while editing:

The new error is due to the fact that you are trying to create a new model on the global in_img , which is not actually used in the previous model creation .. there you actually define the local in_img . So the global in_img is obviously not connected to the top layers in the symbolic graph. And this has nothing to do with loading weights.

To better solve this problem, you should use model.input to reference input.

model3 = Model(input=model2.input, output=[o])