CNTK complains about dynamic axis in LSTM

Question

I am trying to implement LSTM in CNTK (using Python) to classify a sequence.

Input:

• Features of fixed length sequences of numbers (time series)

• Labels are single value vectors

Net:

input = input_variable(input_dim) label = input_variable(num_output_classes) h = Recurrence(LSTM(lstm_dim)) (input) final_output = C.sequence.last(h) z = Dense(num_output_classes) (final_output) loss = C.cross_entropy_with_softmax(z, label) 

Conclusion: The probability that the sequence matches the label

All sizes are fixed, so I do not think that I need any dynamic axis and did not indicate.

However, CNTK is unhappy, and I get:

 return cross_entropy_with_softmax(output_vector, target_vector, axis, name) RuntimeError: Currently if an operand of a elementwise operation has any dynamic axes, those must match the dynamic axes of the other operands 

If (according to some examples) I define a label with a dynamic axis

 label = input_variable(num_output_classes, dynamic_axes=[C.Axis.default_batch_axis()]) 

He no longer complains about this and moves on:

 tf = np.split(training_features,num_minibatches) tl = np.split(training_labels, num_minibatches) for i in range(num_minibatches*num_passes): # multiply by the features = np.ascontiguousarray(tf[i%num_minibatches]) labels = np.ascontiguousarray(tl[i%num_minibatches]) # Specify the mapping of input variables in the model to actual minibatch data to be trained with trainer.train_minibatch({input : features, label : labels}) 

But dying with this error:

  File "C:\Users\Dev\Anaconda3\envs\cntk-py34\lib\site-packages\cntk\cntk_py.py", line 1745, in train_minibatch return _cntk_py.Trainer_train_minibatch(self, *args) RuntimeError: Node '__v2libuid__Plus561__v2libname__Plus552' (Plus operation): DataFor: FrameRange dynamic axis is inconsistent with matrix: {numTimeSteps:1, numParallelSequences:100, sequences:[{seqId:0, s:0, begin:0, end:1}, {seqId:1, s:1, begin:0, end:1}, {seqId:2, s:2, begin:0, end:1}, {seqId:3, s:3, begin:0, end:1}, {seq... 

What do I need to do to fix this?

Answer 1

If I understand this correctly, you have sequences of one-dimensional inputs. If so, then your problems stem from this line

 input = input_variable(input_dim) 

which declares a sequence of input dimensional vectors. If you change it to

 input = input_variable(1) 

then I believe that your initial attempt should work.

Update . The above is not self-sufficient, because the operation of accepting the last element of the sequence produces output whose dynamic axes differ from the default dynamic axes with which the label is created. An easy fix is ​​to define the label after you have defined the output of z , like this

 label = input_variable(num_output_classes, dynamic_axes=z.dynamic_axes) 

This works without any complaints to me. Then I filed some dummy data like this (assuming the mini-channel 4 has a sequence length of class 5 and 3)

 x = np.arange(20.0, dtype=np.float32).reshape(4,5,1) y = np.array([1,0,0,0,1,0,0,0,1,0,0,1], dtype=np.float32).reshape(4,1,3) loss.eval({input: x, label:y }) 

and it worked as expected.