How are neural networks used when the number of inputs can be variable?

Question

All the examples I've seen on neural networks are for a fixed set of inputs that works well for images and fixed-length data. How do you deal with variable-length data with such sentences, queries, or source code? Is there a way to encode variable-length data into fixed-length inputs and still get generalized properties of neural networks?

Answer 1

Usually you extract functions from the data and send them to the network. It is not recommended to take only some data and submit it to the network. In practice, pre-processing and choosing the right functions will determine your success and neural network performance. Unfortunately, IMHO requires experience to develop a sense for this, and no one can learn from the book.

Summing up: "Garbage, trash"

Answer 2

I was there and I ran into this problem. ANN was compiled for a fixed length of the feature vector, as well as many other classifiers, such as KNN, SVM, Bayesian, etc. that is, the input level must be clearly defined and not changed, this is a design problem. However, some researchers prefer to add zeros to fill in the missing gap, I personally think that this is not a good solution, because these zeros (unrealistic values) will affect the weights with which the network will converge. in addition, there may be a real signal ending in zeros.

ANN is not the only classifier; there is more and even better, for example, a random forest. this classifier is considered the best among researchers, it uses a small number of random functions, creating hundreds of decision trees, using package loading, this can work well, the number of selected functions is usually the square of the size of the function vector. these functions are random. each decision tree converges to a solution using majority rules, the most likely class will be chosen then.

Another solution is to use DTW to dynamically change the time, or it is even better to use HMM Hidden Markov.

Another solution is interpolation, interpolation (compensation of missing values ​​along a small signal), while all small signals are the same size as the maximum signal, interpolation methods include and are not limited to averaging, B-spline, cubic .....

Another solution is to use the function extraction method to use the best functions (the most distinctive), this time their fixed size, including PCA, LDA, etc.

another solution is to use function selection (usually after extracting the function), an easy way to select the best functions that give the best accuracy.

what at the moment, if not for those who worked for you, contact me.

Answer 3

Some problems can be solved by a recurrent neural network. For example, it is useful for calculating parity from a sequence of inputs.

A recurrent neural network for parity would have only one input function. Over time, bits can flow into it. Its output also returns back to the hidden layer. This allows you to find parity with only two hidden units.

A normal two-level direct-feed neural network will require 2 ** hidden sequence_length blocks to represent parity. This restriction is preserved for any architecture with two levels (for example, SVM).

Answer 4

I suggest that one way to do this is to add a temporary component to the input (a recurrent neural network) and a stream stream to the network piece at a time (basically creating the equivalent of the lexer and parser neural network). would allow the input signal to be large enough, but would have the disadvantage that there would not necessarily be a stop symbol to separate different input sequences from each other (period equivalent in numbers)

Answer 5

To use the neural network in images of different sizes, the images themselves are often cropped and scaled up or down to fit better into the network entrance. I know that it doesn’t really answer your question, but maybe something like this would be possible with other types of input using some kind of input conversion function?

Answer 6

I'm not quite sure, but I would say I use the maximum number of entries (for example, for words, say, no words will be more than 45 characters (the longest word is found in the Wikipedia dictionary), and if a shorter word is found, set other inputs in the space character.

Or with binary data, set it to 0. The only problem with this approach is that an input filled with white space / zeros / any encounters a valid full-sized input (not so much a problem with words, but with a number).