001-Learn Patterns In Supperpositional Signals by Data Mixing
输入“/”快速插入
001-Learn Patterns In Supperpositional Signals by Data Mixing
1.
Abstract
Automaitcally detecting patterns from speech signals, e.g., a keyword, is important in both theory and practical usage. A notorious problem is that when the signal is mixed with interfering speech, it would be very difficult for machines to detect the target speech patterns, although human can do that task much better, generally known as cock-party effect[1,2]. In this paper, we present a simple mix training approach, that utilizes the superpositional property of speech signals, and let the model try to discover true speech patterns from highly-mixed speech signals. By this extreme training, the model can detect target patterns even if the energy of the pattern is rather weak.
2.
Main
The core of deep learning is to learn high-level patterns from low-level observations, usually by hierarchical process. This is mostly obvious in image processing, where a typeical CNN learns local patterns in each layer at a particular scale, and the pooling mechanism raises the receptive field gradually, and at the top layer learns large-scale, compositional patterns.
A key assumption of the above pattern learning for images is that the patterns at any layer are `exclusive' in space, which means that at a particular position, there is only one pattern. This is because in most cases, objects occurlude each other and so at each location only one object can be seen. This means that at any level of the hierarchy, the composition of the patterns is simple.
Superpossitional signals, such as speech, is totally different. For these signals, in the most cases signals are added together, leading to numerous 'compsotiional' patterns. Most of these patterns are derived from superpositions and are not true patterns. Therefore, if the model is learned with clean signals, it would not work well due to the superposition, especially when the target patterns are weaker than interferering patterns. We therefore must learn to discover true patterns from mixed signals.
One possiblity is to design signal-specific kernels so that only true patterns can produce activation. For example, [3] designs kernes by considering the harmonic structure of speech signals. Although intuitively reasonable, it requires professional knowledge for the data so cannot be well generalized and the success is limited. Many data augmentation methods have been presented, where corruptions are randomly added to the target signal, and the model is instructed to learn the target pattern from the corrupted signal. However, corruption is generally assumed to be weaker than the target signal, unless the focused target is very limited. If the target is weaker than corruption, data aug tends to fail as corruption may contain target patterns as well.
We present a different approach that remains conventional neural net architectures, but training the model by artificially mixed data. A core design is that we let the model detect the existence of any real patterns, no matter how it is mixed with other patterns. This analog to a person in complex conditions who must practice his/her ear more sensitive to speech patterns.The new method is simple and easy to implement, and highly effective in learning any desirable patterns even if they are weak and burried under other signals, or even if two target patterns are co-existing and need to be detected simultaneously.
2.1 Mix training with labelled data
Mathematically, we train the model with a set of classifiers, each detecting a particular pattern. These classifiers are trained with binary cross entropy (BCE):
where j indexes data and i indexes event classes, and
is the output of the i-th classifier with j-th data sample. This can be written as a matrix form:
where Y is the vector of the target label of all the classifiers, and it is usually a one-hot vector that specifies the event of the sample.
In the mix training, we randomly choose two or more samples and mix them together with random weights, with the target label added together but without any weights.
where wj and wk are sampled from [
-1.0], where
is the minimum energy that a pattern is supposed to be recognized. This choice is consistent with the human perception system, i.e., people can perceive a signal only its intensity is beyond a threshold.
We highlight the importance of the random weighting on data and the constant weight of the label during the mixing. This is attributed to the nature of the superpositional signals, for which the patterns should be regarded to exist, no matter what their energy is (only if it is above a threshold).
2.2 AddProb
A limit of the mix training approach is that all the components of the mixture need to be labelled. This is not the case for many scenarios. Usually we may want to mix two or more signals without any label, and we hope it can still guide the training to learn the superposition property. We design an AddProb regularization to materialize the idea, by treating the output of the classifiers as the probability that the corresponding event exists:
The idea is that for any particular event, if any of the mixture components contains it, then it exists. Therefore, the probability that an event exists is
, and this should be the target of the mixture speech
. Note that the main objective of the training has ensured the output of the data is invariant to the weight.