Using MFCC's for voice recognition

Question

I'm currently using the Fourier transformation in conjunction with Keras for voice recogition (speaker identification). I have heard MFCC is a better option for voice recognition, but I am not sure how to use it.

I am using librosa in python (3) to extract 20 MFCC features. My question is: which MFCC features should I use for speaker identification?

In addition to this I am unsure on how to implement these features. What I would do is to get the necessary features and make one long vector input for a neural network. However, it is also possible to display colors, so could image recognition also be possible, or is this more aimed at speech, and not speaker recognition?

In short, I am unsure where I should start, as I am not very experienced with image recognition and have no idea where to start.

Thanks in advance!!

Answer 1

My question is: which MFCC features should I use for speaker identification?

I shall say that use all of them. Technically MFCC features are output from different filter banks. It is hard to say a priori which of them will be useful.

In addition to this I am unsure on how to implement these features. What I would do is to get the necessary features and make one long vector input for a neural network.

Actually when you extract MFCC for N samples you get an array like N x T x 20 T represents the number of frames in the audio signal after processed for MFCC. I will suggest using Sequence classification with LSTM. This will give better result.

Answer 2

In addition to this I am unsure on how to implement these features. What I would do is to get the necessary features and make one long vector input for a neural network.

For each sample, you must have got a 2D matrix of MFCC like N x T X no_mfccs (in your case no_mfccs=20); to make it into one single vector, various researchers take statistics such as mean, var, IQR, etc. to reduce the feature dimension. Some also model it using multivariate regression, and some fit it to a Gaussian mixture model. It depends on the next stage. In your case, you can use statistics to convert into a single vector

OR As told by Parthosarathi, you can use LSTM to preserve sequential information across time frames.

However, it is also possible to display colors, so could image recognition also be possible, or is this more aimed at speech, and not speaker recognition?

I will not recommend you to use spectrogram (image) as a feature vector to neural network because visual images and spectrograms do not accumulate visual objects and sound events information in the same manner.

when you feed image to neural network it assumes that features (pixel values) of an image carry the same meaning regardless of their location. But in case of the spectrogram, location of feature matters a lot.

e.g. Moving the frequencies of a male voice upwards could change its meaning from man to child. Therefore, the spatial invariance that 2D CNN provides might not perform as well for this form of data. To learn more about it refer: What’s wrong with CNNs and spectrograms for audio processing?

Answer 3

You can use MFCCs with dense layers / multilayer perceptron, but probably a Convolutional Neural Network on the mel-spectrogram will perform better, assuming that you have enough training data.