Abstract: In order to improve the recognition performance, a speaker recognition algorithm based on Gaussian valueadded matrix features and improved deep convolutional neural network is proposed. In the algorithm, the adaptive Gaussian mean matrix based on Mel frequency cepstrum coefficient (MFCC) features is first extracted by the maximum posterior probability, and the noise adaptive compensation for features is performed to enhance interframe correlation and speaker feature information. Then, an improved deep convolutional neural network is used to further align the interframe information to improve the feature learning for speaker recognition and the adaptability to the back-ground noise environment. The experimental results show that, compared with Gaussian mixture model-general background model (GMM-UBM) framework and traditional MFCC features, the algorithm proposed in this paper achieves the best recognition accuracy and the least recognition mean square error.