Laser & Optoelectronics Progress, Volume. 56, Issue 3, 031007(2019)
Speaker Identification Based on Multimodal Long Short-Term Memory with Depth-Gate
In order to effectively fuse the audio and visual features in the task of speaker recognition, a multimodal long short-term memory network (LSTM) with depth-gate is proposed. First, a multi-layer LSTM model is established for each type of individual features. Then the depth-gate is used to connect the memory cells in the upper and lower layers, and the connection between the upper and lower layers is enhanced, which improves the classification performance of the feature itself. At the same time, the connection among layer models can be learned by sharing the output of hidden layers and the weight of each gate unit among different models. The experimental results show that this method can be used to effectively fuse the audio and video features and improve the accuracy of speaker recognition. Moreover, this method is robust to external disturbance.
Get Citation
Copy Citation Text
Huangkang Chen, Ying Chen. Speaker Identification Based on Multimodal Long Short-Term Memory with Depth-Gate[J]. Laser & Optoelectronics Progress, 2019, 56(3): 031007
Category: Image Processing
Received: Jun. 13, 2018
Accepted: Aug. 31, 2018
Published Online: Jul. 31, 2019
The Author Email: Chen Huangkang (6161918009@vip.jiangnan.edu.cn), Chen Ying (chenying@jiangnan.edu.cn)