摘要
目标说话人提取任务的目标是在一段混合音频中提取特定说话人的语音,任务设置上一般会给一段目标说话人注册音频作为辅助信息。现有的研究工作主要有以下不足:(1)说话人识别的辅助网络无法捕获学习注册音频中的关键信息;(2)缺乏混合音频嵌入和注册音频嵌入的交互学习机制。以上不足导致了现有研究工作在注册音频和目标音频之间存在较大差异时有说话人混淆问题。为了解决该问题,提出说话人感知的交叉注意力说话人提取网络(Speaker-aware Cross Attention Speaker Extraction Network,SACAN)。SACAN在说话人识别辅助网络引入基于注意力的说话人聚合模块,有效聚合目标说话人声音特性的关键信息和利用混合音频增强目标说话人嵌入。进一步地,SACAN通过交叉注意力构建交互学习机制促进说话人嵌入与混合音频嵌入融合学习,增强了模型的说话人感知能力。实验结果表明,SACAN相比基准方法在STOI和SI-SDRi分别提高了0.013 3、1.069 5 d B,并在说话人混淆相关评估和消融实验中验证了不同模块的有效性。
Target speaker extraction aims to extract the speech of the specific speaker from mixed audio,which usually treats the enrolled audio of the target speaker as auxiliary information.Existing approaches mainly have the following limitations:the auxiliary network for speaker recognition cannot capture the critical information from enrolled audio,and the second one is the lack of an interactive learning mechanism between mixed and enrolled audio embedding.These limitations lead to speaker confusion when the difference between the enrolled and target audio is significant.To address this,a speaker-aware cross-attention speaker extraction network(SACAN) is proposed.First,SACAN introduces an attention-based speaker aggregation module in the speaker recognition auxiliary network,which effectively aggregates critical information about target speaker characteristics.Then,it uses mixed audio to enhance target speaker embedding.After that,to promote the integration of speaker embedding and mixed audio embedding,SACAN builds an interactive learning mechanism through cross-attention and enhances the speaker perception ability of the model.The experimental results show that SACAN improves by 0.013 3and 1.069 5 in terms of STOI and SI-SDRi when compared with the benchmark model,validating the effectiveness of the proposed module in speaker confusion assessment and ablation experiments.
作者
李卓璋
许柏炎
蔡瑞初
郝志峰
Li Zhuo-zhang;Xu Bo-yan;Cai Rui-chu;Hao Zhi-feng(School of Computer Science and Technology,Guangdong University of Technology,Guangzhou 510006,China;College of Science,Shantou University,Shantou 515063,China)
出处
《广东工业大学学报》
CAS
2024年第3期91-101,共11页
Journal of Guangdong University of Technology
基金
科技创新2030-“新一代人工智能”重大项目(2021ZD0111501)
国家优秀青年科学基金资助项目(62122022)
国家自然科学基金资助项目(61876043,61976052,62206064)。
关键词
语音分离
目标说话人提取
说话人嵌入
交叉注意力
多任务学习
speech separation
target speaker extraction
speaker embedding
cross attention
multi-task learning
