摘要
多模态神经机器翻译是指直接采用神经网络,以端到端方式融合图像和文本两种模态信息,以此进行翻译建模的机器学习方法。传统多模态机器翻译,是在将源语言翻译成目标语言时,借助图像中的重要特征信息优化翻译过程。但是观察发现,图像里的信息不一定出现在文本中,对翻译也会带来干扰;与参考译文对比,翻译结果中出现了过翻译和欠翻译的情况。针对以上问题,该文提出一种融合覆盖机制双注意力解码方法,用于优化现有多模态神经机器翻译模型。该模型借助覆盖机制分别作用于源语言和源图像,在注意力计算过程中,可以减少对过去重复信息的关注。在WMT16、WMT17测试集上进行实验,验证了上述方法的有效性,在WMT16英德和英法以及WMT17英德和英法测试集上,对比基准系统BLEU值分别提升了1.2,0.8,0.7和0.6个百分点。
Multimodal neural machine translation refers in this paper to a machine learning method that directly uses neural networks to translate image and text modal information in an end-to-end system. This paper proposes a multimodal machine translation model based on dual attention decoding with coverage mechanism. This model works on the source language and the image respectively by means of the coverage mechanism, which can reduce the attention to past repeated information. This paper verifies the effectiveness of the proposed method over the official evaluation datasets of WMT16 and WMT17. Experimental results show that the method increases the performance of multimodal neural machine translation with 1.2%, 0.8%, 0.7% and 0.6% on the four benchmark datasets of WMT16 En-De/En-Fr and WMT17 En-De/En-Fr, respectively.
作者
李志峰
张家硕
洪宇
尉桢楷
姚建民
LI Zhifeng;ZHANG Jiashuo;HONG Yu;YU Zhenkai;YAO Jianmin(School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China)
出处
《中文信息学报》
CSCD
北大核心
2020年第3期44-55,共12页
Journal of Chinese Information Processing
基金
国家自然科学基金(61672367,61672368,61703293)。
关键词
多模态神经机器翻译
覆盖机制
过翻译及欠翻译
multimodal neural machine translation
coverage mechanism
over-translation and under-translation
