摘要
针对红外与可见光图像的融合,提出了一种基于残差网络和注意力机制的图像融合算法。首先,为了分别从背景层和细节层提取红外与可见光图像的特征信息,使用引导滤波将红外和可见光图像分解为含有低频背景轮廓信息的背景层和含有高频细节纹理信息的细节层,再使用编码网络分别从背景层和细节层中提取特征信息并进行融合,该特征提取方法可提获得更为全面的图像特征信息。其次,为了有效融合红外与可见光的图像特征,利用添加注意力特征融合模块的解码网络对图像进行恢复重建,获得融合图像,该方法从空间注意力和通道注意力两个方向提升网络保留重要特征信息的能力。然后,为了根据图像自身的信息自适应的训练网络,设计了一种基于梯度信息的自适应权重计算方法,以此来调节红外与可见光图像对融合图像造成影响的大小。通过源图像的梯度信息来度量红外与可见光图像的信息量,从而自适应地计算权重。实验结果表明,提出的图像融合算法表现优异,取得了很好的融合效果,与12个算法相比,所提算法在4个客观评价指标上均超过对比算法。基于残差网络和注意力机制的图像融合算法在军事、航空、资源勘探、安防监控等众多领域具有应用价值。
An image fusion algorithm based on residual network and attention mechanism is proposed for infrared and visible image fusion. Firstly, feature information of infrared and visible images is extracted from the background layer and detail layer respectively. In this paper, guided filtering is used to decompose infrared and visible images into the background layer containing low-frequency background contour information and the detail layer containing high-frequency detail texture information, and then the encoder network is used to extract feature information from the background layer and detail layer respectively and fuse them. This feature extraction method can extract more comprehensive image feature information. Secondly, in order to effectively fuse infrared and visible image features. In this paper, the fusion image is reconstructed by using the decoder network which adds the attention feature fusion module. This method improves the ability of networks to retain important feature information from two directions of spatial attention and channel attention. Then, in order to adapt the training network according to the information of the image itself, an adaptive weight calculation method based on gradient information is designed to adjust the influence of infrared and visible images on the fused image in this paper. The gradient information of the source image is used to measure the information of infrared and visible images so as to calculate the weight adaptively. Experimental results show that the proposed image fusion algorithm performs well and achieves good fusion effect. Compared with 12 algorithms, the proposed algorithm outperforms the comparison algorithm in 4 evaluation indexes. The image fusion algorithm based on residual network and attention mechanism has wide applications in many fields such as military, aviation, resource exploration, and security monitoring.
作者
李国梁
向文豪
张顺利
张博勋
LI Guoliang;XIANG Wenhao;ZHANG Shunli;ZHANG Boxun(School of Software Engineering,Beijing Jiaotong University,Beijing 100044,China;Systems Engineering Research Institute,CSSC,Beijing 100036,China)
出处
《无人系统技术》
2022年第2期9-21,共13页
Unmanned Systems Technology
基金
国家自然科学基金(61976017)
北京市自然科学基金(4202056)。
关键词
红外图像
可见光图像
图像融合
残差网络
注意力机制
引导滤波
编码网络
Infrared Image
Visible Image
Image Fusion
Residual Network
Attention Mechanism
Guided Filtering
Encoder Network
