摘要
针对现有基于深度学习的图像修复算法在处理大面积不规则缺损图像时出现局部结构不连通与模糊的问题,提出一种基于边缘和注意力跨层转移的二阶生成式图像修复模型。该模型由边缘修复网络和图像修补网络构成,边缘修复网络在自编码器的基础上结合扩张卷积对缺损图像的边缘二值图进行修复,并将边缘修复图作为先验条件与缺损图像一起输入到图像修补网络,在图像修补网络中,给出注意力跨层转移网络对各尺度编码特征由深到浅进行重构,并将重构特征图跳跃连接至解码层与对应潜在特征融合进行解码,提高各级解码层输出的上下文一致性,减少结构信息和语义特征丢失,最终得到修复图像。在Celeba、Facade、Places2这3个数据集上的实验结果表明,与当前主流算法相比,该方法平均L1损失降低了1.044%~3.801%,峰值信噪比和结构相似性分别提升了1.435~4.486 dB和1.789%~8.755%,不仅能够生成整体语义合理的内容,而且在局部结构连通性和纹理合成方面更符合人眼视觉感受。
To address the problems of local structural disconnection and blurring in existing deep learning-based image inpainting algorithms when processing large-area irregular defect images,a second-order generative image inpainting model based on edge and attention transfer across layers is proposed.The model consists of edge and image repair networks.The edge repair network is based on an autoencoder and is combined with dilated convolution to repair the edge binary image of the defect image.The edge repair image is then input into the image repair network together with the defect image as a prior condition.In the image repair network,a proposed Attention Transfer Network Across Layer(ATNAL)reconstructs the coding features of each scale from deep to shallow and connects the reconstructed feature map to the decoding layer and corresponding potential feature fusion for decoding.This improves the contextual consistency of the output of the decoding layer at all levels,reduces structural information,and eliminates high-level semantic features.A repaired image is finally obtained.Experimental results on the Celeba,Façade,and Places2 datasets show that the average L1 loss from this method is reduced by 1.044%-3.801%as compared with the current mainstream algorithm.In addition,the Peak Signal-to-Noise Ratio(PSNR)and Structural Similarity(SSIM)increase by 1.435-4.486 dB and 1.789%-8.755%,respectively,not only generates content with reasonable overall semantics,it is also consistent with human visual perception in terms of local structural connectivity and texture synthesis.
作者
樊瑶
石英男
柏劲咸
FAN Yao;SHI Yingnan;BAI Jinxian(School of Information Engineering,Xizang Minzu University,Xianyang 712000,Shaanxi,China)
出处
《计算机工程》
CAS
CSCD
北大核心
2023年第6期180-192,共13页
Computer Engineering
基金
国家自然科学基金(62062061)
西藏自治区自然科学基金(XZ202101ZR0084G)。
关键词
图像修复
边缘修复
扩张卷积
注意力跨层转移网络
跳跃连接
image inpainting
edge inpainting
dilation convolution
Attention Transfer Network Across Layer(ATNAL)
skip connection
