摘要
现有的图像美学质量评价方法普遍使用卷积神经网络(CNN)提取图像特征,但受局部感受野机制的限制,CNN较难提取图像的全局特征,导致全局构图关系、全局色彩搭配等美学属性缺失。为解决该问题,提出基于自监督视觉Transformer(SSViT)模型的图像美学质量评价方法。利用自注意力机制建立图像局部块之间的长距离依赖关系,自适应地学习图像不同局部块之间的相关性,提取图像的全局特征,从而刻画图像的美学属性;同时,设计图像降质分类、图像美学质量排序和图像语义重构这3项美学质量感知任务,利用无标注的图像数据对视觉Transformer(ViT)进行自监督预训练,增强全局特征的表达能力。在AVA(Aesthetic Visual Assessment)数据集上的实验结果显示,SSViT模型在美学质量分类准确率、皮尔森线性相关系数(PLCC)和斯皮尔曼等级相关系数(SRCC)指标上分别达到83.28%、0.7634和0.7462。以上实验结果表明,SSViT模型具有较高的图像美学质量评价准确性。
The existing image aesthetic quality evaluation methods widely use Convolution Neural Network(CNN)to extract image features.Limited by the local receptive field mechanism,it is difficult for CNN to extract global features from a given image,thereby resulting in the absence of aesthetic attributes like global composition relations,global color matching and so on.In order to solve this problem,an image aesthetic quality evaluation method based on SSViT(Self-Supervised Vision Transformer)model was proposed.Self-attention mechanism was utilized to establish long-distance dependencies among local patches of the image and to adaptively learn their correlations,and extracted the global features so as to characterize the aesthetic attributes.Meanwhile,three tasks of perceiving the aesthetic quality,namely classifying image degradation,ranking image aesthetic quality,and reconstructing image semantics,were designed to pre-train the vision Transformer in a self-supervised manner using unlabeled image data,so as to enhance the representation of global features.The experimental results on AVA(Aesthetic Visual Assessment)dataset show that the SSViT model achieves 83.28%,0.7634,0.7462 on the metrics including evaluation accuracy,Pearson Linear Correlation Coefficient(PLCC)and SRCC(Spearman Rank-order Correlation Coefficient),respectively.These experimental results demonstrate that the SSViT model achieves higher accuracy in image aesthetic quality evaluation.
作者
黄荣
宋俊杰
周树波
刘浩
HUANG Rong;SONG Junjie;ZHOU Shubo;LIU Hao(College of Information Science and Technology,Donghua University,Shanghai 201620,China;Engineering Research Center of Digitalized Textile&Fashion Technology,Ministry of Education(Donghua University),Shanghai 201620,China)
出处
《计算机应用》
CSCD
北大核心
2024年第4期1269-1276,共8页
journal of Computer Applications
基金
国家自然科学基金资助项目(62001099,61803372)
中央高校基本科研业务费专项资金资助项目(2232023D⁃30)。
