图像语义分割是高分辨率光学遥感图像解译研究领域的重要内容之一,在城市发展规划、国土资源调查、环境监测等众多领域具有广泛应用。遥感图像语义分割系统成功与否的关键问题是能否从数据中分析和提取出符合遥感成像规律的语义特征,传统的应用于语义分割的特征提取方法往往过于依赖研究人员对数据的理解和经验知识。随着遥感成像技术的发展,图像分辨率不断提高,遥感图像中地物目标的各种表观特征,如纹理、颜色等,随光照等各类成像条件的变化而呈现出更为复杂的统计规律,在此情况下,依靠研究人员的经验来设计符合这些统计规律的特征就更加困难。近年来,以卷积神经网络为代表的深度学习方法依靠其强大的特征学习和复杂模型构建能力,在计算机视觉的众多领域中取得了令人印象深刻的应用效果,这类方法能够通过模型训练的过程自动提取到符合数据统计规律的特征,一定程度上避免了传统特征设计方法的不足,因此逐步受到遥感图像解译领域研究人员的重视。 本文将以高分辨率光学遥感图像为数据源,综合利用深度学习等相关理论方法,针对图像中各类地物要素的分割问题,提出具有实际应用价值的像素级语义分割方法。为此,本文重点研究了基于深度卷积神经网络的多尺度特征融合网络模型、多特征金字塔融合网络模型、注意力机制深度网络模型及其优化计算等问题,并将相关理论和优化方法与高分辨率光学遥感图像语义分割框架相结合,构建和提出针对相应问题的模型与算法框架,为高分辨率光学遥感图像语义分割的研究和创新提供了全新的思路。 本文的主要贡献及创新点总结如下: 1.提出了一种基于多尺度特征融合网络的光学遥感图像语义分割方法,该方法充分利用了光学遥感图像的特点,提升现有图像语义分割网络针对遥感图像应用的精度。该方法采用多尺度图像作为输入数据,以解决传统金字塔型空洞卷积结构所提取的图像特征不能完全反映目标多尺度信息的问题。此外,为了解决传统语义分割方法利用双线性差值导致图像细节信息损失的问题,本文考虑引入多尺度编码.解码网络结构,通过多级多尺度特征图的上采样,能够更好地保留和利用图像中的细节信息,从而提高语义分割精度。最后为了提升模型的训练和预测效率,本文研究了如何在现有模型中引入MobileNets轻量级网络,以解决引入上述各种复杂结构导致的高计算复杂度问题。实验表明,该模型所生成的分割图像具有更为真实的细节信息,与其它特征提取方法的对比实验也显示该方法有效地降低了语义分割误差。 2.提出了一种基于多特征金字塔融合深度网络的光学遥感图像语义分割方法,以充分利用图像、数字高程数据等多元数据各自特点,提升语义分割精度。该方法采用一种多元数据特征图融合策略,将光学遥感图像和高程数据的各中间层特征分别对应融合,同时综合利用浅层特征的空间信息与深层特征的语义信息,优化分割效果。此外,针对一些中小型目标的精细化分割问题,本文设计引入金字塔场景解析网络中的金字塔池化结构,针对不同层次的特征,使用金字塔池化结构提取图像的多尺度特征,提高对不同尺度目标的分割精度。我们利用光学遥感图像和高程数据对上述模型进行了实验验证,提升了像素分类精度。 3.提出一种基于注意力机制的遥感图像语义分割方法,以抑制边缘模糊等复杂变化因素对遥感图像语义分割带来的影响。该方法通过在模型训练中对不同区域采取加权取值的方式模仿这种注意力机制,在模型训练中对较难区分的部分施加更大权值,提高图像语义分割的精度。针对遥感图像的不同区域的注意力加权问题,本文提出了通道注意力策略,该策略对不同特征图加权取值,且结合低层细节信息和高层语义信息抑制类别像素的定位误差,精细化分割目标边界,优化分割效果。实验结果表明本方法学习的特征更为鲁棒,可以有效地提高语义分割模型的泛化性能。 关键词:深度学习,语义分割,卷积神经网络,多尺度融合,注意力机制
Image semantic segmentation is one of the important contents in the research field of high-resolution optical remote sensing image interpretation. It is widely used in many fields such as urban development planning, land and resources survey, and environmental monitoring. The key issue of the success of remote sensing image semantic segmentation system is whether it can analyze and extract the semantic features that conform to the characterics of remote sensing imaging. The traditional feature extraction methods applied to semantic segmentation often rely too much on researchers' understanding and experience of data. With the development of remote sensing imaging technology, the image resolution is continuously improved. Various apparent features of the object in the remote sensing image, such as texture and color, are more complicated with various imaging conditions such as illumination. In this case, relying on the experience of researchers to design features that conform to these statistical characterics is more difficult. In recent years, the deep learning method, such as convolutional neural network, has achieved impressive application effects in many fields of computer vision, due to its powerful feature learning and complex model building capabilities. Such methods can automatically extract the features that conform to the statistical characterics of data, and to some extent avoids the deficiencies of the traditional feature design methods. Therefore, it gradually attracts the attention of researchers in the field of remote sensing image interpretation. In this thesis, the high-resolution optical remote sensing image is used as the data source, and the related theoretical methods such as deep learning are used comprehensively. A pixel-level semantic segmentation method with practical application value is proposed for the segmentation of various features in the image. To this end, this thesis focuses on the multi-scale feature fusion network model based on deep convolutional neural network, multi-feature pyramid fusion network model, attention mechanism deep network model and its optimization calculation, and related theory and optimization method. The semantic optical remote sensing image semantic segmentation framework is combined to construct a model for the corresponding problems, which provides a new idea for the research and innovation of high resolution optical remote sensing image semantic segmentation. The main contributions and innovations of this thesis are summarized as follows: 1.An optical remote sensing image semantic segmentation method based on multi-scale feature fusion network is proposed. This method makes full use of the characteristics of optical remote sensing image and improves the accuracy of existing image semantic segmentation network for remote sensing image application. The method adopts multi-scale image input method to solve the problem that the context information extracted by the spatial pyramid-type cavity convolution structure can not fully reflect the multi-scale target information. In addition, in order to solve the problem that the traditional semantic segmentation method uses the bilinear difference to cause the loss of image detail information, this thesis considers the introduction of multi-scale coding-decoding network structure, and multi-level multi-scale feature image upsampling while retaining image detail information. Improve semantic segmentation accuracy. Finally, in order to improve the training and prediction efficiency of the model, this thesis studies how to introduce the MobileNets lightweight network into the existing network to solve the high computational complexity caused by the introduction of the above complex structures. Experiments show that the segmentation image generated by the model has more realistic details, and the comparison with other feature extraction methods also shows that the method effectively reduces the semantic segmentation error. 2.An optical remote sensing image semantic segmentation method based on multi-feature pyramid fusion depth network is proposed to make full use of the characteristics of multi-data such as image and digital elevation data to improve the semantic segmentation precision. The method adopts a multi-data feature map fusion strategy to fuse the intermediate layer features of the optical remote sensing image and the elevation data respectively, and simultaneously utilizes the spatial information of the shallow features and the semantic information of the deep features to optimize the segmentation effect. In addition, for the fine segmentation problem of some small and medium-sized targets, this thesis introduces the pyramid pooling structure in the pyramid scene analysis network, and uses the pyramid pooling structure to extract multi-scale features of images for different levels of features, and improve the target of different scales. We use the optical remote sensing image and elevation data to verify the above model and improve the pixel classification accuracy. 3.A semantic segmentation method based on attention mechanism is proposed to suppress the influence of complex changes such as edge blur on the semantic segmentation of remote sensing images. The method imitates this attention mechanism by taking weighted values for different regions in the model training. In the model training, more weights are applied to the more difficult parts to improve the accuracy of image semantic segmentation. Aiming at the attention weighting problem of different regions of remote sensing image, this thesis proposes a channel attention strategy, which weights the different feature maps, and combines the low-level detail information and high-level semantic information to suppress the positioning error of the class pixels, and fine-divided the segmentation target. The experimental results show that the features learned by this method are more robust and can effectively improve the generalization performance of the semantic segmentation model. Key Words: deep learning, semantic segmentation, convolutional neural network, multi-scale fusion, attention mechanism