Knowledge-based remote sensing image fusion method
-
摘要: 遥感图像融合技术能够将含有互补信息的多源图像进行融合,从而得到内容更丰富、光谱质量更高的图像,是遥感应用的关键和基础。针对遥感影像融合过程中容易出现的光谱失真和空间结构失真问题,该文以注意力机制为基础,利用归一化植被指数(normalized difference vegetation index,NDVI)和归一化水指数(normalized difference water index,NDWI)作为先验知识,构建了基于知识引导的遥感影像融合模型(remote sensing image FuseNet,RSFuseNet)。首先,基于高通滤波能够充分提取边缘纹理细节的优势,构建了高通滤波模块提取全色图像高频细节信息; 其次,提取多光谱图像的NDVI和NDWI信息; 再次,构建自适应挤压激励(squeeze-and-excitation,SE)模块,对输入特征进行重标定; 最后,将自适应SE模块与卷积单元结合,对输入特征进行融合处理。以高分六号遥感影像作为数据源,选择施密特正交化(gram-schmidt,GS)、主成分分析(principal component analysis,PCA)、深度网络结构的图像融合网络(a deep network architecture for pan-sharpening,PanNet)、以卷积神经网络为基础的融合网络(pansharpening by convolutional neural networks,PNN)模型作为对比模型开展实验,实验结果表明: 该文提出模型的峰值信噪比(peak signal to noise ratio,PSNR)指标(40.5)和结构相似性(structural similarity,SSIM)指标(0.98)均优于对比模型,表明这一方法在遥感影像图像融合方面具有明显的优势Abstract: The remote sensing image fusion technology can combine multi-source images containing complementary information to obtain images with richer content and higher spectral quality, thus it is the key and foundation of remote sensing applications. Aiming at the problems of spectral distortion and spatial structure distortion that are prone to occur in the process of remote sensing image fusion, the knowledge-based remote sensing image FuseNet (RSFuseNet) was constructed based on the attention mechanism and using normalized difference vegetation index (NDVI) and normalized difference water index (NDWI) as prior knowledge. Firstly, considering that the high-pass filtering can fully extract edge texture details, a high-pass filtering module was constructed to extract high-frequency details of panchromatic images. Secondly, NDVI and NDWI were extracted from multi-spectral images. Then, an adaptive squeeze-and-excitation (SE) module was constructed to recalibrate the input features. Finally, the adaptive SE module was combined with the convolution unit to perform fusion processing on the input features. The experiment was conducted using Gaofen 6 remote sensing image as the data source, and selecting Gram-Schmidt (GS) transformation, principal component analysis (PCA), a deep network architecture for pan-sharpening (PanNet), and pansharpening by convolutional neural networks (PNN) models as comparative models. The experimental results show that the peak signal to noise ratio (PSNR) index (40.5) and the structural similarity (SSIM) index (0.98) of the RSFuseNet model are better than those of comparative models, indicating that the method in this study has obvious advantages in remote sensing image fusion.
-
Key words:
- image fusion /
- prior knowledge /
- convolutional neural network /
- attention mechanism /
- GF-6
-
-
[1] [3] Xie B, Zhang H, Huang B. Revealing implicit assumptions of the component substitution pansharpening methods[J]. Remote Sensing, 2017, 9(5):443-458. [4] Yong Y, Wan W, Huang S, et al. A novel pan-sharpening framework based on matting model and multiscale transform[J]. Remote Sensing, 2017, 9(4):391. [5] 王晓艳, 刘勇, 蒋志勇. 一种基于结构相似度的IHS变换融合算法[J]. 遥感技术与应用, 2011, 26(5):670-676.[5] Wang X Y, Liu Y, Jiang Z Y. An IHS fusion method based on structural similarity[J]. Remote Sensing Technology and Application, 2011, 26(5):670-676.[6] Yang Y, Wan W, Huang S, et al. Remote sensing image fusion based on adaptive IHS and multiscale guided filter[J]. IEEE Access, 2017(4):4573-4582.[7] W Dong, S Xiao, X Xue, et al. An improved hyperspectral pansharpening algorithm based on optimized injection model[J]. IEEE Access, 2019(7):16718-16729.[8] Xu L, Zhang Y, Gao Y, et al. Using guided filtering to improve gram-schmidt based pansharpening method for GeoEye-1 satellite images[J]. Proceedings of the 4th International Conference on Information Systems and Computing Technology, 2016(64):33-37.[9] Aiazzi B, Alparone L, Baronti S, et al. MTF-tailored multiscale fusion of high-resolution MS and Pan imagery[J]. Photogrammetric Engineering & Remote Sensing, 2015, 72(5):591-596.[10] Vivone G, Restaino R, Mura M D, et al. Contrast and error-based fusion schemes for multispectral image pansharpening[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 11(5):930-934. [11] Alparone, Luciano, Baronti, et al. Spatial methods for multispectral pansharpening:Multiresolution analysis demystified[J]. IEEE Transactions on Geoscience & Remote Sensing, 2016(54):2563-2576.[12] Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Information Fusion, 2007, 8(2):143-156. [13] Lu X, Zhang J, Zhang Y. An improved non-subsampled contourlet transform-based hybrid pan-sharpening algorithm[C]// 2017 IEEE International Geoscience and Remote Sensing Symposium. Fort Worth: IEEE, 2017:3393-3396.[14] James R, Vadivel M. Improvement of spectral and spatial information using modified WAMM and modified bi-cubic interpolation method in non-subsampled contourlet transform domain[C]// 2015 International Conference on Robotics,Automation,Control and Embedded Systems (RACE). Chennai India: IEEE, 2015:1-5.[15] Zhang Z, Luo X, Wu X. A new pansharpening method using statistical model and shearlet transform[J]. IEEE Technical Review, 2014, 31(5):308-316.[16] Wang X, Bai S, Li Z, et al. The PAN and MS Image pansharpening algorithm based on adaptive neural network and sparse representation in the NSST domain[J]. IEEE Access, 2019(7):52508-52521.[17] Yin H T. Sparse representation based pan-sharpening with details injection model[J]. Signal Processing, 2015(113):218-227.[18] Giuseppe M, Davide C, Luisa V, et al. Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8(7):594. [19] Scarpa G, Vitale S, Cozzolino D. Target-adaptive CNN-based pansharpening[J]. IEEE Transactions on Geoscience & Remote Sensing, 2018, 56(9):5443-5457.[20] Yuan Q, Wei Y, Meng X, et al. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3),978-989. [21] Yang J, Fu X, Hu Y, et al. PanNet:A deep network architecture for pan-sharpening[C]// 2017 IEEE International Conference on Computer Vision(ICCV). Venice Italy: IEEE, 2017:5449-5457.[22] Shao Z, Cai J. Remote sensing image fusion with deep convolutional neural network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(5):1656-1669. [23] Azarang A, Manoochehri H E, Kehtarnavaz N. Convolutional autoencoder-based multispectral image fusion[J]. IEEE Access, 2019, 7:35673-35683. [24] Jie H, Li S, Gang S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023. [25] 聂文昌. 融合先验知识的深度学习目标识别与定位研究[D]. 哈尔滨: 哈尔滨工程大学, 2019.[25] Nie Wenchang. Research on target recognition and locationbased on prior knowledge using deep learning[D]. Harbin: Harbin Engineering University, 2019.[26] Pushparaj J, Hegde A V. Evaluation of pan-sharpening methods for spatial and spectral quality[J]. Applied Geomatics, 2016, 9(1):1-12. [27] 杨丽萍, 马孟, 谢巍, 等. 干旱区Landsat8全色与多光谱数据融合算法评价[J]. 国土资源遥感, 2019, 31(4):11-19.doi: 10.6046/gtzyyg.2019.04.02. [27] Yang L P, Ma M, Xie W, et al. Fusion algorithm evaluation of Landsat 8 panchromatic and multispectral images in arid regions[J]. Remote Sensing for Land and Resources, 2019, 31(4):11-19.doi: 10.6046/gtzyyg.2019.04.02. [28] Vivone G, Alparone L, Chanussot J, et al. A critical comparison among pansharpening algorithms[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2565-2586.
-
计量
- 文章访问数: 1504
- PDF下载数: 78
- 施引文献: 0
下载: