A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images
-
摘要: 车辆检测问题是计算机视觉和摄影测量与遥感领域的研究热点。随着深度学习技术的发展,遥感影像车辆检测已在智慧城市和智能交通等领域展开应用。文章系统归纳了现有的基于深度学习模型的遥感影像车辆检测算法,着重从单阶段与双阶段的车辆检测算法进行了归类、分析及比较; 重点梳理了大幅面、复杂背景环境下车辆检测的关键技术,分析主流深度学习模型应用于遥感影像车辆检测的优缺点。利用DOTA和DIOR数据集对YOLOv5,Faster-RCNN,FCOS和SSD算法进行评估,在DOTA数据集上,车辆检测精度分别为0.695,0.410,0.370和0.251; 在DIOR数据集上,车辆检测精度分别为0.566,0.243,0.231和0.154。实验结果表明,目标尺度较小仍是制约遥感影像车辆检测性能的主要因素,深度学习模型应用于小目标检测存在较大的提升空间。最后,基于公开数据集与已有研究算法分析的基础上,给出大幅面复杂背景下遥感影像车辆检测的解决方法及发展趋势。Abstract: Vehicle detection is a hot research topic in the fields of computer vision, photogrammetry, and remote sensing. With the continuous development of deep learning technology, vehicle detection based on remote sensing images has been applied in fields such as smart city construction and intelligent transportation. This study systematically summarized existent vehicle detection algorithms based on remote sensing images and deep learning models and highlighted the classification, analysis, and comparison of one-stage and two-stage vehicle detection algorithms. Moreover, this study summarized the key technologies of vehicle detection in large-scale and complex backgrounds and analyzed the advantages and disadvantages of mainstream deep learning models of vehicle detection based on remote sensing images. Experiments were conducted to evaluate the YOLOv5, Faster-RCNN, FCOS, and SSD algorithms using DOTA and DIOR datasets. The vehicle detection precision based on the DOTA dataset was 0.695, 0.410, 0.370, and 0.251, respectively and that based on the DIOR dataset was 0.566, 0.243, 0.231, and 0.154, respectively. The experimental results show that the small target scale is still the main factor restricting the vehicle detection performance based on remote sensing images and that the application of deep learning models to the detection of small targets is to be further improved. Finally, based on public datasets and the analysis of existing algorithms, this study proposed the solution and development trend of vehicle detection based on remote sensing images in large-scale and complex backgrounds.
-
Key words:
- remote sensing image /
- vehicle detection /
- deep learning /
- analysis method
-
-
[1] 刘金明. 基于深度卷积神经网络的遥感图像中车辆检测方法研究[D]. 开封: 河南大学, 2020.
[2] Liu J M. Research on vehicle detection method in remote sensing images based on deep convolution neural network[D]. Kaifeng: Henan University, 2020.
[3] 刘天颖, 李文根, 关佶红. 基于深度学习的光学遥感图像目标检测方法综述[J]. 无线电通信技术, 2020, 46(6):624-634.
[4] Liu T Y, Li W G, Guan J H. Deep learning based object detection in optical remote sensing images:A survey[J]. Radio Communications Technology, 2020, 46(6):624-634.
[5] Cheng G, Han J. A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117:11-28.
[6] 成喆, 吕京国, 白颖奇, 等. 结合RPN网络与SSD算法的遥感影像目标检测算法[J]. 测绘科学, 2021, 46(4):75-82,99.
[7] Cheng Z, Lyu J G, Bai Y Q, et al. High-resolution remote sensing image object detection algorithm combining RPN network and SSD algorithm[J]. Science of Surveying and Mapping, 2021, 46(4):75-82,99.
[8] Alam M, Wang J F, Cong G, et al. Convolutional neural network for the semantic segmentation of remote sensing images[J]. Mobile Networks and Applications, 2021, 26:200-215.
[9] Ji H, Gao Z, Mei T, et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4):676-680.
[10] 张昭, 姚国愉, 李雪纯, 等. 基于改进Faster R-CNN算法的小目标车辆检测[J]. 科技创新与应用, 2021(4):28-32.
[11] Zhang Z, Yao G Y, Li X C, et al. Small target vehicle detection based on improved Faster-RCNN algorithm[J]. Science and Technology Innovation and Application, 2021(4):28-32.
[12] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2014:580-587.
[13] 南晓虎, 丁雷. 深度学习的典型目标检测算法综述[J]. 计算机应用研究, 2020(s2):15-21.
[14] Nan X H, Ding L. Overview of typical object detection algorithms based on deep learning[J]. Application Research of Computers, 2020(s2):15-21.
[15] Girshick R. Fast R-CNN[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2015:1440-1448.
[16] Ren S, He K, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[17] 罗峰. 基于超分辨率迁移学习的遥感图像车辆检测[D]. 厦门: 厦门大学, 2017.
[18] Luo F. Vehicle detection in remote sensing images based on super-resolution transfer learning[D]. Xiamen: Xiamen University, 2017.
[19] Deng Z P, Hao S, Zhou S L, et al. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3652-3664.
[20] Liu K, Mattyus G. Fast multiclass vehicle detection on aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(9):1938-1942.
[21] 王雪, 隋立春, 李顶萌, 等. 区域卷积神经网络用于遥感影像车辆检测[J]. 公路交通科技, 2018, 35(3):103-108.
[22] Wang X, Sui L C, Li D M, et al. Regional convolutional neural network for vehicle detection in remote sensing images[J]. Journal of Electronics and Information Technology, 2018, 35(3):103-108.
[23] 高鑫, 李慧, 张义, 等. 基于可变形卷积神经网络的遥感影像密集区域车辆检测方法[J]. 电子与信息学报, 2018, 40(12):2812-2819.
[24] Gao X, Li H, Zhang Y, et al. Vehicle detection in remote sensing images of dense areas based on deformable convolution neural network[J]. Journal of Electronics and Information Technology, 2018, 40(12):2812-2819.
[25] 阳理理. 基于人工神经网络的遥感图像车辆检测[D]. 南宁: 广西大学, 2018.
[26] Yang L L. Vehicle detection based on artificial neural network in remote sensing images[D]. Nanning: Guangxi University, 2018.
[27] 孙秉义. 基于遥感图像处理的交通量检测与分析[D]. 上海: 上海交通大学, 2019.
[28] Sun B Y. Traffic volume detection and analysis based on remote sensing images processing[D]. Shanghai: Shanghai Jiaotong University, 2019.
[29] Xia G S, Bai X, Ding J, et al. DOTA:A large-scale dataset for object detection in aerial images[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2018:3974-3983.
[30] 黄国捷. 基于深度学习的遥感图像车辆目标检测[D]. 苏州: 苏州大学, 2019.
[31] Huang G J. Vehicle target detection from remote sensing images based on deep learning[D]. Suzhou: Soochow University, 2019.
[32] Ji H, Gao Z, Mei T, et al. Improved Faster R-CNN with multiscale feature fusion and homography augmentation for vehicle detection in remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(11):1761-1765.
[33] Rottensteiner F, Sohn G, Jung J, et al. The ISPRS benchmark on urban object classification and 3D building reconstruction[J]. ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences, 2012,1-3:293-298.
[34] Razakarivony S, Jurie F. Vehicle detection in aerial imagery:A small target detection benchmark[J]. Journal of Visual Communication and Image Representation, 2016, 34:187-203.
[35] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651.
[36] 梁哲恒, 黎宵, 邓鹏, 等. 融和多尺度特征注意力的融合遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5):668-676.
[37] Liang Z H, Li X, Deng P, et al. Remote sensing images change detection fusion method integrating multi-scale feature attention[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5):668-676.
[38] Yang X, Sun H, Sun X, et al. Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J]. IEEE Access, 2018, 6:50839-50849.
[39] Fu Y, Wu F, Zhao J. Context-Aware and depth wise-based detection on orbit for remote sensing image[C]// 2018 24th International Conference on Pattern Recognition(ICPR).IEEE, 2018:1725-1730.
[40] Li Q, Mou L, Xu Q, et al. R3-Net:A deep network for multioriented vehicle detection in aerial images and videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7):5028-5042.
[41] Zhang Z H, Guo W W, Zhu S N, et al. Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(11):1745-1749.
[42] 林钊. 基于深度学习的遥感图像舰船目标检测与识别[D]. 长沙: 国防科技大学, 2018.
[43] Lin Z. Ship detection and recognition in remote sensing images based on deep learning[D]. Changsha: National University of Defense Technology, 2018.
[44] 刘万军, 高健康, 曲海成, 等. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372.
[45] Liu W J, Gao J K, Qu H C, et al. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372.
[46] 许德刚, 王露, 李凡. 深度学习的典型目标检测算法研究综述[J]. 计算机工程与应用, 2021, 57(8):10-25.
[47] Xu D G, Wang L, Li F. A review of typical object detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2021, 57(8):10-25.
[48] Redmon J, Divvala S, Girshick R, et al. You only look once:Unified,real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2016:779-788.
[49] Redmon J, Farhadi A. YOLO9000:Better,faster,stronger[C]// IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2017:6517-6525.
[50] Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL].(2019-12-25)[2021-12-29]. https://arxiv.org/abs/1804.02767 .
[51] Bochkovskiy A, Wang C Y, Liao H. YOLOv4:Optimal speed and accuracy of object detection[EB/OL].(2020-04-23)[2021-12/29]. https://arxiv.org/abs/2004.10934 .
[52] Liu W, Anguelov D, Erhan D, et al. SSD:Single shot multibox detector[J]. Springer,Cham, 2016, 9905:21-37.
[53] Fu C, Liu W, Ranga A, et al. DSSD:Deconvolutional single shot detector[EB/OL].(2017-01-23)[2021-12/29]. https://arxiv.org/abs/1701.06659v1 .
[54] Tian Z, Shen C H, Chen H, et al. FCOS:Fully convolutional one-stage object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE, 2020:9626-9635.
[55] 李圣琀, 邵峰晶. 基于深度学习的轻量遥感图像车辆检测模型[J]. 工业控制计算机, 2020, 33(6):66-69.
[56] Li S H, Shao F J. Vehicle detection model of light weight remote sensing image based on deep learning[J]. Industrial Control Computer, 2020, 33(6):66-69.
[57] Etten A V. You only look twice:Rapid multi-scale object detection in satellite imagery[EB/OL].(2018-05-24)[2021-12-29]. https://arxiv.org/abs/1805.09512 .
[58] 彭新月, 张吴明, 钟若飞. 改进YOLOv3模型的GF-2卫星影像车辆检测[J]. 测绘科学, 2021, 46(12):147-154.
[59] Peng X Y, Zhang W M, Zhong R F. GF-2 satellite image vehicle detection based on improved YOLOv3 model[J]. Science of Surveying and Mapping, 2021, 46(12):147-154.
[60] 汤田玉. 基于深度学习的高分辨率光学遥感影像车辆目标检测方法研究[D]. 长沙: 国防科技大学, 2017.
[61] Tang T Y. Deep convolutional neural network based vehicle detection methods on high resolution optical remote sensing images[D]. Changsha: National University of Defense Technology, 2017.
[62] 侯涛, 蒋瑜. 改进YOLOv4在遥感飞机目标检测中的应用研究[J]. 计算机工程与应用, 2021, 57(12):224-230.
[63] Hou T, Jiang Y. Resrarch of improved YOLOv4 in remote sensing aircraft target detection[J]. Computer Engineering and Applications, 2021, 57(12):224-230.
[64] 赵鹏飞, 谢林柏, 彭力. 融合注意力机制的深层次小目标检测算法[J]. 计算机科学与探索, 2022, 16(4):927-937.
[65] Zhao P F, Xie L B, Peng L. A deep small target detection algorithm based on attention mechanism[J]. Computer Science and Technolo-gy, 2022, 16(4):927-937.
[66] 王明阳, 王江涛, 刘琛. 基于关键点的遥感图像旋转目标检测[J]. 电子测量与仪器学报, 2021, 35(6):102-108.
[67] Wang M Y, Wang J T, Liu C. Detection of rotating targets in remote sensing images based on key points[J]. Journal of Electronic Measurement and Instrument, 2021, 35(6):102-108.
[68] 唐建宇, 唐春晖. 基于旋转框和注意力机制的遥感图像目标检测算法[J]. 电子测量技术, 2021, 44(13):114-120.
[69] Tang J Y, Tang C H. Remote sensing image target detection algorithm based on rotating frame and attention mechanism[J]. Electronic Measurement Technique, 2021, 44(13):114-120.
[70] 陈俊. 基于R-YOLO的多源遥感图像海面目标融合检测算法研究[D]. 武汉: 华中科技大学, 2019.
[71] Chen J. Research on maritime target fusion detection in multi-source remote sensing images based on R-YOLO[D]. Wuhan: Huazhong University of Science and Technology, 2019.
[72] 谢俊章, 彭辉, 唐健峰, 等. 改进YOLOv4的密集遥感目标检测[J]. 计算机工程与应用, 2021, 57(22):247-256.
[73] Xie J Z, Peng H, Tang J F, et al. Improved YOLOv4 for dense remote sensing target detection[J]. Computer Engineering and Applications, 2021, 57(22):247-256.
[74] 杨治佩, 丁胜, 张莉, 等. 无锚点的遥感图像任意角度密集目标检测方法[J]. 计算机应用, 2022, 42(6):1965-1971.
[75] Yang Z P, Ding S, Zhang L, et al. An arbitrary angle dense target detection method for remote sensing images without anchor points[J]. Computer Application, 2022, 42(6):1965-1971.
[76] 张宏群, 班勇苗, 郭玲玲, 等. 基于YOLOv5的遥感图像舰船的检测方法[J]. 电子测量技术, 2021, 44(8):87-92.
[77] Zhang H Q, Ban Y M, Guo L L, et al. Remote sensing images ship detection method based on YOLOv5[J]. Electronic Measurement Technology, 2021, 44(8):87-92.
[78] 张玉莲. 光学图像海面舰船目标智能检测与识别方法研究[D]. 长春: 中国科学院大学(中国科学院长春光学精密机械与物理研究所), 2021.
[79] Zhang Y L. Research on intelligent detection and recognition metho-ds of ship targets on the sea surface in optical images[D]. Changchun: University of Chinese Academy of Sciences (Changchun Institute of Optics,Fine Mechanics and Physics,CAS), 2021.
[80] Li K, Wan G, Cheng G, et al. Object detection in optical remote sensing images:A survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307.
-
计量
- 文章访问数: 1341
- PDF下载数: 162
- 施引文献: 0
下载: