智能制造

基于特征融合的双模态低辨识度目标识别

展开
  • 东南大学机械工程学院,南京,211189
吴愿,女,1994年生,硕士研究生。研究方向为深度学习、目标识别。

收稿日期: 2019-12-03

  修回日期: 2021-04-13

  网络出版日期: 2021-05-26

基金资助

国家重点研发计划(2016YFD0700905);国家自然科学基金(51975118,U1664258);江苏省重点研发计划(BE2019004-2)

Low-identification Dual Target Recognition Based on Feature Fusion

Expand
  • School of Mechanical Engineering,Southeast University,Nanjing,211189

Received date: 2019-12-03

  Revised date: 2021-04-13

  Online published: 2021-05-26

摘要

针对单一彩色相机对低辨识度目标识别准确率低的问题,提出了一种利用彩色相机和红外热成像仪同时检测自动驾驶目标的方案。为了同时提取彩色图像的颜色特征与红外图像的温度特征,在单模态YOLOv3网络基础上改进网络结构得到双模态YOLOv3神经网络,并设计四种特征融合对比实验以确定最佳融合方案;建立双模态数据集同步采集系统,采集彩色图像与红外图像对并用于双模态网络的训练与测试;使用未经网络训练的验证集得到4种双模态特征融合模型的平均精度值与损失值。实验结果表明,在训练30次后,该双模态网络模型的平均精度值最高可达59.42%。

本文引用格式

吴愿, 薛培林, 殷国栋, 黄文涵, 耿可可, 邹伟 . 基于特征融合的双模态低辨识度目标识别[J]. 中国机械工程, 2021 , 32(10) : 1205 -1212,1221 . DOI: 10.3969/j.issn.1004-132X.2021.10.009

Abstract

Aiming at the problems of low accuracy of low-identification target recognition by a single color camera, a scheme for simultaneously detecting an autonomous driving target was proposed by using a RGB camera and an infrared camera. In order to simultaneously extract the color characteristics of the color images and the temperature characteristics of the infrared images, the network structure was improved on the basis of YOLOv3 network to obtain a dual YOLOv3 neural network, and four feature fusion comparison experiments were designed to determine the optimal fusion scheme. A dual data set synchronous acquisition system was established, which collected color and infrared images for training and testing of the dual network. The validation set without network training was used to obtain four dual feature fusion models of average value and loss value. Experimental results show that the average accuracy of the dual-modal network model may reach 59.42% after 30 trainings.

参考文献

[1]赵德亮. 基于视频的摔倒检测算法设计与实现[D].西安:西安电子科技大学,2018.
ZHAO Deliang. Design and Implementation of Video-based Fall Detection Algorithm[D]. Xian:Xian University of Electronic Technology,2018.
[2]PENG Zhiyong, CHANG Faliang, DONG Wenhui. Vibe Motion Target Detection Algorithm Based on Lab Color Space[C]∥ Chinese Conference on Image and Graphics Technologies. Beijing, 2015:45-54.
[3]GUAN Haike, KASAHARA R, YANO T. Traffic Light Recognition and Dangerous Driving Events Detection from Surveillance Video of Vehicle Camera[J]. Electronic Imaging, 2017(4):3-10.
[4]TEOH S K, YAP V V, NISAR H. A Non-overlapping View Human Tracking Algorithm Using HSV Colour Space[J]. International Journal of Computational Vision & Robotics, 2019, 2(1):97-102.
[5]YAN Yiyun, LUO Le , ZOU Yan, et al. Colored Adaptive Compressed Imaging in YUV Color Space[C]∥3D Image Acquisition & Display:Technology, Perception & Applications. Heidelberg, 2016.
[6]PATAN R J, CHAKRABORTY M, DEVI S S. Blind Color Image De-convolution in YUV Space[C]∥International Conference on Communication & Signal Processing. Melmaruvathur:IEEE, 2016:1551-1555.
[7]YAN Yiyun, LOU Le, ZOU Yan, et al. Colored Adaptive Compressed Imaging in YUV Color Space[C]∥3D Image Acquisition & Display:Technology, Perception & Applications. Heidelberg, 2016.
[8]姜春英,康玉祥,叶长龙,等. 改进PSO-BP-Adaboost算法在尺寸超差故障诊断中的应用[J]. 中国机械工程, 2018, 29(20):2490-2494.
JIANG Chunying,KANG Yuxiang,YE Changlong, et al. Applications of Improved PSO-BP-Adaboost Algorithm in Fault Diagnosis of Dimension Out-of-tolerance[J]. China Mechanical Engineering, 2018, 29(20):2490-2494.
[9]李胜, 张培林, 吴定海,等. 基于渐近式权值小波降噪和Adaboost算法的液压泵故障诊断[J]. 中国机械工程, 2011, 22(9):1067-1070.
LI Sheng, ZHANG Peilin, WU Dinghai, et al. Fault Diagnosis for Hydraulic Pump Based on Gradual Asymptotic Weight Selection of Wavelet and Adaboost[J]. China Mechanical Engineering, 2011, 22(9):1067-1070.
[10]CHARETTE R D , NASHASHIBI F. Real Time Visual Traffic Lights Recognition Based on Spot Light Detection and Adaptive Traffic Lights Templates[C]∥2009 IEEE Intelligent Vehicles Symposium. Xian:IEEE, 2009:358-363.
[11]吴剑锋, 吴群, 孙守迁. 简约支持向量机分类算法在下肢动作识别中的应用研究[J]. 中国机械工程, 2011, 22(4):433-438.
WU Jianfeng, WU Qun, SUN Shouqian. Research on Classification Algorithm of Reduced Support Vector Machine for Low Limb Movement Recognition[J]. China Mechanical Engineering, 2011, 22(4):433-438.
[12]NIE Feiping, WANG Xiaoqian, HUANG Heng. Multiclass Capped P-norm SVM for Robust Classifications[C]∥The 31th AAAI Conference on Artificial Intelligence (AAAI 2017). San Francisco,2017:2415-2421.
[13]权凌霄,郭海,盛世伟,等. 采用“GA+LM”优化BP神经网络的电液伺服阀故障诊断[J]. 中国机械工程, 2018, 29(5):505-510.
QUAN Lingxiao, GUO Hai, SHENG Shiwei, et al. Fault Diagnosis of Electro Hydraulic Servo Valves Based on GA+LM Algorithm Optimized BP Neural Networks[J]. China Mechanical Engineering, 2018, 29(5):505-510.
[14]XIANG Xuezhi, LYU Ning, GUO Xinli, et al. Engineering Vehicles Detection Based on Modified Faster R-CNN for Power Grid Surveillance[J]. Sensors, 2018, 18(7):2258.
[15]REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:Towards Real-time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6):1137-1149.
[16]黎洲,黄妙华. 基于YOLO_v2模型的车辆实时检测[J]. 中国机械工程, 2018, 29(15):1869-1874.
LI Zhou. HUANG Miaohua. Vehicle Detections Based on YOLO_v2 in Real-time[J]. China Mechanical Engineering, 2018, 29(15):1869-1874.
[17]YUAN Man, LIU Zhi, WANG Fan. Using the Wide-range Attention U-net for Road Segmentation[J]. Remote Sensing Letters, 2019, 10(5):506-515.
[18]LEYKIN A, RAN Y, HAMMOUD R. Thermal-visible Video Fusion for Moving Target Tracking and Pedestrian Classification[C]∥ 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, 2007:1-8.
[19]SAPONARO P, SHERBONDY K, KAMBHAMETTU C. Concealed Target Detection with Fusion of Visible and Infrared[C]∥International Symposium on Visual Computing. Las Vegas, 2014:568-577.
[20]ZHAO B, GAO W, SONG Z,et al. Fusion of Infrared and Visible Image Based on Target Extraction and Contourlet Transform[J]. Journal of Information & Computational Science, 2013, 10(15):4751-4761.
文章导航

/

〈 〉