留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于Mask R-CNN结合边缘分割的颗粒物图像检测

李轩 杨舟 陶新宇 王晓杰 莫绪涛 黄仙山

李轩, 杨舟, 陶新宇, 王晓杰, 莫绪涛, 黄仙山. 基于Mask R-CNN结合边缘分割的颗粒物图像检测[J]. 应用光学, 2023, 44(1): 93-103. doi: 10.5768/JAO202344.0102005
引用本文: 李轩, 杨舟, 陶新宇, 王晓杰, 莫绪涛, 黄仙山. 基于Mask R-CNN结合边缘分割的颗粒物图像检测[J]. 应用光学, 2023, 44(1): 93-103. doi: 10.5768/JAO202344.0102005
LI Xuan, YANG Zhou, TAO Xinyu, WANG Xiaojie, MO Xutao, HUANG Xianshan. Particles image detection based on Mask R-CNN combined with edge segmentation[J]. Journal of Applied Optics, 2023, 44(1): 93-103. doi: 10.5768/JAO202344.0102005
Citation: LI Xuan, YANG Zhou, TAO Xinyu, WANG Xiaojie, MO Xutao, HUANG Xianshan. Particles image detection based on Mask R-CNN combined with edge segmentation[J]. Journal of Applied Optics, 2023, 44(1): 93-103. doi: 10.5768/JAO202344.0102005

基于Mask R-CNN结合边缘分割的颗粒物图像检测

doi: 10.5768/JAO202344.0102005
基金项目: 国家自然科学基金(11975023);安徽省高校自然科学研究项目(KJ2020A0238,KJ2019A0049)
详细信息
    作者简介:

    李轩(1997—),男,硕士研究生,主要从事图像检测与深度学习方面的研究。E-mail:1714282873@qq.com

    通讯作者:

    黄仙山(1974—),男,博士,教授,主要从事图像识别与检测方面的研究。E-mail:Huangxs@ahut.edu.cn

  • 中图分类号: TN911.73;TP391.4

Particles image detection based on Mask R-CNN combined with edge segmentation

  • 摘要: 对颗粒物的尺寸检测是生产中重要的环节,使用相机采集图像并处理是常用的非接触检测方法。围绕颗粒物的识别与尺寸检测需求,选用沙粒为检测对象,提出了一种改进颗粒物边界掩膜的Mask R-CNN模型。该模型结合经典的边缘检测技术,并利用深度学习模型预测掩膜,根据边缘分割的结果来得到更高精度的掩膜。使用DenseNet作为检测网络的主干网络,使得整体网络参数量更少,并利用通道注意力机制加强网络的特征提取能力。实验结果表明,改进的网络可以提高检测的精度,且结合图像处理的方式能够改善掩膜尺寸检测的准确度,为颗粒物的工业检测提供了一种有意义的方法。
  • 图  1  Mask R-CNN结构

    Fig.  1  Structure diagram of Mask R-CNN

    图  2  DenseBlock模块和通道注意力模块

    Fig.  2  Structure diagram of Denseblock module and channel attention module

    图  3  DenseAttention网络

    Fig.  3  Structure diagram of DenseAttention network

    图  4  改进掩膜的Mask R-CNN模型

    Fig.  4  Structure diagram of Mask R-CNN model of improved mask

    图  5  图像采集示意图

    Fig.  5  Schematic diagram of image acquisition

    图  6  不同主干网络的训练过程损失曲线

    Fig.  6  Loss curves of training process of different backbone networks

    图  7  不同主干网络的检测效果

    Fig.  7  Detection effect of different backbone networks

    图  8  测试集上不同网络检测的尺寸分布

    Fig.  8  Size distribution of different networks detection on test set

    表  1  不同主干网络的检测精度

    Table  1  Detection accuracy of different backbone networks

    主干网络样本集1AP 样本集2AP样本集3AP 网络权重大小/MB
    ResNet0.9724537290.965826700.968745449106
    DenseNet0.9537215190.929235140.95206029729.5
    DenseAttention0.9761174500.944812880.96388169030.8
    下载: 导出CSV

    表  2  改进前后的IoU对比

    Table  2  IoU comparison before and after improvement

    样本集ResNet
    原模型
    DenseNet
    原模型
    DenseAtt
    原模型
    ResNet
    改进模型
    DenseNet
    改进模型
    DenseAtt
    改进模型
    样本集10.7148100.6626480.7146380.8725220.8490030.866694
    样本集20.6841410.6549700.6985840.8418690.8367590.843355
    样本集30.7187320.6695500.7103280.8526130.8370460.833949
    下载: 导出CSV

    表  3  不同网络的平均检测时间

    Table  3  Mean detection time of different networks s

    样本集ResNet
    原模型
    DenseNet
    原模型
    DenseAtt
    原模型
    ResNet
    改进模型
    DenseNet
    改进模型
    DenseAtt
    改进模型
    样本集10.5089240.5560570.5584420.6887020.7035270.726827
    样本集20.6084560.6343760.6636170.9343230.9307220.991752
    样本集30.6591900.6775580.6802010.9260700.9357170.951665
    下载: 导出CSV

    表  4  不同方法的颗粒累计占比误差的统计标准差

    Table  4  Statistical standard deviation of particles accumulative proportion error of different methods

    方法样本集1样本集2样本集3
    Canny0.2700280.2434270.154735
    Watershed0.2236090.2076370.155501
    UNet0.2564730.2288260.183079
    UNet+Watershed0.3000050.2610200.220387
    ResNet+原模型0.1580700.1532850.120736
    DenseNet+原模型0.3689560.3484260.255952
    DenseAtt+原模型0.1498850.1433150.109891
    ResNet+改进掩膜0.0138070.0255670.012105
    DenseNet+改进掩膜0.0402480.0405120.024433
    DenseAtt+改进掩膜0.0182500.0308760.006074
    下载: 导出CSV

    表  5  不同方法的尺寸分布相关性

    Table  5  Correlation of size distribution between different methods

    方法样本集1样本集2样本集3
    Canny0.1999980.4399370.642170
    Watershed0.2856120.5421200.627450
    UNet0.1285550.3443680.438553
    UNet+Watershed0.0872320.2875990.278089
    ResNet+原模型0.6116060.5749750.694612
    DenseNet+原模型0.4315830.4947010.610243
    DenseAtt+原模型0.6408690.6488560.780750
    ResNet+改进掩膜0.9351480.8943780.983772
    DenseNet+改进掩膜0.7831570.8607520.969065
    DenseAtt+改进掩膜0.9405630.9400950.991209
    下载: 导出CSV
  • [1] CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
    [2] YANG X, REN T, TAN L. Size distribution measurement of coal fragments using digital imaging processing[J]. Measurement,2020,160:107867. doi: 10.1016/j.measurement.2020.107867
    [3] ZHENG J, HRYCIW R D. Soil particle size and shape distributions by stereophotography and image analysis[J]. Geotechnical Testing Journal,2017,40(2):1-14.
    [4] SHELHAMER E, LONG J, DARRELL T. Fully Convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39:640-651.
    [5] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. USA: IEEE, 2015: 234-241.
    [6] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39:1137-1149. doi: 10.1109/TPAMI.2016.2577031
    [7] CAI Zhaowei, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2018: 6154-6162.
    [8] LIU W , ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. [S. l. ]: arxiv, 2016: 21-37.
    [9] REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-08-08)[2022-01-22] https://arxiv.org/abs/1804.02767?fbclid=IwAR3A2nyK8EPa-JcoGp_N6tNqVkKOmy2J1ip5AYcEki5FzkZ62E3z6tbNSy0
    [10] HAFIZ A M, BHAT G M. A survey on instance segmentation: state of the art[J]. International Journal of Multimedia Information Retrieval,2020,9(3):171-189. doi: 10.1007/s13735-020-00195-x
    [11] LI Y, QI H, DAI J, et al. Fully convolutional instance-aware semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2017: 4438-4446.
    [12] HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42:386-397. doi: 10.1109/TPAMI.2018.2844175
    [13] 鄢然, 廖记登, 吴小勇, 等. 基于卷积神经网络的砂石骨料分类方法研究[J]. 激光与光电子学进展,2021,58(20):211-218.

    YAN Ran, LIAO Jideng, WU Xiaoyong, et al. Research on classification method of sand and gravel aggregate based on convolutional neural network[J]. Laser & Optoelectronics Progress,2021,58(20):211-218.
    [14] 张芳, 吴玥, 肖志涛, 等. 基于U-Net卷积神经网络的纳米颗粒分割[J]. 激光与光电子学进展,2019,56(6):137-143.

    ZHANG Fang, WU Yue, XIAO Zhitao, et al. Nanoparticle segmentation based on U-net convolutional neural network[J]. Laser & Optoelectronics Progress,2019,56(6):137-143.
    [15] 朱大庆, 曹国. 基于全卷积网络的砂石图像粒径检测[J]. 计算机与现代化,2020(7):111-116. doi: 10.3969/j.issn.1006-2475.2020.07.021

    ZHU Daqing, CAO Guo. Particle size detection of sandstone images based on full convolutional network[J]. Computer and Modernization,2020(7):111-116. doi: 10.3969/j.issn.1006-2475.2020.07.021
    [16] LIANG Z Y, NIE Z H, AN A J, et al. A particle shape extraction and evaluation method using a deep convolutional neural network and digital image processing[J]. Powder Technology,2019,353,:156-170. doi: 10.1016/j.powtec.2019.05.025
    [17] 李夏霖. 基于机器视觉的颗粒物缺陷检测[D]. 成都: 电子科技大学, 2019.

    LI Xialin. Paticle defect detection based on machine vision[D]. Chengdu: University of Electronic Science and Technology of China, 2019.
    [18] 申志超. 基于YOLO的颗粒状农作物检测算法研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.

    SHEN Zhichao. Research on detection algorithm of granular cropsbased on YOLO[D]. Harbin: Harbin Institute of Technology, 2021
    [19] 谢涛. 基于深度学习的微细粒矿物识别研究[D]. 徐州: 中国矿业大学, 2020.

    XIE Tao. Research on recognition of fine-grained minerals based on deep learning[D]. Xuzhou: China University of Mining and Technology, 2020.
    [20] HUANG G, LIU Z, et al. Densely connected convolutional networks[C]// IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2017: 2261-2269.
    [21] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37:1904-1916. doi: 10.1109/TPAMI.2015.2389824
    [22] JIE H , LI S, ALBANIE S, et al. Squeeze-and-excitation networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. USA: IEEE, 2017 : 99.
    [23] WOO S H , PARK J C , LEE J Y , et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. [S. l. ]: arxiv, 2018: 3–19.
    [24] BANERJEE S, CHAKRABORTI P C, SAHA S K. An automated methodology for grain segmentation and grain size measurement from optical micrographs[J]. Measurement,2019,140:142-150. doi: 10.1016/j.measurement.2019.03.046
  • 加载中
图(10) / 表(5)
计量
  • 文章访问数:  151
  • HTML全文浏览量:  44
  • PDF下载量:  15
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-01-22
  • 修回日期:  2022-03-30
  • 网络出版日期:  2022-08-13
  • 刊出日期:  2023-01-17

目录

    /

    返回文章
    返回