基于Mask R-CNN结合边缘分割的颗粒物图像检测

李轩; 杨舟; 陶新宇; 王晓杰; 莫绪涛; 黄仙山

doi:10.5768/JAO202344.0102005

基于Mask R-CNN结合边缘分割的颗粒物图像检测

doi: 10.5768/JAO202344.0102005

安徽工业大学数理科学与工程学院，安徽马鞍山 243002

基金项目: 国家自然科学基金（11975023）；安徽省高校自然科学研究项目(KJ2020A0238，KJ2019A0049)

详细信息

作者简介:
李轩（1997—），男，硕士研究生，主要从事图像检测与深度学习方面的研究。E-mail：1714282873@qq.com

通讯作者:
黄仙山（1974—），男，博士，教授，主要从事图像识别与检测方面的研究。E-mail：Huangxs@ahut.edu.cn

中图分类号: TN911.73；TP391.4
计量
- 文章访问数: 154
- HTML全文浏览量: 46
- PDF下载量: 16
- 被引次数: 0
出版历程
- 收稿日期: 2022-01-22
- 修回日期: 2022-03-30
- 网络出版日期: 2022-08-13
- 刊出日期: 2023-01-17

Particles image detection based on Mask R-CNN combined with edge segmentation

School of Mathematics and Physics, Anhui University of Technology, Ma'anshan 243002, China

摘要

摘要: 对颗粒物的尺寸检测是生产中重要的环节，使用相机采集图像并处理是常用的非接触检测方法。围绕颗粒物的识别与尺寸检测需求，选用沙粒为检测对象，提出了一种改进颗粒物边界掩膜的Mask R-CNN模型。该模型结合经典的边缘检测技术，并利用深度学习模型预测掩膜，根据边缘分割的结果来得到更高精度的掩膜。使用DenseNet作为检测网络的主干网络，使得整体网络参数量更少，并利用通道注意力机制加强网络的特征提取能力。实验结果表明，改进的网络可以提高检测的精度，且结合图像处理的方式能够改善掩膜尺寸检测的准确度，为颗粒物的工业检测提供了一种有意义的方法。
- 颗粒物检测 /
- 深度学习 /
- 图像分割 /
- 机器视觉 /
- 尺寸分布
Abstract: Particles size detection is an important link in production, and the use of cameras to capture and process images is a commonly-used non-contact detection method. To meet the requirements of identification and size detection of particles, the sand particles were selected as the detection object, and a Mask R-CNN model with the improved boundary mask of particles was proposed. Combined with the classical edge detection technology, the deep learning model was used to predict the mask, and the mask with higher precision was obtained according to the results of edge segmentation. The DenseNet was used as the backbone network of the network detection to reduce the number of network parameters, and the channel attention mechanism was used to strengthen the feature extraction ability of the network. The experiments show that the improved network can improve the detection accuracy, and the combination of image processing can improve the accuracy of mask size detection, which provides a meaningful method for industrial detection of particles.
- particles detection /
- deep learning /
- image segmentation /
- machine vision /
- size distribution

HTML全文

图 1 Mask R-CNN结构

Fig. 1 Structure diagram of Mask R-CNN

下载: 全尺寸图片幻灯片

图 2 DenseBlock模块和通道注意力模块

Fig. 2 Structure diagram of Denseblock module and channel attention module

下载: 全尺寸图片幻灯片

图 3 DenseAttention网络

Fig. 3 Structure diagram of DenseAttention network

下载: 全尺寸图片幻灯片

图 4 改进掩膜的Mask R-CNN模型

Fig. 4 Structure diagram of Mask R-CNN model of improved mask

下载: 全尺寸图片幻灯片

图 5 图像采集示意图

Fig. 5 Schematic diagram of image acquisition

下载: 全尺寸图片幻灯片

图 6 不同主干网络的训练过程损失曲线

Fig. 6 Loss curves of training process of different backbone networks

下载: 全尺寸图片幻灯片

图 7 不同主干网络的检测效果

Fig. 7 Detection effect of different backbone networks

下载: 全尺寸图片幻灯片

图 8 测试集上不同网络检测的尺寸分布

Fig. 8 Size distribution of different networks detection on test set

下载: 全尺寸图片幻灯片

表 1 不同主干网络的检测精度

Table 1 Detection accuracy of different backbone networks

主干网络	样本集1AP	样本集2AP	样本集3AP	网络权重大小/MB
ResNet	0.972453729	0.96582670	0.968745449	106
DenseNet	0.953721519	0.92923514	0.952060297	29.5
DenseAttention	0.976117450	0.94481288	0.963881690	30.8

下载: 导出CSV

表 2 改进前后的IoU对比

Table 2 IoU comparison before and after improvement

样本集	ResNet 原模型	DenseNet 原模型	DenseAtt 原模型	ResNet 改进模型	DenseNet 改进模型	DenseAtt 改进模型
样本集1	0.714810	0.662648	0.714638	0.872522	0.849003	0.866694
样本集2	0.684141	0.654970	0.698584	0.841869	0.836759	0.843355
样本集3	0.718732	0.669550	0.710328	0.852613	0.837046	0.833949

下载: 导出CSV

表 3 不同网络的平均检测时间

Table 3 Mean detection time of different networks s

样本集	ResNet 原模型	DenseNet 原模型	DenseAtt 原模型	ResNet 改进模型	DenseNet 改进模型	DenseAtt 改进模型
样本集1	0.508924	0.556057	0.558442	0.688702	0.703527	0.726827
样本集2	0.608456	0.634376	0.663617	0.934323	0.930722	0.991752
样本集3	0.659190	0.677558	0.680201	0.926070	0.935717	0.951665

下载: 导出CSV

表 4 不同方法的颗粒累计占比误差的统计标准差

Table 4 Statistical standard deviation of particles accumulative proportion error of different methods

方法	样本集1	样本集2	样本集3
Canny	0.270028	0.243427	0.154735
Watershed	0.223609	0.207637	0.155501
UNet	0.256473	0.228826	0.183079
UNet+Watershed	0.300005	0.261020	0.220387
ResNet+原模型	0.158070	0.153285	0.120736
DenseNet+原模型	0.368956	0.348426	0.255952
DenseAtt+原模型	0.149885	0.143315	0.109891
ResNet+改进掩膜	0.013807	0.025567	0.012105
DenseNet+改进掩膜	0.040248	0.040512	0.024433
DenseAtt+改进掩膜	0.018250	0.030876	0.006074

下载: 导出CSV

表 5 不同方法的尺寸分布相关性

Table 5 Correlation of size distribution between different methods

方法	样本集1	样本集2	样本集3
Canny	0.199998	0.439937	0.642170
Watershed	0.285612	0.542120	0.627450
UNet	0.128555	0.344368	0.438553
UNet+Watershed	0.087232	0.287599	0.278089
ResNet+原模型	0.611606	0.574975	0.694612
DenseNet+原模型	0.431583	0.494701	0.610243
DenseAtt+原模型	0.640869	0.648856	0.780750
ResNet+改进掩膜	0.935148	0.894378	0.983772
DenseNet+改进掩膜	0.783157	0.860752	0.969065
DenseAtt+改进掩膜	0.940563	0.940095	0.991209

下载: 导出CSV

参考文献(24)

[1]	CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.
[2]	YANG X, REN T, TAN L. Size distribution measurement of coal fragments using digital imaging processing[J]. Measurement,2020,160:107867. doi: 10.1016/j.measurement.2020.107867
[3]	ZHENG J, HRYCIW R D. Soil particle size and shape distributions by stereophotography and image analysis[J]. Geotechnical Testing Journal,2017,40(2):1-14.
[4]	SHELHAMER E, LONG J, DARRELL T. Fully Convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39:640-651.
[5]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. USA: IEEE, 2015: 234-241.
[6]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39:1137-1149. doi: 10.1109/TPAMI.2016.2577031
[7]	CAI Zhaowei, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2018: 6154-6162.
[8]	LIU W , ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]// European Conference on Computer Vision. [S. l. ]: arxiv, 2016: 21-37.
[9]	REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB/OL]. (2018-08-08)[2022-01-22] https://arxiv.org/abs/1804.02767?fbclid=IwAR3A2nyK8EPa-JcoGp_N6tNqVkKOmy2J1ip5AYcEki5FzkZ62E3z6tbNSy0
[10]	HAFIZ A M, BHAT G M. A survey on instance segmentation: state of the art[J]. International Journal of Multimedia Information Retrieval,2020,9(3):171-189. doi: 10.1007/s13735-020-00195-x
[11]	LI Y, QI H, DAI J, et al. Fully convolutional instance-aware semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2017: 4438-4446.
[12]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42:386-397. doi: 10.1109/TPAMI.2018.2844175
[13]	鄢然, 廖记登, 吴小勇, 等. 基于卷积神经网络的砂石骨料分类方法研究[J]. 激光与光电子学进展,2021,58(20):211-218. YAN Ran, LIAO Jideng, WU Xiaoyong, et al. Research on classification method of sand and gravel aggregate based on convolutional neural network[J]. Laser & Optoelectronics Progress,2021,58(20):211-218.
[14]	张芳, 吴玥, 肖志涛, 等. 基于U-Net卷积神经网络的纳米颗粒分割[J]. 激光与光电子学进展,2019,56(6):137-143. ZHANG Fang, WU Yue, XIAO Zhitao, et al. Nanoparticle segmentation based on U-net convolutional neural network[J]. Laser & Optoelectronics Progress,2019,56(6):137-143.
[15]	朱大庆, 曹国. 基于全卷积网络的砂石图像粒径检测[J]. 计算机与现代化,2020(7):111-116. doi: 10.3969/j.issn.1006-2475.2020.07.021 ZHU Daqing, CAO Guo. Particle size detection of sandstone images based on full convolutional network[J]. Computer and Modernization,2020(7):111-116. doi: 10.3969/j.issn.1006-2475.2020.07.021
[16]	LIANG Z Y, NIE Z H, AN A J, et al. A particle shape extraction and evaluation method using a deep convolutional neural network and digital image processing[J]. Powder Technology,2019,353,:156-170. doi: 10.1016/j.powtec.2019.05.025
[17]	李夏霖. 基于机器视觉的颗粒物缺陷检测[D]. 成都: 电子科技大学, 2019. LI Xialin. Paticle defect detection based on machine vision[D]. Chengdu: University of Electronic Science and Technology of China, 2019.
[18]	申志超. 基于YOLO的颗粒状农作物检测算法研究[D]. 哈尔滨: 哈尔滨工业大学, 2021. SHEN Zhichao. Research on detection algorithm of granular cropsbased on YOLO[D]. Harbin: Harbin Institute of Technology, 2021
[19]	谢涛. 基于深度学习的微细粒矿物识别研究[D]. 徐州: 中国矿业大学, 2020. XIE Tao. Research on recognition of fine-grained minerals based on deep learning[D]. Xuzhou: China University of Mining and Technology, 2020.
[20]	HUANG G, LIU Z, et al. Densely connected convolutional networks[C]// IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2017: 2261-2269.
[21]	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37:1904-1916. doi: 10.1109/TPAMI.2015.2389824
[22]	JIE H , LI S, ALBANIE S, et al. Squeeze-and-excitation networks[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. USA: IEEE, 2017 : 99.
[23]	WOO S H , PARK J C , LEE J Y , et al. CBAM: convolutional block attention module[C]// European Conference on Computer Vision. [S. l. ]: arxiv, 2018: 3–19.
[24]	BANERJEE S, CHAKRABORTI P C, SAHA S K. An automated methodology for grain segmentation and grain size measurement from optical micrographs[J]. Measurement,2019,140:142-150. doi: 10.1016/j.measurement.2019.03.046