基于改进U-net的金属工件表面缺陷图像分割方法

王一; 龚肖杰; 苏皓

doi:10.5768/JAO202344.0102004

基于改进U-net的金属工件表面缺陷图像分割方法

doi: 10.5768/JAO202344.0102004

王一^{1, 2,},
龚肖杰^1, ,,
苏皓^{1, 3}

1.
华北理工大学电气工程学院，河北唐山 063210
2.
唐山市金属构件产线智能化技术创新中心，河北唐山 063210
3.
唐山市半导体集成电路重点实验室，河北唐山 063210

基金项目: 河北省高等学校科学技术研究项目(ZD2022114)；唐山市科技计划项目(21130212C)

详细信息

作者简介:
王一（1981—），男，副教授，博士，主要从事视觉检测与感知技术方面的研究。E-mail：wangyi@ncst.edu.cn

通讯作者:
龚肖杰（1994—），女，硕士，主要从事机器视觉、目标识别方面的研究。E-mail：1692994031@qq.com

中图分类号: TN209；TP391
计量
- 文章访问数: 48
- HTML全文浏览量: 37
- PDF下载量: 9
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-24
- 修回日期: 2022-09-06
- 网络出版日期: 2022-11-29
- 刊出日期: 2023-01-17

Image segmentation method of surface defects for metal workpieces based on improved U-net

WANG Yi^{1, 2
,},
GONG Xiaojie^{1
, ,},
SU Hao^{1, 3}

1.
College of Electrical Engineering, North China University of Science and Technology, Tangshan 063210, China
2.
Tangshan Technology Innovation Center of Intellectualization of Metal Component Production Line, Tangshan 063210, China
3.
Tangshan Key Laboratory of Semiconductor Integrated Circuits, Tangshan 063210, China

摘要

摘要: 针对金属工件表面小尺寸缺陷及受非均匀光照影响的图像缺陷难以分割的问题，提出了一种改进的U-net语义分割网络，实现金属工件表面缺陷图像的精确分割。首先，在U-net网络中融入CBAM（convolutional block attention module）模块来提升图像中缺陷目标的显著度；其次，采用深度超参数化卷积DO-Conv(depthwise over-parameterized convolutional)代替网络中部分传统卷积，增加网络可学习的参数数量；然后，采用Leaky Relu函数代替网络中部分Relu函数，提高模型对负区间的特征提取能力；最后，采用中值滤波及非均匀光照的补偿方法进行图像预处理，减弱非均匀光照对金属工件图像表面缺陷的影响。结果表明：改进后的网络平均交并比、准确率和Dice系数指标分别达到0.8335、0.9332、0.8674，改进的网络显著提升了对金属工件表面缺陷图像的分割效果。
- 表面缺陷 /
- 图像分割 /
- 语义分割网络 /
- 卷积注意力模块 /
- 深度超参数化卷积
Abstract: For the small-size defects of metal workpiece surface and the difficult segmentation of image defects due to non-uniform illumination, an improved U-net semantic segmentation network was proposed to achieve accurate image segmentation of surface defects for metal workpieces. Firstly, the convolutional block attention module (CBAM) was integrated into the U-net netwok to improve the significance of the defective targets in the image. Secondly, part of the traditional convolution in the network was replaced by depthwise over-parameterized convolution (DO-Conv) to increase the number of learnable parameters of the network. Then, the Leaky Relu function was used instead of the partial Relu function in the network to improve the feature extraction ability of the model for the negative intervals. Finally, the median filtering and non-uniform illumination compensation method were used for image preprocessing, so as to reduce the effect of non-uniform illumination on the surface defects of metal workpiece images. The results show that the improved network mean intersection over union, accuracy rate and Dice coefficient index reaches 0.833 5, 0.933 2 and 0.867 4, respectively. The improved network significantly improves the segmentation effect of surface defect images of metal workpieces.
- surface defects /
- image segmentation /
- semantic segmentation network /
- convolutional block attention module /
- depthwise over-parameterized convolution

HTML全文

图 1 U-net网络结构图

Fig. 1 Structure diagram of U-net network

下载: 全尺寸图片幻灯片

图 2 CBAM结构示意图

Fig. 2 Structure diagram of CBAM network

下载: 全尺寸图片幻灯片

图 3 改进的U-net网络结构图

Fig. 3 Structure diagram of improved U-net network

下载: 全尺寸图片幻灯片

图 4 图像采集视觉平台

Fig. 4 Visual platform for image acquisition

下载: 全尺寸图片幻灯片

图 5 图像预处理前后对比图

Fig. 5 Comparison before and after image preprocessing

下载: 全尺寸图片幻灯片

图 6 损失值变化曲线图

Fig. 6 Variation curves of loss values

下载: 全尺寸图片幻灯片

图 7 平均交并比图

Fig. 7 Variation curves of mean intersection over union

下载: 全尺寸图片幻灯片

图 8 准确率

Fig. 8 Variation curves of accuracy rate

下载: 全尺寸图片幻灯片

图 9 分割结果对比图

Fig. 9 Comparison of segmentation results

下载: 全尺寸图片幻灯片

表 1 Dice指标对比

Table 1 Comparison of Dice indexes

No. Approach Dice

1 PW-U-net 0.8463
2 U-net 0.8489
3 DO-U-net 0.8516
4 Att-U-net 0.8512
5 Att-DO-U-net 0.8674

下载: 导出CSV

参考文献(15)

[1]	赵鹤, 杨晓洪, 杨奇, 等. 融合注意力机制的金属缺陷图像分割方法[J]. 光电子·激光,2021,32(4):403-408. doi: 10.16136/j.joel.2021.04.0411 ZHAO He, YANG Xiaohong, YANG Qi, et al. Metal defect image segmentation algorithm combined with attention mechanism[J]. Journal of Optoelectronics·Laser,2021,32(4):403-408. doi: 10.16136/j.joel.2021.04.0411
[2]	HE J W, ZHAO C D, GAO R Y, et a1. Life prediction model of machine tool based on deep learning[J]. International Journal of Plant Engineering and Management,2021,26(1):1-15.
[3]	WU W H, GAO L, DUAN H H, et al. Segmentation of pulmonary nodules in CT images based on 3D-UNEF combined with three-dimensional conditional random field optimization[J]. Medical Physics,2020,47(9):4054-4063. doi: 10.1002/mp.14248
[4]	TAO X, ZHANG D P, MA W Z, et al. Automatic metallic surface defect detection and recognition with convolutional neural networks[J]. Applied Sciences,2018,8(9):1575. doi: 10.3390/app8091575
[5]	ZHANG K H, WANG W D, LUY Z Q, et al. Computer vision detection of foreign objects in coal processing using attention CNN[J]. Engineering Applications of Artificial Intelligence,2021,102:104242-104253. doi: 10.1016/j.engappai.2021.104242
[6]	LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network[J]. IFAC-Papers Online,2018,51(21):76-81. doi: 10.1016/j.ifacol.2018.09.412
[7]	徐建桥, 吴俊, 陈向成, 等. 基于规范化样本拆分的轴承缺陷检测[J]. 应用光学,2021,42(2):327-333. doi: 10.5768/JAO202142.0203006 XU Jianqiao, WU Jun, CHEN Xiangcheng, et al. Bearing defects detection based on standardized sample split[J]. Journal of Applied Optics,2021,42(2):327-333. doi: 10.5768/JAO202142.0203006
[8]	XIAO Z T, LIU B W, GENG L, et al. Segmentation of lung nodules using improved 3D-UNet neural network[J]. Symmetry,2020,12(11):1787. doi: 10.3390/sym12111787
[9]	卢印举, 郝志萍, 戴曙光. 融合双特征的玻璃缺陷图像分割算法[J]. 包装工程,2021,42(23):162-169. doi: 10.19554/j.cnki.1001-3563.2021.23.023 LU Yinju, HAO Zhiping, DAI Shuguang. Glass defect image segmentation algorithm fused with dual features[J]. Packaging Engineering,2021,42(23):162-169. doi: 10.19554/j.cnki.1001-3563.2021.23.023
[10]	YE M, ZHANG W W, CUI G H, et al. Surface defects inspection of cylindrical metal workpieces based on weakly supervised learning[J]. International Journal of Advanced Manufacturing Technology,2022,119(3/4):1933-1949.
[11]	曹富强, 王明泉, 张俊生, 等. 基于深度学习的铸件X射线图像分割研究[J]. 应用光学,2021,42(6):1025-1033. doi: 10.5768/JAO202142.0602003 CAO Fuqiang, WANG Mingquan, ZHANG Junsheng, et al. Casting X-ray image segmentation based on deep learning[J]. Journal of Applied Optics,2021,42(6):1025-1033. doi: 10.5768/JAO202142.0602003
[12]	HOU Y W, LIU Z Y, ZHANG T, et al. C-UNet: Complement UNet for remote sensing road extraction[J]. Sensors,2021,21(6):2153. doi: 10.3390/s21062153
[13]	FU H X, SONG G Q, WANG Y C, et al. Improved YOLOv4 marine target detection combined with CBAM[J]. Symmetry,2021,13(4):623. doi: 10.3390/sym13040623
[14]	王粉花, 张强, 黄超, 等. 融合双流三维卷积和注意力机制的动态手势识别[J]. 电子与信息学报,2021,43(5):1389-1396. doi: 10.11999/JEIT200065 WANG Fenhua, ZHANG Qiang, HUANG Chao, et al. Dynamic gesture recognition combining two stream three-dimensional convolution and attention mechanism[J]. Journal of Electronics and Information,2021,43(5):1389-1396. doi: 10.11999/JEIT200065
[15]	高淑萍, 赵清源, 齐小刚, 等. 改进的MobileNet 的图像分类方法研究[J]. 智能制造学报,2021,16(1):11-20. GAO Shuping, ZHAO Qingyuan, QI Xiaogang, et al. Research on the improved image classification method of MobileNet[J]. CAAI Transactions on Intelligent Systems,2021,16(1):11-20.