基于优化残差网络的复杂纹理表面缺陷检测

林李兴; 夏振平; 徐浩; 宋玉; 胡伏原

doi:10.5768/JAO202344.0102006

基于优化残差网络的复杂纹理表面缺陷检测

doi: 10.5768/JAO202344.0102006

林李兴^1,,
夏振平^{1, 2, ,},
徐浩¹,
宋玉²,
胡伏原²

1.
苏州科技大学物理科学与技术学院，江苏苏州 215009
2.
苏州科技大学电子与信息工程学院，江苏苏州 215009

基金项目: 国家自然科学基金（62002254，61876121）江苏省自然科学基金（BK20200988）

详细信息

作者简介:
林李兴（1997—），男，硕士研究生，主要从事机器视觉检测方面的研究。E-mail：llxgongzuo@126.com

通讯作者:
夏振平(1985—)，男，博士，副教授，主要从事信息显示、机器视觉、图像处理方面的研究。E-mail：xzp@usts.edu.cn

中图分类号: TN27
计量
- 文章访问数: 53
- HTML全文浏览量: 36
- PDF下载量: 7
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-28
- 修回日期: 2022-07-14
- 网络出版日期: 2022-11-17
- 刊出日期: 2023-01-17

Defect detection on complex texture surface based on optimized ResNet

LIN Lixing^1
,,
XIA Zhenping^{1, 2
, ,},
XU Hao¹,
SONG Yu²,
HU Fuyuan²

1.
College of Physical Science and Technology, Suzhou University of Science and Technology, Suzhou 215009, China
2.
College of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, China

摘要

摘要: 产品表面缺陷检测是工业自动化生产的重要环节，准确率是评价自动检测系统可靠性的主要指标。基于复杂纹理表面缺陷检测的特殊性以及对检测方法的实时性、通用性等要求，提出了优化骨干网络并使用迁移学习特征映射构建复杂纹理表面缺陷的检测方法。该方法通过优化残差网络模型并建立仿真数据集的方式进行迁移学习，以解决实际情况中复杂纹理表面产品数据集样本数量少、数据集制作困难、相似问题难以互相兼容等问题。实验结果表明，提出的方法可以准确地检测随机复杂纹理的人造木质板材表面缺陷，平均准确率可达99.6%。现有实验条件下单张人造木质板材的检测时间为305 ms，可以满足在线检测的实时性要求。研究结果可为基于深度学习的复杂纹理表面缺陷检测提供新的思路与理论参考。
- 机器视觉 /
- 复杂纹理 /
- 残差网络 /
- 迁移学习 /
- 缺陷检测
Abstract: Defect detection of product surface is an important part of industrial automatic production and the accuracy is the main index to evaluate the reliability of automatic detection system. Based on the particularity of defect detection on complex texture surface and the requirements of real-time and universal detection methods, a detection method for optimizing the backbone network and using the transfer learning feature mapping to construct the complex texture surface defects was proposed. In this method, the ResNet model was optimized and the simulation data set was established for transfer learning, so as to solve the problems such as the small number of samples in the data set of complex texture surface products, the difficulty of data set making, and the difficulty of similar problems to be compatible with each other. Experimental results show that the proposed method can accurately detect the surface defects of artificial wooden plank with random complex texture, and the average accuracy can reach 99.6%. Under the existing experimental conditions, the detection time of a single artificial wooden plank is 305 ms, which can meet the real-time requirements of online detection. The research results can provide a new idea and theoretical reference for the defect detection on complex texture surface based on deep learning.
- machine vision /
- complex texture /
- ResNet /
- transfer learning /
- defect detection

HTML全文

图 1 迁移学习特征映射的方法示意图

Fig. 1 Schematic diagram of method for transfer learning feature mapping

下载: 全尺寸图片幻灯片

图 2 原始图像经过预处理制作成数据集

Fig. 2 Data set preprocessed by original images

下载: 全尺寸图片幻灯片

图 3 4组表面无缺陷与表面缺陷对比（红色虚线框标志部分为缺陷所在位置）

Fig. 3 Comparison of four groups without and with surface defects, and red dotted line frame is location of defect

下载: 全尺寸图片幻灯片

图 4 仿真数据集构建方法示意图

Fig. 4 Schematic diagram of construction method for simulation data set

下载: 全尺寸图片幻灯片

图 5 不同激活函数在测试数据集上的准确率和收敛比较

Fig. 5 Comparison of accuracy and convergence of different activation functions on test data sets

下载: 全尺寸图片幻灯片

图 6 不同网络深度在测试数据集上的准确率和收敛比较

Fig. 6 Comparison of accuracy and convergence of different network depths on test data sets

下载: 全尺寸图片幻灯片

图 7 改进ResNet18模型结构示意图

Fig. 7 Structure diagram of improved ResNet18 model

下载: 全尺寸图片幻灯片

图 8 迁移学习方法

Fig. 8 Schematic diagram of transfer learning method

下载: 全尺寸图片幻灯片

图 9 迁移学习方法的准确率验证

Fig. 9 Accuracy verification of transfer learning method

下载: 全尺寸图片幻灯片

表 1 真实数据集参数

Table 1 Parameters of real data set

Batch No.	Actual size/m	Size/pixel	Cutting No.
1	1.22×0.175	3×7120×850	350
2	1.22×0.175	3×7120×850	300
3	0.70×0.175	3×7120×850	300

下载: 导出CSV

表 2 实验环境和具体参数

Table 2 Experimental environment and specific parameters

Hardware Environment
Hardware		Model Number
CPU		Inter core i7-10750H
GPU		NVIDIA RTX2060
Memory		24 GB

Software Environment
Software		Name
System		Windows10
configuration		Pytorch 3.6 cuda 10.1

Training parameters
Parameter		Value
Batch size		8
Epoch		25
CUDA		Enable

下载: 导出CSV

表 3 在模拟数据集上检测系统对四类缺陷的实验结果

Table 3 Experimental results of four types of defects by detection system on simulated data sets

Defect types	Pollution defect	Scratches pollution	Breakage defect	Lack of design and color	Average
Recall/%	98.2	99.2	100	100	99.6
Precision/%	100	100	100	100	100
Accuracy/%	98.8	99.5	100	100	99.6
Time/ms	305	305	305	305	305

下载: 导出CSV

表 4 本文方法与其他方法在真实数据集上的结果对比

Table 4 Results comparison of proposed method and other methods on real data sets

Model	ACC	LOSS
ResNet18	83.2%	0.332
DenseNet121	97.2%	0.077
SqueezeNet	95.8%	0.097
MobileNet V3	92.0%	0.118
Our model	98.7%	0.011

下载: 导出CSV

参考文献(18)

[1]	郭慧, 王霄, 刘传泽, 等. 人造板表面缺陷检测图像自适应快速阈值分割算法[J]. 林业科学,2018,54(11):134-142. doi: 10.11707/j.1001-7488.20181119 GUO Hui, WANG Xiao, LIU Chuanze, et al. Research on adaptive fast threshold segmentation algorithm for surface defect detection of wood-based panel[J]. Scientia Silvae Sinicae,2018,54(11):134-142. doi: 10.11707/j.1001-7488.20181119
[2]	朱云, 凌志刚, 张雨强. 机器视觉技术研究进展及展望[J]. 图学学报,2020,41(6):871-890. ZHU Yun, LING Zhigang, ZHANG Yuqiang. Research progress and prospect of machine vision technology[J]. Journal of Graphics,2020,41(6):871-890.
[3]	杨传礼, 张修庆. 基于机器视觉和深度学习的材料缺陷检测应用综述[J]. 材料导报,2022,36(16):1-19. YANG Chuanli, ZHANG Xiuqing. Survey of material defect detection applications based on machine vision and deep learning[J]. Materials Reports,2022,36(16):1-19.
[4]	郭慧, 盛振湘, 王霄, 等. 基于机器视觉的刨花板表面缺陷检测系统[J]. 木材科学与技术,2019,33(3):18-22. GUO Hui, SHENG Zhenxiang, WANG Xiao, et al. A system based on machien vision for detecting surface defects of particleboard[J]. Chinese Journal of Wood Science and Technology,2019,33(3):18-22.
[5]	魏智锋, 肖书浩, 蒋国璋, 等. 基于深度学习的人造板表面缺陷检测研究[J]. 林产工业,2021,58(2):21-26. doi: 10.19531/j.issn1001-5299.202102005 WEI Zhifeng, XIAO Shuhao, JIANG Guozhang, et al. Research on surface defect detection of wood-based panels based on deep learning[J]. Scientia Silvae Sinicae,2021,58(2):21-26. doi: 10.19531/j.issn1001-5299.202102005
[6]	MUDA U R H, HASHIM S Z M, KAMILAH A. Performance evaluation of multivariate texture descriptor for classification of timber defect[J]. Optik-International Journal for Light and Electron Optic,2016,127(15):6071-6080. doi: 10.1016/j.ijleo.2016.04.005
[7]	王宇杰. 基于机器视觉的塑料制品外观缺陷检测[J]. 合成树脂及塑料,2021,38(1):93-96. doi: 10.19825/j.issn.1002-1396.2021.01.21 WANG Yujie. Appearance defect detection for plastic products based on machine vision[J]. China Synthetic Resin and Plastics,2021,38(1):93-96. doi: 10.19825/j.issn.1002-1396.2021.01.21
[8]	ZHANG Yizhuo, XU Chao, LI Chao, et al. Wood defect detection method with PCA feature fusion and compressed sensing[J]. Journal of Forestry Research,2015,26(3):745-751. doi: 10.1007/s11676-015-0066-4
[9]	刘沛津, 王曦, 贺宁. 改进GSO与二维OTSU融合的红外图像多阈值分割方法[J]. 应用光学,2021,42(4):671-677. doi: 10.5768/JAO202142.0402006 LIU Peijin, WANG Xi, HE Ning. Multi-threshold segmentation method of infrared images based on improved fusion of GSO and 2D OTSU[J]. Journal of Applied Optics,2021,42(4):671-677. doi: 10.5768/JAO202142.0402006
[10]	李漫丽, 赵鹏. 基于图像融合的木板表面缺陷检测研究[J]. 液晶与显示,2016,31(9):882-888. doi: 10.3788/YJYXS20163109.0882 LI Manli, ZHAO Peng. Wood surfacce defect detection based on image fusion[J]. Chinese Journal of Liquid Crystals and Displays,2016,31(9):882-888. doi: 10.3788/YJYXS20163109.0882
[11]	贾坡, 田建艳, 杨英波, 等. 基于机器视觉的滚抛磨块缺陷检测方法[J]. 金刚石与磨料磨具工程,2021,41(1):76-82. doi: 10.13394/j.cnki.jgszz.2021.1.0013 JIA Po, TIAN Jianyan, YANG Yingbo, et al. Defect detection method of abrasive block based on machine vision[J]. Diamond & Abrasives Engineering,2021,41(1):76-82. doi: 10.13394/j.cnki.jgszz.2021.1.0013
[12]	廖延娜, 豆丹阳. 基于改进Mask R-CNN的建筑钢筋尺寸检测算法[J]. 应用光学,2020,43(1):100-105. LIAO Yanna, DOU Danyang. Design and research of bridge cracks detection method based on Mask RCNN[J]. Journal of Applied Optics,2020,43(1):100-105.
[13]	GAO Mingyu, QI Dawei, MU Hongbo, et al. A transfer residual neural network based on Resnet‐34 for detection of wood knot defects[J]. Forests,2021,12(2):1-16.
[14]	刘万军, 尹岫, 曲海成, 等. 提高小样本高光谱图像分类性能的变维卷积神经网络[J]. 中国图象图形学报,2019,24(9):1604-1618. LIU Wanjun, YIN Xiu, QU Haicheng, et al. Dimensionality-varied convolution neural network for improving the classification permance of hyperspectral images with small-sized labeled samples[J]. Journal of Image and Graphics,2019,24(9):1604-1618.
[15]	REN Ruoxu, HUNG T, TAN K C. A generic deep-learning-based approach for automated surface inspection[J]. IEEE Transactions on Cybernetics,2017,48(3):1-12.
[16]	URBONAS A, RAUDONIS V, MASKELIUNAS R, et al. Automated identification of wood veneer surface defects using faster region-based convolutional neural network with data augmentation and transfer learning[J]. Applied Sciences (Switzerland),2019,9(22):4898. doi: 10.3390/app9224898
[17]	徐志祥, 关守岩, 杨帆, 等. 一种基于2D-CNN的激光超声表面缺陷检测方法[J]. 应用光学,2021,42(1):149-156. doi: 10.5768/JAO202142.0107002 XU Zhixiang, GUAN Shouyan, YANG Fan, et al. Laser ultrasonic surface defects detection method based on 2D-CNN[J]. Journal of Applied Optics,2021,42(1):149-156. doi: 10.5768/JAO202142.0107002
[18]	汪荣贵, 郑岩, 杨娟, 等. 代表特征网络的小样本学习方法[J]. 中国图象图形学报,2019,24(9):1514-1527. doi: 10.11834/jig.180629 WANG Ronggui, ZHENG Yan, YANG Juan, et al. Representative feature networks for few-shot learning[J]. Journal of Image and Graphics,2019,24(9):1514-1527. doi: 10.11834/jig.180629