[1]邓君,李娜,王亚凯,等.图像分割引导的散堆工件结构光三维位姿估计[J].华侨大学学报(自然科学版),2024,45(6):696-705.[doi:10.11830/ISSN.1000-5013.202401011]
　DENG Jun,LI Na,WANG Yakai,et al.Structured Light-Based 3D Pose Estimation of Piled Workpieces Guided by Image Segmentation[J].Journal of Huaqiao University(Natural Science),2024,45(6):696-705.[doi:10.11830/ISSN.1000-5013.202401011]
点击复制

图像分割引导的散堆工件结构光三维位姿估计()
分享到：

《华侨大学学报（自然科学版）》[ISSN:1000-5013/CN:35-1079/N]

卷:
第45卷
期数:
2024年第6期
页码:
696-705
栏目:
出版日期:
2024-11-15

文章信息/Info

Title:
Structured Light-Based 3D Pose Estimation of Piled Workpieces Guided by Image Segmentation
文章编号:
1000-5013(2024)06-0696-10
作者:
邓君1; 2;  李娜2; 3;  王亚凯1; 2;  高振国1; 2
1. 华侨大学 计算机科学与技术学院, 福建 厦门 361021;2. 华侨大学 计算机视觉与机器学习重点实验室, 福建 厦门 361021;3. 华侨大学 机电及自动化学院, 福建 厦门 361021
Author(s):
DENG Jun1; 2;  LI Na2; 3;  WANG Yakai1; 2;  GAO Zhenguo1; 2
1. College of Computer Science and Technology, Huaqiao University, Xiamen 361021, China; 2. Key Laboratory of Computer Vision and Machine Learning, Huaqiao University, Xiamen 361021, China; 3. College of Mechanical and Electrical Engineering, Huaqiao University, Xiamen 361021, China
关键词:
双目结构光;  点云生成;  点云配准;  位姿估计;  图像分割
Keywords:
binocular structured light;  point cloud generation;  point cloud registration;  pose estimation;  image segmentation
分类号:
TP391
DOI:
10.11830/ISSN.1000-5013.202401011
文献标志码:
A
摘要:
针对散堆工件场景中点云生成耗时久、位姿估计困难、多类工件混合情形难处理等问题,提出图像分割引导的散堆工件结构光三维位姿估计方法,并基于JAkA Zu3 6-DoF机器人开发散堆工件抓取实验系统。采用YOLACT模型获取散堆工件图像中的工件信息,通过自适应阈值筛选待抓取工件,利用双目结构光生成目标工件所在区域的局部点云,并基于投票匹配算法和迭代最近邻算法估计工件位姿。通过搭建的实验系统对文中方法进行测试。实验结果表明:系统完成目标工件位姿估计时间约为3.641 s,其中,点云计算需0.536 s,点云配准需0.446 s;与其他方法相比,文中方法平均可缩小点云规模44%,点云生成时间平均缩减24%,配准成功率提升至100%。
Abstract:
Aiming at the problems of point cloud generation time-consuming, pose estimation difficulty, and multi class workpiece mixing difficult handling in the scene of piled workpieces, a structured light-based 3D pose estimation of piled workpieces guided by image segmentation is proposed, and an experimental system of piled workpiece picking is developed based on the JAkA Zu3 6-DoF robot. The YOLACT model is used to extract workpiece information from the piled workpieces images. The workpiece to be grabbed is filtered through adaptive threshold, the local point clouds in the area where the target workpiece is generated using binocular structured light. The workpiece pose based on the voting matching algorithm and the iterative nearest neighbor algorithm is estimated. The proposed method is tested by the constructed experimental system. The experimental results show that the system takes approximately 3.641 s to complete the target workpiece pose esti-mation. Among them, point cloud computing takes 0.536 s and point cloud registration takes 0.446 s. Compared with other methods, the proposed method can reduce the size of point clouds by an average of 44%, reduce the time of the point clouds generation by an average of 24%, and improve the registration success rate to 100%.

参考文献/References:

[1] SCHONBERGER J L,FRAHM J M.Structure-from-motion revisited[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:4104-4113.DOI:10.1109/CVPR.2016.445.
[2] 黄及远,李敏,谢兵兵,等.双目视觉关键技术研究综述[J].制造业自动化,2023,45(5):166-171.DOI:1009-0134(2023)05-0166-06.
[3] FOIX S,ALENYA G,TORRAS C.Lock-in time-of-flight(ToF)cameras: A survey[J].IEEE Sensors Journal,2011,11(9):1917-1926.DOI:10.1109/JSEN.2010.2101060.
[4] 夏晨旭,郝群,张一鸣,等.基于结构光投影三维重建的人脸特征检测[J].激光与光电子学进展,2023,60(22):186-192.DOI:10.3788/LOP230620.
[5] AHARCHI M,AITKBIR M.A review on 3D reconstruction techniques from 2D images[C]//International Conference on Smart Cities Applications.Berlin:Springer,2020:510-522.DOI:10.1007/978-3-030-37629-1_37.
[6] MAYER N,ILG E,HAUSSER P,et al.A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2016:4040-4048.DOI:10.1109/CVPR.2016.438.
[7] SUNDERMEYER M,MARTON Z C,DURNER M,et al.Implicit 3D orientation learning for 6D object detection from RGB images[C]//European Conference on Computer Vision.Cham:Springer,2018:699-715.DOI:10.1007/978-3-030-01231-1_43.
[8] 牟向伟,孙国奇,陈林涛,等.基于Kinect相机的多视角RGB-D信息融合的甜椒姿态估计研究[J].中国农机化学报,2023,44(10):159-167.DOI:10.13733/j.jcam.issn.2095-5553.2023.10.023.
[9] BRACHMANN E,KRULL A,MICHEL F,et al.Learning 6D object pose estimation using 3D object coordinates[C]//European Conference on Computer Vision.Cham:Springer,2014:536-551.DOI:10.1007/978-3-319-10605-2_35.
[10] KRULL A,BRACHMANN E,MICHEL F,et al.Learning analysis-by-synthesis for 6D pose estimation in RGB-D images[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2015:954-962.DOI:10.1109/ICCV.2015.115.
[11] CRIVELLARO A,RAD M,VERDIE Y,et al.A novel representation of parts for accurate 3D object detection and tracking in monocular images[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2015:4391-4399.DOI:10.1109/ICCV.2015.499.
[12] HODAN T,HALUZA P,OBDR?áLEK ?,et al.T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[C]//IEEE Winter Conference on Applications of Computer Vision.Piscataway:IEEE Press,2017:880-888.DOI:10.48550/arXiv.1701.05498.
[13] JAFARI O H,MITZEL D,LEIBE B.Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras[C]//IEEE International Conference on Robotics and Automation.Piscataway:IEEE Press,2014:5636-5643.DOI:10.1109/ICRA.2014.6907688.
[14] QI C R,SU Hao,MO Kaichun,et al.PointNet: Deep learning on point sets for 3D classification and segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2017:652-660.DOI:10.1109/CVPR.2017.16.
[15] QI C R,YI Li, SU Hao,et al.Pointnet++: Deep hierarchical feature learning on point sets in a metric space[C]//Annual Conference on Neural Information Processing Systems.New York:MIT Press,2017:5099-5108.DOI:10.48550/arXiv.1706.02413.
[16] WANG Yue,SUN Yongbin,LIU Ziwei,et al.Dynamic graph CNN for learning on point clouds[J].ACM Transactions on Graphics,2019,38(5):1-12.DOI:10.48550/arXiv.1801.07829.
[17] WANG Chen,XU Danfei,ZHU Yuke,et al.DenseFusion: 6D object pose estimation by iterative dense fusion[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2019:3343-3352.DOI:10.48550/arXiv.1901.04780.
[18] GAO Ge,LAURI M,WANG Yulong,et al.6D object pose regression via supervised learning on point clouds[C]//IEEE International Conference on Robotics and Automation.Piscataway:IEEE Press,2020:3643-3649.DOI:10.1109/ICRA40945.2020.9197461.
[19] ZHUANG Chuangang,LI Shaofei,DING Han.Instance segmentation based 6D pose estimation of industrial objects using point clouds for robotic bin-picking[J].Robotics and Computer-Integrated Manufacturing,2023,82:102541.DOI:10.1016/j.RCIM.2023.102541.
[20] ZHUANG Chuangang,WANG Haoyu,DING Han.Attention vote: A coarse-to-fine voting network of anchor-free 6D pose estimation on point cloud for robotic bin-picking application[J].Robotics and Computer-Integrated Manufacturing,2024,86:102671.DOI:10.1016/J.RCIM.2023.102671.
[21] PAVLAKOS G,ZHOU Xiaowei,CHAN A,et al.6-DoF object pose from semantickeypoints[C]//IEEE International Conference on Robotics and Automation.Piscataway:IEEE Press,2017:2011-2018.DOI:10.1109/ICRA.2017.7989233.
[22] ROTHGANGER F,LAZEBNIK S,SCHMID C,et al.3D object modeling and recognition using local affine-invariant image descriptors and multi-view spatial constraints[J].International Journal of Computer Vision,2006,66(3):231-259.DOI:10.1007/s11263-005-3674-1.
[23] TULSIANI S,MALIK J.Viewpoints and keypoints[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2015:1510-1519.DOI:10.1109/CVPR.2015.7298758.
[24] RUBLEE E,RABAUD V,KONOLIGE K,et al.ORB: An efficient alternative to SIFT or SURF[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2011:2564-2571.DOI:10.1109/ICCV.2011.6126544.
[25] BAY H,TUYTELAARS T,VAN GOOL L.Surf: Speeded up robust features[C]//European Conference on Computer Vision.Berlin:Springer,2006:404-417.DOI:10.1007/11744023_32.
[26] RUSU R B,BLODOW N,BEETZ M.Fast point feature histograms(FPFH)for 3D registration[C]//IEEE International Conference on Robotics and Automation.Piscataway:IEEE Press,2009:3212-3217.DOI:10.1109/ROBOT.2009.5152473.
[27] TOMBARI F,SALT S,DI STEFANO L.Unique signatures of histograms for local surface description[C]//European Conference on Computer Vision.Berlin:Springer,2010:356-369.DOI:10.1007/978-3-642-15558-1_26.
[28] VIDAL J,LIN C Y,MARTí R.6D pose estimation using an improved method based on point pair features[C]//International Conference on Control, Automation, and Robotics.Piscataway:IEEE Press,2018:405-409.DOI:10.1109/ICCAR.2018.8384709.
[29] SHARP G C,LEE S W,WEHE D K.ICP registration using invariant features[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(1):90-102.DOI:10.1109/34.982886.
[30] YANG Jiaolong,LI Hongdong,CAMPBELL D,et al.Go-ICP: A globally optimal solution to 3D ICP point-set registration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(11):2241-2254.DOI:10.1109/TPAMI.2015.2513405.
[31] 汪霖,郭佳琛,张璞,等.基于改进ICP算法的三维点云刚体配准方法[J].西北大学学报(自然科学版),2021,51(2):183-190.DOI:10.16152/j.cnki.xdxbzr.2021-02-002.
[32] 荆路,武斌,方锡禄.基于SIFT特征点结合ICP的点云配准方法[J].激光与红外,2021,51(7):944-950.DOI:10.3969/j.issn.1001-5078.2021.07.019.
[33] ZHANG Jing,YIN Baoqun,XIAO Xianpeng,et al.3D detection and 6D pose estimation of texture-less objects for robot grasping[C]//International Conference on Control and Robotics Engineering.Piscataway:IEEE Press,2021:33-38.DOI:10.1109/ICCRE51898.2021.9435702.
[34] BOLYA D,ZHOU Chong,XIAO Fanyi,et al.YOLACT: Real-time instance segmentation[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2019:9157-9166.DOI:10.1109/ICCV.2019.00925.
[35] LI Beiwen,AN Yatong,CAPPELLERI D,et al.High-accuracy, high-speed 3D structured light imaging techniques and potential applications to intelligent robotics[J].International Journal of Intelligent Robotics and Applications,2017,1(1):86-103.DOI:10.1007/s41315-016-0001-7.
[36] 熊宗刚.结构光双目三维成像关键技术研究[D].成都:电子科技大学,2021.

备注/Memo

备注/Memo:
收稿日期: 2024-01-11
通信作者: 高振国(1976-),男,教授,博士,主要从事智能制造、机器视觉及无线自组网络等的研究。E-mail:gaohit@sina.com。
基金项目: 国家自然科学基金资助项目(62372190, 61972166); 福建省高校产学合作资助项目(2021H6030)https://hdxb.hqu.edu.cn/
更新日期/Last Update: 2024-11-20