Learning-based object detection and localization for a mobile robot manipulator in SME production

TY - JOUR

T1 - Learning-based object detection and localization for a mobile robot manipulator in SME production

AU - Zhou, Zhengxue

AU - Li, Leihui

AU - Fürsterling, Alexander

AU - Durocher, Hjalte

AU - Mouridsen, Jesper

AU - Zhang, Xuping

PY - 2022/2

Y1 - 2022/2

N2 - Increasing research attention has been attracted to automatic production processes in small and medium-sized enterprises (SMEs) using collaborative robotic systems. In this work, we develop an object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production. In this solution, a 3D point cloud technology is adopted to measure the shape and depth information of targeted objects in SME production, for instance, name tags production and plug-in charging. Deep learning is then employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions and the irregular distribution and structural ambiguity of point clouds. A 2D camera is employed to calibrate the relative positions of the mobile manipulator to workstations. The mobile robot manipulator is equipped with cameras, an in-house developed adaptive gripper, and a learning-based computer vision system developed in this work. The principle and procedures of the proposed 3D object detection and 2D calibration are presented in detail. The automatic name tags production and plug-in charging experiments are conducted to validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulator.

AB - Increasing research attention has been attracted to automatic production processes in small and medium-sized enterprises (SMEs) using collaborative robotic systems. In this work, we develop an object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production. In this solution, a 3D point cloud technology is adopted to measure the shape and depth information of targeted objects in SME production, for instance, name tags production and plug-in charging. Deep learning is then employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions and the irregular distribution and structural ambiguity of point clouds. A 2D camera is employed to calibrate the relative positions of the mobile manipulator to workstations. The mobile robot manipulator is equipped with cameras, an in-house developed adaptive gripper, and a learning-based computer vision system developed in this work. The principle and procedures of the proposed 3D object detection and 2D calibration are presented in detail. The automatic name tags production and plug-in charging experiments are conducted to validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulator.

KW - Localization

KW - Mobile manipulator

KW - Object detection

KW - SME production

KW - STEREO VISION

U2 - 10.1016/j.rcim.2021.102229

DO - 10.1016/j.rcim.2021.102229

M3 - Journal article

SN - 0736-5845

VL - 73

JO - Robotics and Computer-Integrated Manufacturing

JF - Robotics and Computer-Integrated Manufacturing

M1 - 102229

ER -