Learning-based object detection and localization for a mobile robot manipulator in SME production

Zhengxue Zhou, Leihui Li, Alexander Fürsterling, Hjalte Durocher, Jesper Mouridsen, Xuping Zhang*

*Corresponding author for this work

Research output: Contribution to journal/Conference contribution in journal/Contribution to newspaperJournal articleResearchpeer-review

47 Downloads (Pure)


Increasing research attention has been attracted to automatic production processes in small and medium-sized enterprises (SMEs) using collaborative robotic systems. In this work, we develop an object detection solution based on deep learning on 3D point clouds for a collaborative mobile robot manipulator to automate SME production. In this solution, a 3D point cloud technology is adopted to measure the shape and depth information of targeted objects in SME production, for instance, name tags production and plug-in charging. Deep learning is then employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions and the irregular distribution and structural ambiguity of point clouds. A 2D camera is employed to calibrate the relative positions of the mobile manipulator to workstations. The mobile robot manipulator is equipped with cameras, an in-house developed adaptive gripper, and a learning-based computer vision system developed in this work. The principle and procedures of the proposed 3D object detection and 2D calibration are presented in detail. The automatic name tags production and plug-in charging experiments are conducted to validate the object detection, localization algorithms, and tools developed and employed in production cases using the mobile robot manipulator.

Original languageEnglish
Article number102229
JournalRobotics and Computer-Integrated Manufacturing
Number of pages12
Publication statusPublished - Feb 2022


  • Localization
  • Mobile manipulator
  • Object detection
  • SME production


Dive into the research topics of 'Learning-based object detection and localization for a mobile robot manipulator in SME production'. Together they form a unique fingerprint.

Cite this