Digital Twin-empowered Model-Mediated Teleoperation using Multimodality Data with Signed Distance Fields

TY - GEN

T1 - Digital Twin-empowered Model-Mediated Teleoperation using Multimodality Data with Signed Distance Fields

AU - Antonsen, Mads Mørch

AU - Liu, Siwen

AU - Xu, Xiao

AU - Steinbach, Eckehard

AU - Chinello, Francesco

AU - Zhang, Qi

PY - 2024/5

Y1 - 2024/5

N2 - In human-in-the-loop teleoperation systems, the Quality of Task Perception (QoTP) plays an important role, significantly influencing the overall Quality of Experience (QoE). QoTP characterizes the depth of understanding a user possesses regarding remote environments and tasks. To enhance this comprehension, this paper introduces an innovative geometric modeling approach employing Truncated Signed Distance Fields (TSDF) tailored for Model-Mediated Teleoperation (MMT) systems. Our approach combines haptic and visual cues to provide real-time updates to environmental changes. To ensure optimal responsiveness, we devise a model updating scheme that achieves an effective update frequency. Capitalizing on the capabilities of a Digital Twin, the MMT system presented not only uses the existing knowledge of the remote environment but also records its dynamic alterations. Through comprehensive experimentation evaluating both the geometric modeling and the teleoperation framework, our findings underscore the robustness and applicability of the introduced geometric modeling technique.

AB - In human-in-the-loop teleoperation systems, the Quality of Task Perception (QoTP) plays an important role, significantly influencing the overall Quality of Experience (QoE). QoTP characterizes the depth of understanding a user possesses regarding remote environments and tasks. To enhance this comprehension, this paper introduces an innovative geometric modeling approach employing Truncated Signed Distance Fields (TSDF) tailored for Model-Mediated Teleoperation (MMT) systems. Our approach combines haptic and visual cues to provide real-time updates to environmental changes. To ensure optimal responsiveness, we devise a model updating scheme that achieves an effective update frequency. Capitalizing on the capabilities of a Digital Twin, the MMT system presented not only uses the existing knowledge of the remote environment but also records its dynamic alterations. Through comprehensive experimentation evaluating both the geometric modeling and the teleoperation framework, our findings underscore the robustness and applicability of the introduced geometric modeling technique.

UR - http://www.scopus.com/inward/record.url?scp=85193482597&partnerID=8YFLogxK

U2 - 10.1109/HAPTICS59260.2024.10520841

DO - 10.1109/HAPTICS59260.2024.10520841

M3 - Article in proceedings

SN - 979-8-3503-4512-4

T3 - International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS)

SP - 353

EP - 359

BT - 2024 IEEE Haptics Symposium, HAPTICS 2024

PB - IEEE

ER -