Computer Science editorial
Open AccessOA2026
Touch2Robot: Robot Touch in the Human Demonstration Loop
A framework for scalable dexterous data collection via tactile-aware human demonstrations
Shengcheng Luo; Xiaoyang Cheng; Hong Ying; Xiaoying Zhou; Jiaming Jiang; Haoran Guo; Wanlin Li; Ziyuan Jiao; Chenxi Xiao· 2026· DOI 10.48550/arXiv.2609.24660
The core problem
Human demonstrations are a scalable source of manipulation data, but transferring human contacts to a robot hand often introduces instability or infeasibility due to morphological mismatch. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases data collection cost. Touch2Robot addresses this trade-off by letting humans collect demonstrations while observing how the target robot hand would contact the object. The framework captures human hand motion, tactile-glove measurements, and object motion during manipulation. These recordings guide object-specific reinforcement learning (RL) policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. The learned behaviors are distilled into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2R
Innovation
The framework was evaluated on four real-world tasks. Key quantitative results are summarized below.
**Real-Robot Replay Completion.** Touch2Robot improves average real-robot replay completion from 37.9% (visual-only feedback) to 72.1%, a relative improvement of 90.2%.
**Collection Efficiency.** The collection time per replay-successful demonstration is reduced from 58.6s to 18.2s, a 3.2× speedup.
**Contact Reconstruction Accuracy.** Reconstructed target-hand contacts achieve 44.2% against real-robot tactile measurements.
**Downstream Policy Performance.** Policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback.
These results demonstrate that incorporating robot touch into the human demonstration loop significantly enhances both the quality and efficiency of scalable dexterous data collection.
Human demonstrations are a scalable source of manipulation data, but transferring human contacts to a robot hand often introduces instability or infeasibility due to morphological mismatch. Collecting demonstrations directly on the target robot avoids this mismatch but substantially increases data collection cost. Touch2Robot addresses this trade-off by letting humans collect demonstrations while observing how the target robot hand would contact the object. The framework captures human hand motion, tactile-glove measurements, and object motion during manipulation. These recordings guide object-specific reinforcement learning (RL) policies to reproduce the demonstrated object motion while favoring contacts consistent with the recorded human touch. The learned behaviors are distilled into a unified real-time retargeter that maps incoming human observations and object geometry to robot hand configurations. During collection, the predicted robot configuration is synchronized with the tracked object pose in simulation to reconstruct robot-object contacts, which are visualized to help the demonstrator adapt subsequent interactions to the target hand. Across four real-world tasks, Touch2Robot improves average real-robot replay completion from 37.9% to 72.1% over visual-only feedback, while reducing the collection time per replay-successful demonstration from 58.6s to 18.2s. Reconstructed target-hand contacts achieve 44.2% against real-robot tactile measurements, and policies trained on Touch2Robot demonstrations improve downstream Diffusion Policy performance by 29.1 percentage points over visual-only feedback. These results show that bringing robot touch into the human demonstration loop improves both the quality and efficiency of scalable dexterous data collection.
Touch2Robot comprises three main components: (1) multimodal human demonstration capture, (2) object-specific RL policy learning with tactile guidance, and (3) real-time retargeting and contact visualization.
Why it matters
The results highlight the importance of tactile feedback in human-to-robot skill transfer. By visualizing predicted robot-object contacts during demonstration, Touch2Robot enables demonstrators to adapt their interactions to the target hand's morphology, reducing the mismatch that typically plagues visual-only approaches. The substantial improvement in replay completion (37.9% to 72.1%) and the 3.2× reduction in collection time per successful demonstration indicate that tactile-aware feedback not only improves data quality but also accelerates the collection process. The 44.2% for contact reconstruction suggests that the retargeter and simulation-based contact prediction are reasonably accurate, though there is room for improvement. The 29.1 percentage point gain in downstream Diffusion Policy performance underscores the value of high-quality, tactile-consistent demonstrations for learning dexterous manipulation. A limitation is that the approach relies on a tactile glove and simulation, which may not perfectly capture real-world contact dynamics. Future work could explore more accurate tactile sensing and sim-to-real transfer. Overall, Touch2Robot demonstrates that bringing robot touch into the human demonstration loop is a promising direction for scalable dexterous data collection.
Who should read this
CS practitioners and researchers
Opening member content…