Co-Learning of Task and Sensor Placement for Soft Robotics

 

Unlike rigid robots which operate with compact degrees of freedom, soft robots must reason about an infinite dimensional state space. Mapping this continuum state space presents significant challenges, especially when working with a finite set of discrete sensors. Reconstructing the robot’s state from these sparse inputs is challenging, especially since sensor location has a profound downstream impact on the richness of learned models for robotic tasks. In this work, we present a novel representation for co-learning sensor placement and complex tasks. Specifically, we present a neural architecture which processes on-board sensor information to learn a salient and sparse selection of placements for optimal task performance. We evaluate our model and learning algorithm on six soft robot morphologies for various supervised learning tasks, including tactile sensing and proprioception. We also highlight applications to soft robot motion subspace visualization and control. Our method demonstrates superior performance in task learning to algorithmic and human baselines while also learning sensor placements and latent spaces that are semantically meaningful.

 

Authors: Andrew Spielberg*, Alexander Amini*, Lillian Chin, Wojciech Matusik, and Daniela Rus Published in: IEEE Robotics and Automation Letters (RA-L), with presentation in RoboSoft 2021.

 

Full Video:

Featured Product

3MP HDR IP69K Camera for Robotics & Autonomous Vehicles

3MP HDR IP69K Camera for Robotics & Autonomous Vehicles

STURDeCAM31 from e-con Systems® is designed to make robotics and autonomous vehicles safer and more reliable. Powered by the Sony® ISX031 sensor and featuring GMSL2 interface, this compact 3MP camera delivers 120dB HDR + LFM imaging with zero motion blur — even in the most challenging outdoor conditions. Engineered to automotive-grade standards, STURDeCAM31 is IP69K certified, making it resistant to dust, water, vibration, and extreme temperatures. With support for up to 8 synchronized cameras, it enables powerful surround-view and bird's eye systems on NVIDIA® Jetson AGX Orin™.