Poised to race to outer space, SE4’s unique approach eliminates latency challenges in teleoperation; VR actions in 3D simulation can be sent to multiple robotic devices in one process

Los Angeles, Calif., July 30, 2019, SIGGRAPH - SE4 Inc., a company specializing in remote robotics software for high latency environments, is introducing a robot operating system which combines virtual reality (VR), Computer Vision (CV) and artificial intelligence (AI) software to enable highly accelerated remote robotic control. It also provides adaptive operational capability for dynamically changing environments at any distance. The company is initially targeting repetitive tasks such as excavation and construction, with a view towards deployment in robots in outer space, as their software sidesteps the speed of light constraints that cause latency issues.

SE4's innovative technology, called Semantic Control™, eliminates the need for tedious and complex programming steps in controlling robots - such as which joint to move, where... - and removes the need for the involvement of a vast range of programming teams for installation and continued operation. Instead, operators use VR in a safe, simulated 3D environment, where they can "annotate" objects and perform actions on them. AI interprets the actions, and the tasks are organized and sent to the robot as a series of tasks, like a to-do list. At that point the robot itself can determine the best way, and order, to accomplish each task using local information, adapting intelligently to changes in its workspace.
"The majority of robot teaching programs use 2D methods - such as a touchscreen pendant - to interact with 3D environments," said Pavel Savkin, chief technology officer, SE4 Inc. "We control objects in 3D space using VR, which is far more efficient and eliminates the learning curve. It also increases robot and operational safety because tasks are sanity checked first by the simulator. They have to make physical sense before they can be executed."
SE4's unique approach manages the significant problem of latency in the instruction process. By uploading a directive to the robot and providing it with operational flexibility, there is no need for a stream of constant instructions and the resulting delays which in outer space can take 12.5 minutes (on average) one way from Mars to Earth, making direct and real-time control impossible.
"Imagine a robotic construction project on Mars right now - it is like King George telling the colonists in the New World how to build a single structure with a one-month sea delay each way. That's the status of where we are now," said Savkin. "SE4's solution delivers instructions as a to-do list, with dependencies included for flexibility in execution, so that latency becomes manageable. A general directive can go out to a single robot, or to many devices for simultaneous control, using command queueing and automated delegation."
The software is targeted at both new and existing robots equipped with the necessary sensor technology and is designed to make robotics far more useful in a wider variety of settings, including areas of danger or high latency, or inhospitable environments. SE4 will supply its software to companies engaged in repetitive terrestrial work such as construction, where autonomous diggers can be told via a virtual environment which sector to excavate, to what depth, and where to put the excavated earth. Once the robot has received its instructions, the operator can walk away while the system uses computer vision, machine learning and a semantic structure to adapt to environmental changes, for example if the digger were to come across a boulder it might contact the operator for additional guidance.
SE4 will demonstrate its one-of-a-kind robot operating system at SIGGRAPH's Real-Time Live! July 30, 2019.

About SE4, Inc.
SE4 is a robotics company specializing in remote robotics using VR simulation to enable remote operation at vast distances, as well as nearby. SE4's vision is to effectively enable an off-world robot-driven industrial revolution to increase the prosperity and long-term survivability of humanity. The company is part of the NVIDIA Inception accelerator, which supports AI start-ups with tools and expertise. Based in Tokyo, SE4 has received initial investment from Mistletoe Inc. since its founding in 2018. For information, visit se4.space.