Microsoft project to teach robots to see and navigate with the help of simulation

Within the capabilities assigned to robots and autonomous systems for their operation, those associated with seeing and navigating are considered a critical operational requirement.

Both qualities would be essential for an autonomous rescue robot in order to enable it to safely maneuver and move through spaces difficult to access by humans.

This is also the case when developing artificial intelligence agents, which, in order to provide them with a capacity for perception and safe and efficient action, it is necessary to first start with the elaboration of a well-structured engineering, so that a robust environment is generated. where possible the development of both qualities.

However, programming a real-world stand-alone system that can safely scale to scale is a complex task to perform.

This is how in this instance the association between Microsoft Research and Carnegie Mellon University, previously announced in April 2019, with the aim of continuing to generate advances focused on solving real-world challenges in areas such as navigation, autonomous mapping, and monitoring of underground urban and industrial environments.

Considered as part of the fundamental capabilities of robots, the SLAM (Simultaneous Localization and Mapping) is a technique that allows these generate the map of an environment unknown and calculate its trajectory when moving within it.

This technique has made it possible to obtain notable progress and advances, both in methods based on geometry, and in those based on learning.

However, when it comes to generating a robust and reliable SLAM system to be implemented in real world scenarios, the objective becomes a real challenge taking into account the different factors that affect the environment such as changes in light, dynamic objects , scenes without texture, and lack of lighting.

TartanAir: Dataset at the service of robots

And it is that part of the achievement previously obtained with the SLAM has come from geometric approaches.

The fact that a large volume of training data is available, the product of a wide variety of conditions, contributes to the fact that the limits of data-based techniques and algorithms can be crossed.

To solve the obstacles that hinder the performance of SLAM within a real environment, the Microsoft Research team and Carnegie Mellon University developed the TartanAir, a comprehensive data set designed to allow robots to perform navigation tasks, among other activities.

To obtain the TartanAir data set, we proceeded to use photorealistic simulation environments supported by AirSim in which various light and weather conditions were manifested, as well as moving objects.

It should be noted that the work and progress obtained by the members of both teams on the dataset was taken into account to be presented at the International Conference on Robots and Intelligent Systems (IROS 2020) of the IEEE / RSJ.

During the data collection process in the simulation, we can capture information that comes from multimodal sensors, as well as precise labels on the ground, stereo RGB image, segmentation, depth image, optical flow, LiDAR point cloud and the poses of the cameras.

This is how the TartanAir stores a large number of environments with various styles and scenes, encompassing challenging viewpoints and different movement patterns, which can be difficult to record using physical data collection platforms.

Taking into account the set of data present in the TartanAir, Microsoft has decided to organize a visual SLAM challenge whose starting point is the Computer Vision and Pattern Recognition Workshop (CVPR) 2020, which is formed by a monocular track and a stereo track.

Each of the tracks presents a total of 16 trajectories with challenging characteristics designed to cross the limits of the visual SLAM algorithms in which the participants will have the mission of finding the robot and mapping the environment starting from a sequence of monocular / stereo images. . It should be noted that participants will have until August 15 of this year as the deadline to submit entries to the challenge.