System trains AVs before they hit the road

A simulation system to train autonomous vehicles (AVs) to navigate various worse-case scenarios before going on the road has been invented at the Massachusetts Institute of Technology (MIT).

The simulator creates a photorealistic world, dubbed Virtual Image Synthesis and Transformation for Autonomy (VISTA).

Control systems for AVs generally rely on real-world datasets of driving trajectories from human drivers, using the data to learn how to emulate safe steering controls in a variety of situations. However, real-world data from hazardous situations, such as nearly crashing or being forced off the road, are rare.

Some computer programs or “simulation engines” aim to imitate these situations by rendering detailed virtual roads to help train the controllers to recover, but the learned control from simulation has never been shown to transfer to reality on a full-scale vehicle, MIT’s Rob Matheson says.

VISTA uses only a small dataset, captured by humans driving on a road, to synthesise a practically infinite number of new viewpoints from trajectories that the vehicle could take in the real world, he says.

The controller is rewarded for the distance it travels without crashing, so it must learn by itself how to reach a destination safely. In doing so, the vehicle learns to safely navigate any situation it encounters, including regaining control after swerving between lanes or recovering from near crashes.

In tests, a controller trained within the VISTA simulator safely was able to be safely deployed onto a full-scale driverless car and to navigate through previously unseen streets.

A paper describing the system has been published in IEEE Robotics and Automation Letters and will be presented at the 2020 International Conference on Robotics and Automation (ICRA) in Paris on May 31 to June 4, presuming COVID-19 restrictions will be lifted by then.