Objective safety rating of autonomous vehicles coming closer

Although cars with advanced driver-assistance systems are already on sale, it will be years before vehicles drive around fully autonomously, expects De Gelder. Over the past few years, he has worked on several research projects in the field of self-driving vehicles, both on the technology of the vehicles themselves and on testing methods.

‘To be clear, self-driving vehicles have been around for quite some time, but in current autonomous vehicles, humans are still responsible for the task of driving. The extent to which a vehicle can drive autonomously is classified with the levels 0 to 5. Up to level 3, the driver is assisted with, for example, braking, accelerating, or steering. Ultimate responsibility, however, always remains with the human driver. From level 3 onwards, responsibility lies with the vehicle. It must therefore be able to independently solve a lot of situations. And that is precisely where the problem lies.’

Erwin de Gelder presenting his research on the use of realistic driving scenarios in the safety validation of autonomous vehicles.

Problem-solving autonomous vehicle

A bicycle courier zigzagging through traffic, a pedestrian suddenly stopping, a dog crossing the road. Anyone who has ever driven in a busy city knows how great the problem-solving ability of an autonomous driving vehicle must be. ‘There are millions of potential traffic situations that can arise. So, how can we know in advance whether a vehicle can handle them well?’

Erwin de Gelder soon discovered that the current way of testing is far too limited, as practical tests and accident statistics offer too few driving situations. ‘We therefore explored how we can use data from multiple fleets of vehicles to collect many more situations.’

Traffic scenarios from real driving data

The StreetWise methodology, developed by TNO, distils scenarios from real driving data. These scenarios are then used to develop tests for simulation software or on a test track. Erwin de Gelder: ‘A significant additional advantage is that this methodology also allows us to identify unfamiliar situations. Because we still don’t know a lot of situations. Furthermore, every country where we have not yet tested has different infrastructure and specific challenges. Using local data, you can use this methodology anywhere in the world.’

Objective safety assessment of autonomous vehicles

TNO was challenged to not only validate the safety of autonomous vehicles as thoroughly as possible, but also to quantify their safety as objectively as possible. ‘Until now, there has always been a degree of subjectivity when testing safety systems. The person assessing makes his or her own assessment of the risks. This works fine with small systems like seatbelts and airbags. But the complexity of self-driving vehicles calls for a neutral, quantitative safety rating.’

‘With the StreetWise methodology, we will soon be able to express in a single figure how safe a self-driving vehicle is. This will also allow one to make comparisons with accident statistics. Essential for the industry, but also for policymakers, legislators, and inspection bodies.’

Autonomous trucks become StreetWise

TNO’s results with StreetWise did not go unnoticed for long. ‘In discussions with the business community and public authorities, we kept getting confirmation of how important such an objective safety validation is.’ One leading party that quickly expressed an interest in working together was Torc Robotics. The US subsidiary of Daimler Trucks has over 17 years’ experience in self-driving vehicles and is considered one of the sector’s pioneers.

Torc is currently focusing on commercialising autonomous long-haul trucks in the US. ‘Torc wants to use the StreetWise methodology to determine whether their technology for autonomous trucks is safe enough. Therefore, we’re working together to implement our software in Torc’s fleet operation. They use the software to analyse the daily stream of data from the test fleet. This gives their engineers new insights and teaches us a lot as well. We want to further improve our capacity to map out all scenarios and further reduce uncertainties.’

Research on legislation and standardisation for autonomous driving

According to Erwin de Gelder, the partnership with Torc confirms how valuable this research is. ‘Autonomous driving can make our traffic and transport more efficient, more sustainable, and safer. But that does require legislation and standardisation within the industry. I see this research as a typical task for an independent research institute like TNO. We’re in between the business community and the government. Businesses always have a commercial interest, which makes it more difficult for them to remain impartial. TNO’s independent position makes it uniquely suitable to prove whether a vehicle is truly safe enough. That research serves a larger societal purpose.’

According to De Gelder, the research is far from finished. ‘Together with Torc and other potential partners, we want to gain even more knowledge and experience to ultimately be able to determine the safety of autonomous vehicles as accurately and objectively as possible. That’s our mission.’