Autonomy and responsibility debate puts system safeguards under the spotlight

Automated vehicle technology is intended to make driving safer, with driverless cars held up by some as the pinnacle of road safety. However, the rush to put new tech on the market may create additional safety risks, both from the system performance perspective as well as potential driver confusion or misuse. Tesla, the industry's poster child for innovative and automated technology, falls under scrutiny once again with its Autopilot system. On the same day that the US Department of Transportation (DoT) released its updated guidance on automated driving technology, the National Transportation Safety Board (NTSB) issued a stark warning. It has been examining the factors behind the fatal crash of a Tesla Model S driver in Florida back in May 2016. The collision occurred when the Model S ran into a semi-tractor trailer that failed to yield when crossing a divided highway. The National Highway Traffic Safety Administration (NHTSA) had previously investigated the same crash, but the NTSB tends to take an interest when commercial vehicles are involved. In both cases, the question was the role that Autopilot played in the crash. NTSB verdict The NTSB verdict shares out the blame between the truck driver, for failure to yield the right of way, and the Model S driver, whose inattention stemmed from an over-reliance on vehicle automation. Notably, the NTSB also blames Tesla's design, noting that the operational design allowed for the driver's over-reliance on the automation. Specifically, it allowed for "prolonged disengagement from the driving task and enabled the driver to use it in ways inconsistent with manufacturer guidance and warnings." Tesla itself has been careful to emphasise that Autopilot is no replacement for active engagement in the driving experience and that drivers need to remain alert. "System safeguards, that should have prevented the Tesla’s driver from using the car’s automation system on certain roadways, were lacking and the combined effects of human error and the lack of sufficient system safeguards resulted in a fatal collision that should not have happened," commented NTSB Chairman Robert Sumwalt. The NTSB has now issued a handful of safety recommendations to DoT, NHTSA, the manufacturers of vehicles equipped with Level 2 vehicle automation systems, the Alliance of Automobile Manufacturers and the Global Automakers. The recommendations address the need for event data to be captured and available in standard formats on new vehicles equipped with automated systems; manufacturers to incorporate system safeguards to limit the use of automated control systems to conditions for which they are designed and for there to be a method to verify those safeguards; development of applications to more effectively sense a driver’s level of engagement and alert when engagement is lacking; and for manufacturers to report incidents, crashes, and exposure numbers involving vehicles equipped with automated vehicle control systems. Any vehicle can be misused Tesla has long argued that any vehicle can be misused and that it is safer to use Autopilot than not, provided it is used correctly. It has also remained firm that it will not delay the rollout of features that have been shown to improve safety. "At Tesla, the safety of our customers comes first, and one thing is very clear: Autopilot significantly increases safety, as NHTSA has found that it reduces accident rates by 40%," the company told Automotive World. "We appreciate the NTSB’s analysis of last year’s tragic accident and we will evaluate their recommendations as we continue to evolve our technology. We will also continue to be extremely clear with current and potential customers that Autopilot is not a fully self-driving technology and drivers need to remain attentive at all times.” NHTSA’s report on the same incident concluded that “the data show that the Tesla vehicles crash rate dropped by almost 40% after Autosteer installation." NHTSA had also confirmed that it "did not identify any defects in the design or performance of the AEB or Autopilot systems" nor did it find "any incidents in which the systems did not perform as designed." It further backs up the OEM by noting how "over the course of researching and developing Autopilot, Tesla considered the possibility that drivers could misuse the system in a variety of ways," and that "the potential for driver misuse was evaluated as part of Tesla’s design process and solutions were tested, validated, and incorporated into the wide release of the product." As for the accusation that it lacked safeguards to prevent Autopilot from being misused, Tesla claims there are several in place. Before enabling Autopilot, the driver first needs to agree to keep his hands on the steering wheel at all times and to always maintain control and responsibility for the vehicle. When the driver activates Autopilot, he is shown a visual message reminding him to “keep your hands on the wheel” and “be prepared to take over at any time.” When the system is in use, it continues to measure the amount of torque that the driver applies to the steering wheel and if insufficient torque is applied, it delivers an escalating series of visual and audio warnings again reminding the driver to place his hands on the wheel. If the driver repeatedly ignore those messages, then the driver will be locked out from using Autopilot during that trip. A wake-up call Paul Brubaker, Chief Executive and President, Alliance for Transportation Innovation, describes the Tesla crash as "a wake-up call to a lot of folks in the industry that you need to be mindful of expressing the limitations of the technology and warning the drivers. To a certain extent it is also the responsibility of the driver to know what he is buying." While Brubaker is a keen advocate of a light touch when it comes to regulations, he also sees merit to the NTSB's warnings. "They made some conscientious fact-based observations and recommendations. The issue we have now is that this technology is moving at such a rapid pace they have to be careful about recommending any specific technological approach with potential crash avoidance technology that can be employed," he commented. "The suggestion that the OEMs need to be sensitive to limitations to the technology was spot on, and they are sensitive to that." As for crashes of vehicles with highly automated features, the industry is guaranteed to see more of these down the line. "It is not so much that drivers are testing the limits of the technology, it could be other factors involved. This will only help us inform the feedback loop and make the technology better," he explained. "I'm not suggesting we are out there experimenting with people's lives, that's not the case. We are trying to improve safety for everybody. We are losing 100 people every day in the US on the roads. These technologies will undoubtedly improve the safety results if they can be accelerated towards deployment."

Autonomy and responsibility debate puts system safeguards under the spotlight

The NTSB has criticised Tesla for a lack of system safeguards on its Autopilot system, which it believe could have prevented a fatal crash. By Megan Lampinen

M:bility | Magazine – Q4 2018

Special report: Will Mobility as a Service become the new normal?

Special report: Is this the end of the diesel engine?

Strategy update: Renault Nissan Mitsubishi Alliance – 2018 edition

How to efficiently test and calibrate xEVs

M:bility | Detroit

M:bility | Europe

M:bility | California

M:bility | India

Monthly Briefing – September 2018

M:bility | Detroit

Grasp the future of mobility

12-13 March 2019
Dearborn, Michigan

M:bility | Europe

Grasp the future of mobility

9-10 July 2019
Stuttgart, Germany

M:bility | California

Grasp the future of mobility

17-18 September 2019
Santa Clara, CA

M:bility | India

Grasp the future of mobility

26-27 November 2019
Pune, India

It’s time to log in (or subscribe).

Not a member? Subscribe now and let us help you understand the future of mobility.

Single-User Site License

Team Site License

Enterprise Site License

Email Alerts

Related Content

Log In

Single-User
Site License

Team
Site License

Enterprise
Site License