L104 RemovingHumanDriver

Implications of
Removing the
Human Driver
https://goo.gl/YUC5oU
© 2021 Philip Koopman 34

See: https://users.ece.cmu.edu/~koopman/toyota/index.html © 2021 Philip Koopman 35
https://bit.ly/2m9cQRm
Huang fatality;
crash into
concrete median.
Can Humans Safely Supervise Autonomy?
https://goo.gl/ZFCYzD
https://goo.gl/VTFW9d
https://goo.gl/kgRq71
The 94% Human Error False Narrative
 Where did “94%” come from? “94%”
 “The critical reason was
assigned to drivers in an estimated
2,046,000 crashes that comprise
94 percent of the NMVCCS crashes
at the national level.
However, in none of these cases was
the assignment intended to blame
the driver for causing the crash.”
[DOT HS 812 115]
 Looking a little deeper:
https://www.nhtsa.gov/technology-
 74% of driver errors were “recognition” or “decision” errors

innovation/automated-vehicles-safety
 And software driver must handle the 6% of no-driver-involvement crash causes

– Tires, brakes, drivetrain failures
Humans Are Amazing Fault Mitigators
 Other side of the “94%” coin – people prevent crashes too
 Toyota uncommanded acceleration – most saved by human

 89 deaths, 57 injuries as of May 2010
 6,200+ NHTSA complaints [https://www.cbsnews.com/news/toyota-unintended-acceleration-has-killed-89/]
 GM brake issues – most saved by human

 293 injuries, 2111 crashes
 10,861 NHTSA complaints https://www.nytimes.com/1999/07/22/us/gm-admits-brake-flaws-after-inquiry.html
 Will an ADS be as successful at fault mitigation as humans?

 ADS will need to deal with heavy-tail issues © 2021 Philip Koopman 40
Automotive Software Quality Issues
Software
defects &
}
software
integration
problems
Source: Stout 2020 Automotive Defect & Recall Report

IEC is integrated electronic components (hardware) © 2021 Philip Koopman 41
Automotive Software Has Defects
 Small sampling of NHTSA recalls (i.e., confirmed bugs)
 See: https://betterembsw.blogspot.com/p/potentially-deadly-automotive-software.html
 21V-071 Vehicle unexpected pulls to one side during evasive maneuver
 20V-213 Remote smart park continued motion after failsafe activation
 19E-070 Anti-rollback software causes unexpected vehicle motion
 19V-539 Forward collision avoidance does not detect stationary vehicle
 19V-351 Regenerative braking failure reduces deceleration
 19V-075 Transmission unexpected downshift to first gear causes loss of control
 18V-621 Automatic braking cancelled / ABS locks up wheels
 18V-607 Active Lane Keeping Assist does not intervene in lane departure
 17V-713: Engine does not reduce power due to ESP software defect
 17V-686 and MANY others: Airbags disabled
 15V-460 and others: Airbags deploy when they should not
42
© 2021 Philip Koopman
Example Required ADS Fault Handling
 Tire blowout/wheel detachment
 ADS: perform controlled stop (or run-flat tire operations)
 Service brake failure

 ADS: downshift/regen braking, apply parking brake, runaway ramp
 Catastrophic sensor failure

 ADS: dead reckon to stop using most recent object trajectories
 Uncommanded acceleration
 ADS: de-energize engine/motors, apply forceful brakes
 Main battery fire

 ADS: shed electrical load, stop vehicle, passenger evacuation
Controllability Without A Human Driver
 What happens when there is no
??
human to exert controllability?
 Own vehicle human driver?
 Other vehicle human driver?
 Some combination of:

 ADS will need to control faults to
attain C1 or C2
 Vehicle will have to upgrade
subsystems to C3 (“uncontrollable”)  ISO 26262 Driver Controllability:
 C1 = Simply controllable
 Potential for significant ASIL
 C2 = Normally controllable
increase across whole vehicle
 C3 = Difficult / uncontrollable
 Many ADS control requirements
No Human Driver to Blame
 “Computers won’t drive drunk” .. but …
 Drunk/DUI is only 28% of fatalities (US 2019)
[https://crashstats.nhtsa.dot.gov/Api/Public/Publication/813060]
 Automated Driving Systems (ADS) will

likely make different mistakes
– Perception/classification errors
– Brittle in face of surprises (unknown unknowns)
 What happens with ADS “driver error”?

 Every AV crash is a product liability lawsuit
waiting to happen
 Eventually, no human driver to absorb blame
– What about Driver monitor system (DMS) failures?
Operations & Human Interactions
 Drivers do more than just drive
 Occupant behavior, passenger safety
 Detecting and managing equipment faults
 Operational limitations & situations

 System exits Operational Design Domain https://bit.ly/2GvDkUN
 Vehicle fire or catastrophic failure

 Post-crash response
 Interacting with non-drivers

 Pedestrians, passengers
 Police, emergency responders https://bit.ly/2PhzilT

Lifecycle Issues
 Handling updates https://bit.ly/2IKlZJ9
 Fully recertify after

every weekly update?
 Security in general
 Vehicle maintenance
 Pre-flight checks, cleaning
 Corrective maintenance
 Supply chain issues

 Quality fade https://bit.ly/2VavsjM
 Supply chain faults Is windshield cleaning fluid life critical?

Changing Role of
Human Driver
 No human driver to blame for crashes
 ADS handles vehicle equipment failures
 ADS handles non-ADS software failures

L104 RemovingHumanDriver

Uploaded by

Document Informationclick to expand document information

Document Informationclick to expand document information

Copyright:

Available Formats

L104 RemovingHumanDriver

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

L104 RemovingHumanDriver

Uploaded by

Copyright:

Available Formats

Implications of

© 2021 Philip Koopman 34

 74% of driver errors were “recognition” or “decision” errors

 And software driver must handle the 6% of no-driver-involvement crash causes

 Toyota uncommanded acceleration – most saved by human

 GM brake issues – most saved by human

 Will an ADS be as successful at fault mitigation as humans?

Source: Stout 2020 Automotive Defect & Recall Report

 Service brake failure

 Catastrophic sensor failure

 Main battery fire

 Some combination of:

 Automated Driving Systems (ADS) will

 What happens with ADS “driver error”?

 Operational limitations & situations

 Vehicle fire or catastrophic failure

 Interacting with non-drivers

© 2021 Philip Koopman 47

 Fully recertify after

 Supply chain issues

 Supply chain faults Is windshield cleaning fluid life critical?

© 2021 Philip Koopman 49

You might also like