Autonomous Vehicles and Scaling Risk

I want to see mainstream autonomous vehicles (AVs), but I remain bearish about mainstream high-level autonomy.

My reasons for bearishness are not related to the technology that powers self-driving cars, or demand for AVs. Instead, it’s the systemic risk that wide-scale AV deployments create. What will change when we have numerous fully autonomous cars on the road?

Most of the commentary about AVs doesn’t consider second-order effects in their deployment so I’d like to start a discussion on that. (In this post I take high-level autonomy as a technological eventuality and assume a political climate that supports that.)

• Level 4: High Automation. Automated, but geographically restricted, dynamic driving even if a human driver fails to respond.
• Level 5: Full Automation. Unconditional, full-timer performance of all dynamic driving needs.

Many of the arguments for AVs are centered around safety and optimization. Those arguments are hard to disagree with if you are comfortable with change (again setting the tech and political issues aside). First, let’s consider safety. About 3,000 people die per month in the US from traffic accidents (a broader metric than just automotive deaths, but representative here). That 3,000 is a lot, and a number that the AV safety argument draws on, though the fatalities are lower per capita than most other countries. Still, how did we get to the number 3,000 in the US?

If we look at how automotive deaths have trended over the past 100 years, driving has become much safer, especially as measured by deaths per billion vehicle miles traveled (VMT). Chief causes of improvement: better roads, regulations, Ralph Nader-driven safety measures to the vehicles themselves, and fewer vehicle mismatches like fast cars and slow horses sharing the same road.

Those safety and infrastructure measures are not present equally around the world. In some places, traffic deaths are caused more by bad roads and lack of nighttime lighting than anything else. The cars and drivers can only do so much.

As a thought experiment, instead of decreasing, what would it take to increase traffic deaths per capita? With traditional cars, forcing automotive deaths to increase would be nearly impossible. It would take a strange coordinated, long-term purposeful political and corporate attack to eliminate safety regulations like speed limits, seat belts, air bags, car crumple zones, driver intoxication limits, driving age limits, budgets for road repair and clearing, signage, lighting, traffic monitoring, rapid post-crash response, and more. It would in theory be possible to lower automotive safety through those top-down means. But it would not be possible at scale to force or trick individual drivers into killing each other.

Individual, non-scaled shocks to automobile precursors like horses were also common. Many early transportation accidents were caused when horses became spooked, either by other horses or from encountering passing trains, cars, or the occasional marching band.

It’s interesting to look at the safety of horse-drawn buggies today. The nature of the accidents has changed. Today, the anachronous but legal drivers of buggies mostly suffer from accidents when hit by speeding cars and trucks. Many of those crashes today are caused by mismatched vehicle size.

Again, while horses and horse-pulled vehicles were and are less safe than automobiles today, since every horse-driven vehicle required a sentient being to control a sentient being, limits on systemic risk also held. No one can cause horses to create accidents at scale.

The safety argument claims that with AVs the 3,000 monthly traffic deaths of today could be reduced to close to zero. Where humans lose focus, become distracted, get stressed, make mistakes, don’t notice an oncoming accident, fall asleep, drive intoxicated, refuse to buckle their seat belts, drive without maintaining safety features on their cars, AVs would never do that. AVs represent a potential improvement on human driving. Why would anyone argue against that improvement and for the status quo of 3,000 monthly deaths in the US? Who would argue for status quo of 100,000 monthly deaths globally? But what second-order risks are there in this argument?

When it comes to the other argument in support of AVs — optimization — the arguments also avoid second-order effects. The reason for this seems to be similar to why we seldom hear of unintended effects in other types of new business innovation. That is, the businesses that innovate are rewarded for their innovation and presumed ability to scale to a large market. They are not rewarded for considering whether their innovations increase risk elsewhere.

The optimization arguments center on what would be possible with fully autonomous vehicles. And I like to imagine the potential. Self-driving cars could coordinate to distribute traffic, travel much closer together than human-driven cars, free people from the drudgery of unhealthy and boring driving for more productive work, collect and share environmental data, moderate demand through price changes, carry the very young and very old safely, allow people to sleep rather than manage their commute, reduce needed fuel, and on and on.

Over the past decades, systemic shocks to optimization of traditional driving have operated differently than they probably would for AVs. These previous shocks were in the form of oil embargoes, regulation, and company-level shocks affecting commodity prices (sheet metal etc) and import/export duties.