Autopilot and the dilemma: kill the driver or pedestrians?

Autonomous cars are no longer science fiction: vehicles with autopilot modes are appearing on the roads, able to drive without human intervention. For now, car enthusiasts are still more interested in traditional metrics—speed, power, comfort. They eagerly compare, for example, German super sedans BMW M5 F90 and Mercedes-AMG E63 S by their specs. Yet with the advent of autopilot, a completely different topic comes to the forefront—ethics and safety. What should a car do when an accident is unavoidable?

This is no longer a theoretical question from textbooks, but a real problem on the road. Imagine the situation: a self-driving car is speeding along and suddenly a group of people appears in front of it at a pedestrian crossing. It’s too late to brake. There are two possible outcomes: swerve aside and crash into a wall (sacrificing the driver) or keep going and hit the pedestrians. The artificial intelligence algorithm is programmed for a specific decision—and lives depend on that decision.

Philosophers call such a tragic choice the “trolley problem.” It used to be a purely hypothetical experiment: should you divert an out-of-control tram onto another track to save five people at the cost of one life? Now a similar dilemma confronts AI on the road—and its choices will have real consequences.

• Save the passenger. Continue driving, protecting the driver, despite the risk to the group of pedestrians.
• Save the pedestrians. Steer the car into an obstacle or the shoulder, sacrificing the life of the person inside the vehicle.
• Make no conscious choice. Avoid a dedicated choice algorithm—i.e., build in harm minimization without targeting, effectively allowing events to unfold spontaneously.

Some might argue that AI will be so smart it will always find a third, safe way out. But that’s more myth than reality. In practice, even the most advanced sensors and software can’t отменити the laws of physics or eliminate all risks. Critical situations will still happen. Sooner or later, a car may face a choice: whom to protect at the cost of someone else’s life.

Humanity still hasn’t found a single correct answer to this dilemma. However, sociological studies offer an interesting perspective. Surveys show that most respondents agree that, in an extreme situation, an autonomous car should sacrifice its passenger to save a group of people. Yet those same people admit they wouldn’t want to ride in a car that would knowingly sacrifice them for others’ lives. This paradox could become a barrier to bringing self-driving cars onto the roads.

Here is an illustrative result from one public-opinion study on the “autopilot dilemma”:

As we can see, society simultaneously wants robot drivers to save as many people as possible—and at the same time doesn’t want to risk itself. This gap between moral and personal interests creates a dilemma for manufacturers and regulators. Because no matter how you program the algorithm, the decision will be controversial and may provoke outrage.

Who will go to prison if a self-driving car causes a crash?

But if a car on autopilot does cause an accident with fatal consequences—who should be punished? This question is no less complex than the moral choice. The first warning bells have already rung. For example, in 2018 a test autonomous Uber vehicle fatally struck a pedestrian; other deadly crashes involving Tesla on autopilot are also known. Such incidents, where the computer is driving, call into question the traditional principle that places all blame on the driver.

In a “robo-driver” scenario, different approaches to assigning fault are possible. Let’s look at the main ones:

1. The owner (passenger) of the car. Logic: they were sitting inside and activated autopilot. Perhaps the person should have monitored the driving and intervened—therefore they bear some share of fault for inaction.
2. The software (algorithm) developer. After all, it was the programmers who built in the behavior model that led to the tragedy. Should engineers bear criminal liability for the consequences of decisions made by their code?
3. The car manufacturer. The company that released the self-driving vehicle may not have anticipated all risks or may have cut corners on sensors. The manufacturer (through its management) could be held accountable as a legal entity—for example, via lawsuits and fines.

So far, there is no clear legal standard on this. In most countries where autonomous cars are being tested, responsibility is placed on the human behind the wheel. That’s because today’s autopilot systems (Level 2–3) require the driver to remain attentive. Thus, in Tesla-related cases, courts most often find the driver at fault for overestimating the autopilot’s capabilities and failing to intervene when necessary.

However, with the emergence of fully driverless cars (Level 4–5), it logically becomes necessary to shift blame toward the creators of the technology. In Germany, the National Ethics Commission on Automated Driving ruled back in 2017 that if a person no longer controls the car, responsibility for its actions lies not with the passenger but with the manufacturer or the system operator. Some automakers are themselves ready to accept such responsibility—for example, Volvo publicly promised to take full blame once its cars drive in autonomous mode.

Shifting blame to the company has pros and cons. On the one hand, owners will trust autopilot more readily if they know they won’t be made the “fall guy” in the event of an accident. On the other, manufacturers fear taking full responsibility, as it threatens multi-million-dollar claims and even criminal liability for their executives. Special insurance policies or legal limitations may be introduced to balance these risks.

Despite all these uncertainties, autonomous driving technologies are not standing still. New car models are getting ever smarter “brains.” For example, the electric Volkswagen ID.4 crossover is equipped with the Travel Assist suite for semi-autonomous driving, and drivers of this model can boost comfort and practicality by installing various accessories for Volkswagen ID.4. Fully autonomous driving (Level 5) still remains a matter for the future. And that means debates about moral algorithms and legal responsibility will continue—society needs time to reach agreement.

So, the autopilot moral dilemma still has no clear resolution. Should we program machines to save the majority at the passenger’s expense, or to protect their driver under any circumstances? Who should be punished for AI mistakes—engineers, owners, or no one at all? Philosophers, lawyers, and the manufacturers themselves are wrestling with these questions. Perhaps soon we will have to rethink established principles of ethics and responsibility on the roads. Because when a computer is at the wheel, both decisions and guilt take on a completely new meaning—one that humanity still has to learn to live with.