mostlysignssomeportents:

This is an outstanding critique of full-self-driving (FSD) cars

If you’re interested in a critique of full self-driving (FSD) cars that is well-grounded in both theory and technological realism, this piece (by AI researcher Filip Piekniewski) is simply outstanding.

Let me begin this post by discussing why “probabilistic” approach is inadequate for mission critical applications. The crux of the discussion that probabilities are deceptive when what really matters is not the random variable itself, but a certain function of the random variable, often times called payoff function in the context of economical discussions. To illustrate this, imagine you are playing a simplified Russian roulette with a toy gun. The gun has got six chambers, if you hit any of the 5 empty ones, you win a dollar, if you hit the one with a bullet, the revolver will make a fart sound and you loose two dollars. Would you play this game? Obviously the probability of winning is 5/6 and loosing only 1/6, the mean gain from this game is $3, so it’s a no brainer. Everybody would play. Now imagine you play that same game with a real gun and a real bullet. Unless you are suicidal, you would stay away from that kind of entertainment. Why? Neither of the probabilities have changed? Of course what really changed is the payoff function. When you loose, you don’t only loose 2 bucks, but also your life. What if the gun had 100 chambers? Would you play? I know I wouldn’t. What if it had a thousand chambers? Most people wouldn’t have touched that game even if the revolver had a million chambers. That is if they knew with certainty that one of them has a bullet and will cause an instant death. Things are a bit different if players didn’t know about the deadly load. In such case, observing one player pulling the trigger hundreds of times and getting a dollar each time would attract many players. Until one time the gun fires. But if the game is played in such a way that there are multiple independent revolvers and when one goes off, players triggering other guns don’t know about it, you could probably have a large group of players constantly try their luck. And that is exactly what is going on with Tesla FSD. If people knew the real danger the FSD game poses, nobody sane would have attempted it. But because incidents are rare and so far were’t disastrous (in case of FSD, but at least 12 people lost their lives in autopilot related crashes), there is no shortage of volunteers to try. But that is where the government safety agencies need to step in. And just like the government wouldn’t allow people to offer the game of Russian Roulette to uninformed public (even with a revolver with a million chambers), based on the expert knowledge and risk assessment, the FSD experiment needs to be curbed ASAP.

https://blog.piekniewski.info/2021/12/18/farcical-self-delusion/