Boots & Sabers

The blogging will continue until morale improves...


Everything but tech support.

2131, 04 Dec 18

Digital Hallucinations

While it seems that the era of autonomous vehicles is very close, and it might be, we still have a lot of work to do on the technology. And then we have a lot more work to do on our culture, legal systems, etc. to adapt to it.

The passenger registers the stop sign and feels a sudden surge of panic as the car he’s sitting in speeds up. He opens his mouth to shout to the driver in the front, remembering – as he spots the train tearing towards them on the tracks ahead – that there is none. The train hits at 125mph, crushing the autonomous vehicle and instantly killing its occupant.

This scenario is fictitious, but it highlights a very real flaw in current artificial intelligence frameworks. Over the past few years, there have been mounting examples of machines that can be made to see or hear things that aren’t there. By introducing ‘noise’ that scrambles their recognition systems, these machines can be made to hallucinate. In a worst-case scenario, they could ‘hallucinate’ a scenario as dangerous as the one above, despite the stop sign being clearly visible to human eyes, the machine fails to recognise it.

Those working in AI describe such glitches as ‘adversarial examples’ or sometimes, more simply, as ‘weird events’.


2131, 04 December 2018


  1. jonnyv

    Everyone thinks they are the best driver, and everyone thinks the other guy is the biggest idiot on the road. I would take 100% autonomous driving RIGHT NOW in all vehicles over what I see on the freeway every day. Put everyone on a level playing field.

    I hope in my lifetime I see it illegal to drive “manually” on highways. There are going to be deaths, and people trying to scare others about “hacking” your automated vehicles, but frankly people can screw with cars now if they really want.

    These vehicles are going to be great for people with special needs and the elderly. And it is gonna SUCK for anyone who makes their living on the road driving. And I am fine with all of that.

  2. MjM

    Owen sez: “While it seems that the era of autonomous vehicles is very close…”

    Actually, it’s probably 30-50 years away, if ever.   Until some devises sensors/cameras that can in discern 3D in at minimum 100 degrees radius in real time at speed – never mind the rear view –  vehicle automatons will remain far inferior to just average human real-time perception.

  3. dad29

    I think MJM has it.  “Guide-wire” cars may show up sooner, traveling along cables in the roads.  But they, too, have limitations.  Poor ol’ JohnnyV will have to keep ducking and dodging for quite some time.

  4. MjM

    Things I have big problems with auto-autos; physical and logical failure, and human error.

    As the article Owen cites explains, logical failure is probably the biggest hurdle in AI/nueral net development. For one, you have to not only teach the machine TO think, you have to teach it HOW to think. And that is where the Human Error factor kicks in. We in fact, do not even know with certainty how our own brains really work, and here we are trying to train inanimate objects to that level. Humans have a tendency to be quite vain in believing we are the summit of all-knowing (see: global warmists) when in truth humans have always lived by simple trial and error.

    For example, I’m sure you’ve seen the fear-mongering car commercials that highlight auto-braking. Saves you from serious injury and a long hospital stay, so the story goes. It’s a nice idea. But what about that concrete truck behind you? Your 2,000lb Subaru may make the stop in 125ft. But that 20,000lb truck with it’s 40,000lb soggy load isn’t, guaranteed. So, does that Subaru (or whatever) allow you to override the auto-brake by stomping on the gas so you can move left or right to avoid the issue in front while also avoiding being squashed like a wine grape from behind? Does the auto-brake even sense rear distances? Did Subaru even think of that situation? I don’t know. The commercials doesn’t let on about that.

    I wonder because I have been in that exact situation more than once.

    And lastly, as one who has been involved with IT/Telecomm/hardware/software since the days of IBM360s and Apple I, I tell you now I will never trust the full control of my travel safety solely to human-programmed man-made hardware. I’ve been fixing the failures of that shit for too damn long.

    I’ll wait for the flying cars we were promised…. in the 70’s.

Pin It on Pinterest