The plaintiffs in Toyota’s Unintended Acceleration lawsuit had someone with knowledge in building embedded software had a look at Toyota’s source code:
possible bit flips, task deaths that would disable the failsafes, memory corruption, single-point failures, inadequate protections against stack overflow and buffer overflow, single-fault containment regions, thousands of global variables. The list of deficiencies in process and product was lengthy.
I often tell myself and my students: medicine is the most human of all the sciences that is stuck with the least human of all the experiments: and that is the randomized trial.
Randomization doesn’t exist because doctors are malign or medicine is nasty it exists precisely for the utterly opposite reason: because we hope too much.
We’re so hopeful, that we want things to work so badly-especially against cancer-we want things to work so badly that we’ll trick ourselves to believing that they’re working.
And there’s nothing as toxic or as lethal as that trick: the trick of hope.
— Dr. Siddhartha Mukherjee in PBS’ Cancer: The Emperor of All Maladies
“People still aren’t realizing quite the magnitude that the problem could get to,” Raj said. “And it’s not just surveillance,” he said. “People are using voice services all the time. And where does the voice go? It’s sitting somewhere. It’s going somewhere. You’re living on trust.” He added: “Right now I don’t think you can trust anybody.”
Also when all the voice data gets automatically transcribed, made keyword-searchable, flagged and presented to agents as “potentially interesting” there’s basically no way of producing any sort of indication for suspicion other than pointing at a black box and mumbling something vaguely resembling “correlation.”
“When the NSA identifies someone as ‘interesting’ based on contemporary NLP [Natural Language Processing] methods, it might be that there is no human-understandable explanation as to why beyond: ‘his corpus of discourse resembles those of others whom we thought interesting’; or the conceptual opposite: ‘his discourse looks or sounds different from most people’s.'”