Talking Technology! (On Technological Entanglement)

I've confessed before to an annoying fondness for saying "I told you so." Still, I can't help but note that an article in the New York Times this morning affirms arguments that are regular themes here.

The article is a profile of computer scientist Peter G. Neumann, who has warned for decades that the complexity of the computer systems we rely on makes it "virtually impossible" (as the Times put it) to assure that they'll run reliably and safely. 

Neumann is recognized as one of the nation's leading experts on computer security. Nonetheless the hardware and software industries have consistently ignored his predictions that the products they sell, as they proliferate, will become increasingly vulnerable to breakdown and attack. His mantra is that complex systems break in complex ways, a principle that's been vindicated by legions of computer bugs and viruses, regular reports of massive data breaches and thefts, and growing fears of cyber warfare.

“His biggest contribution is to stress the ‘systems’ nature of the security and reliability problems,” said Steven M. Bellovin, chief technology officer of the Federal Trade Commission. “That is, trouble occurs not because of one failure, but because of the way many different pieces interact.”

Neumann's current project, Clean Slate, has been funded by the Pentagon's Defense Advanced Research Projects Agency (DARPA). It's an effort to rethink computer design from the ground up. The goal: an entirely new race of machines that's simpler, more stable, and less easily violated. 

If that sounds idealistic, it is, and Neumann knows better than to expect eager assent from the digital establishment.

“I’ve been tilting at the same windmills for basically 40 years,” he tells the Times' John Markoff. “And I get the impression that most of the folks who are responsible don’t want to hear about complexity. They are interested in quick and dirty solutions.”

Again, forgive me for pointing out how clearly this echoes themes I harp on regularly in this space (though certainly not original to me). Recent examples include "On immovable technologies," "Recycling Ellen Ullman," and "Technological Autonomy: Greasing the rails to Armageddon."

The broader point to be emphasized is that the systemic nature of technology – and the fallibility that's an inevitable consequence of that systemic nature – is not only an issue in the world of computers. It applies to the  technological society as a whole. Peter Neumann's argument that the complexity of digital systems makes them unstable and insecure can be extended to the inconceivably complex web of technological systems that now dominate the ecosystem we inhabit.

If it's difficult to imagine starting over with a "clean slate" in the world of computers, how much more difficult is it to imagine rethinking and rebuilding the entire technological edifice on which we depend, regardless of how unstable that edifice might be? As I've said before, it's not a problem we seem prepared to contemplate.

©Doug Hill, 2012