Heartbleed's Companions: There Will Be Consequences

The Heartbleed bug appears to be a contender for Greatest Internet Security Fuckup of all time, but it also fits in the category of Here We Go Again.

At this point discoveries of massive data breaches have become pretty much routine. A study cited by Farhad Manjoo in the New York Times reported that 814 million data records were exposed in 2013, and that was before Heartbleed was outed.

It’s not data breaches, per se, that concern me, however. Rather, I’d like to point out a couple of Heartbleed’s broader implications.

The first is that we miss Heartbleed’s most important lesson if we think it applies only to the question of Internet security. The designers of all sorts of technologies regularly assure us, and themselves, that their machines and their systems are absolutely safe and secure. Just as regularly we discover they’re wrong. Recent examples include the Deepwater Horizon oil spill, the Fukushima meltdowns, the chemical leak that poisoned West Virginia’s Elk River and Michael Lewis’s revelations about how the stock market has been rigged by flash traders.

Deepwater Horizon

We are similarly assured that we need not fear the even greater risks posed by such developing technologies as nanotechnology and synthetic biology, both of which have the potential to unleash unexpected consequences of Biblical proportions. (See Bill Joy’s famous article in Wired, "Why the Future Doesn’t Need Us,” for details.)

Synthetic biology is especially relevant here because its advocates are consciously basing, at least in part, both their hopes for advancing the technology and their confidence in its safety on the open source computing model that gave us Heartbleed.

The thinking in both cases is that there’s safety in numbers. Open source advocates argue that more people looking at computer code makes it more likely that openings for hackers will be prevented or closed. Synthetic biologists argue that the more people who know how to manipulate strands of DNA, the more prepared we’ll be to respond to the accidental release of harmful organisms or a bioterrorist attack.

The Heartbleed bug doesn’t prove that the open source philosophy is false, but it does demonstrate that it’s not perfect. As Farhad Manjoo put it, Heartbleed showed that “the Internet is still in its youth, and vulnerable to all sorts of unseen dangers, including simple human error. Today’s digital systems are complex and penetrate every corner of our lives. It is impossible to lock them down.” 

Manjoo argues that the Internet is less likely to correct its lapses than other large-scale industries have been because of its ubiquity, its complexity and its interdependence, and also because the human beings getting rich off the web pay more attention to building the applications that will make them rich than they do ensuring that those applications are safe. I agree with all those contentions except that the Internet is different from other large-scale industries, and the suggestion that industries become "locked down" as they mature.

Manjoo credits Upton Sinclair’s The Jungle and Ralph Nader’s Unsafe at Any Speed with helping alert the public and lawmakers to unsafe conditions in the “chaotic, unruly days” of the meatpacking and automobile industries, respectively. (Never mind that Nader’s book came out in 1965.) Those unsafe conditions have since been rectified, Manjoo says by “a combination of regulation and industrywide cooperation.”

Mary Barra, CEO of General Motors, is sworn in to testify before the House Energy and Commerce subcommittee on Oversight and Investigation

Well, yes and no. Certainly there have been improvements, but Manjoo seems not to have noticed that executives of General Motors have spent a lot of time recently testifying before Congress, trying to explain why they failed to correct a flaw in their ignition systems that killed 13 people over the past decade. And if Manjoo thinks The Jungle solved the problems in the meatpacking industry, he hasn’t read Eric Schlosser’s Fast Food Nation.

The point — painfully obvious yet persistently ignored — is that breakdowns in large-scale technological systems are inevitable, and breakdowns produce consequences. The complexity, ubiquity and interdependence of those systems contribute to that inevitability, as do the oversights and mendacities of the human beings who design and run them (not to mention the oversights and mendacities of the human beings who use them). The scale of consequences will vary from insignificant to catastrophic, but there will be consequences.

This brings me to the second point I’d like to make about Heartbleed.

In the days after the bug was revealed, most of the blame was pinned on the open source engineers who failed to detect it before the updated software was released. This seems to be the sort of explanation intended to make us all feel better. If only proper procedures had been followed, everything would have been fine.

Again, though, the oversight that opened the Heartbleed door is hardly an isolated incident. You don’t have to be a volunteer working on open source software for free to miss a flaw that will cause problems. Plenty of paid professionals do, too. Indeed, the Heartbleed bug was overlooked for more than two years by any number of major companies and institutions using the OpenSSL software that carried it, among them Google, Amazon, Cisco, Facebook, Netflix, Yahoo, the Pentagon and the FBI.

“Given enough eyeballs, all bugs are shallow,” says open source software guru Eric S. Raymond.

The Heartbleed fiasco shows that the bug that lays you low can hide in plain sight, no matter how many people are looking. It also affirms Farhad Manjoo's point that Internet companies pay more attention to profits than they do to security.

All systems are fallible, and all systems are vulnerable. Anyone who says different is lying.

###

Note: This post originally stated that the Heartbleed bug was discovered by an engineer at Google. Other reports tell a different story, so I've eliminated the reference.

Earlier posts related to this subject can be found here, here and here.


 
Image credits: Heartbleed tshirt: Martin Mulazzan. Eyeball: Thinkstock 

©Doug Hill, 2014


 

Marry in Haste, Repent at Leisure

 

The New York Times ran an article a few days ago that points toward one of the more important lessons of our time, a lesson that will almost certainly be ignored. 

Headlined “Data-Driven Tech Industry Is Shaken by Online Privacy Fears,” it described how upset members of Silicon Valley’s elite have been by the revelations of the National Security Agency’s Prism program.

The piece, by David Streitfeld and Quentin Hardy, was nicely written, with a generous serving of appropriate irony. “The dreamers, brains and cranks who built the Internet hoped it would be a tool of liberation and knowledge,” they began. “Last week, an altogether bleaker vision emerged with new revelations of how the United States government is using it as a monitoring and tracking device.”

Then came the kicker: “In Silicon Valley, a place not used to second-guessing the bright future it is eternally building, there was a palpable sense of dismay.”

That nails it. No one is more convinced that technology is the gateway to a new Eden than the technologists themselves, and no one is more surprised than they are when things turn out to be more complicated than expected.

The reason I find the Times article so significant is that the privacy concerns at the heart of the Prism imbroglio are only the tip of the technological iceberg. The Internet is far from the only cutting edge technology that presents tremendous opportunities for intentional abuse or unintentional disaster, and Silicon Valley’s engineers and scientists are far from the only ones who have routinely ignored the dangers.

There’s a second irony here that Streitfeld and Hardy didn’t mention. While the security branches of government have joined the profiteers and thieves in exploiting the power of the Internet for questionable ends, realistically our best hope of protection from those ends lies in the hands of­­—you guessed it—the government. Libertarians will disagree, but there’s abundant evidence to suggest that Silicon Valley’s failure to rigorously defend the public interest in its corner of the technological universe is the rule rather than the exception. Thus the answer to the question, “Who’s watching the technological store?” is basically, “Nobody.” 

Remember Bill Joy’s famous essay in Wired, “Why the Future Doesn’t Need Us”? It's surprising to realize that 13 years have passed since it appeared. In it, Joy warned of three specific technologies that concerned him: robotics, genetic engineering, and nanotech. Like the Internet, each holds tremendous promise, but each also holds tremendous risks. Joy urged that the risks be seriously addressed before it’s too late, and said he remained optimistic we would find ways to do so. 

As far as I can tell, the momentum toward exploitation of all three technologies continues unabated. If anything, it’s accelerated. I'm not aware of any concurrent momentum toward establishing effective precautions.

Joy issued another plea for restraint five years later, less known, but relevant to the discussion here. It was an an op-ed piece in the New York Times, co-authored with the futurist Ray Kurzweil and headlined “Recipe for Destruction.” In it they denounced the decision by the U.S. Department of Health and Human Services to publish online the genome of the 1918 influenza virus responsible for the deaths of 50 million people worldwide. To publicly release such information in an era when the techniques of synthetic biology are widely available, they said, was tantamount to advertising “the design of a weapon of mass destruction.” Joy and Kurzweil called for an "international dialog" on ways to prevent lethal genetic codes from "falling into the wrong hands." They also called for “a new Manhattan Project" to develop specific defenses against new biological viral threats, natural or human made.

Christine Daniloff/iMol

In connection with the book I’m writing, two months ago I emailed Joy to ask him how much progress he’d seen toward the sorts of safeguards he and Kurzweil proposed. Here’s his response:

Doug:

Since the article, I have been focused on investments for sustainability. I haven't been tracking the progress on what we suggested. I know of nothing substantial that has been done to address any of these. But then again, I'm not "in the loop" on all such things, so perhaps something has been done; to find that out would be a pleasant upside surprise.

Best,

Bill

There’s a couple of implicit suggestions in Joy’s response, beyond what he says explicitly. First, although he’s careful to say he’s not aware of anything substantial having been accomplished to address his concerns, I think it’s fair to assume he would be aware of any significant efforts in that regard, had they materialized.

Second, the fact that Joy’s attentions are directed toward other endeavors represents a big part of the problem. All of us have our attentions directed elsewhere – they have to be. We can’t spend full time trying to see that the potential dangers of a whole range of incredibly powerful technologies are being adequately addressed. Nonetheless, thousands of people are spending full time, day after day, week after week, trying to find ways to exploit those incredibly powerful technologies. Often they're well paid for doing so; almost always they’re hoping for a payoff at the end. Undoubtedly some of them are working carefully; undoubtedly others aren’t. The problem is that the balance between ambition and restraint seems radically tilted toward risk and irresponsibility.   

As I say, like it or not, our best bet for oversight is the government. Not surprisingly, we can't take much comfort in that fact. For example, in 2010, after a lengthy series of hearings on synthetic biology, the Presidential Commission for the Study of Bioethical Issues found “no reason to endorse additional federal regulations or a moratorium on work in this field at this time." This prompted an open letter signed by the leaders of more than fifty environmental organizations calling the Commission’s conclusions hopelessly inadequate. "We are disappointed that 'business as usual' has won out over precaution in the commission's report," the letter said. "Self regulation amounts to no regulation.”

The story is much the same with nanotechnology. The National Nanotechnology Initiative, which is responsible for coordinating the activities of 15 federal agencies that distribute government money for nanotech research and development, has been the focus of ferocious criticism for spending almost all of its funds promoting nanotech’s commercial prospects while paying virtually no attention to its safety. A survey, meanwhile, found that some 60 percent of American nanotechnology companies have ignored government recommendations regarding safety precautions in their workplaces.

Citing that survey, the President’s Council of Advisors on Science and Technology said it's "critical" that appropriate federal agencies "engage" with companies to increase their awareness of safety issues and their ability to address them. And how should this engagement proceed? Why, in a "non-regulatory capacity," of course!

Like Bill Joy, I can’t claim to have kept track of every attempt, either by government or by industry, to monitor and regulate the dangers inherent in synthetic biology – or in nanotechnology and robotics. I have other things to do. I think it’s a safe bet, however, that neither government nor industry have pursued their responsibilities in those areas as aggressively as the NSA has pursued its surveillance of the Internet, despite the fact that their potential for evil are at least as severe, and probably more so. 

It’s true, I’m sure, that the web can be an effective tool for uncovering terrorist plots involving other technologies—someone hoping to unleash a genetically engineered virus might well leave tracks there, for example. As a pronounced civil libertarian, it feels odd to say it, but I hope the NSA is watching out for those types of threats. These are the devil's bargains our technologies lead us into.

I’ll close by noting the comments made nearly half a century ago by a scientist who can be considered Bill Joy’s predecessor in the role of technological Cassandra. Norbert Wiener was the founder of cybernetics, and in that role made foundational contributions to the digital technologies that are so forcefully reshaping our world today. Unlike many technologists, however, he worried a lot about the uses to which some of his theories might be put, so much so that he turned down many offers of corporate and military research contracts, at significant cost to his career.

Wiener harbored an undisguised contempt for the “gadget worshipers” among his colleagues who rushed to exploit their knowledge without due consideration of the consequences likely to ensue. They fail to appreciate, he said, that “a sense of the tragic" is a prerequisite to the exercise of scientific and technological power. The scientist with an appreciation of the tragic, he said, "will not leap in where angels fear to tread, unless he is prepared to accept the punishment of the fallen angels.” 

Technology, he added, is a two-edged sword, “and sooner or later it will cut you deep.”

 

©Doug Hill, 2013