The Security Challenges of Ever More Complex Tech

The convergence of radically disruptive technologies presents unprecedented systemic security challenges. Have our systems become too complex to manage?

As our society is inundated with new technologies, security often takes a back seat. As the example of Russian interference in the 2016 election makes clear, security challenges often arise from a failure of imagination. Despite the presence of an ever-growing security industry, as a whole we're still far too reactionary about security, usually focusing initially on the benefits of any new technology, and only after new tech is adopted and problems begin to emerge realizing that we need to catch up on security.

The implications are huge, and go not just to personal privacy, the security of our financial, medical, and other most important data, but to our collective security, and the very integrity of our democratic society. Election interference is only one thread in a larger, disturbing narrative of using the global information system as an instrument of warfare.

Wired has some great reporting on recent cyberattacks, among them:

The Untold Story of NotPetya, the Most Devastating Cyberattack in History

How An Entire Nation Became Russia's Test Lab for Cyberwar


These stories are eye-opening, and they made me realize that we are witnessing the realization of some of our worst fears come true. Stephen King has very prominently written about machines that come to life and terrorize humanity (Trucks, The Mangler, etc.), and it seems that his grim vision is now increasingly coming to pass. What's horrifying is that no supernatural force is needed - technology has advanced to the point that machines can be controlled remotely as if they had a mind of their own, and advances in AI are bringing us rapidly closer to truly autonomous machines that think and act on their own.

A new Europol report highlights the criminal threats posed by the new wave of hypertechnologies - AI, 5G, quantum:

AI, quantum computing and 5G could make criminals more dangerous than ever, warn police (ZDNet)

Do Criminals Dream of Electric Sheep: How Technology Shapes the Future of Crime and Law Enforcement (report itself [PDF])

The larger question, from a systems perspective, is whether our societies can maintain security in the face of rapidly advancing technology. Some security experts, like the eminent Bruce Scheier, think the answer is no. He argues that because technology is at its core a force multiplier, it makes it possible for a small group of malicious actors to inflict increasingly greater damage, and the asymmetrical nature of the dangers and benefits means that the good guys will always be one step behind.

I fear that we are rushing into repeating the mistakes of the past, and adopting new technologies too rapidly and without a thorough understanding of all the ways that they could go wrong. 

One issue is that the focus is usually on technologies individually, and not enough on how they interact and relate to other technologies and parts of our already complex techno-administrative-economic-social system. As we rapidly advance the technology landscape, we make the system more complex in ways that are difficult to understand ahead of time, without many years to use the new tech and understand, in the real world, what new issues it might introduce. And the problem is made worse because we often don't even attempt to understand the systemic and interaction effects of new technologies before rushing to adopt them.

We see this happening with the new wave of tech, where the hype largely focuses on the security advances that are possible: (unbreakable cryptography, better malware detection, etc.)

The report linked above highlights some of these issues, and provides a good point of discussion. I am also working on a longform piece about complexity in (software) technology that goes into more detail on some of these questions. More to come soon...