The thought had never struck us.

This is the opening sentence of "Dual Use of Artificial Intelligence-Powered Drug Discovery" by Urbina, et al., published in Nature Machine Intelligence in 2022. The central conceit of the paper is that recent advances in generative AI pose an emerging dual-use technological threat.

Dual-use technologies are those which are identified as having both salutary peace-time use, as well as the capacity for facilitating warfare, terrorism, or nefarious activities not otherwise specified. The designation of a technology as 'dual-use' is typically made by a state or international body, and may subject the research, development, dissemination, sale, and export of that technology to regulatory scrutiny and restriction.

Stable Diffusion... of Terror

In Urbina, et al., the authors provide a case study in inverting the logic of machine learning models aimed at identifying novel therapeutic candidates, such that the models emit candidate toxins instead. This paper garnered considerable pop-scientific attention, conjuring images of AI-enabled Al-Qaeda, lone-wolf basement hackers, and GosNIIOKhT GPU farms crunching the next Novichok agent.

The authors of the paper repeatedly refer to their naïveté regarding the possibility that therapeutic chemoinformatic software could be abused to generate new chemical or biological weapon candidates. The authors of the paper are all experts in rational drug design, so they can't feign shock when they turn up a dial labelled 'toxicity' and get plausibly toxic molecules, nor can they realistically pretend this is a new state of affairs especially created by recent advances in generative AI. Yet they do.

Much of the pop-science attention this paper received can likely be attributed to a general ignorance in the lay public of the concept of a quantitative structure-activity relationship, or QSAR. QSARs connect the structure of a molecule to predictions regarding pharmacological figures of merit, such as the therapeutic potential of a molecule, its bioavailability, and the extent to which it does not kill the patient. QSAR is by no means a new concept, and has been the traffic of drug design for decades. Lipinski's Rule of Five, which establishes quantitative connections between molecular structure and likely oral activity of a drug candidate, was first promulgated in 1997. The minimisation of unwanted toxicity is obviously an important figure of merit in rational drug design, and therefore the tools for screening drug candidates for potential toxicity are absolutely foundational.

It's basically Mister Potato Head

Key to the idea of QSAR is that molecular structure has some predictable bearing on how a molecule interacts with the body. Molecules are not random jumbles of atoms. They are collections of concatenated fragments, which can be generated, compared, and analysed using the tools of graph theory. The Shulgins discovered an embarrassment of psychedelics—hundreds—based on conceptually simple modifications of only two types of fragment. In so doing, they wrote what were retrospectively labelled by the US DEA as 'pretty much cookbooks on how to make illegal drugs'. Some structural fragments are powerful predictors of toxicity. All but one of the nerve agents listed in the Wikipedia 'Chemical Agents' template are based around one of two different fragments, and you don't need to be Stanley Goodspeed to recognise them. And they all inhibit the enzyme that lets your muscles relax after contracting. Who could have guessed?

Urbina and coworkers specifically 'chose to drive the generative model towards compounds like the nerve agent VX', and were rewarded with candidate molecules with extremely high predicted toxicity. These molecules have high predicted toxicity because they look like VX. A reasonably precocious highschooler could achieve the same with pencil and paper, based on the elementary observation that most known nerve agents look like VX. What's the story here? Talk is cheap. Compute is cheap. Pencils and paper are cheap. Employing highschoolers to design nerve agents is cheap.

Commonsense Recreational™ Nuclear™ Weapon™ Control

Camille Flammarion, in his 1894 novel "La Fin du Monde", expresses a concern that the democratisation of publishing and a nihilistic cultural malaise could lead to a situation where irresponsible newspapers published the chemical structures of poisons for anyone to read. This anxiety is replicated with a sense of moral exasperation and horror at least three times on the talk pages of nerve agents on Wikipedia, where chemical structures and synthesis details are freely available. Considerable resources are available online regarding the theory and design of nuclear weapons. Genetic and protein sequences of a range of deadly pathogens are easily accessible from public databases. Given the glut of information available on the creation of weapons of mass destruction, it seems like there is nothing stopping anyone creating them in the comfort of their own home or motor vehicle, except for all of the other barriers.

Urbina, et al. conclude their paper with an urgent wake-up call for regulation, education, safety restrictions on APIs, codes of conduct, and even propose a phone reporting hotline for dangerous misuses of AI-facilitated drug design. Contrariwise, it is the opinion of this author that the vibe has not shifted, the balance of terror has not been upset, and that no fundamentally new threat has been offered up. Yet in our present moment, the addition of yet more machine intelligence to a field already predicated on machine intelligence seems to always create an emergency with a managerial solution, more excuses for more top-down control, and ever more slots for interdisciplinary AI safety experts.