The Most Important "No" in Human History

I’ve just learned about Vasily Arkhipov. He is credited with preventing World War III.

In 1962, the US and USSR had been stockpiling nuclear weapons. There was a tense moment when the USSR sent submarines to Cuba, off the coast of the US. The crew of the submarine knew this was provocative, and the sub was loaded with nuclear-armed torpedoes, capable of causing strikes the size of the Hiroshima attack. US started dropping (non-nuclear) depth charges on the submarine, and since it was cut off from USSR communications at this point, it had to decide what to do.

From the perspective of the captain, they assumed World War III had started, and they should shoot their nuclear torpedo at nearby US ships. What choice did they have? That was what they had been tasked to do. The second-in-command refused this plan. Despite the immense pressure from his captain, and the policy officer, he kept a cool head and refused.

The submarine would have kaboomed a range of nearby US ships. The US would have seen this, and responded with a large array of its own nuclear warheads. It would have rained down terror over the USSR, dropping hydrogen bombs over Eastern European cities such as Kyiv and Moscow, and Chinese cities such as Beijing and Shanghai. The USSR may then have responded with strikes on London, Paris, and New York.

The strikes would have killed up to 500 million people directly, and potentially caused a nuclear winter. The dust kicked up into the atmosphere would likely have reduced the sun reaching fields, causing crops to fail globally and large famines.

There would have been huge numbers of deaths, and complete derailment of all the societal progress since 1962, My parents wouldn’t have been born, and the Beatles wouldn’t have written Something or a Day in the Life.

The first thing we learn is that history is full of near misses. Forethought have written about how AI is likely to rapidly accelerate tech progress, which causes huge policy challenges -- if nuclear tech had happened way faster, we may have struggled to deal with the policy and ended up in this winter scenario.

Secondly, as the Anthropic v Pentagon debacle recently covered, humans-in-the-loop matter! Even on small scales, life-or-death decisions should rest on humans.

The second thing is, anti-conformity training 1 could be hugely important. Arkipov was willing to stand up for his opinion, but humans are naturally terrible at this.. There are studies that show if everyone around you is ignoring smoke alarms, then you will sit through them, even if you see smoke billowing under the door. Other studies show that if your peers all believe that one stick is shorter than the others, even if you know deep down they’re wrong, you’ll likely agree. The result is that decision-makers are naturally primed to do dumb stuff, like allow their commanders to send missiles and start WWIII. To avoid situations like this in the future, humans being good at this could be important.

Anthropic v Pentagon gave us another example of this recently. Dario Amodei broke down an important contract on a dispute over the legal smallprint— taking heat from the government while doing it. Surely other execs in Anthropic advised him against this, urged him to continue the contract and dissent later. Dario being good at saying ‘no’, resisting conformity, has meaningfully changed the conversation on domestic surveillance and autonomous weapons.

History is a thin tightrope. We are entering an era of AI and rapid progress where the stakes are higher than ever. So the next time you feel the itch to agree and keep the peace, consider practising standing up for your opinion. Get good at using your ‘no’ - the future might need it.