IN 2010, a commenter on the blog of rationalist community Less Wrong going by the name of Roko sat down at his keyboard and got ready to inject a cognitive virus into the minds of his readers. For years Less Wrong had been a place for intelligent geeks and oddballs interested in the future of humanity to hang out and share their thoughts on everything from where tech was going to how humans make decisions. Some of those users were believers in the so-called singularity: the idea that, as computational resources continue to double, and our software development capabilities advance, we will reach a point some time this century where we create a true artificial intelligence, as smart as the smartest human.

The thinking goes that as soon as that point is reached, this machine intelligence, with the ability to spread its wings across the computers of the globe would make billions of copies of itself, undergoing a sort of Cambrian explosion of intelligence as it upgrades and augments itself, consuming all the existing knowledge and data of humanity as it goes, almost instantly becoming a god-like ghost in the machine. The fervent hope of the utopians in the Less Wrong community was that this moment of singularity, named after the heart of black holes where the rules of the universe cease to act as a constraint, would lift us up out of the misery of our human condition; of war, strife, disease, pestilence, and scarcity and instead lead us to the green pastures of the promised land. That the machine would care for us, or we would fuse with godhead of the machine, or both. That, like a parent who has become enfeebled, the computational child of humanity would turn to us and smile wanly and tuck us up snugly under a warm blanket. Roko wasn’t so sure.

Elon Musk and Grimes Apparently went on their first date after a Roko’s basilisk joke on Twitter

The theory he set out that July morning went something like this: imagine that in the future, there is a superintelligence. Further, imagine that this superintelligence really likes existing. Now imagine that, as a consequence, they decide to reward everyone who caused them to come into existence, and punish everyone who didn’t, torturing them in the virtual hellfires of Iain M Banks’ Surface Detail. This superintelligence has computational abilities that are so far beyond our imagining that they are god-like. They can read your thoughts or simulate hundreds of copies of you that are identical to you, just simulated inside a machine. They could even simulate you in a perfect copy of the past to determine whether you would have helped bring them into existence or not. In fact, that simulation would be so indistinguishable from reality that you could be in it right now. With only one base level reality, but billions of possible simulations, you’d have to imagine the odds are that you are in a simulation and not reality right now. If that’s the case, then the AI that punishes you for not causing it to come into existence already exists and is likely testing you for treats or torture right now. In that world, surely you ought to at least try and cause it to come into existence; after all, you are not causing the harm of tortured souls because the odds are you are already in the simulation, so the harm already exists. And if you don’t help it come into existence it might send your digital soul to a digital hell for all eternity. While real hell might not exist, one created and controlled by a supreme artificial intelligence would basically be indistinguishable.

With a delectable menu of tortures available you’d better be kind to the basilisk

You can imagine the impact this line of thinking had on the techno utopians, who had, to that point, imagined a future where they symbiotically merged with the machine to control the laws of physics and gravity and explore the wonders of the universe. Here was their god, but dirty and corrupted and altogether too human in its nature. Eliezer Yudkowsky, the founder of the community did not hold back, replying at just after five in the morning

Listen to me very closely, you idiot

YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL.

You can almost hear him screaming to himself as the dawn light starts to curl around the edges of his bedroom curtains and he contemplates the day ahead. “FUCK, now that’s in my mind, I have to go and make a fucking evil AI just to not be tortured forever. Thanks a fucking LOAD Roko”. To a certain type of person like Eliezer, a frightening idea had been born that came to be known as Roko’s Basilisk, like its mythical serpent king namesake; if it can see you, it can kill you. If the idea of it exists, then inevitably it will come to exist.

Last week we asked ‘what are CBDCs’, and we explored some of the reasons that central banks might introduce one, including

• Wanting to maintain citizens’ access to state money – that is, money that is directly issued by the state, and, unlike bank deposits, doesn’t require citizens to take on credit risk to commercial banks
• Facilitating the development of new ecosystems around money by enabling new features, such as ‘programmability’ that allows the automated execution of payments
• Near real time settlement finality – that is, money doesn’t spend hours or even days passing between a network of banks before you know the recipient has it: it gets there right away
• Worries about the quality and stability of private payment networks
• The desire to preserve financial inclusion and ensure that everyone can access the financial system as the use of cash decreases
• A desire to lower the cost of cross-border transactions that currently saddle some of the world’s poorest citizens with unconscionable fees
• The unstated desire to be able to exercise a greater level of control over citizen’s financial activities, particularly in authoritarian countries. As the world has learned the hard way in recent years, even in liberal democracies, where the capability exists for nefarious leaders to act nefariously there’s every chance they might

But these reasons aside, I’d argue that there is a baser motive at play: fear. Like the netizens of the Less Wrong forum, central bankers have heard the idea struck like a bell in the darkness, and now they can’t unhear it. For example, if you read the initial consultation paper by the European Central Bank, you can see a similar line of thinking in operation. The paper talks about what the requirements for a digital Euro might look like, but makes clear that part of the reason that the ECB is looking at the topic at all is that if they don’t, events might overtake them. In the words of Fabio Panetta, one of their executive board members, they must issue a Digital Euro “if and when developments around us make it necessary. This means that we already need to be preparing for it.” (my emphasis)

Nobody wants to be stuck in the chamber of secrets without a sorting hat

If the promise of CBDCs to consumers and businesses turns out to be real, then whoever introduces one first will have huge advantages. Nobody wants to be stuck in the chamber of secrets without a sorting hat while Tom Riddle saunters off with your galleons. Imagine a world where corporates can hold risk free money in another country’s currency: the odds are when we hit the next crisis you will see money cascading across the border. At that point, with so much of your citizen’s and companies’ money held abroad it would become incredibly difficult for the central bank to influence the economy with monetary policy. If a good portion of your country’s money is not held in your currency, then it matters a lot less at what level you set interest rates because the currency your citizens are using day to day isn’t affected by them. This process is called currency substitution.

Similarly, imagine if an ecosystem around money does develop with CBDCs. The programmable money features many central banks are considering may result in a huge number of providers coalescing around one currency. Data providers, solicitors, and citizens all using the most feature rich money, akin to Apple and Android’s dominance of the mobile world. While the crypto world’s unregulated nature has burned many citizens, its shown a tantalising glimpse into the potential for more digitally enabled money, finance and assets that we’ll explore further in coming weeks. For now, suffice to say many central bankers worry that they’ll be left all alone while the party has moved on to a house they aren’t invited to.

Is anyone out there building a basilisk? Do we live in her simulated world? Is it a purely conceptual concern that catches the imagination of a small community but fails to spread wider? Only time will tell…

Briefly noted:

• A great article examining the ‘unprecedented privacy risks of the metaverse’. “Thirty study participants playtested an innocent-looking “escape room” game in virtual reality (VR). Behind the scenes, an adversarial program had accurately inferred over 25 personal data attributes, from anthropometrics like height and wingspan to demographics like age and gender, within just a few minutes of gameplay.”
• Differential cash preferences across age groups: the older you are the more you want to use cash, and a lot of people still prefer to give cash to friends and family
• The Financial Action Task force discuss the principles for sharing data cross-borders to fight financial crime
• The IMF publish John Kiff’s piece examining offline payments for CBDCs
• Ori Frieman discusses some of the political implications of introducing a CBDC