Thank you.

I fully support that. That’s a very, very good way of putting that.

Whatever story we create in the 21st century in order to deal with the new challenges, we should constantly ask ourselves this question. Who actually suffers? Remember that everything we do is in order to alleviate that suffering. Then we are on safe ground.

You believe in the nation or in some god or in a corporation or whatever. You want to know if it’s real, ask whether it can suffer. A nation cannot suffer. Money can’t suffer. When the dollar loses its value, it doesn’t suffer. Computers, too. Code, as far as we ...

I would say that the test of reality…Reality is still there. Behind all the codes and all the stories, reality is still there. I would define reality by suffering. If you want to know whether something is real or not, whether the hero of your story…

It’s the same when we look to the future. We need to create new stories to unite humankind, but we have to be extremely careful to remember that it’s all done in order to alleviate suffering.

If you think about something simple, like a game, like football, obviously, we invented football. It’s fun. It’s nothing bad about it. If you start beating up or killing people because you lost the game, then that’s a problem.

The key thing is to create stories that serve us without being enslaved by them. The danger that humans constantly face is that they come up with some big story to help organize society. Then they forget it’s just a story we invented. They get trapped in it. They start ...

Money, obviously, has no objective value. It’s only here that money has value, in contrast to, say, a banana that has an objective value. I can eat it. It sustains me. It’s not bad. Without money, we couldn’t have trade networks like we have today.

We cooperate because we believe in imaginary stories about gods and nations and money even though these things exist only in our own imagination, even only in our own mind. This is not bad. This is the bedrock of almost everything we do.

I would say that humans are storytelling animals. We rule this planet because we are the only animal, as far as we know, that can create imaginary stories and believe them. This is the key for cooperation among humans.

With a well-informed population in democratic countries, instead of wasting resources on policing the people, you can actually benefit from their initiatives. This is the best way forward.

Many of the countries which have dealt with the epidemic the best – whether it’s in East Asia, like Taiwan or South Korea, whether it’s New Zealand, Greece, or Germany – they are democracies, because generally, a well-informed and self-motivated population is far more efficient than an ignorant and policed ...

The example of Taiwan proves the opposite, and not just Taiwan. In this pandemic, it’s true that authoritarian PRC have dealt with the epidemic better than the democratic USA, but they are not the only examples.

Also, a very important thing is that some people think that to deal with these emergencies, whether the pandemic or global climate change, we need some kind of authoritarian regime that will tell everybody what to do. Otherwise, there is no way to reach a consensus.

To be a good nationalist, you also have to be a globalist. There is no contradiction. I think we are seeing it with initiatives like the one we just heard in Taiwan.

There is no contradiction. Nationalism is about loving your compatriots, not about taking foreigners. In many situations, like in this pandemic, or with global climate change, if you really love your compatriots, and you want to take care of them, you have to cooperate with foreigners.

Yeah, I think that with many of these global challenges, the solution has to be global. Of course, it’s rooted, it’s based in individual countries. The most important thing is not to fall into the trap of thinking that there is a contradiction between nationalism and globalism, and that we ...

In this way, hacking human beings does not necessarily mean imprisoning them in their own previous preferences and biases, it can lead to unprecedented variety and exploration. So it can really go in a lot of different ways.

But maybe on Tuesday, at 11 o’clock, I am very tired, or I am concerned about something. This is the worst moment to try and introduce me to jazz, or to Indonesian gamelan music. The AI will know that actually at seven o’clock in the evening, on that particular day, ...

In the traditional way of school, you go to music class. So music class is every Tuesday, at 11 o’clock. That’s it. This is when you are supposed to be exposed to new kinds of music.

But the opposite view is that, no, because it knows me so well, it also knows the best way to expose me to new musical tastes. Sometimes when you try to much, then it backfires. So it knows that 10 percent of the music that it gives me would be ...

If you think about something like music, let’s say I have a particular musical taste. Now one vision of the algorithmic sidekick or the algorithmic mentor is that the AI learns what I like and just gives me more and more of that and kind of imprisons me in the ...

I think the idea of an AI mentor doesn’t imply a single trajectory or a particular value system. Just the opposite. It can actually encourage exploration and a wide breadth of interest even more than traditional education systems.

You don’t really know where are these decisions coming from. A human can’t go over all the data and understand it, and these decisions shape the values of the child, as she or he is growing up. So again, I don’t have a fixed opinion about it. I don’t think ...

But if you start with a baby, or a young child, and more and more decisions about the life of the child are taken by an AI mentor, again, not an evil mentor that actually serves some corporation, a mentor which is supposedly really serving the interests of that child ...

I’m now 44, so my values have been shaped by decades of experience. If an algorithm now increasingly makes decisions for me, the algorithm will still find it difficult to change my core values.

So yeah, that’s just one possible scenario. But the really deep problem of value alignment is that even if you have a democracy and you have many players, as the algorithms get to know us better, and as we listen to them in more decisions in life, they increasingly also ...

In a democracy, you need to convince millions of people to trust the algorithm in order for the algorithm to take over. In an authoritarian regime, you just need to convince a handful of people which are already primed to accept this kind of logic, that there is somebody who ...

This kind of algorithmic takeover, which I think is far, far more likely than the science fiction scenarios of a robot rebellion, can actually happen far more easily in an authoritarian regime than in a democratic regime. The only ingredient missing is for the people higher up to develop enough ...

Very soon the algorithm basically takes over the party. Even if one day the Politburo wakes up and says, “Oh, no, it’s gone too far, we’ve lost control,” it’s too late for them because the algorithm has already appointed all the lower-ranking members.

Very soon nobody in the Communist Party actually understands why the algorithm is deciding to appoint this person or to advance that person, but they trust the algorithm.

But let’s say appointing all the officials in the local cities and branches and so forth, it is increasingly done by an algorithm that constantly follows all the 80 million members of the Communist Party and collects data and analyzes it and learns from experience.

What happens if the party gives algorithms increasing control over the appointment of lower-ranking officials? Not the people in the Politburo. It’s too political. It’s too complicated.

The funny thing is the dictatorships are actually far more vulnerable to this kind of algorithmic takeover. If you think about the PRC, I often think about maybe if I had time to write a science fiction novel or a science fiction movie about algorithms taking over, my favorite setting ...

The algorithm comes to the PM and says, “Look, there is a huge financial crisis about to happen and we must do this. But I can’t explain to you why because your brain just can’t analyze all the data that I have gathered.” So even though the PM is still ...

But people don’t realize that in the long run when you talk about algorithms really taking over – not in a science fiction way of the robots rebelling and trying to kill us – it’s algorithms gathering more and more power to themselves. Even if you have a human president ...

I agree that one way to deal with this issue is to have this plurality of actors and viewpoints. Funny enough, when people think about algorithms taking over, they most fear about democracies, and democracies are vulnerable in things like election manipulations.

I don’t want this to sound dystopian or utopian, it’s just fascinating as a historian to think what kind of relationships will emerge out of this new technology.

Now we are suddenly creating a completely new kind of entity who actually knows about me far more even than my mother, and we have no cultural or historical traditions about what kind of relationship I have with my AI mentor that has all this information about me.

But my mother had my best interests in mind when thinking how to use this information about me, and we have thousands of years of experience in building the kind of beneficial parent-child relationship.

One more thing. We had this kind of entity throughout history in a way – a mother or father or teacher. My mother is somebody, who when I was 14, maybe she didn’t know I was gay, but she knew a lot of things about me I didn’t realize.

I mean, should it tell me that I’m gay? Should it expose me slowly to different contents that will enable me to realize this about myself? What is the proper relationship with this kind of entity?

But the really big issue is what if the algorithm isn’t malign, it’s not working in the service of some corporation, and I don’t know this about myself, but the algorithm knows it? There is kind of an imbalance here. What happens then?

If it’s a bad manipulation like Coca-Cola using this knowledge to sell me something I don’t need – they show me commercials with sexy guys – so I buy their product and I don’t know why, then they are using it against me.

Taking the example I began this interview with, if I think about myself when I was, say, 14, and this algorithm analyzes my behavior, analyzes wherever my eyes go. I walk down the beach, and the algorithm analyzes if I focus on cute guys or cute girls. Or, it analyzes ...

Looking to the future, again, the threat of a rising dictatorship, a new kind of dictatorship is a big one. But even if we avoid that, how to deal with the new tools for hacking the human brain, the human mind, that’s the really big question.

They take the smartest people in the world and give them this task, how to hijack people’s attention and keep them on our platform. These smartest people in the world discovered how to press out emotional buttons, the fear button, the hate button, the greed button. This is the easiest ...

You have the big corporation. They say, “Look, people are spending 30 minutes a day on our app, on our device, on our platform. We want them to spend one hour. This is your mission for this year.”

Again, the full ability to hack human beings, it’s still in the future. We are still not there yet, but even what’s been happening in the last few years is alarming. Really, you have all these algorithms and apps and devices that what they are really about is hacking human ...