Hey, how are you doing?

Pretty good, pretty good. Can you hear me? Is the connectivity OK?

Yeah, I can hear you pretty well.

That’s great.

Thanks for making time for me. I really appreciate it.

That’s just fine.

How are things going in Taiwan?

It’s a Lunar New Year, so it’s like a nine-days holiday.

I’ve been off the grid -- that is to say, off Internet -- like, maybe checking 20 minutes of mail in the past five days or so. It’s refreshing to be back.

Nice. Well, Happy New Year. I’m really happy that we have a chance to reconnect. I really enjoyed your talk in Madrid in the Open Government Conference some time back. I’m part of this very big project in Finland, where we develop democratic innovations.

The aim of the project is to tackle bubbles and biases in democratic participation. I’m leading a part of the project where we develop these democratic innovations in a hub called Democracy Accelerator.

Our goal is foster democratic innovation, study them, and then also collect case studies from all over the world, what has been done so far. Of course, then, your work there in Taiwan is really interesting in many aspects.

I’m really glad that we have a chance to talk now. I was curious to know if you had a chance to tell me a little bit more about the VR experiments that you have been doing. I’ll send you a link to the project website.

If you could recap a little bit what you have been doing, and then also, what is the current status? Are you working on something in VR in democracy currently?

VR in democracy and AR in democracy is one of the areas where we want to investigate where we could use these technologies for democratic values.

Certainly. Can you see my iPad screen?

Yes, I can. That’s really cool. Thanks for sharing.

OK, great, because the nature of VR means that there’s a lot of visuals. Glad that it has worked.

I like that. Thank you.

I’ll show you my office first.

This is my office. It’s in Taipei, called the Social Innovation Lab. I show this because we have a dedicated VR space in the lab, with a Vive and Oculus in this space.

This is where some of our early prototypes were developed. This space, I consider itself a part of the immersive experience, because the visual installations, as you can see -- for example, here, this soccer field -- is designed by people with a Trisomy-21 difference.

It turns out that they view the world through geometric means, not as text or numbers, but geometry. They started as an art therapy project, but when we realised them as public installation of art, it kind of naturally primes people, even before they enter any immersive environment, that there is a possibility of a creative space around.

The setting that embeds the VR is important as well. This is the first thing I want to note. It also helps that when people enter this place, they will see some self-driving tricycles around.

These PEVs are from MIT, and people tinker on them all the time. If people don’t like that it looks like cyclops, like one eye pictured here, they can experiment the build with two eyes. It can make eye contact now, as well.

Again, it shows a kind of not exactly virtual reality, more like a mixed reality, with a collective intelligence approach, that primes people into the mindset that, "Oh, this is actually whatever you see in VR is not a fiction." It is not interactive fiction, but rather interactive nonfiction, for lack of a better term.

They could be realised without losing much fidelity. Those robotic avatars that you encounter in VR do have their real space, physical space counterparts as well. That’s the introduction. One-sentence introduction.

Now, as for the actual experiments we did, we did around four different experiments. One that you have heard, perhaps, is called Holopolis. holopolis.pdis.tw is the website. In Holopolis, we mostly test the interactive part of the Polis infrastructure.

I don’t know whether you’ve heard of it, but it’s a visualisation of people’s sentiment around common ideas. We use Polis for many different things. This is the very first place where we used Polis as a two-dimensional technology. This is the Uber case, that shows people their different groups.

Most importantly, when the same facts is shown and curated to people, we ask people, "How do you feel about those facts?" without jumping into interpretations. Usually, the UI in two dimensions looks something like this.

You see a statement from your fellow citizen, you press agree or disagree, and then your avatar moves toward people with similar ideas, similar thoughts. Then it shows you this beautiful report. This report, the most important thing here is that it shows that people only disagree on a few things.

Most people, much to their surprise, agree on most of the things, most of the time. It’s just social media tends to focus on the divisive parts, or popular media nowadays. Showing people that this chart itself, and saying that we hold people’s rough consensus as agenda to face-to-face meetings, that, by itself has potential.

The thing with Polis is that it is very textual, in the sense that if you are not a person versed in textual modalities of just typing in statements and pressing yes or no for 10, 12, statements, then the contribution is less than satisfactory.

For people who are more of an interactive modality, we use immersive technology in the Holopolis project. There’s two prototypes. One is immersive rather than mixed. In immersive reality we use highfidelity.io, or Hi-Fi, which is the open source VR platform. highfidelity.io.

Oh, I can’t see your...Oh, here it is, High Fidelity.

Yes. We build the space where people who are less textually versed, for example, they may be 12 years old, and they may wear those VR Vive head mount and enter into a space where they can speak to my avatar.

I have 3D-scanned myself into the space, and then we just hold on a natural conversation, but with two changes.

The first thing is that I lowered my avatar to the same height as the schoolchildren. They feel that I am kind of their classmates, because, in reality, I’m like 1.8 meters tall.

They really do have to look up to me — literally — when we facilitate the conversations like this. When our avatars are of the same height, and we put each other in a familiar environment to them, then that means that they feel much more free to speak aloud.

That’s the one intervention that we did. The other thing is that when we started talking about things like this, we don’t use the attenuation as in physical space. Basically, it’s like Skype, there’s only directionality of audio. There is no distance of audio.

When I read aloud a statement, many, many different rooms can hear in the same time. I hear them too, but with equal attenuation, and I can serialise them too; I don’t have to hear a cacophony.

To the participants, they basically are interacting with a semi-scripted avatar of me. That is the kind of one-sentence summary.

We are also bringing these principles to the physical space as well. In the Social Innovations Lab, there is a room designed for projections. Basically, I tour around Taiwan, like to the remote places, to the rural places, to indigenous tribes to talk with elders, and so on. You can see that this space is a typical meeting room.

When I travel to a region, back in the Social Innovation Lab, we get up to 12 different ministries related to the local development. They are all public servants in the central government.

With the right arrangement of projectors, they just see through my eyes what the local peoples are and what their issues were, what are the common reactions to those agenda topics crowdsourced before the meeting.

Then, when they have questions, instead of shuffling the text back and forth, they just hold a face-to-face conversation.

There is also a green screen in the Social Innovation Lab, dedicated to indigenous languages.

In Taiwan, there is 16 nations, with 22 languages. As of two years ago, they’re all national languages. Meaning that if I go to the tribe and talk to the Amis elders, first, we have to prepare our open policymaking kit in the Amis language.

That’s amazing.

As something that indigenous co-ops...There are co-ops who specialise in information technology work. The IT co-ops of the indigenous tribe first need to understand and accept the tools.

Second, this place, the Social Innovation Lab, has a green screen and a professional interpreter who talks to those tribal children virtually every morning to build a virtual language circle. Using green screen and so on, they were able to build a semi-immersive environment of showing the traditional lands of those cultures.

So that even people who are not of the Amis nation, for example, can feel that they are in the Amis nation, and talking through the lens of the Amis nation understandings, the environ. The whole milieu is set up so that even I don’t speak Amis.

My open government textbooks and everything, comics actually, are translated to Amis. The Amis people can read it before we enter such a deliberation. Whatever utterances that I speak currently is interpreted by Amis interpreter.

The local people feel that they are in a...It’s the same idea as lowering my avatar to the 12-years-old, but this time kind of casting my avatar into a pseudo-Amis that is scripted a little bit and read aloud by a professional Amis translator.

I don’t know how to say about this, but the movie "Avatar" actually does a pretty good job. Just think toward that direction. I think that more or less describes the effect of what we’re building towards in the Social Innovation Lab.

One in VR, and one outside, but using composite technologies so that they receive the projections as some kind of aesthetic image. This is a regular tour. I do this every other week or so.

There’s many early experiments as well. I talked to two mayoral candidates about building a deliberative space for public construction; although they did not become mayors, one of them become our Deputy Premier, so we’ll see how that goes. [laughs]

The original intention is to place people in future versions of public constructions, so that they don’t have to blindly follow an architect’s conversation or not.

They can decide for themselves, immersed in that place. It requires, of course, a lot of modeling. So far in the past couple years, we’ve just building the base databases. Of course, the self-driving cars also need that database.

We also build, for example, the historical buildings in the Taiwan model library, the Taiwan Digital Model Library. When people really need to talk about the future of these historical buildings and so on, they can go to the TDAL, the Taiwan Digital Archive Library, I think that’s the website.

It’s pretty impressive. It’s ready to import into any VR space, but we have yet to use any one of them in a real deliberation or conversation.

Sounds great. That’s really impressive. In terms of research, what type of research questions would be of interest to you, in terms of the projects you have done? If there was any research to be done that you thought would be relevant, what type of questions you would like to see addressed?

We’ve been working with some visiting researchers, actually. human geographer. Like cultural anthropology, but focused on space instead of people. I hope I got that right.

She did quite a bit of ethnographic hanging out in Taiwan, as well as in Madrid, and comparing the participatory space-building in the two places. I think her focus is on the collective empowerment and understanding of this technology.

With VR, as well as with AI-moderated conversation, the tension is always that even though we really do open source everything, we really do practice development in the open, and so on, it is, frankly speaking, not as well-understood as paper ballot or as a physical town hall.

Which people were raised up knowing how that works, since they were seven years old. When adults were seven years old, there is no VR or AI-moderated conversation, for that matter. The dispersion of know-how -- and not just know-how, know-why -- and how quickly people can get accustomed to set up something like this, but for their lower-scale ideas.

Like not as fully developed as a full policy, but just a gut feeling. I shouldn’t say lower-scale, but less developed ideas. Just feeling the agency, the empowerment, to hold up such conversations, even without a twenty-person team, like my office, to back it up.

I think that was the main interest of just, how do you democratize this technology without having it be a de facto authoritarian agenda-setting environment? How to make it truly an agora, rather than some pre-defined space in a castle is, I think, her main focus. Personally, I’m very interested in that dimension, as well.

I’m curious about, one of the research interests that I have in this project is to see how could we use VR in a way that would help people to understand the implications of policy decisions or different facts, different background information that help them make policy decisions.

I’m thinking about buzzing out different ideas about creating VR in unity environments to show some simulations of policy decisions, or 360-degree video to show some environments that these politic decisions are about.

Then studying the implications of these different environments, visual as compared to some more traditional ways to give information to people. That’s what I’m trying to explore. What would be the most meaningful direction to take?

Right. I can use a few examples, because in our office we have 360 cameras that are just one-touch to use. It’s very easy to use, so we use them all the time. We use them during our regular face-to-face deliberation meetings, but people can participate using this kind of actually just regular YouTube technology.

This is how our collaborative meetings look like. It’s just standard ideation stuff, where we built the tax filing software together. Anyone can, even if they are not in Taipei, they can participate virtually. Truth to be told, most of the people didn’t use a VR headset on YouTube.

They do use, for example, their iPad, which they can rotate around, and the spatial audio also helps. First of all, the source of that information is not synthetic. It is just an authentic conversation with the VR device mounted at the middle of the conversation space, so that it doesn’t give bias to the camera personship.

That is one way we’ve been using this. Very mundane, but very effective. The second this is that when we’re talking about the conservation of marine life in a marine national park, there’s teams who contribute videos -- and later, 360 videos -- so that people understand how is it like to be a diving person, a person who fishes, or so on in that marine space.

That is kind of far away. Very few people who deliberate, even if they are in the Pescador Island, have been underwater. Again, this is crowdsourced or participant-sourced, in the sense that we help them to get equipment, but they do the narrative themselves.

There’s also a case where we talk about the motorcycles, whether they can turn left, or they have to wait for a red light and turn left using an L shape, like bicycles. It also helps that the motorcycle people, who did the petition, use a Go Pro to wide angle head mount what he actually sees during those two different ways, in various different settings of public roads.

I remember the video. It’s not really VR, but with one wide-angle it could be like 175 degrees, and it is immersive enough that when we put it on our large projector for people who didn’t ride any motorcycles before in their lives, like me, I can actually understand perfectly what they are talking about.

That usually when we prime people before our collaborative discussion, each side, each stakeholder group, can bring one five-minute -- at most ten -- video of a VR, 180, or whatever, or just PowerPoint from any dimension.

We rarely curate them ourselves. Rather, we work with specialists to give them the tools to make convincing, informative videos, but always starting from their first-person viewpoint. That’s our practice so far, but I must stress it’s not democratic enough. It still takes much more time than using your phone to shoot something.

I like the idea of this 175 video, because there’s a debate around, do we even need 360 degrees? Maybe 180, or 200, 175, would be enough for the feeling of sense of presence. I find that really meaningful.

There’s [indecipherable 24:22] technologies I’ve heard of you capture such video, and then also watch it. The other interest of mine in this project is crowdsourcing. In Finland, we have a done massive amount of crowdsourced policymaking.

I can show you, we have crowdsourced [indecipherable 24:39] lawmaking privacy. If you go to a website called www.thefinnishexperiment.com -- I’ll share you the link here -- you’ll see all the projects we have done.

We have studied many different aspects, people’s motivation factors to participate in crowdsourced policymaking. These projects work so that any ministry in Finland who wants to crowdsource people’s feedback, they will give instructions online for people to participate.

Then it’s publicly, freely open for anybody to participate. Then the government officials input the crowd’s input, and then decides how it’s going to be applied in a law.

The domain, I don’t think the domain works, Finnish experiments?

Let me see. I think the network is getting worse. I’m calling from home to...

I’m just saying that the website, I don’t get quite get the website. When I type finnishexperiment.com, I couldn’t open it... It’s fine. I can just listen.

I’m trying to look up what’s going on with the website. Anyway, I want to take the next step now, take these models further, and study more of maybe digital storytelling in crowdsourced policymaking, and so on.

First, I’d like to hear, because I recall you mentioned we the maverick talk about crowdsourcing -- also, you mentioned here a couple of times about crowdsourcing videos from people -- have you used crowdsourcing in policymaking, policy decisions, or urban planning projects? Do you have any cases like that?

Yes. We held around 40 what we call collaborative meetings. They are, strictly speaking, all crowd-initiated. This is the core values of my office. I just want to put it out there, because for many crowdsourcing activities, the civil servant’s workflow...

The Finnish Experiment, I got it. thank you. [laughs]

Oh, yeah. Sorry about that.

No problem. It’s the one Finnish experiment. It’s not just any Finnish experiment.

It was supposed to be one and only crowdsourced experiment, but then it just has continued.

Now, there is many, but it’s still called The Finnish Experiment.

Exactly. That’s the spirit of reincarnation. Anyway, what I am getting at is that from my understanding, the civil servant’s workflow here is really key, because...

I love this graph, by the way.

OK, cool.

I love this graph. I love it.

This is the core values and the practices that we do. The green ones are the artifacts that people see. So far, we’ve been talking about the green artifacts. I want to take a couple minutes to talk about the core values, because these are really what drives our experiments. Taiwan’s experiments, but also global ones.

The key here is actually that I think during crowdsourcing, the trust between citizen and government is enhanced. If you do it in an accountable way, this is widely agreed. The core value does not need to be disputed.

What we have found is that if we are not in the flow of work for the public servants -- back when I was not a digital minister, but just adviser to the Minister in charge of Cyberlaw -- we found that if we do this outside of the flow of work of the public servants using expert intervention teams or whatever.

Then actually to the civil servants, they learn the effectiveness, but they don’t learn the intricacies of crowdsourcing. By the time that the experts moves away to some other things, the civil servants lack the judgment to actually embed this into their workflow.

We insist that all the live streaming, for example, are controlled, are set up, by public servants. We coach them, but we don’t take over. All the small round table discussions were led, or at least co-led, by a dedicated team of participation officers.

There’s a team of POs in all the 32 ministries. Their work is to embed these elements piecemeal into their workflow so that workflow can be streamlined, and they can get higher quality life, basically. I think even though most of the crowdsourcing of citizen science materials talk about cultivating the civil society organizations and so on, that is only the visible part.

The internal part is as important. A concrete example, I usually use the example of the AirBox. I don’t know whether you have seen this g0v before. If you are in Madrid, you probably have heard, at least, of the g0v initiative.

The initiative is a Taiwan-started initiative that basically says if you don’t like how the government displays the air quality, or if you think the government only has 60 different air quality measurement sites, and you want something that’s closer, the g0v motto is, “Don’t ask why nobody is doing this public service. Be that nobody that provides this public service.”

For each and every government website that ends in gov.tw, the civic hackers -- me included -- just build a g0v.tw. You just change an O to a 0 in your browser bar. Suddenly, you get into the shadow government that is crowdsourced and citizen-led.

A concrete example is that more than 2,000 people in Taiwan set up these very cheap, less than €100 air quality measurement boxes in their homes, in their balcony, and so on. They not just get the readings, but they contribute the readings to a professor in Academia Sinica, in Taiwan’s highest research facility.

That then works with the g0v people to visualize these back, so that we can see at a glance what the air quality is like in a more micro observation kind of way. We also get to see Taiwan’s digital gap. [laughs] In any case, these are legitimacy threatening move.

If people’s friends participate in this measurement and get one number, and the official environmental agency gives another number, then people, of course, are going to believe their friends, even though they are lower precision.

In many Asian countries -- actually, in pretty much all non-Taiwan, East Asian countries -- this kind of work is not encouraged. In some more severe regimes, actually, the leaders do get disappeared for making things like this.

In Taiwan, we don’t beat them — we join them. We held a series of meetings, and committed a lot of budget to useful visualization, and useful interaction from those crowdsourced numbers. From the consultation, we hear three things.

The first thing is that they need these sensors in industrial areas, where people don’t usually have the permit to enter, but actually are where the pollution came from, or at least popularly thought as where pollution came from.

We set up hundreds of spots in the lamp lights exactly in the industrial areas, to complement these small areas of shortcoming in the people’s repertoire. The second thing is that they want a point here, which is here in the Taiwan Strait.

They want to tell domestic air pollution versus non-domestic pollution. The citizen scientists cannot actually build a site there, even with drones. They run out battery. It’s actually very far away. We are building wind power turbines with oversea partners.

We changed the contract to say all the oversea wind power stations offshore need to carry those meteorological air measurement devices. The third thing is that they want a high-speed GPU to do analysis of the crowdsourced vis-à-vis government sources to collaborate their equipment.

We set up a, what we call, collective intelligence, ci.taiwan.gov.tw. It also stands for civil IoT, to show that this is not a ministry, local, or national, or whatever. It is just Taiwan. It is a shared database. We have one website, like ai.taiwan.gov.tw, si.taiwan.gov.tw, bio.taiwan.gov.tw, smart.taiwan.gov.tw, and so on for such national agenda.

Essentially, offered high-speed computing for free for anyone who can contribute numbers here. They initially didn’t trust that the government will not change their numbers. In the consultation, they raised this concern.

We worked with a university to build distributed ledger technology. I think it’s one of the very few valid use of distributed ledger technology in public policy, in that the people check in their numbers to the ledger before uploading to the national high-speed computing center.

So that they know that we cannot modify the number, because it’s on a public, blockchain-like ledger. The third thing that they ask through this centralized computation, after we deliver it, we run a competition.

The team who took the first place is also crowdsourced social media. They took PTT, which is the Taiwan equivalent of Reddit. It’s the public forum. They built a bot called Air Pollution Buster, that basically look at any conversation that may be off-topic, that may be outdated, related to air quality.

The bot just very helpfully posts and replies with a visualization of what the air quality is actually like. They build a machine translation apparatus. Instead of translating Taiwanese to Finnish or whatever, they translated press release language to netizen language.

I think it is a real breakthrough, because then the automated press releases, the press release that the public officials wrote, but nobody really see, because they are kind of boring, they were translated by this neural network into one-sentence, bite-sized summaries that raised people’s attention.

Then the payload, the body itself, of course, is still from the press release, or a summarized version of the press release. The topic is guaranteed to get people’s attention, because it’s translated with a GAN network.

Theoretically, when people reply to that clarification bot, we can also translate their netizen speak back to official language, but we haven’t done that yet. I think it’s an interesting line of research that is like what we did in immersive reality with indigenous people.

In a sense, digital natives are indigenous people. That’s one example I would like to raise.

I like that. That’s really fascinating. I’m just making notes here. This is wonderful. Thanks a lot for sharing all this information. I’m going to follow up with all these links that you shared. Is there any other resources I could find online about these cases?

Have you done case studies? Are there articles, blog posts, written resources that I could look into in more detail?

Yes, we publish, I think one of the early prototypes, using Polis and so on. We publish on the social archive. I can send you the link of the paper. It’s not virtual reality yet. It sets the ground. I can also refer you to Shu-Yang Lin, who did the Holopolis research in the collective intelligence part in Madrid.

I don’t know whether you already know Shu-Yang, but she is now based in the UK, in Bristol, I think. Two of our designers are going to be based in the UK in a couple months. She’s already in Bristol last week.

They’re still part of PDIS, still part of our office. I think time zone-wise, it should be much more easily for you collaborate, and even meet each other.

Yes, that would be wonderful. I’m in the US now, but then soon, I will be going to Helsinki for a while before I come back here. That would be wonderful. I’ll be visiting the UK in a couple of months, too.

Another thing -- thank you so much. I appreciate all these -- I wanted to ask you if you would be interested, we would like to invite you to visit us in Finland in this tackling bubbles and biases of democratic participation project.

It would be really great for our Finnish audience to hear about your work in Taiwan. They would be very much interested in it, if you would be interested in giving a talk, and maybe having some meetings with interested parties in Finland. Would that be of interest to you?

Yes. Can I send a robotic double?

What is that?

Can I send a robotic double? When Madrid invited me, they built me a 360 robot called Galatea. I just sit in Taipei, put on my glasses, and immerse myself in the environment of Medialab-Prado.

That experiment really works really well, because I can turn locally. I don’t have to wait for the video to come back. I don’t know if you have the bandwidth. I’m sure that this could be made to work, because it’s already made to work in Madrid.

When is the meeting, sorry? Which month?

That would be up to you. We would organize the event your talk. Whenever would work for you. For instance, just to name some times, the end of March, early April, May, or next fall.

It would be really valuable to get more public attention to these topics in Finland from somebody who has been working on these topics for such a long time. You have so much insight and perspective.

My suggestion is that we ask Shu-Yang or Fang-Jui, and see if they have time to travel physically to Finland, and set up a robotic avatar, so that I can talk alongside you. I think this works particularly well, because then we get to show the -- what was the word? -- telepistemology of connecting spaces, just like the initial demos that I just show you.

I think it works particularly well if we’re able to pre-agree on a setting that shows, for example, we can project things from the Taiwan Digital Model Asset Libraries so that we’re immersed in the same environment.

If you have a 360 scan of the hall, or the venue, like Medialab-Prado, you can send it to me beforehand. I can immerse myself there also. The thing is that, just to make it a real demonstration of the technology we’re talking about.

I like the idea. I’m thinking about, this would be at the University of Helsinki. It would be really nice to have the immersive aspect to it, but I don’t think that the technology is there at this point, unfortunately. We would need help in setting that up.

Do you have sufficient number of portable devices — such as Oculus Go — that you can maybe rent locally? I think that is the most easy way.

Yeah. Could you send me the information, what’s needed for this robotic performance, or appearance, so then I can look into that?

OK, sure. I’ll send you just the actual cases that I’ve been to, both in immersive modalities, as well as in robotic modality. I spoke in Geneva in a United Nation meeting through a "Double 2" robot. That solves an interesting political problem, because the robot doesn’t need passports to enter Un.

I’ve been attending quite a few UN meetings in this way. It’s an interesting hack.

I like that. [laughs] That would be really cool. I like the idea. Let’s work towards making that happen.

Sure.

This sounds great. Thanks a lot. I really appreciate your time. Let me know if ever I can help you out, or if there is any information I can provide you with that you might need.

OK, cool. I will send you all the links, as well as the contact info for our two colleagues that are going to be based in the UK.

Yes. Sounds good. Thank you so much. I appreciate it.

Thank you so much. Cheers.

Thanks. Happy New Year.

Happy New Year. Bye.

Thank you. Bye.