Because this is something that we got to ask and I said oh great question, like I have thoughts about this but we don’t have like a here is our suggested way in which you can use this in a formal fashion. I think it’s a fair answer to say, ...

I’d be very curious if you’d be able to share the the testing steps.

Right, right. Yeah, I just want to share that with you because I think this will naturally queue into some of the avenues there. Cool. Anything else from anyone?

Yeah, Guide rails.

Which I think is probably better than AI trying to synthesize is AI guiding humans to say, hey can we find common ground right now.

You can just review it with the skin but I would love to share that with you so if you have anything you want to see…

I guess the last thing here that we didn’t touch on was the decolorization tool kit. I did send… I did put the link of Bruno ‘s project on depolarized, which is…it actually has a lot of the some of the aspects on you know is it establishing common ground. ...

Right. I would love to meet Ronnie too and talk more about this too. And I’ll take a look at the Join Plus app.

That bridge is something I’m very interested in like… how can there be stronger attribution by the central decision-making authority to the original idea. Curious to hear ideas on that front on how we can establish…

I have… I don’t want to say worked with more so dealt with this in the context of US federal agencies where there is regulatory feedback, but the way in which the government signals accountability is actually quite misguided. They have to read the content and the content is not ...

Right. I mean that is where I would look to you all for example for maybe the first case in the world for like actually having a government party that is actively listening to this form of input.

Yeah. Um, cool. Anything else? About five minutes… anything else that jumps out here that we want to touch on while everyone ‘s…

Right, right. That’s why I don’t see that as an immediate thing but once it exists it will enable a lot more people. Yeah, so if you want to make an RFC for it and then say hey here’s an experiment that would also be great. But let’s not start ...

Like, one is the transcription and diarization tooling for example, that was relevant for civil as well. I’m curious, Peter, your thoughts on this because you guys already have tried to whisper for…

Basically, like one thing I blow is there’s some features that will come across any of you that want to contribute on developing them or making RFCs, that is also a great way for like… if we have RFs that come from you guys that would be actually wonderful because ...

But I don’t know what’s your timeline on that side but you was pretty straightforward at this point because that way there is more of a shelling point for more people to not have to deploy an instance but they can just create…

Two questions I have is my understanding last time we chatted was that these might not feel as fast for example. HCS article that he was interested in he wanted there to be a hosted version. Does this get hosted in Taiwan or do we host it for now? Like, ...

Yeah. It has been a while since I last chatted with her when I was in the UK. Basically, we won’t fork the two repos. Talk to the city. We should name them in the right way. But then the problem is people get confused because they are after the ...

I think it would be great. We can do a follow-up chat with Shannon as well.

So, I think this is worthwhile to explore for sure. Anything jumps out to… I’m just shifting gears because I think there is plenty of food to chew on on this avenue, is there anything else that is of interest for moda? We talked and made a priority list last ...

I studied this a fair amount in college in the context of why do more liberal political perspectives tend to fracture more. And it ended up you know in the elections in Tunisia. Like, One conservative party and 10 different liberal parties, then the conservative party wins just by having ...

You can ask this to everyone that is both in and outside the cluster, see how they see it. It may be so that people in the cluster don’t think they’re well-represented but the ones outside the cluster might think that the cluster works well.

Right. That is a valid cluster. It can be very specific, so I retract that statement. I think it would be very interesting. I would probably make it in Europe’s paper. I am not really a paper person, I am more interested in prototypes, but I am sure someone would ...

Yeah. That is why I said. That is not a good cluster. Or, it is okay for someone to say, you can be very clear and I am not sure what to do. Clarification is very obvious.

I can imagine even using a separate model to score them, and a human to be able to validate this. Like, those are very measurable. The thing is, if a cluster is… I was going to say, what if a cluster is not specific? Or a cluster is not diverse?

I am interested in the Faraday example, let’s start with this. Maybe this is is easier to do. You said how well calibrated… And that is a great sentence that has a lot of nuance. If I double click on that, what else is in there? How do we measure ...

Yeah. Assuming that will be the case.

Similar to the way we did it for accuracy of the summary, like, how would this serve? Because here we are doing accuracy of the summary, right? We are turning async clusters into a representation and we can ask does this represent. Now, if the same thing is happening for ...

That also lets us evaluate the success and accuracy of the model. There is some experiment design work that I would love help with. That is the part that is not clear to me at this point. So… We can brainstorm again.

That makes sense. That makes sense because you guys already have a context on how you want people to enter the debate. So, let’s not produce something ahead of time for it, but once the event happens, then we can start the cycle So, the summaries can be gathered into ...

One thing we talked about was we would have an open-ended survey possibly before the first deliberation. Do you think this is feasible or possible?

Right. This is the graph you drew that could not be translated with the translate button, but I figured that is what it means. And Isabel and I talked about this. This is I would love to have a little more high-definition on what’s the sequence of questions would be ...

So, I would love to hear ideas on where we go from this towards…

There was a project that came out of DeepMind last year. I’m sure you all saw it. And it’s like bike lanes are good. And I’m like yeah okay everyone agrees bike lanes are good. I’m interested in like should we have less road lanes and more bike lanes. Like, ...

I’m curious what kinds of experiments we can have that go from here towards bridge making. Where we can actually measure the models ability to actually have formed the bridge, rather than find the most watered-down consensus.

This is what I was texting you yesterday on things I want to touch on. I completely agree which is why these are the things I think are worthwhile to build language models I think this can be potentially helpful if our goal is to be able to do things ...

In some ways this is where I want to switch us… We are just brainstorming here, right? Like, where do we go from these two to be able to come to an exploration or a leap that will give us more insight towards bridge making. That’s what I’m really curious ...

And then we can represent… We can come up with two different things, right? One is representing the contributors. While the other is representing cluster 1, 2, 3. And that is these are two very different ways to represent. Here we have four LLMs. Here we have three LLMs. But ...

Say we have a couple clusters that comes out of a couple different speakers. And we see that speaker A talked a lot about this. And there is a little bit of speaker B here. This one has more division. This one is actually only one other speaker and it ...

Yeah, definitely. Sure.

Sure. I don’t actually have something to draw right now…

Say we make a couple language models that represent different viewpoints. And they will have these kinds of gaps.

And this lets us be able to evaluate and otherwise very qualitative. Like a lot of people shy away from qualitative judgment making mainly because it is hard. I don’t think it is being hard is a good reason to not try it. So, not a lot of people take ...

So, I’ll take two minutes. If you can click on the Figma of reliability experiments and AI summarization. So, part of our work is… Currently, the first thing that the reports are tackling is the ability to summarize. We are building this app called Open Questions, which should be a ...

I understand. Well, I wish it wasn’t the case. But the world isn’t short of needing for bridge making So, I’m sure we could come up with a case that’s not in Taiwan if necessary.

How do we go from here, like say we reach that step, what do you envision to go towards having that be an experiment in the journey towards bridge making and this misrepresentation being a feature in the book? What comes next from there?

So, my question would be then, I’m shifting the agenda order a little bit, but this is a natural sequence. We have four to five-hour transcripts, say one hour each. So, we have five different people. We have clusters that are produced out of this. We have language models that ...

100%. Right. That’s what I wanted this meeting to be about first, and I’m glad it’s coming back today.

Right, right. I think this is a great first prototype… I like there to be a reason for a prototype, rather than just ideation, too. Actually Audrey, I’m curious, because we talked about this a little bit as well for the extras repo. Any kind of feature orientation you’re interested ...

Yes, that makes sense.