Insights from running Yes/No Debates

After coming up with the Yes/No debate format in 2018, I ran such debates both myself or facilitated them between other people, be it in workshops or online.

Workshops were held with audiences from:

Online debates were run on Reddit.

Spoken debates

Structure

Spoken debates at workshops were organized like this:

  • Participants filled out a form with ideas, stating whether they agree to them or not (and how much, on a +2 to -2 scale). We used a lot of items from Julia Galef’s List of unpopular ideas.
  • Participants then were put together into random pairs.
  • Their first task was to find an idea
    • to which they had opposing views, and
    • they wanted to debate about.
  • They were made familiar with the the rules
  • and started debating.

Learnings

Allow intros

First learning here: It is hard to start asking if you don’t know anything about the other person’s reasoning on the issue. So in later runs, participants were allowed to start with a short input talk to each other.

Channel meta-talk

Another observation: It happened that participants “deviated from the protocol”, i.e. started meta-talk. Like:

  • asking back about how a question was meant
  • giving a counter-argument instead of asking a question
  • complaining about the restrictions of the format :)

Generally, meta-talk is not bad, especially if it’s about clarifying a question. But it should not take over the conversation and therefore, being able to call a moderator was useful.

Moderators can also be useful when participants cannot agree whether a question contained a false premise, although this did not happen so far. (BTW, from all possible answers, false premise was given the least.)

Asking is harder than answering

For me, the most interesting insight was that coming up with a question was much harder than answering one. I saw two reasons for this

  • First, crafting a question forces you to realize what actually makes you believe what you think: What are the relevant facts for you, and in what way do they lead to your belief?
  • And secondly, you have to put yourself into your partner’s mind, remember what they have answered before, and come up with a reasonable follow-up question.

This is apparently much harder than simply answering with a Yes or a No (or Depends, … etc).

But I see this asymmetry as a good thing: In regular debates, participants crave being allowed to speak, and then often fill that airtime with as many as possible thoughts and arguments that they have, not checking whether they are relevant for their debating partner.

So at Yes/No, the incentive is different: When you speak, you must ideally say things your partner agrees to (answers Yes or Depends), otherwise your airtime is over. That way, seeking as much as a possible common ground or at least steelmaning is encouraged. And strawmanning gets caught immediately.

Asking in spoken debates seems harder than in written ones

However, there seemed to be also a difference between spoken and written debates regarding formulating a question:

In spoken debates, even if participants agree that it is OK to take your time, being silent for 1 minute or more while thinking seems awkward. Not so in written debates (on Reddit), where it is even OK to pause and think for a whole day.

So in practice, participants in spoken debates feel much more pressured to quickly come up with a new question. This can harm their quality.

Written debates

Structure

For written debates, a call for volunteers was sent out to 3 rationalist-minded subreddits. Volunteers were asked to fill out a form that included a list of ideas and tell, to which of them they (strongly) agreed or disagreed (or did not want to debate them).

A total of 16 participants joined.

From the answers, I matched pairs of participants, based on ideas they had opposite views on.

For each debate, the participant agreeing with the idea created the initial post, giving their intro. Their partner, disagreeing with the idea, then replied with their opposing summary, and the debate started with the OP asking the first question.

Eventually, 5 debates (with 10 participants) started.

After a week, participants were asked to fill out a feedback form.

Feedback

Turnout was 8. This is of course a very small sample size, yet I still want to summarize the feedback:

  • The vast majority could easily follow the rules.
  • A few participants found the rules limiting.
  • Around half of them could understand their partner’s position better.
  • Compared to previous debates, around half of the participants had a better overall experience with the Yes/No debate.

Quotes from participants

One participant enjoyed this:

It was surprisingly good to have a focal point to respond to. Even if they posted a wall of text explaining their position, the question was the only thing I felt required to address, so it took less mental energy to debate than it would have taken in a different format.

A few times this improvement around role-changing was suggested:

I would prefer multiple discrete points with the ability to say agree/disagree/magnitude with short blurb on why for each

We ended up not totally following the rules for who would ask the next question, because there were some long stretches of “Yes” answers so the question-asker did not flip per the rules, but we changed anyway.

Multiple questions at a time would help speed it up

What next?

Change the rules?

I have pondered about changing the rules according to the feedback, e.g. related to role-changing. But I am reluctant, yet only for the lack of a consistent, better idea. If roles shall not change by the answers, we need another trigger, e.g. time or number of questions asked.

However, as a general remark: Treat the rules as something that serves you, and not the opposite. They are flexible, as long as this leads to more insights into your disagreement. Roles-changing for instance can also happen (or not happen) if both participants agree to it.

Volunteers constantly needed

With the subreddit, we have found a great platform for running Yes/No debates. Starting one is as easy as simply posting an idea you agree with, with a short summary of your arguments. And joining a debate is as easy as replying to a post.

So I will be happy if more people do it, and even more, if you fill out the feedback form after your experience.