3

Update June 22, 2020: With no objection and consensus among the mods, we've decided to elevate this to the CM team.


Background

Currently, it takes five votes within 14 days from users with the sufficient privileges to close a question (you can read more of the specifics about closing and reopening questions here in the help center). This is the default across the network.

Recently, Stack Overflow reduced this threshold to three. There is a TON of discussion on that post and others about why the threshold should be lowered, but the ultimate goal was increase the efficacy of individual actions. In other words, when a user with the close/reopen privileges votes, there should be a good chance that this will result in a question being closed/reopened.

Why lower the threshold?

Why does this matter? Imagine that you've voted to close a few questions, hoping to maintain a high quality of questions on the site. You notice that it rarely does any good. Bad questions just sit there until your close votes age out. Why would you continue voting to close? If we lower the threshold, there's a better chance that enough votes will accumulate to close the question, and you'll be encouraged to keep voting.

The opposite is true as well -- if you notice that closed questions are getting edited and improved, but your reopen vote never makes a difference, why would you try? A lower threshold means that not only will more questions be closed -- but more closed questions will be reopened as well!

This will not only help improve the quality of questions on the site, but it also encourages users to edit closed questions, since it increases the chances that those edits will trigger a question to be reopened.

Did it work?

Yes. SO ran a two week experiment where they tracked efficacy (how many questions completed the close/re-open process before the votes timed out) and actioned of count (how many times a close/reopen vote resulted in closing or reopening, respectively). This table summarizes their results:

enter image description here

Basically, when they lowered the threshold, there was a significant increase in the chances that when a user cast a close or reopen vote, the question would eventually be closed or reopened (respectively) before their vote timed out. AND, any vote increased the chances that a question would be reviewed by enough other users to reach a verdict (perhaps a user votes to close but a closure isn't justified -- the lower threshold increases the chances that other users will engage the question and reach a "leave open" verdict).

Is there precedent for other sites doing this?

Yes. After the SO experiment, Hardware Recommendations.SE ran one of their own. They had similar results, and ultimately lowered their threshold to one vote. That's a bit extreme for us, but Hardware Recs gets a high volume of questions, so being able to clean those up faster was a priority.

What next?

Let us know if you like this idea -- a simple up or down vote will suffice, or feel free to opine in an answer.

We'll wait until 12 June 2020 (two weeks from today) to allow plenty of time for feedback. At that point, we'll then escalate this question to the Stack Exchange community management team (using the new process started in March) to implement the change.

| |
  • 1
    I like that you specified the date at which we move on to the next phase. đź‘Ť – Nic May 29 at 16:28
3

Yes, we should change the close vote threshold to 3.


Can we quantitatively determine whether this is a good idea? What statistics can we gather to support the change? It's hard for us to run an experiment on this site because we get less traffic than Stack Overflow. We can, however, look back retrospectively at what has happened in the past.

I prepared a query on Stack Exchange Data Explorer (SEDE) that shows all the questions that were successfully closed or a close attempt was made. A total of 106 questions are listed, including some that were originally closed and later reopened.

Community Autonomy

The community is autonomous if it has the power to act without moderator intervention. Here's a histogram of number of close votes accumulated before the question was closed. The orange bar represents questions that completion of the close vote by community members. It's clear that autonomy is low right now.

36 questions closed with one vote, 11 questions closed with two votes, 11 questions closed with three votes, 19 questions closed with four votes, 30 questions closed with five votes.

Of the 106 questions that were ever closed:

  • 36 were closed unilaterally by â—†moderators.
  • 50 were closed as a joint effort.
  • 20 were closed by members of the community.

Scenario: Close after 3 votes

enter image description here

All posts that received at least 3 close votes from community members are highlighted in orange. All the posts with 3 close votes have 2 from community members and 1 from a â—†mod so I haven't counted them here, but it's likely some of those would have also been closed by community members before moderator intervention.

Of the 106 questions that were ever closed:

  • 36 would have been closed unilaterally by â—†moderators.
  • At most 22 would have been closed as a joint effort.
  • At least 48 would have been closed by members of the community.

It's obvious that lowering the voting threshold makes the community more autonomous; here I have quantified how much the power would shift.

Accuracy

However, simply closing more questions is not a good metric for success. Ideally the right questions are closed, and they stay closed. For that, we should consider which questions were re-opened.

  • Zero questions (out of 18) were reopened after receiving exactly 3 close votes from community members (and a final vote from a moderator).
  • Zero questions (out of 10) were reopened after receiving exactly 4 close votes from community members (and a final vote from a moderator).
  • Two questions (out of 20) were re-opened after receiving exactly 5 close votes from community members, but only after material edits were made.

These are stunningly good numbers. They show that reducing the close vote threshold would not have introduced any new errors (a close followed by a reopen).

Conclusion

Reducing the close vote threshold to 3 would improve community autonomy, and would likely not lead to an increase in improperly closed questions.

| |
  • What about questions that were edited (and improved) after being closed? – LShaver May 29 at 17:48
  • 1
    @LShaver There were 2 of those; I described them in my last bullet point of the accuracy section. I'll try to rephrase it to be more clear about what I was measuring. – Nic May 29 at 18:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .