Should we lower the close/reopen vote threshold from five to three?

Question

Update June 22, 2020: With no objection and consensus among the mods, we've decided to elevate this to the CM team.

---

Background

Currently, it takes five votes within 14 days from users with the sufficient privileges to close a question (you can read more of the specifics about closing and reopening questions here in the help center). This is the default across the network.

Recently, Stack Overflow reduced this threshold to three. There is a TON of discussion on that post and others about why the threshold should be lowered, but the ultimate goal was increase the efficacy of individual actions. In other words, when a user with the close/reopen privileges votes, there should be a good chance that this will result in a question being closed/reopened.

Why lower the threshold?

Why does this matter? Imagine that you've voted to close a few questions, hoping to maintain a high quality of questions on the site. You notice that it rarely does any good. Bad questions just sit there until your close votes age out. Why would you continue voting to close? If we lower the threshold, there's a better chance that enough votes will accumulate to close the question, and you'll be encouraged to keep voting.

The opposite is true as well -- if you notice that closed questions are getting edited and improved, but your reopen vote never makes a difference, why would you try? A lower threshold means that not only will more questions be closed -- but more closed questions will be reopened as well!

This will not only help improve the quality of questions on the site, but it also encourages users to edit closed questions, since it increases the chances that those edits will trigger a question to be reopened.

Did it work?

Yes. SO ran a two week experiment where they tracked efficacy (how many questions completed the close/re-open process before the votes timed out) and actioned of count (how many times a close/reopen vote resulted in closing or reopening, respectively). This table summarizes their results:

Basically, when they lowered the threshold, there was a significant increase in the chances that when a user cast a close or reopen vote, the question would eventually be closed or reopened (respectively) before their vote timed out. AND, any vote increased the chances that a question would be reviewed by enough other users to reach a verdict (perhaps a user votes to close but a closure isn't justified -- the lower threshold increases the chances that other users will engage the question and reach a "leave open" verdict).

Is there precedent for other sites doing this?

Yes. After the SO experiment, Hardware Recommendations.SE ran one of their own. They had similar results, and ultimately lowered their threshold to one vote. That's a bit extreme for us, but Hardware Recs gets a high volume of questions, so being able to clean those up faster was a priority.

What next?

Let us know if you like this idea -- a simple up or down vote will suffice, or feel free to opine in an answer.

We'll wait until 12 June 2020 (two weeks from today) to allow plenty of time for feedback. At that point, we'll then escalate this question to the Stack Exchange community management team (using the new process started in March) to implement the change.

Answer 1

This change is live now.

It's worth noting that, over the course of a month, you don't get a lot of close-worthy questions - at least, you don't close a lot of them. Of those, all of them were closed by the moderators either alone or with votes from other users.

This is visible in the chart below - With the exception of August 2020, when 12 questions were closed, you have less than half that in the average month that get closed (between 1-5). If you're trying to find the "moderators closed" and "moderators reopened" lines, they're identical to the "all closed" and "all reopened" lines.

As such, this change may help out here. Hopefully it will also help with getting the percentage of questions that are nominated for closure, handled - while it's often in the 100% range, there are times it's much lower, in the 50-80% range. This doesn't mean there's a big issue with posts that should be closed, going unclosed - since so few posts get closed here, it's really only 2-3 posts per month when it's low.

Hopefully having the votes needed to close set at 3 will help get everything up to 100% but you'll still need to keep an eye on the review queues to catch those posts while they're in review.

Answer 2

Yes, we should change the close vote threshold to 3.

---

Can we quantitatively determine whether this is a good idea? What statistics can we gather to support the change? It's hard for us to run an experiment on this site because we get less traffic than Stack Overflow. We can, however, look back retrospectively at what has happened in the past.

I prepared a query on Stack Exchange Data Explorer (SEDE) that shows all the questions that were successfully closed or a close attempt was made. A total of 106 questions are listed, including some that were originally closed and later reopened.

Community Autonomy

The community is autonomous if it has the power to act without moderator intervention. Here's a histogram of number of close votes accumulated before the question was closed. The orange bar represents questions that completion of the close vote by community members. It's clear that autonomy is low right now.

Of the 106 questions that were ever closed:

• 36 were closed unilaterally by ◆moderators.
• 50 were closed as a joint effort.
• 20 were closed by members of the community.

Scenario: Close after 3 votes

All posts that received at least 3 close votes from community members are highlighted in orange. All the posts with 3 close votes have 2 from community members and 1 from a ◆mod so I haven't counted them here, but it's likely some of those would have also been closed by community members before moderator intervention.

Of the 106 questions that were ever closed:

• 36 would have been closed unilaterally by ◆moderators.
• At most 22 would have been closed as a joint effort.
• At least 48 would have been closed by members of the community.

It's obvious that lowering the voting threshold makes the community more autonomous; here I have quantified how much the power would shift.

Accuracy

However, simply closing more questions is not a good metric for success. Ideally the right questions are closed, and they stay closed. For that, we should consider which questions were re-opened.

• Zero questions (out of 18) were reopened after receiving exactly 3 close votes from community members (and a final vote from a moderator).
• Zero questions (out of 10) were reopened after receiving exactly 4 close votes from community members (and a final vote from a moderator).
• Two questions (out of 20) were re-opened after receiving exactly 5 close votes from community members, but only after material edits were made.

These are stunningly good numbers. They show that reducing the close vote threshold would not have introduced any new errors (a close followed by a reopen).

Conclusion

Reducing the close vote threshold to 3 would improve community autonomy, and would likely not lead to an increase in improperly closed questions.