In the beginning, there was Christianity.SE

Congratulations community, we are now just over half-way through the minimum stay in public beta! We have lots of participation and a full seven weeks of asking and answering. Honestly we have a lot of good content and some wonderful users.

However, many of us who have been here from the beginning have become increasingly concerned at the prevalence of low quality questions. At first we stood by - hoping that we could get the low quality basic questions out of the way and grow into expert, high quality questions. Unfortunately this hoped for direction is not materializing and we have realized that Christianity.SE is not improving. Instead, it seems that as we gain new users, the number of low quality questions seems to increase while the number of high quality questions diminish.

Over the past several weeks, several people have raised this issue in chat. Other concerns about low quality can even be seen much earlier in our history in meta posts like How can we get better expert-level questions?. This problem has become even more tangible since Biblical Hermeneutics went into private beta. Those of us who have participated in that beta so far have been impressed by the high quality of that site.

With all of this in mind, several of us have tried to nail down what our problem is and we have a possible solution to propose. After several long discussions, we’ve concluded the problem is two-fold: unfocused questions and unsupported answers.

Unfocused Questions

The primary problem that we’ve identified that makes a question low quality is a lack of focus. Questions like Is it a sin to get a tattoo? invite low quality responses due to their unfocused nature. Pure opinions and speculation would be completely valid for this type of question. However, opinions and speculation make for very low quality answers. Unfocused questions like these have drawn a large number of low quality answers and encouraged a general trend in opinion-based answers.

The fact that unfocused questions lead to poor quality answers can be clearly seen if we compare the previous example question (about tattoos) to a high quality, focused question such as What is the Mormon doctrine regarding becoming a god?. Clearly this second question invites high quality answers that are focused and well supported. Anyone that posts an answer to the second question will post high quality, supported answer based on doctrine*. Low quality posts (speculation, opinion, or unverifiable doctrine etc.) quite simply do not answer the question and can easily be identified, downvoted, flagged or removed as appropriate.

* Thus we see the other advantage of focused questions: they make it harder for trolls to get a word in edge-wise :)

Unsupported Answers

While unfocused questions are clearly the primary problem (as they lead to unsupported, low quality answers), we have also seen a lot of low quality answers in general. This answer in response to an excellent question regarding Catholic prayers is currently the highest voted answer. However, the answer itself is completely unsupported and purely opinion.*

*I [Richard] suggested this example but am not encouraging you to downvote it, it is after all my answer!

If we compare that low quality, unsupported answer with this answer regarding the translation of Elohim, we can see that what makes it good is its thoroughly support using biblical references and external sources. Obviously, these are the types of answers that make StackExchange a better place.

What could happen?

Our goal when we committed to Christianity.SE was to see this site succeed. (I’m sure that is true of almost everyone reading this long post.) We wanted Christianity.SE to be a place where people could come and ask questions or find answers to the theological and doctrinal issues that they struggle with.

We believe that the quality of the questions and answers around here needs to be raised. We believe this is essential in order for Christianity.SE to become a haven on the internet for intelligent questions and for doctrinal experts. We believe that without raising the bar on the questions and answers, this site may never graduate from public beta and may eventually end up failing as a StackExchange site.

Therefore, we believe we need to refine our quality guidelines in order to raise the bar on both questions and answers.

So what now?

Lets hear some voices! Do you agree that something needs to be done? Or do you think things are fine the way they are? This is an opportunity to either agree or disagree with our conclusion. This is not our site; it does not belong to the moderators or even just those with the highest reputation. It is a community effort and the whole community needs to pitch in to help choose a direction.

We will deal with how to fix this in later meta posts. At this point we are just looking to establish a baseline that there is a problem. Please use the answer section here to voice your support or disagreement and explain why.

Furthermore, you can help out with this effort by getting involved.

share
1  
How is "Is it a sin to get a tattoo?" unfocused? It sounds very very focused to me--in fact probably too focused. Can you explain what a "focused" question (on the same topic) would look like to you? –  Flimzy Oct 11 '11 at 20:05
3  
@Flimzy The problem is that it is not answerable with doctrine, only with assorted opinions. In order to be focused it would need to be edited to request the view of a specific tradition. Then good quality answers would be able to reference the doctrinal statements of what tradition for why the answer is yes or no. –  Caleb Oct 11 '11 at 20:12
6  
Also, the fact that it's unfocused invites low quality answers. Compare "Is it a sin to get a tattoo?" to the question "What is the Episcopalian doctrine regarding tattoos?" The second is more focused and lends itself to higher quality answers. –  Richard Oct 11 '11 at 20:18
1  
I don't see a problem. Low quality questions and answers get less votes (or voted down). The moderators (and others) seem to be doing well closing and deleting poor questions. I'm not convinced we need a fundamental re-evaluation at this stage. –  Wikis Oct 12 '11 at 12:04
2  
@Wikis I disagree. Some of our top voted questions are some of the lowest quality ones on the site. These are the kinds of bikeshed questions that SO and Prog.SE had in their early days, they attract a ton of views and votes, but they aren't really useful towards creating an expert site. –  wax eagle Oct 12 '11 at 12:06
    
@waxeagle: "Some of our top voted questions are some of the lowest quality ones on the site." How can you then define "low quality"? Sure, unfocused and unsupported are "low quality" - is that what you are thinking of or something else? –  Wikis Oct 12 '11 at 12:16
1  
@Wikis - unfocused primarily. The issue is that unless you are sighting in on a particular tradition or doctrine (or even a particular tradition's view of a particular doctrine) you are going to have a hard time coming up with answers that A. answer the question and B. are useful. –  wax eagle Oct 12 '11 at 12:17
2  
@Wikis What wax eagle is saying is true. Our top voted question regarding evolution and Christianity is totally open ended. Indeed the top five are low quality questions. To see this: look at each question and think: What would be a wrong answer here? Any one person's opinion is valid because the question isn't asking about facts, but opinion. We're want this to be a site about facts: a site for experts. –  Richard Oct 12 '11 at 12:35
    
Follow up post 1: What makes a good focused question? –  Caleb Oct 12 '11 at 14:52
add comment

5 Answers

Yes there is a problem, and that problem is exacerbated by the existence of BH.SE as a separate site. The problem is confounded by the fact that some beliefs and practices - doctrines if you like - have a wealth of reference material while others have relatively little.

The older, more traditional denominations place a great emphasis on their long histories and written doctrines, and much more has been written about them than about things like the New Church Movement. I think the point I'm trying to make is this:

We are not Wikipedia.

Wikipedia famously has a criterion for inclusion which is "verifiability, not truth". This works well to meet that project's aims of consolidating information that's already published; but that's not SE's purpose. We're here to help people, to answer questions, to give the right answer - and that "right answer" doesn't necessarily have a shed load of reference material to back it up, especially where newer developments are concerned.

So I agree that there's a problem but I wouldn't want us to end up insisting that every statement in every answer needs to cite a reliable source as a solution.

share
3  
The main problem here is that Christianity as defined by this site (not changing) is too broad for most questions to have a single "right" answer without giving each question some extra focus. Can you give an example question that would have a right answer that would NOT be able to reference a verse, tradition or doctrine? We have not suggested requiring bibliographies, so an answer that states "The new Church Movement doctrine on x is y" is valid without links (even if links are preferred), but just saying 'y is right' is rarely an acceptable thing across the scope of the whole site. –  Caleb Oct 12 '11 at 10:24
    
I wasn't the one that downvoted this answer (I try not to do that on issues where I have a competing opinion) but my rebuttal wouldn't fit in comments so I did write a rebuttal answer. –  Caleb Oct 12 '11 at 11:40
    
@Caleb I've provided an example question in a comment on your rebuttal answer so as not to create a fork in the discussion –  Waggers Oct 12 '11 at 12:10
1  
+1 for "that problem is exacerbated by the existence of BH.SE as a separate site" Many of the questions and uncertainties that people have is in a misunderstanding of the original language. –  David Stratton Nov 4 '11 at 7:31
add comment

I've been harping about this for a while, but what really turns me off to the majority of questions here is how utterly untestable they are: "what does X in the Bible mean?", "what's the Biblical basis for X?", "what do Christians think about X?"

These are soft questions: if you spent at least a modicum of effort to quote any arbitrary amount of Biblical passages, you get a ton of upvotes. But they don't actually teach people who don't already agree with the answer's interpretation anything new. If I'm not also familiar with the Biblical interpretation an answer uses, how do I know it's correct? I can't test it: the interpretation is not cited, or worse: completely synthesized by the answerer.

The value in Stack Exchange is providing not only expert answers, but verifiable answers. You can see this in action on a site like Stack Overflow. How do I know an answer is correct? I can test it myself. Obviously, testing an answer in the same vein as Stack Overflow doesn't work for cerebral sites like Christianity.SE, but we should be relying on the next best thing: reliable secondary sources. I know an answer is correct because I can read the secondary source myself and confirm it.

Ah, you may be thinking: "well why should I trust Augustine or Luther over a random Christianity.SE user? It's all just opinions, man!" That's the problem: questions that just ask for interpretations are, for the purposes of verifiability, opinions. For someone who doesn't already have a background in the question, random Christianity.SE user's interpretation is no better or worse than the interpretation of whale like Augustine or Luther.

So the solution isn't to allow people to provide personal synthesis: instead, we need to stop asking open-ended interpretation questions. Questions must provide parameters for testability. If you'll excuse my reliance on questions I've answered (I do so because they are ones I believe to be testable), I'm talking about questions like these:

Not questions like these:

Yes, these are popular, "fun" questions that let people exercise their intellectual curiosity and taste for discourse, but they don't provide answers. I'd much rather see a less-popular, dry Q&A site that actually teaches people things than a wildly-popular, fun discussion board that's only a step above Reddit or Yahoo! Answers. The hope of a fun discussion site where I can riff and wax intellectual about a topic is not why I invest my time in Stack Exchange.

share
5  
+1 for the dry expert site vs. fun discussion board point! I think the SE model gives us the chance to be the most factual, most expert-oriented Christianity site on the internet. Now that doesn't mean the site can't be wildly popular and fun too, but that's a secondary objective! –  dancek Oct 11 '11 at 22:49
1  
What's wrong with asking questions that the doctors of the Church have tried to answer? I mean seriously, the onus is on answerers not the askers to not just spout their own interpretations or ideas. –  Peter Turner Oct 12 '11 at 1:06
4  
"dry expert site" = receipe for low visits and low questions and low answers. Which means: failure. Just throwing in a point that's always overlooked. –  Jürgen A. Erhard Nov 1 '11 at 21:32
add comment

Yes, we can!

(reverse the trend on low-quality posts.)

Seriously, we have a lot of good questions, too. And we do tend to get excellent answers on good questions. It's just that the good questions get too little attention because of the bad(ly-focused) ones. I agree we have a definite problem.

I personally have asked many questions that are vague, unfocused or in general don't encourage good answers.*

Similarly, I've posted bad answers.* But I think we should focus on the questions, as not all questions are even possible to answer well.

Me, I'll need to give up or greatly revise many of my own posts if we're to change guidelines. But I think it's worth it. The SE system has been proven excellent in crowdsourcing factual answers to questions. My dream from the beginning has been to leverage that ability, and create a high-quality Christianity site. One where knowledge really trumps ignorance. One where trolls and ungrounded opinions have little room, whereas elsewhere on the internet they roam free.

So I say, let's do this! We already have much high-quality content and much activity. If we can come up with revised guidelines that encourage good content and filter out bad, we have a good chance of making the site a great success!

* I've posted excellent questions and answers, too, but they're too numerous to list... ;)

share
add comment

Yes, there is a problem and we can turn it around. (This is my second answer here, this one takes a different angle as a counter point to @Waggers answer).

One of my early concerns on the site was this line in the stock FAQ:

Your questions should be reasonably scoped. If you can imagine an entire book that answers your question, you’re asking too much.

Given the long history of Christianity and it's propensity for disagreements and elaborate explanations of the minutest details, none of us could imagine a point doctrine that did not have at least one book devoted to the topic. I even went so far as to suggest removing that line.

In retrospect I think we should have paid more attention the warning because it turns out one of our largest problems is with popular questions that are unanswerable and it turns out generally have whole shelves of books on the topic.

Waggers suggested "newer developments". This is partially my doctrinal bias (Reformed Protestant) but I don't think there are any such things. Ecclesiastes tells us there is nothing new under the sun. Several posts here have shown how old heresies just resurface in different forms. New developments are typically NOT doctrinal, they are technological or archiological. These developments are typically well documented. In our modern day and age few beliefs slip through the cracks without somebody else documenting them.

Note that I am not suggesting that every answer must have a bibliography, but I think it is reasonable to require that answers do at least represent doctrine as believed by somebody other than self-proclaimed-opinion. With the diversity around it should never be hard to find some group that holds any given doctrine. Not identifying specific doctrines only leads to communication nightmares where we are using different terms to fight over the same ground.

I would not request every answer have written references to printed doctrines of the New Church Movement, but an answer that makes a doctrinal statement should identify itself as "The position of the NCM on this issue is X." If you can source it with references to statements of faith or other documents great. If you can show "and this is how they defend that doctrine using scripture" then more bonus points. But just answering "X" without any form of support is not a useful or constructive answer.

share
1  
I think that addresses questions about denominational doctrine well. But not all questions are about doctrine, or even come from a single denomination's perspective. I agree that where this is the case, the answer should state as much. But a question like "What does the title of [famous worship leader]'s latest album mean/refer to" doesn't fit that bill. –  Waggers Oct 12 '11 at 12:08
4  
@Waggers We definitely need to keep in mind that there are three types of questions: What is the belief or application of X? (doctrine) What is the meaning of the Biblical passage Y? (exegesis) What is the difference between ESV and NASB? (factual) All expert quality questions must fall into one of these three categories or else they will fall into opinion. If we want to make this a solid site, we have to prevent opinion from being acceptable. We believe it's possible to set some guidelines that prevent opinion. What your asking about is factual and must be addressed. –  Richard Oct 12 '11 at 12:43
1  
@Waggers: About your specific example ... I would suggest that that question on a case by case basis might be answerable with a factual referenced answer or it would be close material. If the album title made any reference to a Christian doctrine, a pointer to that doctrine would be acceptable. If the author made any statement about the meaning that can be found or referenced or if the words or rest of the album show what his meaning is likely to be, then those are factual answers. If the title is a humpty-dumptyism or the artist meant for it to be ambiguous then this is not a good question. –  Caleb Oct 12 '11 at 12:48
    
@Caleb Thanks for that. I was thinking of something like Matt Redman's "The Father's Song" - which is more a metaphor than a specific song - but could appear to be referencing something specific. Of course the only decent reference for explaining it would be an interview with Matt where he explains the meaning himself - but it is possible to answer the question without such a reference –  Waggers Oct 12 '11 at 12:56
add comment

Yes, I agree this is a significant problem and we need to reform our guidelines in order to foster an environment for experts!

I have a number of questions that are unreasonably broad and difficult to answer without creating a voting contest. They encourage opinion-only answers. I think in order to provide a reasonable environment for high quality answers, these should be edited to ask for the doctrines of particular tradition or a comparison between a specific set.

I also have a number of answers that are just me documenting my own beliefs. I think it is unreasonable to consider these high quality and they should be prompted for references to who believes this and in some cases, why.

share
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .