HN Explorer

How do you prevent factions from taking control of a community-driven site?

21 points by ekanes ↗ HN
Hey folks, we've built a "poll the world" type site, and it's starting to get some traction.

Along with growth comes the inevitable community challenges which seem to occupy more and more time.

Polls are voted up by the community, and instead of a down button we have a "mark as inappropriate" to help monitor for abuse. Some folks have started to use that link to try to get rid of polls they don't agree with.

I'm a fan of PG's "Things you can't say" premise (http://www.paulgraham.com/say.html), and I'd love to try to guide the site in that direction.

The ideal would be that abusive questions are modded down, but "questions you don't like" aren't. It doesn't mean they'll be voted up and become popular enough to run, but it means they won't be squashed either.

I guess one option would be to remove the "mark as inappropriate" completely, and/or have trusted community members able to remove questions if needed.

Thoughts? Any advice would be much appreciated.

Thanks!

29 comments

[ 2.9 ms ] story [ 79.0 ms ] thread
jward
This isn't an easy problem. One thing you could do is mod the moderators. Have each persons vote count a certain amount. People who report true spam you bump up, and people who are abusive get bumped down, perhaps to the point where their opinion doesn't matter. Then require a minimum threshold before you have to deal with it. Also have this ranked against the up votes and popularity.

On imageboard.net we have it setup so more than one person has to mod something down to hide it. It still doesn't work that great, but it's better than nothing.

ekanes
Thanks, that's very helpful.
greendestiny
Your site is: http://www.ask500people.com/ ? Don't be shy about a link, it looks like quite a cool site.

Something to consider is what these people are trying to express with their use of the 'mark as inappropriate' and how you can incorporate that. I think reddit proves that you cannot simply tell people how or why to vote a certain way - you can only try and set up systems where the consequence of voting matches what you're trying to do. Are these people trying to filter out a certain kind of poll? Are they signaling a breach of etiquette?

Maybe you could leave 'mark as inappropriate' and simply have it count and display the number of marks, and leave it entirely up to human editors to decide whether to keep polls or not. Ultimately when people are trying to express something they really just want to be heard.

ekanes
Yep, that's the site. Glad you like it. :)

I've been thinking similarly about 'mark as inappropriate' - that it could give people a way to express themselves about the question, even if it doesn't have a large effect on the site. We could have:

- mark as abusive/spam - duplicate question (another ongoing issue) - I don't like this question

(edited to fix formatting)

greendestiny
Yeah I like it. Maybe it complicates things but it'd be interesting to see if it pleased the users.
michael_nielsen
It's not quite what you asked, but Teresa Nielsen Hayden has written an excellent essay, "Some things I know about moderating conversations in virtual space", at http://nielsenhayden.com/makinglight/archives/006036.html#00...

Some of the ideas in that essay may be useful to you. Her point 11 is particularly worth pondering:

"11. You can't automate intelligence. In theory, systems like Slashdot's ought to work better than they do. Maintaining a conversation is a task for human beings."

Interestingly enough, Jimmy Wales, founder of Wikipedia, has made essentially the same comment. He went so far as to describe himself as "battling programmers" who want to automate various aspects of Wikipedia's community.

ekanes
Great stuff thanks.
rzwitserloot
One thing you can do, is to announce that any inappropriate use of 'mark as inappropriate' results in an automatic 1 year ban of usage of the feature. And then follow up on that. It should keep manual sorting through misuse down to a manageable task, and the threat itself might stop people.

Then again, it might just generate whines and asshats that create multiple accounts.

icky
Maybe a better approach would be to silently gather statistics and find out the degree of correlation between any given user's "inappropriate" rating and the actual (as determined by you) inappropriateness of any given item.

Then, if someone has, say, a NEGATIVE correlation between their inappropriateness rating and the actual inappropriateness of said item, you can still use that person's ratings as a useful source of information... just not the way they intended ;-)

rzwitserloot
Extra points for creativity but that assumes that 'content is inappropriate' abusers do not use the 'mark inappropriate' feature for actual inappropriate content. I don't have any quotes or statistics here, but common sense suggests to me that abusers will mark real inappropriate stuff and stuff they just don't agree with.

By keeping mum, you also lose the scare effect, though OTOH that might keep the "I'll just create a bunch of accounts" folks down.

A third risk is that legitimate marks no longer get made because the potential marker doesn't want to risk the ban. However, if the ban is ONLY on the use of the mark as inappropriate button and not for anything else, then I doubt it would actively scare users.

icky
> By keeping mum, you also lose the scare effect, though OTOH that might keep the "I'll just create a bunch of accounts" folks down.

Exactly: if someone is making consistently bad flaggings on one account, you don't want to tip them off and have them flee to a new account.

> A third risk is that legitimate marks no longer get made because the potential marker doesn't want to risk the ban. However, if the ban is ONLY on the use of the mark as inappropriate button and not for anything else, then I doubt it would actively scare users.

You'll note that my proposal does not include a ban of anything, not even the "mark as inappropriate" button. I just suggest letting the bad users keep putting in their bad ratings, and using that information in the opposite manner as they intended.

You could even have some sort of Bayesian (naive or Markovian, depending on your computing resources and scaling needs) classifier set up to see what words or phrases in a post make any given user more likely to mis-rate. Then you could separate out a user's probably-bad ratings from their probably-good ones.

sosuke
The only way this would be an issue is if "mark as inappropriate" was not editor reviewed. You can add in a message that states that an editor will look at the claim before taking action and then you can set the submission appropriate yourself.
tjic
As soon as you can "fix" human nature, you'll have this one solved.

People communicate for lots of reasons...and, unfortunately, transmitting information isn't the most important. People posture, they seek out status, they build alliances...all of these things are deeply wired into us, because primates (and other critters) who are successful at these things have more descendents.

You are not a beautiful snowflake.

You are the descendent of lots of individuals who "abused" social situations for posturing and alliance building.

angryprofessor
You might try looking for controversy. I've thought about this more for reddit/slashdot comments than for polls, but here are my ideas:

For comments on a point based site (such as reddit, this site if it gets popular), there are signs that a post is being downmodded by people who disagree. For instance, -10 points, 10 replies. Or 10 upmods, 12 downmods.

In your case, this would probably be 10 "innappropriates", and another 20-30 people actually using the poll. You might also look for people clicking "innappropriate" and then voting in the poll.

ekanes
> You might also look for people clicking "innappropriate" and then voting in the poll.

Very interesting thanks.

Kaizyn
1) You can select your audience by tailoring the site more specifically to groups of people who are less prone to this problem, or

2) Employ user moderators and moderate the moderators.

cmars232
A per-user, shareable kill list and bayesian filter. Let them have their revolution, it will be ignored!
oditogre
Simple: Make it inconvenient. Require anybody who marks a post inappropriate to fill in a textbox explaining themselves.

Apply a basic 'stupid filter' to submissions (just to strip out submissions which are gibberish or vulgar) and make it clear to people that you will be filtering on those things.

Set a reasonable limit - 250 characters or something, so editors don't have to read essays on why a submission is inappropriate.

A little inconvenience goes a long way in shutting down site abusers. ;)

ekanes
... and perhaps combined with making it clear that your reason is public would be interesting as well. Great food for thought thanks.
henning
God help you when the Ron Paul fans discover your site, irrespective of the merits of Paul's policies.
kirse
When it comes to sites like Reddit, I wonder if limiting the number of votes a user can contribute per day might help.

This way votes become a limited resource and you've got to use them carefully and really choose what you think is the best news item of the day.

ekanes
Interesting idea, but it wouldn't fit the different user types - someone on the site all day needs more votes for the same level of scarcity as someone who's on the site 20 mins a day. Otherwise one person has scarcity and one has plenty.
kirse
I think it would fit perfectly! Someone on the site all day needs to get off and go do something else productive =)