"Rate My Startup" Post Policy

224 points by pg ↗ HN
"Rate My Startup" posts have increased dramatically in the last couple months, to the point where spammers started to post links with "Rate My Startup" prepended. Obviously we can't just allow any newly-created account to post a link to any site, no matter how offtopic it would otherwise be, with the claim that it's their startup (or weekend project, or whatever). And yet I would not want to ban RMS posts entirely. We need some way to separate legit ones, and it seems like the simplest policy is to allow them from established accounts but not newly created ones.

I'd been planning to consider ways to embody this in the software, maybe with a new type of item that (like polls) had a karma threshold, but I've been postponing dealing with it till I was done reading YC applications. In the meantime for the past several weeks I've been killing RMS posts that weren't from established accounts.

I really need to get back to reviewing applications, since today is the last day. In a couple days I'll revisit the matter.

139 comments

[ 3.8 ms ] story [ 184 ms ] thread
I don't think it is unreasonable to ask people to invest time in the community before they can request the community to invest time evaluating their product.
I agree, but it's important that what qualifies as investment matters. Perhaps a function of comments, votes, submissions and time the account has been around.

As an aside, how would we distinguish between RMS posts and regular posts? Would the spammers simply revert to normal posts? I think if this is the case then perhaps we should reconsider the goal.

If the goal is to stop spam masquerading as RMS, then it seems to me that accounts should require more longevity or community contribution to post an RMS post than a regular post. As long as we define etiquette for posting an RMS post, then it should be feasible to use a combination of following that etiquette, and flagging posts that don't follow said etiquette.

If the issue is stopping spam, then perhaps that's separate to RMS posts and should be considered as a wider whole.

You have almost 1000 karma. Do you have a startup you want to ask about? Try posting it; he's not going to kill it. Otherwise: what's the problem?
I don't care about my karma a great deal (although I've written about a personal karma-related goal before for me to gauge how engaged I am in this community, it's a guide for me rather than a score per se).

I care about other people's startups, but not spam.

I'm reminded of the first time I saw PG speak, it was on bayespam at MIT -- he noted that if SPAM can come up with interesting and engaging email, he'll happily read it. I imagine that as soon as someone is posting interesting content, RMS or not, PG and others won't think of them as 'spammers' anymore, rather contributors.
If SPAM is interesting and engaging, is is SPAM?
If someone sent me 500 interesting and engaging emails every day, I wouldn't have any time to get real work done.
Thats why I took HN out of my RSS reader!
Mine was pretty interesting to the 10 people who's commented and the 23 who'd voted it up. It was still closed.
As a side effect, if you're legit, not a spammer but are new to HN (0 karma), then maybe one of the paths to getting an RMS is to be able to convince at least 1 non-new HN member with enough karma to stake their reputation and post an RMS on your behalf. Therefore the RMS is already vetted by someone real that's been around HN for a bit.

Crowdsourcing RMS spam control sounds like it could work well. Like how investors stake their reputation by inviting other co-investors, established HN'ers would have something to lose if they posted RMS spam.

Cue the PPARMSFM (please post a Rate My Startup for me) posts :)
This is actually a case where "Cue", not "Queue," is correct.
Thanks...I guess queue is such an integral part of my career I forgot about alternatives!
Unless there are several of those posts, and they have to wait in line.
Agree. I commented on a recently killed "Rate My Startup" submission asking the reasons behind killing it. http://news.ycombinator.com/item?id=1861551. Now I know.
That was my RMS - honestly have just been a reader and less of a participator. It was highly suggested that I seek HN for feedback on my newly launched site, thought little of posting it from my newly created account. Tons of great feedback in the time it was alive though. (Thanks everyone.) Lesson learned.
I'm about 2 months away from having something to rate, so I started commenting/writing/posting just so that when the time came I wouldn't be "that guy"...
As long as you say interesting stuff...

OTOH, if you say stuff that's considered interesting here, you are probably not trying to sucker us into reviewing your app.

You and I think eerily alike.
What about just requiring a certain number of active days being signed up? Is there data that shows spam is ever posted by long running users?
Poor for stopping spam. It's pretty easy to write a little bot that just logs in to ycombinator on an active basis. Data would be good though.
Sorry, I meant days since signup, not time since last login. I haven't see any data that suggests spammers are using old accounts, but I'm not sure.
Even if spammers aren't using old accounts, if something like that was implemented, spammers would just add

  sleep( n * 86400 )
to their bots. Once those n days were up, there would be yet another wave of spam.
I think this could work. If you make it something like 60 days that's not too harsh on new users, but will slow spammers down. If a spammer does wait 60 days and post an obvious spam link then gets banned they are stuck waiting again.

The only problem here is a spammer that creates a ton of accounts and then uses them all at once after the 60 day threshold.

I completely agree. I am a HN newbie and wouldn't expect people to care about any of my projects until I spent time offering analytical and positive contributions here first.
As a non-technical, first time entrepreneur I am finding it difficult to add value to HN. I have been lurking for ~6 months, a member for 1 month and I made my first post yesterday.

With that said, I have learned a ton here and HN is easily the best resource I have ever found on any subject.

edit:

Potential solution: require participation in other user's RMS before you can post your own RMS. Even though I am non technical and don't have much entrepreneurial experience, I can still give an opinion from the view of a customer.

If one makes a habit of participating in RMS threads and giving thoughtful feedback, I'm sure the karma threshold would take care of itself. Good idea.
Surely you have a background in something unique or interesting, that others here don't have. At the very least, you have your perspective. I don't think it's difficult to provide insightful, valuable content to HN, in both submissions and comments. I've noticed that the topics here tend to rotate over time (meaning one week is all hard core programming, and the next week is boot strapping). Also, check out the new section, where the submissions tend to be more varied (in my experience). Find a topic or two that you have something to contribute to and comment on it.
(not grandparent, but similar situation) I've been here for a while, and I think it is difficult to participate at a level that gets lots of karma. I limit by intake of HN with the "Hacker News 100" feed, so that you all can filter my content for me (I'm going to miss good articles anyways, so I might as well not miss the popular ones). This also means that most things have already been said, or the conversation is dead, and even if I asked a great question, I wouldn't get a response.

This hasn't really bothered me, until the average karma/post started getting shown, which showed me that I get upvotes on less than half the things I say. Now that pg is also pointing out the correlation between new accounts and undesirables, I'm worried about...something (not sure what).

You could definitely add value, even from the perspective that in a generally technical community, you have a non-technical perspective.

If you are generally clever and motivated, there's few situations where you won't have input. Even asking well structure questions to learn on a topic is a good contribution, and can help people out.

Your potential solution might work, but I think it allows for the kind of spammy comments the Android App store gets a lot

> Awesome App! 5 stars! Get paid apps 100% free from xyz.com!

Investing time does not equal karma. I have posted about 10 comments so far and only have 2 karma. Most of the threads I post in never make it past 20 total posts. Requiring people to have a certain amount of karma will make them only post more comments to achieve said karma. And the more unnecessary posts there are, the lower the quality is of those posts.
I can definitely imagine lurkers creating accounts to show off their latest project, so auto-killing posts from newly created accounts doesn't seem like the best way.

Perhaps make it policy that all RMS submissions be text submissions explaining the idea to give it a bit of authenticity, and every RMS submission with a link in the title be auto-killed. I don't imagine spammers would go through the effort of writing out a description of their service, asking for feedback, etc. Also, adjusting the flag threshold to 5 on text submissions could help. People tend to be more accepting and less likely to flag text submissions because the chances of them being spam are less likely (due to a higher degree of effort put into them as opposed to linking to some affiliate site), so a reduced flagging threshold might get rid of those submissions more quickly.

If there was an established rule about who got to submit RMS posts, lurkers would know that and create accounts.
An option might be to send RMS posts from users with low karma into a slashdot-esque crowdsourced moderation. Randomly show it to 10 users with a high enough karma on any of the list pages (frontpage, newest, classic, etc.). It's then only made public if a high percentage of such "moderators" approve of it (not necessarily counting "approval" as an upvote, just meaning "this is legit"). The percentage could be pretty high, say 80%, and it could optionally be log(karma)-weighted or similar. 100% might be risky; I have no idea.

I have to say I'm not a huge fan of the slashdot moderation system in general, but it could work within this sort of limited scope. Allow people to opt-out altogether via a setting if necessary.

Tangent: I was amazed at the first spammy RMS submission I encountered. It had actually been upvoted. Spammers self-upvoting, or people wholesale upvoting any RMS they see without looking at it first?

Rating without looking at the underlying content is horrible. Perhaps it could be mitigated by tracking if the link was followed, weighting authenticated upvotes more heavily than unauthenticated ones (where "authenticated" means a click-through to the content was observed).

You could track a link click either by setting a cookie client-side with javascript, or by linking to a trampoline page on HN that tracks the click, then redirects to the real URL.

I have, on occasion, voted up a link on HN without clicking through iff I have already read the article in question on, say, reddit. (I mouseover to make sure it's the same URL, though, and not just title and domain.)
And spammers could create lots of accounts, wait until each is mature, and then unleash it. How can you tell the difference between a spammer that creates a stealth account and a lurker that still doesn't have anything to say, but is creating an account so he can eventually participate fully?
@PG What do you consider an established account on HN?
What constitutes an established account? I rarely comment on posts but have had an account for 2.5+ years and regularly upvote posts and comments. Is my account "established"?
I have no first hand knowledge, but I would be surprised if a 2.5 year old account with 250 of karma (like you do) is not considered established enough for this purpose.
What about a 2.5 year old account with no karma? You'd be surprised at how many lurkers there are. I know quite a few (and was one myself up until a short while ago).

EDIT: added emphasis

Karma's really a proxy for community participation, albeit kind of one-sided since you don't get increased karma for up-voting someone else (and rightfully so). I don't think it would be unreasonable to say that if you don't participate, then you don't get to solicit the help of others.

But the bar is much lower than that. For the purpose of just keeping spam accounts out, I think the combination of a karma threshold with average non-karma accruing activity over time (such as upvoting others, clicking through on articles, etc) could be an effective filter.

Like myself. Been around for a couple of months now and this is my second comment!
Agreed. I lurked here for about one year before I made an account. Even after I made an account, I still rarely comment.
I as well. I've been 'lurking' on hacker news for almost two years. It doesn't mean I don't want to participate, it's just that whenever I do have something to say it's typically been addressed already by another user (the topics I'd probably post on are the hot-topics that draw more eyes.) And I'd rather not add redundancy to comments. To me the natural game mechanics that arise from 'gunning for karma' by being first to post (relevantly) takes a lot of effort.
I started posting less when the karma ratio became a big deal. But you can still pick up a point or two here and there by making encouraging comments to people far down a thread. This does add value to the community, but it's not a big shot-in-the-arm like a popular submission or a relevant "first post". They do, however, add up.
That's basically where I'm at (speaking of redundancy).

I'm usually afraid to say something wrong, since I know that there are always experts on the topic of the post. It makes more sense to me to let said experts explain a particular topic than to possibly disseminate wrong information, even though I might have a useful perspective to add.

People shouldn't be afraid to say something wrong. If you say something wrong, people should correct you and everyone will learn (including readers that don't know anything about the subject).

The issue is if people that see that you're wrong, instead of correcting you, just downvote you. In my opinion, a wrong statement should not be buried with downvotes, it should stay at 1 point with a reply that corrects it and that get a lot of upvotes.

I wouldn't, actually. I've got comparable karma (221) after about two years, and I can't even consistently downvote items. I noticed down arrows a while ago for comments, but recently I've noticed that they only appear on certain comments, and others are upvote only. My assumption has been that there is some sort of inflation at play.

Not that I'm complaining of course. I think I've only used my downvote powers once normally. The only other time I've used such powers was when I noticed some spam comments, and chose instead to flag them on the noobcomments list.

I honestly think that something like my level of karma is too high a limit for posting things like an RMS. This place isn't reddit, so I take my comments on here a lot more seriously. As such, I don't comment nearly as often since I don't usually feel I have a comment of sufficient intelligence to contribute.

Really, I'm quite amazed anytime I see my karma increase here. I have a definite inferiority complex when reading some of the great comments here.

With experience like that, building any measure of karma could be a slow and arduous process for the typical new user. Unfortunately, I'd imagine a lot of people working on their first project would have the same sort of inferiority complex, yet they would be exactly the sort to benefit the most from an RMS sort of post.

In fact, I'm building (very slowly, thanks to inexperience and a bit of RSI) my first "major" project, and I had hoped to show it off here first, but now that same "I'm not worthy" feeling is making me feel like it wouldn't be welcome.

You cannot downvote comments older than 24 hours. Submissions can't be downvoted at all.
Also I believe you cannot down-vote comments you reply to or that reply to you or something of the sort. That is to avoid flame wars.

I think the grandparent (xatax) is being far too timid, I am sure 221 karma is enough to do anything on here that any other non-moderator is allowed to do.

Except that PG increased the karma required to downvote from 201 to 501
Odd, I didn't think the post I referred to was that old.

I guess it also makes sense not to have submission downvotes, since the really bad posts will just get killed, and the iffy posts just won't get enough points to get very visible.

Thanks for the explanation.

I'm sure number of votes and length of membership could be incorporated into some sort of algorithm to determine that?
How about a simple Bayesian classifier for legitimacy? Parameters: account open date, date of first comment, date of last comment, karma, std dev of time between comments.

Good for drawing crooked lines between accounts of a similar age, eg. there is a signature for legitimacy that appears on short (two day) and long (two year) time scales. Talk about specific thresholds seems misplaced when it can be computed from a small training sample.

The easiest threshold is karma, as for polls.
Just wondering, I've been able to down vote for a few months, but recently, my karma hasn't gone down, but I can no longer down vote comments.

Why would that be?

There's an enforced ratio of downvotes to upvotes that you may have hit. Also: comment more.
Hmm, encouraging commenting as a means, not as an end, is engendering a culture of under valued contributions...

I've been a member since late 2009, but I've made 2 or 3 comments so far. My case is not atypical. Maybe there's a better way?

The karma needed to downvote moved from 200 to 500.
Is this a request for input?

I personally haven't come across any "rate my start-up" posts I'd considered spammy, but maybe that's because the measures that are currently being taking have been effective (marking as dead, lack of upvotes for spammy posts). Mostly I've only seen that kind of spam in comments.

As for RMS posts in general, I really enjoy seeing what other people have done. So I would be opposed to an outright or unnecessarily restrictive ban.

I suspect it's at least partly a heads-up in case a reasonable story was mistakenly killed.
(comment deleted)
I'd rather see a fairly low karma and a few weeks or months account age thresholds, but then I'm biased, being mostly a lurker here.
Spammers would just create accounts months in advance, and then use them when they mature. They're not above that sort of thing for high-traffic sites like this.
In fact they do it constantly.
So a hacker who has been reading HN everyday while coding his product, but does not post often; is not allowed to post RMS anymore?
I'm in your boat. I read HN pretty religiously but don't have much time to comment or follow up on comments. Every time I even think about doing that I'd almost always rather spend more time obsessing about my product. Thus I barely have any karma points. But every once in a while, I like to post my latest project.
Ditto. I'm building, grinding on customer discovery, reading HN to stay informed (and maybe a little entertained). Also, most of what I read on HN is stuff that is outside my comfort zone, and therefore I am not as able to comment as I would be if I stuck to reading topics with which I am already very familiar.
You "barely have any karma points"?? Your profile says you have 453... that's going to be far, far, far over any threshold pg is likely to set.
You could ask a community member with an established account to consider posting your startup as suggested earlier.
So here's my counterpoint to that: Then that established community member gets even more karma points for posting that. You still have that same problem where I, the perpetual lurker who occasional contributes a comment or a post every now and then, never gets to build their karma points.

It's still like WoW. You put in the time and you get the exp/karma/level what have you.

But how are "rate my startup" posts different from any other posts?

I mean, if you invent special treatment for RMS-posts then why wouldn't the spammers simply go back to submitting their stuff without that prefix?

I imagine a number of people blindly up vote the RMS posts to encourage the trend, though I'd be curious to see the data behind this compared to other post types.
They're different because they can be to sites that would otherwise be offtopic.
Reviewing of startups and providing various perspectives is something very special this community offers for starting entrepreneurs. We should take care to keep that interesting and not to devalue it by too many / spammy posts.
I'm new to HN and I wouldn't post my startup link, as I've yet to contribute enough to the community to earn the time other members would need to review my startup. That being said, I do believe that a karma threshold for RMS posts would be a valuable asset to HN.
(comment deleted)
i think this XKCD points out the issue rather well. http://xkcd.com/810/

Is there any kind of captcha on the submission for RMS? If not perhaps a direct link to the other RMS's and a requirement to post there.

Agree. Also add a moderation queue. It doesn't go live as an RMS unless you pass either captcha + moderation or karma check. A great way to gain karma would be to review the moderation queue.
How do you stop spammers from building up their karma by reviewing that moderation queue by upvoting their own spammy submissions?
(comment deleted)
The system just needs to remove the possibility of a closed loop that could be entirely automated. Some part of the loop of gaining or loaning karma must involve content creation that is rated by known good human actors.

Timing is actually one of the simplest techniques to game. Create the account and wait. Content creation is one of the best ways to ensure no gaming is happening.

Meta-moderation? But thats a lot of effort, and you'd really have to want to see feedback.
I think it's better to put "rate" or "RMS" in the header/menu to distinguish from others.
how about you just disallow all frontpage submissions until a certain karma level? make people comment before they're allowed to submit links or text content, since those are the easiest to game.

comments are easier for the community to moderate and its easier to pick out spammier ones, since they're clearly off topic or include random inappropriate links.

Because then spammers would start to post comments instead, and those are harder to catch.
are they? we kind of tend to smash down comments that are even slightly off topic or non-contributory.

they might sneak in on the less popular submissions, but i kind of feel like it won't get by any more of a problem than it already is. if comments were more a effective way to spam, they'd already be a problem.

alternatives: disallow links in comments until a certain karma/age. disallow both comments and submissions (ha) until you've voted on enough statistically different things. make it a game by creating a new metric that people can be measured on based on the number of spammers they've sought out and identified via flagging, and create a new top list for it.

Maybe another section like the "new" section?

But don't name it rms, that means something else on any hacker site.

Indeed I was phreaking out thinking pg was deleting Stallman's posts !! ;-)
What's the point of "flag"? By killing legit RMS posts from low-point posters, you're creating a lot of ill will, particularly when they follow up with "Why was this thread killed?" posts.
Jiminy! People: he's just saying I'm too busy to fix this the right way today, so you'll all have to make do with this band-aid.

Can we stop with the star-crossed imaginings of startup questions lost to the ages because of karma thresholds? If you have a real product you want to rate, and you can't post it, just find someone who posts regularly and who has their contact info in their profile. Then: send them an email and ask for help.

If that's too much of a hurdle for you, I have some bad news for you about entrepreneurship...

If you do have a startup and are new to the site you're welcome to send me an e-mail (it's in my profile). I don't mind spending a few minutes looking through your site and posting an RMS on your behalf if everything is legit.

Be aware that I'll look on archive.org to see whether your site has been up for five years :-)

> If you have a real product you want to rate, and you can't post it, just find someone who posts regularly and who has their contact info in their profile. Then: send them an email and ask for help.

I like this idea, and from what other have said about this, I got the following idea: How about letting regulars promote/vouch for RMS posts. That way, the person behind the site can post it and it will still be clear that it is not spam.

Perhaps in addition to the user's credentials you could analyze the url they are submitting. Has it been submitted before?
crazy idea: maybe also check to see when the domain was registered? more than a year old doesn't sound like a 'startup'. it could be another metric in the decision engine.

in fact.... i sense a new startup/weekend project. 'spam link analyzer service' which incorporates user defined metrics (e.g. HN metrics)

No, a lot of people sit on good domains for years before doing anything with them.
I don't think it's a good idea as domains may be placeholders for quite a bit of time. Especially if startup is not a weekend project.
If this was implemented, I'd imagine the quality of names for RMS candidates would drop off a cliff!

I've got to imagine that everyone who reads these forums has at least 2-3 domains that they've owned for a while for future projects.

An effective way of dealing with spammers is letting the community moderate it. You can add a "Mark as spam" link to every post and only make it visible to users with a certain amount of karma or whatever metric. Basically giving trusted members the option to moderate.

On top of that, record the IP address of the poster, and display the post but only to the spammer. Basically hiding it from everybody else. The spammer will think the post is still active. Everybody else will not see it.

I see a few things happening: we've got a separate link for "Ask HN" topics, another one was created for "offer HN" topics, and now we're talking about having one for RMS topics.

Perhaps it's time for a more generic categorizing system of some kind? You could make it so that people can't post in certain categories without meeting a certain karma threshold.

As a fellow hacker in the trenches I like to see what others are doing. The rate my startup posts are a great way to see what is going on and some of the current trends. I have learned a lot from the posts, not only evaluating the site myself but the feedback that is received. Some of it has been spammy or half-baked. Maybe a karma threshold is in order. Better yet as a site of mentors maybe a RMS should be sponsored by a “senior” member. Karma of the sponsor would be on the line if it is spam as judged by his down voting peers but share the rewards of karma if it is a worthy site.
Welp... just registered today so that hopefully when I try to post my own startup thread, I can. I'll try to post more from now on instead of lurking but I'm pretty busy. :|
Yeah - same here... Aren't lurkers users too?! I feel separate but equal.
Many answers to this new policy proposals mention some version of the argument that if you don't participate, you shoudn't be able to solicit the help of others.

While this is true to some extent, it undervalues the fact that RMS posts create value for everyone, since the comments are then available for everyone (not just the poster).

What qualifies as an "established" account? I have been here for a while but don't post that often.
Maybe it's time to have spam url filter setup.