Tags concede defeat, fragmenting one dysfunctional community into a whole spectrum of subcommunities. How will we handle the tag for "good enough for '08 Hacker News"?
If you can sideline all the people who want to bullshit about how the ipad2 is the future of personal computing or the quality of the writing at TC, you've already won.
No, what Reddit does is fragmenting community. How much does the tag system in StackOverflow or Quora fragment them? Tags helped people to somehow filter their already overwhelmed input stream, focus on what they really interest in, and more importantly, ban the items they genuinely don't like to see.
That was my thought too, they both have quite stringent moderation, but that doesn't solve the increasing the quality of participation, unless of course it's decline is what drove some people away.
Nope, I got an email from him the other day telling me not to editorialise in submission titles after I posted that Nissan Leaf HTML5 extravaganza with the heading "Behold: We don't need flash for anything anymore. Ever".
Well, I guess because I was editorialising!! I found it sensational at the time. That was the first time I had looked at a site and said to myself "well that must be Flash(TM)", and then a second later in my head thought "well fair enough, you'd HAVE to use Flash(TM) to get that kind of experience for a major brand", then right-clicked on the page only to see the standard context menu... no "Zoom in"!!
I was really quite blown away. I could have just put it on my blog then posted that to HN (which is actually what pg said I should have done when he emailed me), but my initial thoughts, were that I really just wanted to share this with my pals on HN (similar to the way someone else had posted Ben the Bodyguard as a "Gorgeous HTML5 implementation" or something) and that putting it on my blog then linking to my blog from HN would just be viewed by the community as me jockeying for traffic and being self-promotional!
For what it's worth: I feel safe saying that most high-karma users of HN have a variety of severe concerns with it. My experience asking this question over email has generally been one of getting gigantic essay-length responses.
In my official capacity as "representative of people dorky enough to have karma this high", we do officially declare: stuff's broken. Needs unbreaking.
Well, based on what I've seen so far, PG usually asks for datasets that support the point that one is making about the degradation of the community. If the essays that you have are just long rants, they will be easy to ignore. However, if there exists a dataset that supports those various points, as well as suggestions on how to change HN, PG usually seems very reasonable about trying them out.
If there is one thing that causes an internet community to break down, its when a subsection of that community has too muh communication out-of-band in another medium that the rest of the members are unaware of. Especially if that other medium is more synchronous than the primary medium.
Many forums have been driven to civil war by a clique plotting on a secret IRC channel. (Note: I am not suggesting that is happening here! Just that I've seen it before.)
It was an innocent observation: I was sharing an anecdote about my experiences on other internet communities.
The mention of the fact that the higher-karma, "ingroup" members of HN seem more likely to communicate with each other on twitter and in private emails than the rest of the population reminded me of bad situations I had seen on other forums.
I then thought how glad I was that here on HN it seems to happen on twitter which is still largely public. You commented quickly here about private conversations regarding HN with other high-karma members. It is reassuring that you were open, and happy to be open about this.
This gives me faith in what I find to be an excellent community that the inevitable psychological group-dynamics type phenomena are playing themselves out here in a relatively benign way, due to its high calibre membership. Your later comment "I'm too high drama for that" came across as an amusingly wry confirmation that you feel the same way.
As I said, this put me in mind of situations I have seen in lesser communities where this dynamic had led to ugly scenes. I posted about them. I then felt bad that you or someone else might misinterpret this as an accusation against them. Thats why I added my parenthetical note. Perhaps I should have reconsidered the whole thing.
For all the newer people, I encourage everybody to come by the HN secret cabal IRC channel, #startups. I found that the term "secret cabal" really sounds much more exciting than the actual channel ;)I try to go by at least once a week, though. It's fun to be in more of a real-time discussion.
I would also encourage people to look up folks' email and email them with questions and such. Great way to make new friends. Many a time I've tracked somebody down from an HN discussion only to have it continue via email. (It also brings the noise factor down over here, which is always a good thing)
I know a lot of guys follow everybody they can from HN on Twitter. I don't know if I would personally go that far -- Twitter has a hecuva lot of noise and I get easily distracted -- but it works for some.
So yes, there's a lot of out-of-band chat going on, but as far as I know, none of it is exclusive. If you'd like to participate more, the same channels are open for anybody. I know I keep my Twitter, Facebook, and email addresses in my profile for just that very reason.
There seems to be a regular effort to help HN posters connect with each other. Doesn't seem like a month or two goes by without another site that offers that.
Still my favorite site on the whole internet, but yeah, things have definitely changed.
Stories, yeah, they're a bit worse but that's not the main thing. I skip stories that don't appeal to me, and you can always go a few pages deep if you want more stories.
It's the comment quality sliding down that's scary to me... it's becoming much more okay to engage in standard-internet-level discussion here, which is to say, much less civil than HN used to be.
Perhaps liberally killing any comment that's primarily insulting, sarcastic, pedantic would help?
For instance, I just saw a link to this two year old thread today, look how civil and friendly it is:
The only halfway insult on there - "Dunno but she has a Twitter page. Not sure why PG would quote such a weirdo." - was voted to zero. And the 2-3 jokes/sarcasm stood still at 1, none of it voted up.
Also, I'd love to see people who just consistently left substance-less comments get put on auto-kill, but I know that'd never happen for a ton of reasons.
I'm thinking of one guy who just posts about how outraged he is about anything remotely politics related, and nothing else. Joined in the last six months. Actually, that probably describes a lot of users now, which might be part of the problem...
You couldn't get anywhere near that today. Discussing tough and interesting subjects civilly with some of the smartest people I know wasn't what brought me to hn (startups were), but it was a big factor in keeping me here. I knew if I posted some crazy half-assed startup idea that might piss people off that the community could talk about it logically and dispassionately. That allowed each of us to open up more.
I blame karma. I know it existed back then too but I don't think people talked about it. There should be no "top users" page. Having a measuring stick drives competition and there should be no metric to civil discussion besides the rewarding feeling of discovery and helping people.
Admittedly, it wasn't voted up, but there was some agreement with it. You're looking back wistfully at a point where another user thought HN had fallen off precipitously.
I don't see a significant difference between HN now versus about 3 years ago (when I first started lurking), and I believe the community could and (on most days) would have the very discussion you've linked. There are cycles where many of the submissions aren't interesting to me. I usually chalk those up to slow news days. And there have been situations where I've been appalled and infuriated by (what I perceived as) the poor quality of articles/comments that the community was voting up. Some of that is the community offending my own biases. Some of that is just bad days. You can't discern the climate by just looking at today's weather.
The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse. But I'll allow that others are more sensitive to HN's quality. Perhaps these reflective posts are what keep HN's quality high enough for my tastes. Or maybe, as I suspect, they are more about the tastes and perceptions of the long-time users changing. It's more than a little reminiscent of the "golden age" thinking of older generations, where the past was simpler and more innocent because they were younger and insulated from the time's complexity and injustice.
You're looking back wistfully at a point where another user thought HN had fallen off precipitously.
Yes, I think that's what made it a good example.
The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse.
Yes, it's almost an inside joke. One of the "classic" HN stories is the meta story where we all lament the decaying state of HN. The "naval gazing" stories. You learn either to ignore them or vent (politely) After watching them for a while, I think they serve to help release a lot of frustration people have.
I would note that many of the folks early on who said HN was fine are now also saying there are problems.
To me, HN's core problem was the problem you always have when speaking to more than five people: trying to anticipate reactions. You end up trying to cover every possible response, and it kills the tone. One of the reasons my comments run on so long is that I have to cover every obvious way they could be misunderstood. You're stuck either not replying to anybody, writing long comments, or getting in endless debates. All of that is distasteful to me.
One of the hard lessons I had to learn blogging was not to tease people. I used to put pictures of girls in bikinis up or joke around about making money tricking people online -- to me I was just playing. I live by my reputation: being an internet porn mogul is not on my list of future plans. But there were a heckuva lot of folks who didn't understand my posts, who thought that made me some kind of evil person. It was really sad to have to give up the playful part of my personality simply to get along with people because of the many ways things can be misunderstood. I hate that.
But the real problem is that everybody's personality will piss somebody off somewhere. I know I probably rub thousands of people the wrong way just because I'm the way I am. Others do the same. So then I end up commenting something like "Looks like it is raining outside" and I get 3 or 4 downvotes. Sure, the upvotes outweigh them, but it's just silly nonsense. Vengeance voting.
In my official capacity as commenting wind-bag, I'll second that.
HN started as a way for YC guys to communicate. It was about startups, the tips and techniques to make one, to grow one. Once PG opened it up, it became "stuff hackers like", which meant cool deep tech stuff. But the tech stuff was there because after a while, all the startup stuff just ran together.
Then "stuff hackers like" kept drifting wider and wider. I was happy with that, but now that the traffic numbers are through the roof, what I'm seeing is the 1% snark factor is out of control. If each member of HN said something truly snarky only once a month, there would still be thousands of snarky comments a day on here. It's just too much noise.
HN has always been driven by emotion: the only reason to click that button is to express your emotional response, good or bad. Sure you can rationalize that in various ways, but in principle it's about how the thing makes you feel.
Now we have the love-fest stories, the hate-fest stories, the snide jokes that get upvoted, the trashing of "rate my startup" posts (I still can't believe how some of the startups are treated). It's becoming less of a community and more of a mob.
I think maybe the story titles might look mostly the same, perhaps, but the quality of the stories and the quality of the conversations have changed quite a bit.
Excellent point and thanks for all your thoughtful comments.
I use to comment regularly on HN (member since Apr 2007 and former #1 on leaderboard in the ancient days, haha), but have found that I very rarely do any more.
When I look at my thought process now when thinking about posting something that may be of value, I find myself very reluctant. This is due to the snarkiness and hot-headed emotion that often disregards logical arguments as worthless and instead values breathy spittled anger.
Cowards become brave behind anonymity.
Who enjoys "lawyering" their comments? Basically trying to anticipate and address all the possible concerns hot-head users will bring up. Yet, I feel like I've had to do that more and more. In the end it seems an exercise in futility - particularly when I can just move on to interacting directly with the thoughtful hackers I care about and avoid those that spit when they talk.
I can understand the lament of the OP since I've really enjoyed the community and associating with the gentleman-hackers I've met here.
Thanks especially to those who have added true value here - I've learned a lot from the thoughtful ones who share and help.
Maybe upvotes/downvotes could be weighted by the karma of the person giving them, thus putting more emphasis on those with high karma (which might sound elitist, but the only reason those people have high karma is because people think they talk sense).
This would concentrate power slightly in the hands of those long time dedicated commentators ('the community'), and slightly lower the strength of newcomers who are still getting used to the ettiquette ('the mob').
Hopefully this would go someway to lowering the effect of newcomers, until they rack up enough karma themselves to 'graduate' into the higher karma ratings.
Agreed -- I've always enjoyed my debates with you. I'd go so far as to say I sort of miss them. But I've now grown so frustrated by the pedantic replies that I get on HN nowadays that I've had to start deleting my comments after I write them. And I've stopped visiting as often because it takes too much effort to write a response that I feel avoids the trollish responses I fear I will get.
One of the final straws for me was when the title of my submission about being replaced as CEO was changed -- a topic of significant relevance to this audience I think. What frustrated me most was that I had picked a useful title and the not only was the new title worse in my eyes, but it was surrounded by rather inane posts about the "top X reasons for Y" or something about Apple or Google. It just didn't add up for me.
And on a tactical level, there is unquestionably a tendency for people to respond to the top post rather than start a new top-level thread. The lack of collapsing of threads is quite frustrating on long topics.
In my official capacity as "a relative newcomer", I agree that stuff's broken. I think one of the challenges the site is facing is a question of how much to continue to cater to the old guard and those who share their perspectives, and how much to evolve? The start world in general (and YC in particular) is at a very different place than when HN started ...
I also think there are problems (though I've not spoken with you specifically). However, I think this is a universal problem that doesn't necessarily need a solution. Most sites seem to have a group of hardcore participants with axes to grind. Top Wikipedians have lots of issues with Wikipedia. Top eBay sellers have problems with eBay. Even on MetaFilter it's not always plain sailing..
It seems there's always been reluctance to add features, for the sake of simplicity. Personal messages, for example, would have been useful in many, many instances, but instead we find ourselves checking out the plain-text profile and finding alternate methods of communication.
This limitation has also sprouted ancillary sites attached to the HN Tree of Life, such as searchyc.com, hackermonthly.com, and hnrecap.com as mentioned in the post.
In a similar vein, carving out a sub-HN seems to be: a) downloading the source code, b) bringing it online at another domain and c) announcing via "Tell HN".
All in all, unless someone with >10^5 karma decides to take the time and add some community features to HN (for various values of "community" and "features"), we're all going to continue and see more noise and many different signals.
As an aside, I wholeheartedly appreciate the name, "Bloomfilter."
As a low-karma, long-time lurker, I'm not sure I've ever really seen the kind of submissions he is looking for. Can someone provide examples of submitted content that would meet his criteria of deeply technical discussion worthy news?
I keep reloading this comment because I'm in the same boat as jmm57 and would love to see some examples as well.
Out of curiosity, is Hacker Monthly (http://hackermonthly.com/) what al3x and other long-time members want? Or is it the kind of discussion & content seen on a StackOverflow?
As a longtime MeFite and a longtime member of this community, I believe that most of the issues could be dealt with by having obvious and active moderation.
As MetaFilter, not only do we know who the mods are, we know which mods are on call at what times. (And there's 24/7 coverage.) HN relies very heavily on a flagging system, but it's just not as responsive to stuff that is broken as is a human who's responsible for what's on the front page and what's in the comments. Having a handful of humans who are responsible for curating the front page (and possibly also pinning really good stories from new onto the front page) would solve most of these problems. Is this less democratic? Sure it is. Would the unfairness be worth it? In my opinion, yes.
This problem just isn't solvable with code; it takes benevolent dictators.
What's the threshold HN crossed where it started needing active moderation? Because '08 HN threads were just better than '11 threads. Is it just a number of users, past which no set of guidelines restrains pathological conversations? Why? If you can trace it back to what thing happens when you get to your 50,000th user, maybe there's some passive moderation mechanism you can identify to keep it in check.
I'm not sure it's number of users--- Kuro5hin's downhill slide roughly coincided with a net loss of users, and its heydey was probably also its most populated era.
'08 HN seemed pretty similar to me, though, though I was just a lurker. It was a mixture of good technical content, startup-scene celebrity watching, too many TechCrunch posts, a vaguely political thread every few days (sometimes with a weak "hacking" justification), and the periodic thread once every 3-4 weeks about how HN is dying / has too many political posts / is turning into reddit. Just fewer total comments and votes (by a good margin).
Theory on the threshold: the point at which groups that social engineer social news sites decide the site is worth their time. This is also a point where the population has grown to a certain point and the discourse has regressed, providing a larger and more easily manipulated audience.
For me, I noticed it when people started posting things from investor blogs and the VCs and angels themselves started commenting here. I think that spurred some small handful of users to start showing off how smart they were in hopes that they'd get a response from fredwilson or whoever, and that just naturally spread the way it did. (There's a certain high when you get fredwilson or msuster to respond to your comment on their blogs; I think it's the same dynamic here.) I'm not blaming the VCs at all here - but that's when I started noticing it. (joshu and yegg are the exceptions that prove the rule because they're hackers first.)
Also, I've been told the Something Awful community is surprisingly pretty awesome, although I'm not a participant there. If there's some intersection of structure/moderation between MeFi and SA that HN could implement, I think that'd be a great place to start.
sighs We complain because we care. It doesn't mean that we have all the answers or we're pointing fingers. It means we're so invested in finding a solution to this problem that we won't just move on. Given just how good HN was (and remains), minute changes are more noticeable. I don't think a comparison of then-and-now really gives us useful data. There's just a gestalt, a feeling, you get that something's not quite the same. The closest I can think of is when you're out to dinner with someone you've fallen out of love with; your routine isn't any different, but you just know that it's not the same.
Something Awful and Metafilter both charge money to join.
Something Awful is pretty awesome...but is double-sided. The people from Something Awful are the same people who founded 4chan. ADTRW (Anime Death Tentacle Rape Whorehouse) is one of the boards of Something Awful and they were the first people on /b/ the day it opened along with 4chan and have set the bar to where it is since.
They are basically a grown up 4chan...amazing things like Copenhagen Suborbitals http://www.copenhagensuborbitals.com/ but aren't all pretentious and srs business like Hacker News likes to pretend it is.
SomethingAwful was (is?) good because of the membership fee, and because of the viciously strict moderation.
You could have your account deleted, with no warning, for:
* Persistent lack of punctuation or grammar
* Posting tired memes
* Persistent inability to cite sources for claims (in some subfora)
* Dozens of other things I can't remember
* Basically anything the mods decided to ban your for
The reason this worked is because the mods were drawn from the community. The admins would look at a subforum, see who was trusted on there, had good judgement and was there a lot. They would then make that person a mod, without asking them or even telling them! That person would just log in next time to be confronted with lots of extra widgets on the UI.
Because the community was so cohesive across the site, that person would then know immediately it was their duty to post a thread saying "Hi, I'm XYZ, your new Moderator" and christen their new powers by stickying the thread.
Again, they were not told to do this, but they would immediately and without fail, because the criteria for being chosen as a mod means that anyone who is chosen would be aware of this tradition, and in fact every other aspect of their role, having learned by example.
The SA forums were operating lots of quite deep game mechanics stuff nigh on ten years ago. It's very interesting. Most of the measures wouldn't be appropriate for here though, a lot of them involved strategically fostering antagonism for the benefit of wider community cohesion. It really worked over there for a long time though.
EDIT: I seem to remember that for a long time they only took payment by actual credit cards, visa and so on, and not debit cards, paypal etc. This was deliberate to prevent kids from signing up.
I think there's a real disconnect between HN and Metafilter. Metafilter is about making good posts. There are tens of thousands of active members and only a few dozen posts per day because you are supposed to make good posts. The motto is "best of the web" and it is a serious motto.
By design (I think?) HN is not about making good posts; it's exactly the opposite. It's about posting whatever link you happen to stumble across. The hope is that maybe the good posts will rise to the top via voting, and that comments will add value to bad posts. This is a really different idea! I'm not personally convinced that it's possible to get a high volume of visible good posts using this idea in an open general-interest community, because no matter what you do, there will always be more people ready to vote for popularized, lurid, and flamebait posts than good posts.
I think that distinction has much more of an effect on content than moderation. (Although if you've decided that you only want good posts, you probably need moderation to enforce your decision.)
HN would be better if it were invite-only like Dribbble. I'm not a member of Dribbble, but it's a good example of why restricting community growth is beneficial.
We're too late for that here. I don't think PG has enough bandwidth or interest to truly solve the problem. New users will continue to join, adding noise to the signal, unless HN changes course. It's going to become more generic and more biased the longer the site stays open.
I hesitate to suggest more moderation as some posters suggest. I'm already uncomfortable with the murmurs of unfair moderation in the system here.
* Site X residents refer to themselves as the New Wave of whatever. Much better than older Site W because of features/members/dynamic/demographics 1, 2 and 3!
* Site X's reputation spreads to former hot new sites T, U, V and W. Site X begins to attract more and more new users.
* Site X denizens begin linking articles at T, U, V, W and vice versa.
* Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th
"What does Site X think about AlphaGamma?" post and begin to slap down newbies.
* Someone reminisces out loud about the Golden Days of Site X.
* Discussions on Site X become more and more about Site X. Extremely intelligent individuals begin to earnestly argue that their proposed feature will save Site X from itself.
* Someone proposes or launches Site Y. A how new community begins to form there ...
I've been watching this same story play itself out since Slashdot circa 1998.
This comes across as snide. Slashdot circa 2011 is largely unreadable. So is Digg, so is main Reddit. The cycle may be real, but so is the phenomenon driving it.
Sure. All sites experience regression to the mean[1].
Usually this starts happening about 12 months after I decide to become a regular. In HN's case I've come in halfway through the cycle, about 3 months ago, so apparently the magnitude of my destructive powers is increasing.
>I've been watching this same story play itself out since Slashdot circa 1998.
And every social subculture in history. See: the constant evolution of the social scenes around music genres. It's just a fact of social psychology we have to live with.
"Dubstep was sooo good back in 02 before the students found out about it! I mainly go to future-garage raves now man..."
On reflection it resembles flocking behaviour. I imagine that website members can be modelled as simple agents with two rules:
1. A cool site has few, impressive members.
2. Move towards the cool sites.
Given an arbitrary distribution of "impressive" members, this should cause a constantly moving flock of members to move from site to site regardless of its purpose.
It strikes me that a logical step to prevent #2 is "allow only new contributing members that have been vouched for by n old members".
Yes, I'm that elitist. And yes, I'm well aware I probably wouldn't get voted in :)
But here's the other part - keep the site open for reading. That means it (hopefully) self-selects for quality, and other people can observe a high-quality discussion.
It'll be interesting to see how that model holds up to another 2-3 years. The few invite-based communities I've been a part of started to degrade as the social graph grew out and started to pick up the occasional undesirable.
It was harder to get rid of them because they knew someone who knew someone, and trying to push them out had a network effect. It might be obvious to someone removed by 3+ levels that a bad element is bad, but the person who invited them might not see it.
I am seriously considering pruning branches, not just leaves of the social graph to keep communities healthy.
I.e. if you invited a bunch of bad apples, you will be pruned too. I have no idea if it would work, but it's pretty obvious that open social graphs beyond a certain threshold are not conducive to high-level discussions.
There is a story making rounds about hiring policies of Saint-Petersburg Mint (back in Royal Russia times). Every new hire not only needed three recommendations from current employees, but that should the new person ever get fired, those who recommended him would be let go as well.
Brutal, but brilliant... even if it is not necessary true :)
But here's the other, other part: keep the site open for writing as well, but make the non-elites' contributions only visible when you are logged in as a non-elite. Then the elites can continue their excellent discussions unhindered, and any passing stranger can browse the elite discussions: yet everyone can have their say in the agora.
Let's not forget it happens in the actual cities we live in as well! Hipster-ism gives way to gentrification and then the cool kids move because there are way too many yuppies. Same things happens with suburbs - they get too crowded and people start moving to the country to make new suburbs.
This is just human nature, it manifests itself in the clothes we wear, the cars/bikes we drive, the places we live, it certainly also applies to social websites. I'd be surprised if it doesn't - we'll see how facebook fares, or if there will be a coolkid exodus as the rest of the world gloms on.
It's basically a (sub)cultural arms race that everyone fights in.
Rent control has been a pretty effective way of preserving the identities of communities in cities. It's not perfect, but it helps. Concepts similar to rent control can and should be applied to sites trying to preserve certain characteristics.
In my experience (SF, LA) the rent control is there to prevent the gouging of yuppies, not to keep the poor people in. The idea is the city wants to keep rent just high enough to retain the rich people for the tax and spending base but low enough that the landlords to squeeze them out, which they most likely would do if given the chance.
It might be different in places other than California though, like New York.
Lots of landlords are stupid and will raise rent beyond the market clearing rate just to make a quick buck, resulting in empty units and shitty communities.
No, actually it's rent control that gives way to that. Landlords don't rent at a loss, and prefer keeping their apartments empty, resulting in shitty communities, then squatters, etc.
Most economists believe that a ceiling on rents reduces the quality and quantity of housing available.[41][42] This view is based on analysis of empirical evidence as well as the understanding generated by theoretical models.[42] Economists from differing sides of the political spectrum, such as Paul Krugman[43] and Thomas Sowell,[44] have criticized rent regulation as poor economics which, despite its good intentions, leads to the creation of less housing, raises prices, and increases urban blight.
It probably works both ways, depending on exactly how low or high the rents are held. I've lived in some really, really nice areas that are rent controlled (SF Pacific Heights, Santa Monica, Downtown San Diego), apparently others have lived in relatively crappy neighborhoods where rent is also controlled.
Now that I think about it further, it seems obvious that rent control would be a tool that can be used by a municipality to control rents in either direction, depending on their goals.
Here in SF the geniuses of the Lembi Group bought a bunch of apartment buildings on credit (something like 1 billion dollar's worth), used questionable/borderline illegal tactics to squeeze people out of rent controlled units, and then jacked the prices way up. Then the real estate bubble popped and they went bankrupt.
Off-topic, but rent control has terrible problems. It lead to some grave NY neighbourhoods problems from the 60s to the 80s; back in the 20s in Paris...
A big part of political-economic maturity (for me at least) realizing that just because the supply/demand lines from your Econ 101 class don't line up, doesn't mean that something is worthless. It could be that you're not measuring everything. Culture matters, preserving social mobility matters.
Well, to be perfectly exact, in the real-world, for a limited duration, controlled rent can work fine. It just has been applied without discernment too many times.
Sure, you can screw up anything, and I certainly wasn't saying "everything should be rent-controlled".
Just kinda irks me when someone comes in with the Econ 101 argument regarding a policy that's being implemented by people with advanced degrees in city planning. These people took Econ 101, and know more about the subject than you or I. They could still be wrong, of course, but it's not for a silly simplistic reason like that.
While I basically agree with you--I too have seen this cycle many times before--what I don't understand is why people feel the need to slap down newbies in an attempt to recapture some since-passed golden age.
If I get bored of a site or simply get nothing more out of it, I just move on.
If I see topics that are boring or simply have no interest to me, I just ignore them.
If I see rude, insulting comments or personal attacks, I just flag them and move on.
I really don't understand why people need to express so much hate.
People who are deeply invested in a site want to defend their view of how it should be. Change is threatening, and it's easy to react with knee-jerk hostility.
Slapping down newbies takes on a lot of forms, too. This entire submission and the accompanying threads comprise a giant screw-you to relative newbies like myself.
I don't kid myself that I'm the kind of person that always remains civil in discussions, because I'm passionate about what I believe in. I know I'm hit-or-miss here, and often the way I communicate my point of view dilutes it and comes across as being a dick. I get that, and I've learned a lot about how to communicate with others from HN and other Web sites. If I'm the problem, though, which this submission and other comments seem to want me to believe, then I'm wondering what the solution is. Agree all the time? The burden is on me to improve, not the community to avoid me.
Every time I see a navelgazing submission like this I'm yanked right out of the community and I'm instead thinking about the community instead of the topics that I come to Hacker News to read and discuss.
What people don't seem to understand about community sites such as this one is that if you are discussing the community ad infinitum, you've already lost. A bunch of high-karma users, and one commenter in particular reminding us that "on behalf of the elites, this site sucks now due to you," is exactly the kind of thread that is contributing to the problem.
If the high-karma users here really want a walled, invitation-only garden where a circlejerk of valley gossip takes place and everybody agrees with each other, this is a pretty good way to go about it. It's really disappointing, to be honest.
> Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.
This implies it's hostility to "old hat discussions" that have already been around the block that people are worried about.
I don't think that's primarily it. The civility levels have slipped a lot... this place used to be as civil and friendly as meeting a bunch of people at a dinner party where everyone admires the host.
Still super-civil compared to the rest of the internet, but the levels have come down a lot. Raw outrage, profanity, and outright insults used to get downvoted to oblivion even if people agreed with the general argument the commentor was making. Rudeness really wasn't tolerated, so new members learned quickly that you had to think for a minute longer before replying.
It's still a great site, but it's not elitism and reminiscing that the old guard is scared of. It's comment quality slipping to rest-of-internet level.
How about, instead of a single community, the site runs as a set of "cohort" communities that each only allow direct sign-ups for their first year in existence, but then run a bit like leagues thereafter? If people like you on HN/2011, you might be invited into HN/2010, then HN/2009, etc. It could also be automatic/karma-based: if you earn enough karma on HN/2011, you start again on HN/2010 with 1 karma point (though you can also stay on HN/2011, marked as an alumnus/boddhisattva.
When you have two groups that have no reason to not be the same group, the fact that they're separated breeds resentment. When you tell them that there's a criteria they were ranked upon to put them into these groups (in this case, age + karma), they understand. For example, there's no big rivalry between grad students and undergrads; one graduated to become the other.
>For example, there's no big rivalry between grad students and undergrads; one graduated to become the other.
All undergraduates know that they will soon be at least graduates, if not later postgrads. The transition is inevitable.
Any kind of stratified internet community would have to have the eventual prospect of transfer from one strata to the other, as an inevitable consequence of good/bad participation.
As soon as it becomes set in stone, or the transfer becomes a deliberate choice rather than an inevitable consequence of behaviour, I think people will forget about the criteria that originally divided them and slowly regress to hatred of the other.
Just because this has been the typical website lifecycle in the past, doesn't mean that it has to be the typical website lifecycle in the future. We increasingly have better and better tools to manage how people interact with sites and communities.
Upvoting/downvoting, filtering, disemvoweling, moderation, tagging, reputation systems, etc. are just a smattering of the tools we have available and with time it's likely that we will see more and better tools for community management available in the future.
The most important thing is for "us" to define exactly what current and past features of the community we want to preserve and also decide what kind and amount of "chaos" we want to permit to allow the site to evolve and improve. It makes the most sense for PG, YC founders and very high-karma users to be responsible for defining these community characteristics, ideals and values.
The problem is that many of those tools are affected negatively by the regression to the mean as well. An average comment, that would never be upvoted during the early elite, will be upvoted by average people during the mean. Reputation follows.
Eigenvalue/SVD upvoting. Comments and articles are ranked for you according to their ranking with similarly voting/submitting/commenting users.
The frontpage (depending on how you set it up) could look like Reddit's, based on the majority of users, but to logged-in users it'd be more refined to their tastes. Perhaps you could look at the site from others' perspectives.
Defining the model, computing likenesses and finding the appropriate material could be quite resource intensive if you didn't think long and hard about the implementation.
(You'd might also want to avoid up and down arrows, because "like" and "dislike" quickly supplant "good comment" and "bad comment".)
The very first time I saw a site like this, many years ago, that is what I assumed all of the voting was about. I know a lot more now than I did then why resource wise it is the way it is, but I occasionally think about how one might go about doing a more customized version.
Superficially at least, it seems to be a good way to put everyone into a community whose size relates to the commonness of their preferences. I think this would occur as long as comments/articles beneath a certain threshold just weren't visible to a particular user.
Unfortunately it seems like a lot of calculating to do per hit, but in thinking about it, it doesn't seem to hurt Netflix too badly.
If it's too much complexity to track each user's preferences in terms of every other user's, you could also take a Bayesian filtering approach, similar to junk mail controls, and compute a custom front page based on each individual user's prior upvotes/downvotes.
In either case, the function of the voting mechanism will ultimately shift from generating the public front page to generating a custom view for each user. At that point, you could re-purpose the public front page as a selection mechanism to draw in the highest-quality users - perhaps by choosing a cohort of existing users to have public-upvote rights, or even by just applying old-fashioned editorial controls.
* We all (and I'm aware I'm a pretty young account myself) vote on stories and comments to rank their quality
* Newer users vote less in line with the historic ethos because they're less aware of it
* All our votes have equal power (the downvote filter here is a crude effort to suppress that but isn't powerful enough to be fully effective)
* Network effect means a successful site's growth will be closer to exponential than linear
* With exponential growth the new who are unaware of the community standards are quickly able to overpower the existing who are aware
Hence almost inevitable degradation in quality which people here are complaining about and which I can partially see myself.
Two ways to partially address this spring to mind:
* Express vote power as a function of reputation - an upgrade to the downvote system. This has the problem that it provides a direct incentive to align with groupthink and post the easy but commonly held stories and threads so as to attract repuation and so increase one's voting power and.... It also, I think, requires floating point reputation and scoring, and so increases computational load.
* Display voting based on a similarity function - 'users who liked this also liked....'. This permits cohorts of users to work en masse without prior coordination to lower the ranking of content they find less interesting / valuable and so get what they want more easily. This means though that it increases fragmentation and reinforces groupthink again by design, which is clearly not ideal. It's also again significantly more computationally expensive.
TL;DR: Computation can be thrown at partial improvements but they're expensive to operate, open to gaming and likely to reinforce groupthink. I'm not sure it's socially or technically possible to keep the noisy brats out without severely damaging what attracted the Grand Old Users in the first place. Welcome to social groups...
What about disabling voting until a user reaches a certain amount of karma? Of course, you'd have to give the upvote ability to the original members (hand selected?) so that the system could actually get off the ground.
It would be similar to the "invite only" community idea where older users have to actually approve new members, but less restrictive. Perhaps even have upvotes on users instead of just comments. It would be similar to an "invite", but just for allowing them to vote. I'd say also keep the hellbanning that we have here. There will sadly be some false positives, but it seems to work very well for keeping out the worst.
Perhaps upvoting users could affect the reputation of the folks doing the upvoting. If you upvote a bad user into the community, your reputation suffers as well.
Now, this does nothing to solve the problem of groupthink, and in fact somewhat enforces it. However, I think some amount of groupthink is inevitable if you want to create a strong sense of community. Because you're trying to cluster users around a core set of values, it's probable that they'll think similarly in other ways as well.
Edit: I think multidimensional voting helps as well. Ignoring the comments voted "funny" on /. helps to filter out what people liked just because it was amusing. I think "agree/disagree" voting would be useful as a separate thing, because too often that winds up conflicting with upvoting/downvoting for quality.
Interestingly, this isn't a problem related to any particular category of site. Hell, look at 4chan: bitching about newfags and how /b/ isn't good anymore, how /b/ was never good, different factions with different ideas on how to fix the problem (or what the problem is, or if there even is one), spinoff sites where the early adopters migrate to...
This isn't an HN problem, it's an Internet problem. pg could probably do some things to ward off dilution, but it won't last forever. Interestingly enough, Something Awful seemed to be pretty damned successful in this regard by doing the unthinkable and charging people to post there. I'd hate to suggest it as a solution, but it has been effective in the past.
SA has internal problems too. There's a constant slow cycle where the individual topic forums become boring and straightlaced because the users get proud of how mature they are, so they start new subforums where people are allowed to make "bad" but interesting posts as long as they pretend to be doing it ironically. Then everyone moves into those and it collapses, because they really did become bad posts.
Sometimes they move offsite instead, usually because the site admins didn't actually like the topic enough to make more forums for them, or because the admins banned all the users because they had formed secret IRC cabals conspiring against them.
The last stage is that the admins change the forum CSS so it's unreadable. I haven't figured out that part yet.
The gist of that (forums "cancer") is that those users "infected" with it have their posts become harder to read outside of those subforums by lightening the font. A quick mouseover returns the text color to black.
In my opinion, the method doesn't really work because they aren't reading their own posts. It should probably change everyone else's text color instead of their own. Greasemonkey scripts and the like would still fix it, but it would be a lot more annoying to the shitposters that way.
Still, for a site that's been up since the Dark Ages ('99), and relies almost entirely on its users to keep it relevant, it's been remarkably successful.
Just joking! I totally agree with you. Both in regards to /. but also with websites in general. In fact, the Internet in general. It started out (for me) idealistic and cool and slowly has got dumber and more mass market as normal people started using it. Ah well. We lost something, we gained something. The cool is gone, but there's a lot more of the typical stuff and the Internet is making some of the real world changes I hoped it would only because it is now so ubiquitous (like the change in the Middle East)
I think all discussion sites struggle to identify quality discussion. The whole Karma idea that /. created was very very cool and has worked reasonably well over the years. Certainly, their discourse is wildly better than digg's, for example, and I would submit that Karma is a large part of that. But the problem with Karma is that it promotes group-think and that's where /. is today. Alhough: you could say just the same about HN. Anyone who doesn't love LISP, Python & PG is automatically on the outer. Hating Java is cool, too.
Perhaps group-think is an inevitable requirement for communities to form? Perhaps without some level of shared unquestioned axioms, groups will not form in the first place - just in the way a Christian and an Atheist can struggle to have a conversation about right and wrong because their starting assumptions are just so far apart? I don't know - I'm not sure if there is an answer to this kind of thing, I think that maybe it is just human nature at work.
I agree. One thing tho that never seems to come up: we people from older waves ignore the fact that we spent the last $N years growing, getting better, getting deeper into stuff, while the site has not grown with us. So even if the site hasn't changed, our perception of the topics and such has -- intro to X used to be deep, since we were little, but now that we are big, we just see a puddle :)
If you are right, a website could automate the cycle as follows:
1) Every user belongs to at most two (or k) classes.
2) Each class can have a finite number of users n.
3) When a class fills up, a new one is opened.
4) There are always two (or k) open classes at a time, and new users join both of those two (or k) classes.
5) The classes are staggered so they don't close at the same time; e.g. if k=2, n=10000, the first 5000 people to join belong to class 1 and class 2, the next 5000 belong to class 2 and class 3, the next 5000 to class 3 and class 4, and so on.
6) Users can only post to classes they are in, and can only comment on posts in classes they are in (or maybe newer ones too).
7) By default, users only see posts to classes they are in (or maybe their classes + older ones). They can opt in to see more if they want.
The idea might work better if it is combined with some more onerous mechanism to join classes late - a fee, a requirement for an endorsement from a member or someone. Charging a fee for access to the more '1337' earlier classes might be a good way to monetise the site; the fees could either be flat, or could escalate (exponentially?) with the age of the class and / or the number of people who pay to join it late (possibly with a discount applied as active members leave).
Classes might get too small due to attrition - coalescing older classes when they get too small might be a solution to this - it might also make sense to increase class sizes (exponentially?) as the site gets more popular.
Could this phenomenon be in any way attributable to nostalgia? Personally, I find any online community loses attractiveness after a certain period of time.
I think if we could see random frontpages from days a few years ago, we'd find that the top stories weren't that different, and that there was the same "jack of all trades, master of none" aspect to the site that Alex complains about. It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.
Maybe I'll write something to regenerate past front pages, so we can check if things are different now. That should be possible, because news.arc has always logged vote times.
It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.
As the infamous "How to Become A Hacker" states "The world is full of fascinating problems waiting to be solved"[1]. There is no mention of those problems having to be "deeply technical".
I certainly see the tremendous variety of topics that make the homepage of HN as an asset in keeping with the true Hacker Mentality.
The non-technical questions on HN have poor associated discussions though. As OPs go further from tech/startups (and towards econ and policy), commenters display a poorer understanding of basic concepts.
Bottom line: sure, hackers care about many problems. But that doesn't make up for the poor discussion quality surrounding many non-tech articles here.
The non-technical questions on HN have poor associated discussions though
I've seen what looks like fantastic advice and discussion on a range of non-technical topics (particularly things like venture capital/financing and laws around starting a company and that kind of thing).
Of course, not being an expert in any of that stuff I'm at a loss to make a judgement as to it's veracity ;)
Of that sampling, it hit the apex for me with Jan 2009 page. The Mar 2009 page is when there was an influx, and we started posting Erlang stories as a joke to stem the flow.
That Erlamg screenshot is hilarious. Only one story on the front page didn't have Erlang in the title, why HN had been slow...was it due to the amount of Erlang submissions?
What brings a tear to my eye is that, back in those days, it was hard for me to tell at first that it was a deliberate plot. It felt quite natural to have multiple discussions of Erlang on the front page.
I was part of that influx (I registered 5 days before pg's Erlang post). I apologize for making the page worse for the rest of you ;-). Honestly, the Erlang posts contributed to keeping my interest (though I didn't pay much attention during the following summer as I was busy coding).
Political and other front-page-news posts with little "hacker" content, blogs by irrelevant magazine columnists, posts about how X product/market/phenomenon is dooooomed, random questions about how the site works because the FAQ is a tiny link in the footer...and lots of posts by people horribly concerned that those damn noobs will stink the place up.
I really haven't noticed HN get any worse in the two years I've been around.
Maybe you should compare random /active pages from years back. I don't know, I only found out about /active this last month, but I'm guessing the problem will be more visible there.
Most of us that have been around since the beginning (this is my second account) are too busy to participate as regularly as we once did. I'd rather spend time on my business than get hooked on discussions here as much as I did in the past. I will jump in here every so often and I still think HN is relatively great. Doesn't everybody remember the big debate over even linking to Techcrunch articles? We have always been whining about what's wrong to realize that this place has consistently been pretty great.
And sure, we might be like that dance club that's a bit too mainstream. But pretty soon those early cool kids grow up and are also not cool enough to discover the next hot thing. The next HN will come about and it won't be somebody from here who starts it.
I agree. This place has always been mostly good with a sprinkle of fluff pieces every now and then. The one thing that has changed a bit is the comments. Sadly the comments are being muddled a bit by the new kids on the block. Not that the new kids are doing anything intentionally bad, nor are they really bad comments, but I do remember a time where the people replying to comments were actual domain experts in a certain field or they wrote the application that you use every day, etc.... It's a problem with getting a larger user base. I do think that this site has aged about as gracefully as possible though.
Additionally, I think newer users are conditioned by other sites to use voting as a way to agree or disagree with what is said, rather than as a way to rate the quality of the point being made, regardless of what the point is. Which I find very frustrating, and I know people say 'whining about declining quality has always happened', but I really feel that this voting behavior used to be less pronounced.
I think newer users are conditioned by other sites to use voting as a way to agree or disagree with what is said, rather than as a way to rate the quality of the point being made, regardless of what the point is.
It hasn't just been new users. Rather it has been nearly everyone since the beginning. See what the founder said 1103 days ago about downvoting for disagreement:
I think this may be due in part to people learning how to target Hacker News readers with their article and blog post headlines, so while the front page would look similar, the newer linked content is more fluffy.
I mean, there has to be some explanation like this, because it just seems so clear from regular reading that the quality has gone down. Then again, maybe I'm projecting the decline of comments onto the content.
The fact that HN hasn't been overrun by marketing zombies is why I'm not too worried about its direction. If linkbait titles are the big complaint about submissions, it's a good sign.
You (pg) will not need this since you have the data.. ;-).. but for anyone else interested in doing analysis over time, HackerSlide stores JSON representations of the HN front page every hour (starting Oct 15 2010).
No. At least, no more API than what I just explained ;-) It's not my data. It's publicly available because the main http://hackerslide.com/ site uses it and if you want to grab it every now and then, that's fine by me.
If anyone grabs the same file over and over or anything crazy like that, they'll be blocked, but if you just poach the data and store your own copies, go for it :-)
That's always a serious concern. It's not surprising that many bloggers either heavily moderate or don't even allow comments on blog posts. It's a difficult issue to solve for sure. Better comments means you need better people offering thoughtful and insightful comments.
There was a submission a couple of weeks ago of a site that randomly arranged previous HN submissions of a certain threshold from the past with each refresh. Was really useful and sadly neither Searchyc.com or Google is helping with finding the post.
Perhaps the submitter sees this comment and shares the link here again?
I have recently begun to realize that there are 2 aspects to "website degeneration". One is the obvious new people showing up because they heard it's cool psuedo python paradox situation.
The other comes from the old-timers themselves. I'm not particularly that old-timer, but in the 3something years I've been here, I can say I have experienced this:
I've grown.
The "5 awesome vim tips" or "super deep closure stuff" articles that today annoy me as fluff were deep and new to me back then. HN was a fantastic site introducing me to amazing things, and it is introducing a lot of people to the same amazing things now. As such, the quest for truly new content needs to go deeper, but the people who need the lighter content are greater in number and enthusiasm, ensuring the fluff rises.
When the older crowd doesn't notice their growth, or the lack of change in content (or the combo) you start to get "it's gone down-hill" type comments.
edit: I mean the quest for truly new content for me needs to go deeper, not that the site necessarily needs to go that way.
A similar effect: The average age on the internet is younger than you, and will be for the rest of your life. Think about what effect that has on the density of quality user generated content.
Umm, what? In the limit of everyone going online the average age of internet users becomes the average age of the population at large. Perhaps younger internet users would be more active, especially on social media sites, but that's a different matter.
Current estimates put the average age on the internet at about 37.
His point is that while you used to love some content (teen angst poems, political ramblings by 17 year olds, etc) as you've grown older, you're probably smarter and more mature. The quality of teen angst poem hasn't necessarily declined, but your view of teen angst poems has changed and you view them as worse because you're smarter and more mature.
I think this is a very plausible explanation, we are all at different points in the learning process and can only enjoy certain material. All but the most trivial programming articles are above my head right now, and so while I might enjoy them a lot; the HN community would say they are fluff pieces.
" One of the joys of being a kid is that experiences are new and therefore more intense. For example, I'm about to stick my nose in a jar of mustard and inhale deeply! Let's see what it's like. WHOOP!!
See, whed you are oder, you dake your sinuses fo granded.
Or may be after a while our perception about something changes disproportionate to the actual change?
That's basically boredom--and it can happen even if you consume something good for a long time. That "good thing" doesn't change so much as your perception of it.
Hi, I'm taking on this challenge. Al3x, can you send me an email at jkwon.work@gmail.com?
Also, I'm taking suggestions for seed users. There will also be a HN Karma cutoff where everyone above a threshold can join. You can nominate HN users or yourself here.
Where is the data that shows HN has degraded? We aren't seeing kitten pictures. A scan of the front page shows the mix of articles being programming, startups, tech.
I'm not sure what Alex wants? More discussion around PG's hackers and painters?
Very well written article. I think he has some good points. I'd really like to see a 'deeply technical' alternative to HN and hope he'll find the good guys !
Forking of sites has the same problems as forking open-source projects. It exacerbates conflict, forces people to choose sides, and ultimately both forks typically end up poorer because of members lost.
Instead, the best solution is to evolve Hacker News as a product.
My personal opinion is that we should put Hacker news in the hands of the YCombinator alumni. Founders and first employees (CEO, CTO, lead designer, first engineer hire and first design hire) of YC startups would probably make the best moderators and admins.
In fact, I would say that it's probably time that PG spin off YC as a full-time startup, assigning control of the design and codebase to one talented UI designer, one talented developer and one talented product manager.
For the site to keep growing in a way that maintains quality, it needs more functionality that it has. The two features that lack the most are filtering and combinatorial game mechanics.
Filtering is necessary so it is easy for the the hardcore tech articles to be easily found by high-karma members, so they can vote those articles up. If it's not findable, it's not voteable. Filtering is also necessary for people to extract the most value out of hacker news. Most users don't want 100 front-page articles everyday. They probably want 10-20 of the highest value articles. Less is more.
Combinatorial game mechanics like those on StackOverflow would help as well. Upvoting/downvoting is limited in that it will always fall victim to the masses. Giving special voting/tagging/burying rights to distinguished members (very high-karma users and YC founders and employees) would go a long way to helping eliminate the crap.
I think I speak for most members here, when I say that I don't want Hacker News to be a democracy. I want it to be a technocracy. I want the smart and accomplished people to control what is good and should be visible to all. I've got only 260 karma points, and personally I don't think that should be enough karma points to allow me to upvote a submission. 500+ karma points should be the threshold to be able to vote an article to the frontpage.
Say your valued members have a certain mindset, ethos and style. Let's call it YC-meme. Everyone with YC-meme contributes in a manner that exemplifies YC-meme. If someone comes along and doesn't have that meme, their posts tend to not be what the memesters want. So, members use their voting powers to reduce that person's karma. Eventually they either leave, get ignored by all, or become infected with YC-meme.
Now, open up the community to everyone. At first, YC-meme is still dominant. Only 1% of people without it are able to write posts in a style similar enough to gather karma. Let's say they have RD-meme, which has about 80% overlap with YC-meme. Great.
The problem is, soon, there isn't 1% of the community with RD-meme, there's 10%. Who can (and do) re-enforce RD-meme posts, because it's what they want. People can even get lucky and simply contribute 'good' posts, long enough that they get a decent karma score, and so can re-enforce different memes.
What makes it WORSE is that what's acceptible is what's visible, so with more RD-meme posts, RD-meme posts become more likely, as the probability that they get 'punished' decreases.
End result: Community-dilution runaway.
There are a few 'cures' I can think of, but most are fairly prescriptive:
1. A concrete set of rules and behaviors that anyone can reference to moderate from. Gross.
2. A very high bar for moderation, with active participation. Given that the people most likely to have YC-meme are also likely to be busy, this might be doomed by design.
3. "Proof" required for contribution. Great way to kill off your community.
4. Invitations. Either so easy to get as to be meaningless, or so hard as to stagnate the community.
5. Close ranks to just people at/gone from YC. Sadly, that'd mean I was out. May still be the best alternative.
Maybe the community is self-aware enough to start consciously moderating better. Perhaps a post like this every so often might serve as a gentle reminder. That'd be the best outcome, but needs to be chosen by a majority of the user base. Do YOU want a stricter HN?
2. A very high bar for moderation, with active participation.
The reason this is doomed is: there is currently much more stuff needing moderation than the selected few can handle. They would soon wear out and you'll end up with a site with almost no moderation at all.
The trash needs to be stopped at the front door. When it gets into the house, it takes too much effort to clean up.
both forks typically end up poorer because of members lost.
I strongly disagree. Off the top of my head: XFree86/Xorg, Mozilla/Firefox, Debian/Ubuntu, Rails/Merb... these were all cases where the existing community had calcified in some key area and a fork was necessary to create a space where important work could get done.
And in most of those cases, not only was the fork stronger than the trunk in (some) specific areas, the fork actually led to the trunk getting stronger. Not only were patches merged, but entire projects, community values, development practices, etc were.
XFree86 is an exception, but organizational paralysis was what necessitated the fork, so it's not surprising they were also unable to respond to the fork.
But I just don't buy your notion that forking is somehow unhealthy and destructive. It's one of the most valuable ways we have to keep open source code, ideas, organizations and practices fresh, healthy, and vibrant.
I agree that there are exceptions. In the case of HN, I don't see how forking would be of benefit, especially since HN is so strongly linked to YCombinator, the startup incubator/school that tends to attract the most talented up-and-coming startups.
Well, obviously posts/discussions like this can actually be contributing to the "problem" as some see it, but I'll make one point I don't see made here.
For me personally, I've learned a lot and grown a lot over the course of the 4 years I've been lurking and occasionally contributing here. So for me, a smaller percentage of the stories/articles/posts/discussions appear as insightful as they once did. I don't mean to knock HN in any way, in fact my point is that that fact is not a "problem" to me. New users are joining everyday and everybody who makes the effort to learn and contribute gets something out of HN.
All of this has happened before, and all of this will happen again. HN is succumbing to the problem pg tried to address: dilution. Thing is, reddit already came out with an excellent solution with their subreddit system. This simply wouldn't be an issue if there were different sections for tech, hacking, programming, startups, science, finance, and general interest. Keep all of the deeply technical stuff in one place, the cruft in another. Let's face it, the people who are complaining about lack fo deep tech are also likely to read and enjoy one of Spolsky's blog posts. There's no need to ban the latter to protect the former, just keep them in separate sections.
Now, HN isn't trying to grow, so there's no need to have user-created subreddits (sections, I suppose). Just make 8 or so that people care about, and add another if there's sufficient demand.
I really shouldn't be crediting reddit with this, as the solution existed long before them. All HN needs to do is follow the forum model and have different sections. It's too big to only have the front page.
How to solve the signal/noise problem? Amplify the signal.
Call it undemocratic, but insight and perspicacity is not uniformly distributed so it's absurd that pg/$whoever_you_respect's upvote on an article counts as much as anyone else.
As a simple experiment, it would be interesting to see a view of the frontpage based only on the upvotes of people who are above a certain avg-comment karma threshold (since the site is predicated on karma as a quality indicator) and the idea that people who write insightful comments won't upvote crap stories.
The problem with using karma is that it is affected by the regression to the mean. As more "average" people come in, more "average" people will give them karma. Solve that problem, and you might have a solution to the whole problem (I, for one, am not smart enough for that ;).
Alternatively, perhaps we can let users select (follow) a set of like-minded people and make the up-votes of the people they pick weigh more on a "customized" front page.
But then people may have not like having their up-votes be so transparent..
But the interesting issue here is that those who write interesting commentary may not be good at discerning good quality. (according to whatever standard we're holding)
This problem applies to all communities using votes to sort user-generated contents: HN, Reddit, StackOverflow, Quora, etc.
I think something like PageRank can really help. The same need is to rank things based on a directed acyclic graph. The only difference is we have multiple kinds of vertexes: users, submissions, comments.
What about using baysian filtering? We already have a lot of data on good and bad comment styles, baysian filtering could give us an indication if a comment is violating HN guidelines.
I am not so optimistic about submissions because the data we currently have is tainted by submissions we don't want. However by using the technique you have described, we could probably achieve better results.
If you really wanted to make the site more like it was, you could look at the voting history of people here at some distant time in the past and find other members whose votes highly correlate with the old timers. Some people just quietly upvote, so they don't have much karma. If you want the upvoting to be more like it used to be, award people who vote similarly like how older users used to vote.
It would help take care of the effect where older uses leave due to changing interests etc.
Note: I don't actually think it's a good idea. I think just figuring out _what_ you want exactly, in vaguely quantifiable role, is sort of a hard problem. I suspect a lot of people have different 'golden ages' in mind when they think of HN.
When I came here originally it was pretty startup heavy, which was sort of remote from what I was up to at the time. I liked it when it got a little away from startup business related stuff (my perception) and toward more generic technical issues. My point is even if everyone agreed that ait has gone done hill (an open question), not everyone might agree on what it should be like. It's not a democracy of course, but even as an individual for my own preferences that seems as hard as the technology issues.
The basic complaint is that social sites grow into mobs. The solution is rather obviously: halt growth. You have to limit the number of active users to some vaguely Dunbar-ish number or you inevitably wind up with a lowest common denominator mob.
Metafilter did this, right? For a couple years they said "No new accounts."
I think scaling a social site to a very large number of members without deteriorating badly is impossible. It's a matter of human nature and mobs.
283 comments
[ 6.4 ms ] story [ 443 ms ] threadI'm sure that post itself rankled our OP.
I was really quite blown away. I could have just put it on my blog then posted that to HN (which is actually what pg said I should have done when he emailed me), but my initial thoughts, were that I really just wanted to share this with my pals on HN (similar to the way someone else had posted Ben the Bodyguard as a "Gorgeous HTML5 implementation" or something) and that putting it on my blog then linking to my blog from HN would just be viewed by the community as me jockeying for traffic and being self-promotional!
Gods know it was a terrible decision, but it showed some active participation by pg.
In my official capacity as "representative of people dorky enough to have karma this high", we do officially declare: stuff's broken. Needs unbreaking.
Many forums have been driven to civil war by a clique plotting on a secret IRC channel. (Note: I am not suggesting that is happening here! Just that I've seen it before.)
The mention of the fact that the higher-karma, "ingroup" members of HN seem more likely to communicate with each other on twitter and in private emails than the rest of the population reminded me of bad situations I had seen on other forums.
I then thought how glad I was that here on HN it seems to happen on twitter which is still largely public. You commented quickly here about private conversations regarding HN with other high-karma members. It is reassuring that you were open, and happy to be open about this.
This gives me faith in what I find to be an excellent community that the inevitable psychological group-dynamics type phenomena are playing themselves out here in a relatively benign way, due to its high calibre membership. Your later comment "I'm too high drama for that" came across as an amusingly wry confirmation that you feel the same way.
As I said, this put me in mind of situations I have seen in lesser communities where this dynamic had led to ugly scenes. I posted about them. I then felt bad that you or someone else might misinterpret this as an accusation against them. Thats why I added my parenthetical note. Perhaps I should have reconsidered the whole thing.
I would also encourage people to look up folks' email and email them with questions and such. Great way to make new friends. Many a time I've tracked somebody down from an HN discussion only to have it continue via email. (It also brings the noise factor down over here, which is always a good thing)
I know a lot of guys follow everybody they can from HN on Twitter. I don't know if I would personally go that far -- Twitter has a hecuva lot of noise and I get easily distracted -- but it works for some.
So yes, there's a lot of out-of-band chat going on, but as far as I know, none of it is exclusive. If you'd like to participate more, the same channels are open for anybody. I know I keep my Twitter, Facebook, and email addresses in my profile for just that very reason.
Do you mean through here: http://www.ircnet.org/ ?
If so, either I'm doing something wrong or the Secret Cabal is not in session tonight (darnit).
Stories, yeah, they're a bit worse but that's not the main thing. I skip stories that don't appeal to me, and you can always go a few pages deep if you want more stories.
It's the comment quality sliding down that's scary to me... it's becoming much more okay to engage in standard-internet-level discussion here, which is to say, much less civil than HN used to be.
Perhaps liberally killing any comment that's primarily insulting, sarcastic, pedantic would help?
For instance, I just saw a link to this two year old thread today, look how civil and friendly it is:
http://news.ycombinator.com/item?id=554889
The only halfway insult on there - "Dunno but she has a Twitter page. Not sure why PG would quote such a weirdo." - was voted to zero. And the 2-3 jokes/sarcasm stood still at 1, none of it voted up.
Also, I'd love to see people who just consistently left substance-less comments get put on auto-kill, but I know that'd never happen for a ton of reasons.
I'm thinking of one guy who just posts about how outraged he is about anything remotely politics related, and nothing else. Joined in the last six months. Actually, that probably describes a lot of users now, which might be part of the problem...
http://news.ycombinator.com/item?id=236808
You couldn't get anywhere near that today. Discussing tough and interesting subjects civilly with some of the smartest people I know wasn't what brought me to hn (startups were), but it was a big factor in keeping me here. I knew if I posted some crazy half-assed startup idea that might piss people off that the community could talk about it logically and dispassionately. That allowed each of us to open up more.
Those days are gone.
I blame karma. I know it existed back then too but I don't think people talked about it. There should be no "top users" page. Having a measuring stick drives competition and there should be no metric to civil discussion besides the rewarding feeling of discovery and helping people.
http://news.ycombinator.com/item?id=237127
Admittedly, it wasn't voted up, but there was some agreement with it. You're looking back wistfully at a point where another user thought HN had fallen off precipitously.
I don't see a significant difference between HN now versus about 3 years ago (when I first started lurking), and I believe the community could and (on most days) would have the very discussion you've linked. There are cycles where many of the submissions aren't interesting to me. I usually chalk those up to slow news days. And there have been situations where I've been appalled and infuriated by (what I perceived as) the poor quality of articles/comments that the community was voting up. Some of that is the community offending my own biases. Some of that is just bad days. You can't discern the climate by just looking at today's weather.
The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse. But I'll allow that others are more sensitive to HN's quality. Perhaps these reflective posts are what keep HN's quality high enough for my tastes. Or maybe, as I suspect, they are more about the tastes and perceptions of the long-time users changing. It's more than a little reminiscent of the "golden age" thinking of older generations, where the past was simpler and more innocent because they were younger and insulated from the time's complexity and injustice.
(Sorry for rambling there.)
Yes, I think that's what made it a good example.
The other cycle I've noticed is these efforts at self-policing and reflection. I'm almost annoyed by them because I don't see HN as getting worse.
Yes, it's almost an inside joke. One of the "classic" HN stories is the meta story where we all lament the decaying state of HN. The "naval gazing" stories. You learn either to ignore them or vent (politely) After watching them for a while, I think they serve to help release a lot of frustration people have.
I would note that many of the folks early on who said HN was fine are now also saying there are problems.
To me, HN's core problem was the problem you always have when speaking to more than five people: trying to anticipate reactions. You end up trying to cover every possible response, and it kills the tone. One of the reasons my comments run on so long is that I have to cover every obvious way they could be misunderstood. You're stuck either not replying to anybody, writing long comments, or getting in endless debates. All of that is distasteful to me.
One of the hard lessons I had to learn blogging was not to tease people. I used to put pictures of girls in bikinis up or joke around about making money tricking people online -- to me I was just playing. I live by my reputation: being an internet porn mogul is not on my list of future plans. But there were a heckuva lot of folks who didn't understand my posts, who thought that made me some kind of evil person. It was really sad to have to give up the playful part of my personality simply to get along with people because of the many ways things can be misunderstood. I hate that.
But the real problem is that everybody's personality will piss somebody off somewhere. I know I probably rub thousands of people the wrong way just because I'm the way I am. Others do the same. So then I end up commenting something like "Looks like it is raining outside" and I get 3 or 4 downvotes. Sure, the upvotes outweigh them, but it's just silly nonsense. Vengeance voting.
Hey now I've rambled too, so we're even :)
HN started as a way for YC guys to communicate. It was about startups, the tips and techniques to make one, to grow one. Once PG opened it up, it became "stuff hackers like", which meant cool deep tech stuff. But the tech stuff was there because after a while, all the startup stuff just ran together.
Then "stuff hackers like" kept drifting wider and wider. I was happy with that, but now that the traffic numbers are through the roof, what I'm seeing is the 1% snark factor is out of control. If each member of HN said something truly snarky only once a month, there would still be thousands of snarky comments a day on here. It's just too much noise.
HN has always been driven by emotion: the only reason to click that button is to express your emotional response, good or bad. Sure you can rationalize that in various ways, but in principle it's about how the thing makes you feel.
Now we have the love-fest stories, the hate-fest stories, the snide jokes that get upvoted, the trashing of "rate my startup" posts (I still can't believe how some of the startups are treated). It's becoming less of a community and more of a mob.
I think maybe the story titles might look mostly the same, perhaps, but the quality of the stories and the quality of the conversations have changed quite a bit.
I use to comment regularly on HN (member since Apr 2007 and former #1 on leaderboard in the ancient days, haha), but have found that I very rarely do any more.
When I look at my thought process now when thinking about posting something that may be of value, I find myself very reluctant. This is due to the snarkiness and hot-headed emotion that often disregards logical arguments as worthless and instead values breathy spittled anger.
Cowards become brave behind anonymity.
Who enjoys "lawyering" their comments? Basically trying to anticipate and address all the possible concerns hot-head users will bring up. Yet, I feel like I've had to do that more and more. In the end it seems an exercise in futility - particularly when I can just move on to interacting directly with the thoughtful hackers I care about and avoid those that spit when they talk.
I can understand the lament of the OP since I've really enjoyed the community and associating with the gentleman-hackers I've met here.
Thanks especially to those who have added true value here - I've learned a lot from the thoughtful ones who share and help.
This would concentrate power slightly in the hands of those long time dedicated commentators ('the community'), and slightly lower the strength of newcomers who are still getting used to the ettiquette ('the mob').
Hopefully this would go someway to lowering the effect of newcomers, until they rack up enough karma themselves to 'graduate' into the higher karma ratings.
One of the final straws for me was when the title of my submission about being replaced as CEO was changed -- a topic of significant relevance to this audience I think. What frustrated me most was that I had picked a useful title and the not only was the new title worse in my eyes, but it was surrounded by rather inane posts about the "top X reasons for Y" or something about Apple or Google. It just didn't add up for me.
And on a tactical level, there is unquestionably a tendency for people to respond to the top post rather than start a new top-level thread. The lack of collapsing of threads is quite frustrating on long topics.
For now, I'm mostly done here.
This limitation has also sprouted ancillary sites attached to the HN Tree of Life, such as searchyc.com, hackermonthly.com, and hnrecap.com as mentioned in the post.
In a similar vein, carving out a sub-HN seems to be: a) downloading the source code, b) bringing it online at another domain and c) announcing via "Tell HN".
All in all, unless someone with >10^5 karma decides to take the time and add some community features to HN (for various values of "community" and "features"), we're all going to continue and see more noise and many different signals.
As an aside, I wholeheartedly appreciate the name, "Bloomfilter."
Out of curiosity, is Hacker Monthly (http://hackermonthly.com/) what al3x and other long-time members want? Or is it the kind of discussion & content seen on a StackOverflow?
http://news.ycombinator.com/item?id=1321443
As MetaFilter, not only do we know who the mods are, we know which mods are on call at what times. (And there's 24/7 coverage.) HN relies very heavily on a flagging system, but it's just not as responsive to stuff that is broken as is a human who's responsible for what's on the front page and what's in the comments. Having a handful of humans who are responsible for curating the front page (and possibly also pinning really good stories from new onto the front page) would solve most of these problems. Is this less democratic? Sure it is. Would the unfairness be worth it? In my opinion, yes.
This problem just isn't solvable with code; it takes benevolent dictators.
'08 HN seemed pretty similar to me, though, though I was just a lurker. It was a mixture of good technical content, startup-scene celebrity watching, too many TechCrunch posts, a vaguely political thread every few days (sometimes with a weak "hacking" justification), and the periodic thread once every 3-4 weeks about how HN is dying / has too many political posts / is turning into reddit. Just fewer total comments and votes (by a good margin).
Some random examples: http://news.ycombinator.com/item?id=179755 , http://news.ycombinator.com/item?id=105739 , http://news.ycombinator.com/item?id=109052 , http://news.ycombinator.com/item?id=80234 , http://news.ycombinator.com/item?id=437727
Also, I've been told the Something Awful community is surprisingly pretty awesome, although I'm not a participant there. If there's some intersection of structure/moderation between MeFi and SA that HN could implement, I think that'd be a great place to start.
sighs We complain because we care. It doesn't mean that we have all the answers or we're pointing fingers. It means we're so invested in finding a solution to this problem that we won't just move on. Given just how good HN was (and remains), minute changes are more noticeable. I don't think a comparison of then-and-now really gives us useful data. There's just a gestalt, a feeling, you get that something's not quite the same. The closest I can think of is when you're out to dinner with someone you've fallen out of love with; your routine isn't any different, but you just know that it's not the same.
They are basically a grown up 4chan...amazing things like Copenhagen Suborbitals http://www.copenhagensuborbitals.com/ but aren't all pretentious and srs business like Hacker News likes to pretend it is.
You could have your account deleted, with no warning, for:
The reason this worked is because the mods were drawn from the community. The admins would look at a subforum, see who was trusted on there, had good judgement and was there a lot. They would then make that person a mod, without asking them or even telling them! That person would just log in next time to be confronted with lots of extra widgets on the UI.Because the community was so cohesive across the site, that person would then know immediately it was their duty to post a thread saying "Hi, I'm XYZ, your new Moderator" and christen their new powers by stickying the thread.
Again, they were not told to do this, but they would immediately and without fail, because the criteria for being chosen as a mod means that anyone who is chosen would be aware of this tradition, and in fact every other aspect of their role, having learned by example.
The SA forums were operating lots of quite deep game mechanics stuff nigh on ten years ago. It's very interesting. Most of the measures wouldn't be appropriate for here though, a lot of them involved strategically fostering antagonism for the benefit of wider community cohesion. It really worked over there for a long time though.
EDIT: I seem to remember that for a long time they only took payment by actual credit cards, visa and so on, and not debit cards, paypal etc. This was deliberate to prevent kids from signing up.
By design (I think?) HN is not about making good posts; it's exactly the opposite. It's about posting whatever link you happen to stumble across. The hope is that maybe the good posts will rise to the top via voting, and that comments will add value to bad posts. This is a really different idea! I'm not personally convinced that it's possible to get a high volume of visible good posts using this idea in an open general-interest community, because no matter what you do, there will always be more people ready to vote for popularized, lurid, and flamebait posts than good posts.
I think that distinction has much more of an effect on content than moderation. (Although if you've decided that you only want good posts, you probably need moderation to enforce your decision.)
We're too late for that here. I don't think PG has enough bandwidth or interest to truly solve the problem. New users will continue to join, adding noise to the signal, unless HN changes course. It's going to become more generic and more biased the longer the site stays open.
I hesitate to suggest more moderation as some posters suggest. I'm already uncomfortable with the murmurs of unfair moderation in the system here.
* Hot new community forms at Site X.
* Site X residents refer to themselves as the New Wave of whatever. Much better than older Site W because of features/members/dynamic/demographics 1, 2 and 3!
* Site X's reputation spreads to former hot new sites T, U, V and W. Site X begins to attract more and more new users.
* Site X denizens begin linking articles at T, U, V, W and vice versa.
* Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.
* Someone reminisces out loud about the Golden Days of Site X.
* Discussions on Site X become more and more about Site X. Extremely intelligent individuals begin to earnestly argue that their proposed feature will save Site X from itself.
* Someone proposes or launches Site Y. A how new community begins to form there ...
I've been watching this same story play itself out since Slashdot circa 1998.
Usually this starts happening about 12 months after I decide to become a regular. In HN's case I've come in halfway through the cycle, about 3 months ago, so apparently the magnitude of my destructive powers is increasing.
[1] and also the snide.
And every social subculture in history. See: the constant evolution of the social scenes around music genres. It's just a fact of social psychology we have to live with.
"Dubstep was sooo good back in 02 before the students found out about it! I mainly go to future-garage raves now man..."
On reflection it resembles flocking behaviour. I imagine that website members can be modelled as simple agents with two rules:
1. A cool site has few, impressive members.
2. Move towards the cool sites.
Given an arbitrary distribution of "impressive" members, this should cause a constantly moving flock of members to move from site to site regardless of its purpose.
3. A member's threshold of minimum community coolness below which they will leave, is correlated to their impressiveness.
Yes, I'm that elitist. And yes, I'm well aware I probably wouldn't get voted in :)
But here's the other part - keep the site open for reading. That means it (hopefully) self-selects for quality, and other people can observe a high-quality discussion.
Then again, Something Awful is $10...
These terms are 100% arbitrary and invented an hour ago, but these seem like good definitions.
It was harder to get rid of them because they knew someone who knew someone, and trying to push them out had a network effect. It might be obvious to someone removed by 3+ levels that a bad element is bad, but the person who invited them might not see it.
I.e. if you invited a bunch of bad apples, you will be pruned too. I have no idea if it would work, but it's pretty obvious that open social graphs beyond a certain threshold are not conducive to high-level discussions.
It avoids unnecessary disruptions. Not all branches will shake if a leaf falls.
Brutal, but brilliant... even if it is not necessary true :)
This is just human nature, it manifests itself in the clothes we wear, the cars/bikes we drive, the places we live, it certainly also applies to social websites. I'd be surprised if it doesn't - we'll see how facebook fares, or if there will be a coolkid exodus as the rest of the world gloms on.
It's basically a (sub)cultural arms race that everyone fights in.
It might be different in places other than California though, like New York.
Most economists believe that a ceiling on rents reduces the quality and quantity of housing available.[41][42] This view is based on analysis of empirical evidence as well as the understanding generated by theoretical models.[42] Economists from differing sides of the political spectrum, such as Paul Krugman[43] and Thomas Sowell,[44] have criticized rent regulation as poor economics which, despite its good intentions, leads to the creation of less housing, raises prices, and increases urban blight.
Now that I think about it further, it seems obvious that rent control would be a tool that can be used by a municipality to control rents in either direction, depending on their goals.
A big part of political-economic maturity (for me at least) realizing that just because the supply/demand lines from your Econ 101 class don't line up, doesn't mean that something is worthless. It could be that you're not measuring everything. Culture matters, preserving social mobility matters.
Just kinda irks me when someone comes in with the Econ 101 argument regarding a policy that's being implemented by people with advanced degrees in city planning. These people took Econ 101, and know more about the subject than you or I. They could still be wrong, of course, but it's not for a silly simplistic reason like that.
http://blog.bumblebeelabs.com/social-software-sundays-2-the-...
If I get bored of a site or simply get nothing more out of it, I just move on.
If I see topics that are boring or simply have no interest to me, I just ignore them.
If I see rude, insulting comments or personal attacks, I just flag them and move on.
I really don't understand why people need to express so much hate.
I don't kid myself that I'm the kind of person that always remains civil in discussions, because I'm passionate about what I believe in. I know I'm hit-or-miss here, and often the way I communicate my point of view dilutes it and comes across as being a dick. I get that, and I've learned a lot about how to communicate with others from HN and other Web sites. If I'm the problem, though, which this submission and other comments seem to want me to believe, then I'm wondering what the solution is. Agree all the time? The burden is on me to improve, not the community to avoid me.
Every time I see a navelgazing submission like this I'm yanked right out of the community and I'm instead thinking about the community instead of the topics that I come to Hacker News to read and discuss.
What people don't seem to understand about community sites such as this one is that if you are discussing the community ad infinitum, you've already lost. A bunch of high-karma users, and one commenter in particular reminding us that "on behalf of the elites, this site sucks now due to you," is exactly the kind of thread that is contributing to the problem.
If the high-karma users here really want a walled, invitation-only garden where a circlejerk of valley gossip takes place and everybody agrees with each other, this is a pretty good way to go about it. It's really disappointing, to be honest.
This implies it's hostility to "old hat discussions" that have already been around the block that people are worried about.
I don't think that's primarily it. The civility levels have slipped a lot... this place used to be as civil and friendly as meeting a bunch of people at a dinner party where everyone admires the host.
Still super-civil compared to the rest of the internet, but the levels have come down a lot. Raw outrage, profanity, and outright insults used to get downvoted to oblivion even if people agreed with the general argument the commentor was making. Rudeness really wasn't tolerated, so new members learned quickly that you had to think for a minute longer before replying.
It's still a great site, but it's not elitism and reminiscing that the old guard is scared of. It's comment quality slipping to rest-of-internet level.
[1] https://secure.wikimedia.org/wikipedia/en/wiki/Realistic_con...
All undergraduates know that they will soon be at least graduates, if not later postgrads. The transition is inevitable.
Any kind of stratified internet community would have to have the eventual prospect of transfer from one strata to the other, as an inevitable consequence of good/bad participation.
As soon as it becomes set in stone, or the transfer becomes a deliberate choice rather than an inevitable consequence of behaviour, I think people will forget about the criteria that originally divided them and slowly regress to hatred of the other.
Upvoting/downvoting, filtering, disemvoweling, moderation, tagging, reputation systems, etc. are just a smattering of the tools we have available and with time it's likely that we will see more and better tools for community management available in the future.
The most important thing is for "us" to define exactly what current and past features of the community we want to preserve and also decide what kind and amount of "chaos" we want to permit to allow the site to evolve and improve. It makes the most sense for PG, YC founders and very high-karma users to be responsible for defining these community characteristics, ideals and values.
I don't have an answer to this off the top of my head, but succinctly describing the problem as you just did oftentimes can be half the battle.
"What system can allow a site to absorb N new users without causing regression to the mean, with the constraint the membership is open?"
I'm pretty sure the answer is: no such system exists.
The frontpage (depending on how you set it up) could look like Reddit's, based on the majority of users, but to logged-in users it'd be more refined to their tastes. Perhaps you could look at the site from others' perspectives.
Defining the model, computing likenesses and finding the appropriate material could be quite resource intensive if you didn't think long and hard about the implementation.
(You'd might also want to avoid up and down arrows, because "like" and "dislike" quickly supplant "good comment" and "bad comment".)
Superficially at least, it seems to be a good way to put everyone into a community whose size relates to the commonness of their preferences. I think this would occur as long as comments/articles beneath a certain threshold just weren't visible to a particular user.
Unfortunately it seems like a lot of calculating to do per hit, but in thinking about it, it doesn't seem to hurt Netflix too badly.
In either case, the function of the voting mechanism will ultimately shift from generating the public front page to generating a custom view for each user. At that point, you could re-purpose the public front page as a selection mechanism to draw in the highest-quality users - perhaps by choosing a cohort of existing users to have public-upvote rights, or even by just applying old-fashioned editorial controls.
* We all (and I'm aware I'm a pretty young account myself) vote on stories and comments to rank their quality
* Newer users vote less in line with the historic ethos because they're less aware of it
* All our votes have equal power (the downvote filter here is a crude effort to suppress that but isn't powerful enough to be fully effective)
* Network effect means a successful site's growth will be closer to exponential than linear
* With exponential growth the new who are unaware of the community standards are quickly able to overpower the existing who are aware
Hence almost inevitable degradation in quality which people here are complaining about and which I can partially see myself.
Two ways to partially address this spring to mind:
* Express vote power as a function of reputation - an upgrade to the downvote system. This has the problem that it provides a direct incentive to align with groupthink and post the easy but commonly held stories and threads so as to attract repuation and so increase one's voting power and.... It also, I think, requires floating point reputation and scoring, and so increases computational load.
* Display voting based on a similarity function - 'users who liked this also liked....'. This permits cohorts of users to work en masse without prior coordination to lower the ranking of content they find less interesting / valuable and so get what they want more easily. This means though that it increases fragmentation and reinforces groupthink again by design, which is clearly not ideal. It's also again significantly more computationally expensive.
TL;DR: Computation can be thrown at partial improvements but they're expensive to operate, open to gaming and likely to reinforce groupthink. I'm not sure it's socially or technically possible to keep the noisy brats out without severely damaging what attracted the Grand Old Users in the first place. Welcome to social groups...
It would be similar to the "invite only" community idea where older users have to actually approve new members, but less restrictive. Perhaps even have upvotes on users instead of just comments. It would be similar to an "invite", but just for allowing them to vote. I'd say also keep the hellbanning that we have here. There will sadly be some false positives, but it seems to work very well for keeping out the worst.
Perhaps upvoting users could affect the reputation of the folks doing the upvoting. If you upvote a bad user into the community, your reputation suffers as well.
Now, this does nothing to solve the problem of groupthink, and in fact somewhat enforces it. However, I think some amount of groupthink is inevitable if you want to create a strong sense of community. Because you're trying to cluster users around a core set of values, it's probable that they'll think similarly in other ways as well.
Edit: I think multidimensional voting helps as well. Ignoring the comments voted "funny" on /. helps to filter out what people liked just because it was amusing. I think "agree/disagree" voting would be useful as a separate thing, because too often that winds up conflicting with upvoting/downvoting for quality.
This isn't an HN problem, it's an Internet problem. pg could probably do some things to ward off dilution, but it won't last forever. Interestingly enough, Something Awful seemed to be pretty damned successful in this regard by doing the unthinkable and charging people to post there. I'd hate to suggest it as a solution, but it has been effective in the past.
I think a paraphrase of this sentence has appeared in HN comments 100's, perhaps 1000's of times.
Sometimes they move offsite instead, usually because the site admins didn't actually like the topic enough to make more forums for them, or because the admins banned all the users because they had formed secret IRC cabals conspiring against them.
The last stage is that the admins change the forum CSS so it's unreadable. I haven't figured out that part yet.
In my opinion, the method doesn't really work because they aren't reading their own posts. It should probably change everyone else's text color instead of their own. Greasemonkey scripts and the like would still fix it, but it would be a lot more annoying to the shitposters that way.
Just joking! I totally agree with you. Both in regards to /. but also with websites in general. In fact, the Internet in general. It started out (for me) idealistic and cool and slowly has got dumber and more mass market as normal people started using it. Ah well. We lost something, we gained something. The cool is gone, but there's a lot more of the typical stuff and the Internet is making some of the real world changes I hoped it would only because it is now so ubiquitous (like the change in the Middle East)
I think all discussion sites struggle to identify quality discussion. The whole Karma idea that /. created was very very cool and has worked reasonably well over the years. Certainly, their discourse is wildly better than digg's, for example, and I would submit that Karma is a large part of that. But the problem with Karma is that it promotes group-think and that's where /. is today. Alhough: you could say just the same about HN. Anyone who doesn't love LISP, Python & PG is automatically on the outer. Hating Java is cool, too.
Perhaps group-think is an inevitable requirement for communities to form? Perhaps without some level of shared unquestioned axioms, groups will not form in the first place - just in the way a Christian and an Atheist can struggle to have a conversation about right and wrong because their starting assumptions are just so far apart? I don't know - I'm not sure if there is an answer to this kind of thing, I think that maybe it is just human nature at work.
1) Every user belongs to at most two (or k) classes.
2) Each class can have a finite number of users n.
3) When a class fills up, a new one is opened.
4) There are always two (or k) open classes at a time, and new users join both of those two (or k) classes.
5) The classes are staggered so they don't close at the same time; e.g. if k=2, n=10000, the first 5000 people to join belong to class 1 and class 2, the next 5000 belong to class 2 and class 3, the next 5000 to class 3 and class 4, and so on.
6) Users can only post to classes they are in, and can only comment on posts in classes they are in (or maybe newer ones too).
7) By default, users only see posts to classes they are in (or maybe their classes + older ones). They can opt in to see more if they want.
The idea might work better if it is combined with some more onerous mechanism to join classes late - a fee, a requirement for an endorsement from a member or someone. Charging a fee for access to the more '1337' earlier classes might be a good way to monetise the site; the fees could either be flat, or could escalate (exponentially?) with the age of the class and / or the number of people who pay to join it late (possibly with a discount applied as active members leave).
Classes might get too small due to attrition - coalescing older classes when they get too small might be a solution to this - it might also make sense to increase class sizes (exponentially?) as the site gets more popular.
Maybe I'll write something to regenerate past front pages, so we can check if things are different now. That should be possible, because news.arc has always logged vote times.
As the infamous "How to Become A Hacker" states "The world is full of fascinating problems waiting to be solved"[1]. There is no mention of those problems having to be "deeply technical".
I certainly see the tremendous variety of topics that make the homepage of HN as an asset in keeping with the true Hacker Mentality.
[1] http://www.catb.org/~esr/faqs/hacker-howto.html#believe1
Bottom line: sure, hackers care about many problems. But that doesn't make up for the poor discussion quality surrounding many non-tech articles here.
I've seen what looks like fantastic advice and discussion on a range of non-technical topics (particularly things like venture capital/financing and laws around starting a company and that kind of thing).
Of course, not being an expert in any of that stuff I'm at a loss to make a judgement as to it's veracity ;)
"As OPs go further from tech/startups..."
I'm definitely no funding savant, either!
2007AUG15 ~ http://www.flickr.com/photos/bootload/1119991071/
2008FEB08 ~ http://www.flickr.com/photos/bootload/2250089864/
2009JAN18 ~ http://www.flickr.com/photos/bootload/3205568708/
2009MAR12 ~ http://www.flickr.com/photos/bootload/3347109733/
2010JUN24 ~ http://www.flickr.com/photos/bootload/4728476663
And pg's appeal for sanity: http://news.ycombinator.com/item?id=512280
I really haven't noticed HN get any worse in the two years I've been around.
That has been a theme on HN longer than I realized.
And sure, we might be like that dance club that's a bit too mainstream. But pretty soon those early cool kids grow up and are also not cool enough to discover the next hot thing. The next HN will come about and it won't be somebody from here who starts it.
It hasn't just been new users. Rather it has been nearly everyone since the beginning. See what the founder said 1103 days ago about downvoting for disagreement:
http://news.ycombinator.com/item?id=117171
I mean, there has to be some explanation like this, because it just seems so clear from regular reading that the quality has gone down. Then again, maybe I'm projecting the decline of comments onto the content.
http://hackerslide.com/data/YYYY-MM-DD-HH.json gets you what you want. For example: http://hackerslide.com/data/2010-10-15-23.json
Do you provide this as an "api" and allow people to build apps on top of it? If so, are there any limits?
If anyone grabs the same file over and over or anything crazy like that, they'll be blocked, but if you just poach the data and store your own copies, go for it :-)
Perhaps the submitter sees this comment and shares the link here again?
The other comes from the old-timers themselves. I'm not particularly that old-timer, but in the 3something years I've been here, I can say I have experienced this:
I've grown.
The "5 awesome vim tips" or "super deep closure stuff" articles that today annoy me as fluff were deep and new to me back then. HN was a fantastic site introducing me to amazing things, and it is introducing a lot of people to the same amazing things now. As such, the quest for truly new content needs to go deeper, but the people who need the lighter content are greater in number and enthusiasm, ensuring the fluff rises.
When the older crowd doesn't notice their growth, or the lack of change in content (or the combo) you start to get "it's gone down-hill" type comments.
edit: I mean the quest for truly new content for me needs to go deeper, not that the site necessarily needs to go that way.
Current estimates put the average age on the internet at about 37.
I think this is a very plausible explanation, we are all at different points in the learning process and can only enjoy certain material. All but the most trivial programming articles are above my head right now, and so while I might enjoy them a lot; the HN community would say they are fluff pieces.
Calvin and Hobbes, 16 JULY 1992
" One of the joys of being a kid is that experiences are new and therefore more intense. For example, I'm about to stick my nose in a jar of mustard and inhale deeply! Let's see what it's like. WHOOP!!
See, whed you are oder, you dake your sinuses fo granded.
Some of us prefer to.
http://waybackmachine.org/20070221033032*/http://news.ycombi...
The first one saved was on Feb 22, 2007, just after the site launched:
http://replay.waybackmachine.org/20070221033032/http://news....
Notice that the second link is to pg's announcement post.
Second cached page, Feb 26, 2007:
http://replay.waybackmachine.org/20070226001637/http://news....
And so forth:
http://replay.waybackmachine.org/20070405032412/http://news....
http://replay.waybackmachine.org/20070406195336/http://news....
Bounce on the ▸ icon to page from day to day.
This one has a gem on it from paul:
http://replay.waybackmachine.org/20071001102019/http://news....
The archive is not complete, but it does give a reasonable glimpse into the recent past of HN.
[Edit: I think I'll go crazy before getting the line breaks to look correct.]
That's basically boredom--and it can happen even if you consume something good for a long time. That "good thing" doesn't change so much as your perception of it.
Also, I'm taking suggestions for seed users. There will also be a HN Karma cutoff where everyone above a threshold can join. You can nominate HN users or yourself here.
I'm not sure what Alex wants? More discussion around PG's hackers and painters?
I had a comment which I guess I should link to rather than repost: http://hackerne.ws/item?id=1934605
Instead, the best solution is to evolve Hacker News as a product.
My personal opinion is that we should put Hacker news in the hands of the YCombinator alumni. Founders and first employees (CEO, CTO, lead designer, first engineer hire and first design hire) of YC startups would probably make the best moderators and admins.
In fact, I would say that it's probably time that PG spin off YC as a full-time startup, assigning control of the design and codebase to one talented UI designer, one talented developer and one talented product manager.
For the site to keep growing in a way that maintains quality, it needs more functionality that it has. The two features that lack the most are filtering and combinatorial game mechanics.
Filtering is necessary so it is easy for the the hardcore tech articles to be easily found by high-karma members, so they can vote those articles up. If it's not findable, it's not voteable. Filtering is also necessary for people to extract the most value out of hacker news. Most users don't want 100 front-page articles everyday. They probably want 10-20 of the highest value articles. Less is more.
Combinatorial game mechanics like those on StackOverflow would help as well. Upvoting/downvoting is limited in that it will always fall victim to the masses. Giving special voting/tagging/burying rights to distinguished members (very high-karma users and YC founders and employees) would go a long way to helping eliminate the crap.
I think I speak for most members here, when I say that I don't want Hacker News to be a democracy. I want it to be a technocracy. I want the smart and accomplished people to control what is good and should be visible to all. I've got only 260 karma points, and personally I don't think that should be enough karma points to allow me to upvote a submission. 500+ karma points should be the threshold to be able to vote an article to the frontpage.
Say your valued members have a certain mindset, ethos and style. Let's call it YC-meme. Everyone with YC-meme contributes in a manner that exemplifies YC-meme. If someone comes along and doesn't have that meme, their posts tend to not be what the memesters want. So, members use their voting powers to reduce that person's karma. Eventually they either leave, get ignored by all, or become infected with YC-meme.
Now, open up the community to everyone. At first, YC-meme is still dominant. Only 1% of people without it are able to write posts in a style similar enough to gather karma. Let's say they have RD-meme, which has about 80% overlap with YC-meme. Great.
The problem is, soon, there isn't 1% of the community with RD-meme, there's 10%. Who can (and do) re-enforce RD-meme posts, because it's what they want. People can even get lucky and simply contribute 'good' posts, long enough that they get a decent karma score, and so can re-enforce different memes.
What makes it WORSE is that what's acceptible is what's visible, so with more RD-meme posts, RD-meme posts become more likely, as the probability that they get 'punished' decreases.
End result: Community-dilution runaway.
There are a few 'cures' I can think of, but most are fairly prescriptive:
1. A concrete set of rules and behaviors that anyone can reference to moderate from. Gross. 2. A very high bar for moderation, with active participation. Given that the people most likely to have YC-meme are also likely to be busy, this might be doomed by design. 3. "Proof" required for contribution. Great way to kill off your community. 4. Invitations. Either so easy to get as to be meaningless, or so hard as to stagnate the community. 5. Close ranks to just people at/gone from YC. Sadly, that'd mean I was out. May still be the best alternative.
Maybe the community is self-aware enough to start consciously moderating better. Perhaps a post like this every so often might serve as a gentle reminder. That'd be the best outcome, but needs to be chosen by a majority of the user base. Do YOU want a stricter HN?
The reason this is doomed is: there is currently much more stuff needing moderation than the selected few can handle. They would soon wear out and you'll end up with a site with almost no moderation at all.
The trash needs to be stopped at the front door. When it gets into the house, it takes too much effort to clean up.
I strongly disagree. Off the top of my head: XFree86/Xorg, Mozilla/Firefox, Debian/Ubuntu, Rails/Merb... these were all cases where the existing community had calcified in some key area and a fork was necessary to create a space where important work could get done.
And in most of those cases, not only was the fork stronger than the trunk in (some) specific areas, the fork actually led to the trunk getting stronger. Not only were patches merged, but entire projects, community values, development practices, etc were.
XFree86 is an exception, but organizational paralysis was what necessitated the fork, so it's not surprising they were also unable to respond to the fork.
But I just don't buy your notion that forking is somehow unhealthy and destructive. It's one of the most valuable ways we have to keep open source code, ideas, organizations and practices fresh, healthy, and vibrant.
For me personally, I've learned a lot and grown a lot over the course of the 4 years I've been lurking and occasionally contributing here. So for me, a smaller percentage of the stories/articles/posts/discussions appear as insightful as they once did. I don't mean to knock HN in any way, in fact my point is that that fact is not a "problem" to me. New users are joining everyday and everybody who makes the effort to learn and contribute gets something out of HN.
It's what brings me back 17 times a day.
Now, HN isn't trying to grow, so there's no need to have user-created subreddits (sections, I suppose). Just make 8 or so that people care about, and add another if there's sufficient demand.
I really shouldn't be crediting reddit with this, as the solution existed long before them. All HN needs to do is follow the forum model and have different sections. It's too big to only have the front page.
Call it undemocratic, but insight and perspicacity is not uniformly distributed so it's absurd that pg/$whoever_you_respect's upvote on an article counts as much as anyone else.
As a simple experiment, it would be interesting to see a view of the frontpage based only on the upvotes of people who are above a certain avg-comment karma threshold (since the site is predicated on karma as a quality indicator) and the idea that people who write insightful comments won't upvote crap stories.
Stackoverflow operates under this principle, more or less, but it doesn't seem to entirely stop the slow decline.
But then people may have not like having their up-votes be so transparent..
I think something like PageRank can really help. The same need is to rank things based on a directed acyclic graph. The only difference is we have multiple kinds of vertexes: users, submissions, comments.
I am not so optimistic about submissions because the data we currently have is tainted by submissions we don't want. However by using the technique you have described, we could probably achieve better results.
It would help take care of the effect where older uses leave due to changing interests etc.
Note: I don't actually think it's a good idea. I think just figuring out _what_ you want exactly, in vaguely quantifiable role, is sort of a hard problem. I suspect a lot of people have different 'golden ages' in mind when they think of HN.
When I came here originally it was pretty startup heavy, which was sort of remote from what I was up to at the time. I liked it when it got a little away from startup business related stuff (my perception) and toward more generic technical issues. My point is even if everyone agreed that ait has gone done hill (an open question), not everyone might agree on what it should be like. It's not a democracy of course, but even as an individual for my own preferences that seems as hard as the technology issues.
Metafilter did this, right? For a couple years they said "No new accounts."
I think scaling a social site to a very large number of members without deteriorating badly is impossible. It's a matter of human nature and mobs.