A suitably joking response for a "study" that didn't need to be taken too seriously (and wasn't taken very seriously by most media outlets, including the one that led a story with it). As we discussed yesterday here on Hacker News[1], the study methodology was not sufficiently validated to convince most people that Facebook will massively lose users, even if all the data were correct. More likely, the study's model was just flat wrong.
The other big news was that Facebook was testing ads outside of their own platform. The possibility of taking on Google's Adwords and adding more than a few billion dollars in revenue should have increased the price?
You're layering a lot of your believe about Facebook onto that assumption - maybe people saw that news and concluded Facebook would fail in external ads and lose a bunch of money, which decreased the stock price. It's hardly unambiguously good news (like record profits or something).
One of the top comments on that page is that they loved the tone of article. Personally I hate sarcastic responses (no matter how correct you are), which sound very unprofessional. I hope to see less of these types of responses in the future.
Why do you expect them to be professional? Obviously, the response was in jest and wasn't some formal response by FB's PR group. This was just done by a couple FB workers who laughed at Princeton's methodology. To expect a certain level of professionalism for someone's FB post seems a bit absurd and pedantic.
Hmmmm ... I think I believe both articles! Higher education and social media are both in decline. Who needs critical thinking and the ability to gossip 24/7? I'm going to go rewatch Idiocracy (http://www.imdb.com/title/tt0387808/).
Finally...a rational statement about what is really going on. I do think Facebook will survive, as infrastructure for the internet, but the history of all social networks says they have a time limit before they are replaced by something cooler. Right now, there is nothing better, so no one is jumping ship en masse.
For anyone that hasn't seen it, Idiocracy goes after the present by exaggerating the future.
It's a sad but true trend that people are too busy playing World of Warcraft to understand much less resist what is going on around them: automating of warfare[0], climate change[1], police militarization[2], sources of terrorism (including JSOC ("American Taliban") killing American citizens without a trial in countries without a declared war)[3] and peak oil [4]. That's a lot of heavy shit that that isn't being addressed, which defers consequences to be much more painful sooner or later. With commodities, often the consequences are spread out gradually (epic rise in cost of gasoline) whereas others are not so gradual.
That seems far more important than whether some pre-press study is important or not.
I hope this becomes a bigger story than the original. I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled. A survey with a small sample size, outsized extrapolations and numbers that don't match the accompanying conjecture.
There should be consequences for the people who publish these things. People have a tendency to believe anything that someone in a lab coat says, especially if it supports their point of view or anecdotal experience. In many cases the people who do the research present it with few qualifications while not standing behind assumed implications. If someone publishes sensational and link baity findings they should say, unequivocally, "I'm willing to stake my reputation on the idea this trend is real and will continue" or "These are just data and I'm not willing to say that they have any bearing on reality".
Facebook may not have been right to dignify the initial post with a response, but I hope it works for the best. They say that some attention is better than no attention at all. It's important that this applies to self promotion and persona creation and not science. If somebody has something crazy to say, they should start a personal blog. Those who want to intentionally attract media attention should present themselves as such, instead of pretending to be doing any kind of meaningful experiment and hypothesis testing.
Nostalgia for anonymity. And also, sometimes you make a throwaway and then need to respond to a response and then you respond to something else while logged in and then you have 50 karma points and well, why not?
well not necessarily die by 2018, just the fact that social media is mostly moving fads. everyone wants to be the social network that high school and college girls post their vacation photos to, if you are that then you will likely be somewhat relevant for the lifetime of these girls.
Or maybe it's the opposite: if a teenager has an account with lots of pictures, he/she will probably want to kill that account as soon as they get to adult life to avoid awkward moments. I think this explains why many young kids are leaving FB.
The bigger issue is kids don't want to use the uncool social networks their parents use. Secondarily, that might come into play later as an afterthought.
what the heck is "real" science? A lot of the greatest science started the same exact way. Making connections that nobody else could see. Finding purposefully playful, surprising analogies. Doing things so ludicrous that everyone said would not work. Exploring ideas and sharing them.
The worst part for me is that the two authors are Ph.D. candidates in MAE (Mechanical and Aerospace Engineering) [1]. They aren't even tenure track faculty at the school and aren't really in a position to carry the name of the school on their shoulders. Yet from what it seems, it will be the school itself that will be hit with most of the public shaming burden (ex: the FB post's headline).
As a Princeton alum, I'm a bit saddened at the course of events :(
4500 undergrads, 1500 graduate students, and hundreds of faculty, and you think that every little paper they give or study they conduct is a product of the whole in a meaningful sense?
The study was practically just a thought experiment by two grad students looking at something outside of their expertise. It _only_ got any attention, because it was about Facebook, and so the media saw a chance to sell more ads. For the same reason, they used the metaphor of pars pro toto as a rhetorical ploy, because Princeton vs. Facebook sells more ads. Like many among the unthinking masses, you got punked by it.
The students also published it online before it was peer-reviewed. I think the grandparent is more referring to the typical corporate cop-out of "the actions of a few employees", even though the damage is done regardless of who did it--and the employees (or students) did it under the name of the organization.
And it is standard for authors to list their university affiliations on papers whatever the quality of the paper regardless of whether it is a pre-print or not. Look at every other paper on arXiv, they all have affiliations, and there is a ton of crap on arXiv that will never pass pre-review.
These students didn't do anything wrong in putting the paper online here. Now if they did a PR to the media to promote their work and mis-portrayed it as peer reviewed or authoritative, that is a different matter. But putting a pre-print on arVix with proper affiliations is just standard procedure these days.
They submitted their paper to a well known, well regarded preprint service (arXiv), which is pretty normal for a number of fields.
Then, presumably, the paper would go to peer review. Where one would hope the reviewers would point out a few glaring flaws (like that an SIR model without either the creation of new susceptibles or waning immunity must go to 0 and they are 'dooming themselves to success').
The paper would get rejected, or heavily revised, and the world would keep spinning.
They didn't actually do anything wrong. If you want to blame someone, blame an overexcited media that grabbed something off a preprint server and ran with it, knowing full well arXiv has no 'Is this right' quality checks, and isn't peer reviewed.
> The paper would get rejected, or heavily revised, and the world would keep spinning.
Which means they did something wrong. Sure it might have been just a mistake, not quite the cardinal sin the GP makes it out to be, but you don't have to publish papers with flawed arguments on ArXiv.
It's one thing to write up a silly LaTeX PDF with wild speculations, put it on your webpage, but you don't need to submit every brainfart to ArXiv either, or do you?
Beyond that, the paper isn't pants-on-head crazy. The objections most people have to it can be solved by adding a single term to one of the equations, and some very straightforward sensitivity analysis.
Which is, you know, exactly the kind of thing pointed out in peer review.
The bottom line is any paper made public will be examined and possibly turned into a story, so evidence and conclusions better be up to snuff or risk damaging one's and the institution's reputations.
The problem is with the media and the way things were spun. Correcting that should not involve stifling the workflows scientists use. The GitHub analogy is a good one as what you're suggesting is the equivalent of only pushing perfect code.
I am a current Princeton Ph.D. student in CS. Let me say that if one sees an article saying "Princeton study finds..." they are more likely to pay attention to it. Thus, unfortunately, we must also bare the brunt of bad press like this.
On the OTHER hand, this was a non-peer reviewed thought experiment by some MAE students outside their area. It would be akin to me turning in an assignment for a class and the media ran with it saying "Princeton study..."
Unfortunately, readers are going to read this rebuttal as an attack on Princeton, which it really isn't; if anything it's only funny because of how strong Princeton really is.
> There should be consequences for the people who publish these things.
The question you pose here isn't whether or not something is "good science", it's about who gets to decide what constitutes "good science" and "bad science".
Science has been able to handle itself quite well over the centuries since it has come into existence. Publishing something, in your name, as these authors have done, is an implicit declaration of what you phrased as "I'm willing to stake my reputation on the idea this trend is real and will continue".
The spirit of free inquiry, on which science is based, demands that we hold nothing sacred and immune from questions. Good science, almost by definition, requires "bad science".
Be careful what you wish for, because you just might get it.
> There should be consequences for the people who publish these things.
Should there be consequences for people who publish buggy code on github? Because that situation is completely equivalent to what these phd students did. ArXiv isn't a repository of peer-reviewed accepted papers, it's just a repository of papers, just like how github is repository of code.
The people to blame here aren't the students themselves. They just wrote up a mildly interesting observation and put it up on a repository. The people to blame are journalists and internet dumbasses who used this paper to serve their own selfish ends.
> I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled.
There's no denying the existence of a lot of peer-reviewed junk science but sometimes there are perfectly good reasons why papers with faulty conclusions are accepted. The question addressed by the paper might be interesting, the experimental methodology might be interesting, the data sources used in the research might be interesting, or the editors/program committee might just want to highlight the lack of progress in an area that deserves more attention.
Publishing something that is completely fault-free isn't a desirable or even an achievable goal. This is a fundamental misunderstanding of the purpose of scientific publishing. The goal of scientific publication is to advance the field by stimulating discussion at the boundaries of human knowledge. This can often be done even with "buggy" results. Think of it as the scientific equivalent of fail often, fail early and fail fast mantra that drives many startups.
I'm not dismissing your frustration though because the problem you've described is a real one. We aren't very good at communicating scientific results to laypersons. But unlike you, I don't have a glib solution to a problem I don't fully understand. Magazines like the communications of the ACM with accessible, professionally-edited articles written by experts in the field are probably one step in the right direction, but there is a lot more that needs to be done here.
What matters is the effect. If someone puts up code on GitHub which purports to compute the Fibonacci series, but mistakenly does a `rm -r ~/*` then surely there should be a way to flag that code or author ("consequences").
If you find a flaw in a published paper, the normal course of action is to e-mail the authors letting them know what you think is wrong. If authors think this is a devastating flaw, they will retract their paper. Even if they don't, you can still publish a note yourself explaining why these results are flawed. This happens all the time and the 'consequences' you desire are already built into the system. Flawed papers don't do very well on citation or influence metrics.
But much more importantly, the flaw in your mental model is thinking that claims in published papers are expected to be correct. They aren't. The goal of a paper is to advance the field by stimulating discussion and new research. The unfortunate consequence of this is that laypersons can't directly use research papers as an authoritative source describing our current understanding of a subject.
The github analogy still holds up. Non-programmers shouldn't be downloading code off github, compiling and running it because they have no way of evaluating whether the code really does what it claims to do. They're probably better off getting applications from curated app stores. Accessible scientific publications written by recognized experts in the field are the equivalent of curated app stores. That is where laypersons should be looking to understand the state of the art in scientific knowledge.
This is just kind of ridiculously fallacious. Surely a slightly inaccurate paper is not comparable to malicious code. That's not a fair comparison to make.
What matters is the effect, but the person responsible for the effect should be the person deploying the code. Not all code on github is close to "done". If you take my alpha Fibonacci-sequence code and choose to run it on your nuclear power plant, I'm responsible for the mess - you are.
> There should be consequences for the people who publish these things.
Lab coats are there to bounde ideas off each other and sometimes write stupid things that will get corrected by other lab coats.
Consequences should be there for the people who find such articles and take them for a media spin. It's not the paper that is doing the damage, it's the news articles, like the Guardian story. It is the responsibility of a journalist to make sure they're not posting utter bullcrap to millions, and when they fail to do that, there, here should come consequences.
Could you provide an example? I haven't seen a single case of the thing I was meaning by my comment, so either we misunderstood each other, or I haven't checked hard enough.
What drove Cannarella and Spechler, two mechanical engineering Ph.D. students, to prerelease a paper about network theory and epidemiology, without coauthoring/consulting with epidemiologists or people experienced in viral communications theories, is beyond me. But the title "Debunking Princeton" seems to suggest this should be considered representative of all of the quality of research Princeton outputs, which is certainly not the case. Many posts have been made questioning the strength of the paper in question, and the fact that one with such a link-bait title is rising on HN is unfortunate in my view.
Once again, a reminder to everyone that there are no peer review requirements for papers posted to arXiv. There is no evidence that the original paper was ever accepted by any journal or conference, and not surprisingly given the speculative nature of the study, the advisor of the two Ph.D.-candidate coauthors declined to place his name on the paper. So as an institution, Princeton is no more responsible for this paper than Obama is responsible for that drink machine being broken down the hallway (thanks Obama), even though it happens to be a drink machine affiliated with his country. In that regard, with all due respect towards Mr. Develin, I'm going to have to "debunk" "Debunking Princeton."
Is this a clever meta-troll? Or are you actually taking the title seriously?
It should be obvious that the title itself is part of the self-parody. The whole thing is a giant exercise in reductio ad absurdum. In fact, the 'joke' only works based on Princeton's reputation as an excellent university, if that makes you feel any better.
2 princeton students write a paper on Facebook, declining
1 facebook employee write a post on Princeton, declining
Perfect match... Dude, it's not about the scientific contribution of the paper... Do I need to emphasize this obvious thing to a Princeton alumni?... If I do, I do worry that Facebook post is speaking the truth...
I don't think that an expert is viral communications theory would have done any better.
Almost everything involving networks is a fad. Networks are an interesting mathematical structure, but on their own, they rarely enable one to build a compelling statistical model. E.g. people join Facebook for very complex social reasons, that go beyond local interactions between individuals. The image of Facebook has changed significantly since it started, and new products have entered the market, like snapchat and kik.
The original article isn't very long. If you want to discuss it, you could, I dunno, try reading it? Yes, it is from google trends. Just like the data on yahoo and google itself that they compared it to. So no, there is no rational basis to assume that the data indicates people are getting more familiar with the internet.
Sassy, but I feel like Princeton was deserving of a response like this. It was a pretty poor study that had a lot of lazy data and speculation behind it.
The exact values for the axes are proprietary to Facebook. (in fact, in ggplot2 which he used for the first 2 graphs, you explicitly have to tell it to not show the units on the axes, and it's very unintuitive)
So, if this all snark, does that mean there's no "strong correlation between the undergraduate enrollment of an institution and its Google Trends index"? Because that would be a pretty interesting correlation.
I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps.
>I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean?
Google trends represents share of search volume, not absolute volume [1]. As the higher education bubble continues to grow, ivy league schools have not grown as fast as other schools, and there are new schools being created as well. All of this activity contributes to Princeton's (for example) share of search decrease within their sector, and overall.
It is really easy to find something that correlates with your data, when your data follows a simple trend. Now if princeton enrollment was going up and down a lot, and the google trends followed the same shape, that would be something.
md224 wrote: "I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps."
I have seen this in most nerdy topics associated with early internet adoption in Western English speaking countries -- or at least that is my theory. Basically as other countries come online and late adopters come online, they do not search for the same types of topics earlier adopters looked for.
This trend for academic or technical topics decreasing as a percentage of Google trends is universal, see here:
The article jokes about "the scientific principle 'correlation equals causation,'" but I feel like I see people make this mistake all the time, particularly with statistical concepts like expected value, and epidemiological risk factors.
Statistics is descriptive--it's not predictive. It tells you about the data you have. It doesn't tell you why the systems produced that data, and whether they might produce very different data under different conditions.
Personally I didn't even check what the FB link was since it was clearly a link to a Facebook album. For anyone interested in data munching, a graphic is worthless, you need the numbers. And more data, always
Not sure what you expect them to publish beyond MAUs and DAUs by region. Those are pretty standard ways to measure engagement and if I were Facebook I'd be loathe to publish more granular data just to "quiet the critics".
Seems like Facebook is responding to the media interpretation of the Princeton paper rather than the paper itself. I saw no problems with the paper: it applied an epidemiologically-inspired statistical technique to Facebook using a defunct precursor as exemplar. The limitations of this technique are obvious, but it's an interesting idea.
People do "research" like this all the time - you throw shit at the wall and see what sticks. Most of the stuff that ends up on the floor never receives any attention at all, so you don't hear about it on first-tier news sites. But when it's about Facebook, it goes viral, and suddenly is the subject of intense scrutiny. They didn't bring this to the UN for a call to action. They didn't start a company around it. They just applied an idea to some data and wrote it up. And now the entire Internet is making fun of some exaggerated version of their idea, summarized by Huffington Post hit-mongers.
Personally, I applaud these guys for putting in the work to test out a theory. If it's not correct, it will go in the bin with the other ten million papers with flawed theories, premises, methods, or other aspects that have been published in the last day or two.
Well, sure. They have a right to explore it, just as Facebook has a right to make fun of it. Maybe it's not the original authors' fault it blew up, but that doesn't mean Facebook has to ignore the consequences.
The problem is less about the technique, but about the flawed assumptions about the data that they used, and how they assumed that it was a valid basis for a paper. That undermines the whole thing, no matter the rigor of the analysis.
The medium matters, too. I'm all for people discussing crazy ideas and seeing what sticks, but when they wrote a paper about it and submitted it to arXiv.org ("Submissions to arXiv should conform to Cornell University academic standards."), they staked their reputation. Why wasn't this just a blog post?
I think it's important to add that "flawed" is not the same as "invalid." If their assumptions are incorrect, the conclusions of the paper are wrong. You have to start with some assumptions, otherwise your theory wouldn't be a theory, but rather data. Science is advanced by such assumptions and theories, even if the results are "wrong." It all goes on the pile.
As for why it's a paper and not a blog post - I think the conventions of academia are there to be questioned too. Why not form it into an abstract-methods-discussion format if that's what they think best suits it? Why does it have to be made into a more accessible and friendly blog format? Seems to me that open access will reduce the "scariness" of having a formal paper out there, as well as reduce the worth of same. Broadening access has that effect, and it looks like these guys are early adopters of a new, less formal process for describing the evaluation of theories like this one.
I guess what I'm asking is why this paper gets singled out for special derision when there are plenty of papers out there just as flawed (though no less "valid"). I think it's just a run-of-the-mill bit of science that, like many others, will simply be disproven in a positive way by experts. It seems ugly to submit it and its writers to such public pillorying.
I think that discussing this in academia is worthy. One paper can start a discussion, another paper can improve on it or disprove it and so forth. I think that putting out a paper with a controversial conclusion is not wrong, it is merely provocative. I hope there is more study on this and that it is serious.
I also do not think they are completely wrong. I think there is general consensus that in North America we have likely reached peak Facebook engagement -- unless there is a game changer. The question is fairly open at this point in time as to whether it stays steady, decreases slowly or fast from this point.
writing can help you organize your thoughts and that can lead to new epiphanies. Writing helps you explore, helps you think, and helps you find ideas. Some people really hate that though. They don't like non-linear thinkers who use analogies and metaphors. They like to start with the proof. Other people enjoy surprising, sometimes ludicrous connections and analogies.
Some people also don't want facebook to fail. If you go deep into their comment history, many of them have argued that facebook cannot be the next myspace.
It's worth exploring and discussing, and if academia wants to get involved, their contributions are welcome. I just think that they shouldn't be surprised when the lede is about "Princeton researchers" by mass media.
> I think there is general consensus that in North America we have likely reached peak Facebook engagement
I disagree. There are signs that US teens are decreasing engagement or maybe not even registering: http://istrategylabs.com/2014/01/3-million-teens-leave-faceb.... However, other age groups are apparently doing well, and there's no sign that 80% of users will leave by 2017.
On the whole, they continue to grow. The question is whether the teens are an early indicator that will apply to all demos/geos and that they will all decline, or if there's maturation in the product and that it better fits an older demo and the teens will grow into FB in later years. Or maybe something else.
I see this as Facebook's willingness to use its unfair influence to discredit anyone who dares challenge it. In other words, "if you have a page here, we won't hesitate to use it against you".
Facebook takes a lot of flack: it's today's standard whipping boy. In this case, it did respond to a fundamentally flawed research paper, but I doubt it would have bothered if that paper hadn't been so widely and so shoddily reported.
I though it was quite generous of Facebook to resort to gentle mockery rather than shred its numerous stupidities.
149 comments
[ 4.0 ms ] story [ 188 ms ] thread[1] https://news.ycombinator.com/item?id=7104904
It's a sad but true trend that people are too busy playing World of Warcraft to understand much less resist what is going on around them: automating of warfare[0], climate change[1], police militarization[2], sources of terrorism (including JSOC ("American Taliban") killing American citizens without a trial in countries without a declared war)[3] and peak oil [4]. That's a lot of heavy shit that that isn't being addressed, which defers consequences to be much more painful sooner or later. With commodities, often the consequences are spread out gradually (epic rise in cost of gasoline) whereas others are not so gradual.
That seems far more important than whether some pre-press study is important or not.
References:
[0] http://www.ted.com/talks/daniel_suarez_the_kill_decision_sho...
[1] California is having the worst drought in five decades. North Texas is having the worst drought in over a century. http://droughtmonitor.unl.edu/
[2] https://www.aclu.org/militarization
[3] http://dirtywars.org/
[4] http://www.theguardian.com/environment/peak-oil
You mean "not all linkbait is created equal?"
There should be consequences for the people who publish these things. People have a tendency to believe anything that someone in a lab coat says, especially if it supports their point of view or anecdotal experience. In many cases the people who do the research present it with few qualifications while not standing behind assumed implications. If someone publishes sensational and link baity findings they should say, unequivocally, "I'm willing to stake my reputation on the idea this trend is real and will continue" or "These are just data and I'm not willing to say that they have any bearing on reality".
Facebook may not have been right to dignify the initial post with a response, but I hope it works for the best. They say that some attention is better than no attention at all. It's important that this applies to self promotion and persona creation and not science. If somebody has something crazy to say, they should start a personal blog. Those who want to intentionally attract media attention should present themselves as such, instead of pretending to be doing any kind of meaningful experiment and hypothesis testing.
Many who are published deserve obscurity. And some who have obscurity deserve more funding. Can you give it to them?
Do not be so eager to deal out censorship; for even institutional review boards cannot see all ends.
As a Princeton alum, I'm a bit saddened at the course of events :(
[1] http://www.princeton.edu/~spikelab/people.html
4500 undergrads, 1500 graduate students, and hundreds of faculty, and you think that every little paper they give or study they conduct is a product of the whole in a meaningful sense?
The study was practically just a thought experiment by two grad students looking at something outside of their expertise. It _only_ got any attention, because it was about Facebook, and so the media saw a chance to sell more ads. For the same reason, they used the metaphor of pars pro toto as a rhetorical ploy, because Princeton vs. Facebook sells more ads. Like many among the unthinking masses, you got punked by it.
http://en.wikipedia.org/wiki/ArXiv
And it is standard for authors to list their university affiliations on papers whatever the quality of the paper regardless of whether it is a pre-print or not. Look at every other paper on arXiv, they all have affiliations, and there is a ton of crap on arXiv that will never pass pre-review.
These students didn't do anything wrong in putting the paper online here. Now if they did a PR to the media to promote their work and mis-portrayed it as peer reviewed or authoritative, that is a different matter. But putting a pre-print on arVix with proper affiliations is just standard procedure these days.
They submitted their paper to a well known, well regarded preprint service (arXiv), which is pretty normal for a number of fields.
Then, presumably, the paper would go to peer review. Where one would hope the reviewers would point out a few glaring flaws (like that an SIR model without either the creation of new susceptibles or waning immunity must go to 0 and they are 'dooming themselves to success').
The paper would get rejected, or heavily revised, and the world would keep spinning.
They didn't actually do anything wrong. If you want to blame someone, blame an overexcited media that grabbed something off a preprint server and ran with it, knowing full well arXiv has no 'Is this right' quality checks, and isn't peer reviewed.
> The paper would get rejected, or heavily revised, and the world would keep spinning.
Which means they did something wrong. Sure it might have been just a mistake, not quite the cardinal sin the GP makes it out to be, but you don't have to publish papers with flawed arguments on ArXiv.
It's one thing to write up a silly LaTeX PDF with wild speculations, put it on your webpage, but you don't need to submit every brainfart to ArXiv either, or do you?
Beyond that, the paper isn't pants-on-head crazy. The objections most people have to it can be solved by adding a single term to one of the equations, and some very straightforward sensitivity analysis.
Which is, you know, exactly the kind of thing pointed out in peer review.
On the OTHER hand, this was a non-peer reviewed thought experiment by some MAE students outside their area. It would be akin to me turning in an assignment for a class and the media ran with it saying "Princeton study..."
Unfortunately, readers are going to read this rebuttal as an attack on Princeton, which it really isn't; if anything it's only funny because of how strong Princeton really is.
The question you pose here isn't whether or not something is "good science", it's about who gets to decide what constitutes "good science" and "bad science".
Science has been able to handle itself quite well over the centuries since it has come into existence. Publishing something, in your name, as these authors have done, is an implicit declaration of what you phrased as "I'm willing to stake my reputation on the idea this trend is real and will continue".
The spirit of free inquiry, on which science is based, demands that we hold nothing sacred and immune from questions. Good science, almost by definition, requires "bad science".
Be careful what you wish for, because you just might get it.
If you want to blame someone, blame an uncritical media.
Should there be consequences for people who publish buggy code on github? Because that situation is completely equivalent to what these phd students did. ArXiv isn't a repository of peer-reviewed accepted papers, it's just a repository of papers, just like how github is repository of code.
The people to blame here aren't the students themselves. They just wrote up a mildly interesting observation and put it up on a repository. The people to blame are journalists and internet dumbasses who used this paper to serve their own selfish ends.
> I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled.
There's no denying the existence of a lot of peer-reviewed junk science but sometimes there are perfectly good reasons why papers with faulty conclusions are accepted. The question addressed by the paper might be interesting, the experimental methodology might be interesting, the data sources used in the research might be interesting, or the editors/program committee might just want to highlight the lack of progress in an area that deserves more attention.
Publishing something that is completely fault-free isn't a desirable or even an achievable goal. This is a fundamental misunderstanding of the purpose of scientific publishing. The goal of scientific publication is to advance the field by stimulating discussion at the boundaries of human knowledge. This can often be done even with "buggy" results. Think of it as the scientific equivalent of fail often, fail early and fail fast mantra that drives many startups.
I'm not dismissing your frustration though because the problem you've described is a real one. We aren't very good at communicating scientific results to laypersons. But unlike you, I don't have a glib solution to a problem I don't fully understand. Magazines like the communications of the ACM with accessible, professionally-edited articles written by experts in the field are probably one step in the right direction, but there is a lot more that needs to be done here.
But much more importantly, the flaw in your mental model is thinking that claims in published papers are expected to be correct. They aren't. The goal of a paper is to advance the field by stimulating discussion and new research. The unfortunate consequence of this is that laypersons can't directly use research papers as an authoritative source describing our current understanding of a subject.
The github analogy still holds up. Non-programmers shouldn't be downloading code off github, compiling and running it because they have no way of evaluating whether the code really does what it claims to do. They're probably better off getting applications from curated app stores. Accessible scientific publications written by recognized experts in the field are the equivalent of curated app stores. That is where laypersons should be looking to understand the state of the art in scientific knowledge.
If we were to continue the GitHub analogy, it would be great if ArXiv had a public "issue" list.
In HN-speak, Arxiv is like a Svtle blog.
Lab coats are there to bounde ideas off each other and sometimes write stupid things that will get corrected by other lab coats.
Consequences should be there for the people who find such articles and take them for a media spin. It's not the paper that is doing the damage, it's the news articles, like the Guardian story. It is the responsibility of a journalist to make sure they're not posting utter bullcrap to millions, and when they fail to do that, there, here should come consequences.
So what's been thrown in your face, the published paper or the out-of-breath news article with the click-bait headline?
Studies are only ever useful in aggregate, anyway. It's about metaanalysis, not just what any singular study says.
Once again, a reminder to everyone that there are no peer review requirements for papers posted to arXiv. There is no evidence that the original paper was ever accepted by any journal or conference, and not surprisingly given the speculative nature of the study, the advisor of the two Ph.D.-candidate coauthors declined to place his name on the paper. So as an institution, Princeton is no more responsible for this paper than Obama is responsible for that drink machine being broken down the hallway (thanks Obama), even though it happens to be a drink machine affiliated with his country. In that regard, with all due respect towards Mr. Develin, I'm going to have to "debunk" "Debunking Princeton."
[Full disclosure: I am a Princeton alumnus.]
It should be obvious that the title itself is part of the self-parody. The whole thing is a giant exercise in reductio ad absurdum. In fact, the 'joke' only works based on Princeton's reputation as an excellent university, if that makes you feel any better.
[Full disclosure: I am a facebook employee]
Almost everything involving networks is a fad. Networks are an interesting mathematical structure, but on their own, they rarely enable one to build a compelling statistical model. E.g. people join Facebook for very complex social reasons, that go beyond local interactions between individuals. The image of Facebook has changed significantly since it started, and new products have entered the market, like snapchat and kik.
Meaning: people that can't type "facebook.com" on the address bar and type it into google then click?
So... based on that, we can assume people are getting more familiar with the internet maybe?
I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps.
EDIT: Here are some comparisons:
http://www.google.com/trends/explore#q=%2Fm%2F05zl0%2C%20%2F...
Interesting how the initial spread seems to narrow.
http://www.google.com/trends/explore#q=harvard%2C%20princeto...
Google trends represents share of search volume, not absolute volume [1]. As the higher education bubble continues to grow, ivy league schools have not grown as fast as other schools, and there are new schools being created as well. All of this activity contributes to Princeton's (for example) share of search decrease within their sector, and overall.
[1]en.wikipedia.org/wiki/Google_Trends
It is really easy to find something that correlates with your data, when your data follows a simple trend. Now if princeton enrollment was going up and down a lot, and the google trends followed the same shape, that would be something.
I have seen this in most nerdy topics associated with early internet adoption in Western English speaking countries -- or at least that is my theory. Basically as other countries come online and late adopters come online, they do not search for the same types of topics earlier adopters looked for.
This trend for academic or technical topics decreasing as a percentage of Google trends is universal, see here:
http://www.google.com/trends/explore?hl=en-US&q=Science,+/m/...
Statistics is descriptive--it's not predictive. It tells you about the data you have. It doesn't tell you why the systems produced that data, and whether they might produce very different data under different conditions.
(Oh, you don't have metrics to prove that? Is that because users are jumping ship?)
(Oh, you missed that? Is that because you didn't actually read the post?)
(Read the post, didn't miss it.)
People do "research" like this all the time - you throw shit at the wall and see what sticks. Most of the stuff that ends up on the floor never receives any attention at all, so you don't hear about it on first-tier news sites. But when it's about Facebook, it goes viral, and suddenly is the subject of intense scrutiny. They didn't bring this to the UN for a call to action. They didn't start a company around it. They just applied an idea to some data and wrote it up. And now the entire Internet is making fun of some exaggerated version of their idea, summarized by Huffington Post hit-mongers.
Personally, I applaud these guys for putting in the work to test out a theory. If it's not correct, it will go in the bin with the other ten million papers with flawed theories, premises, methods, or other aspects that have been published in the last day or two.
Example: "The amount of time you spend sick with the flu is constant no matter how many of your friends ALSO contract the flu"
The medium matters, too. I'm all for people discussing crazy ideas and seeing what sticks, but when they wrote a paper about it and submitted it to arXiv.org ("Submissions to arXiv should conform to Cornell University academic standards."), they staked their reputation. Why wasn't this just a blog post?
As for why it's a paper and not a blog post - I think the conventions of academia are there to be questioned too. Why not form it into an abstract-methods-discussion format if that's what they think best suits it? Why does it have to be made into a more accessible and friendly blog format? Seems to me that open access will reduce the "scariness" of having a formal paper out there, as well as reduce the worth of same. Broadening access has that effect, and it looks like these guys are early adopters of a new, less formal process for describing the evaluation of theories like this one.
I guess what I'm asking is why this paper gets singled out for special derision when there are plenty of papers out there just as flawed (though no less "valid"). I think it's just a run-of-the-mill bit of science that, like many others, will simply be disproven in a positive way by experts. It seems ugly to submit it and its writers to such public pillorying.
I also do not think they are completely wrong. I think there is general consensus that in North America we have likely reached peak Facebook engagement -- unless there is a game changer. The question is fairly open at this point in time as to whether it stays steady, decreases slowly or fast from this point.
Some people also don't want facebook to fail. If you go deep into their comment history, many of them have argued that facebook cannot be the next myspace.
http://www.amazon.com/Surfaces-Essences-Analogy-Fuel-Thinkin...
> I think there is general consensus that in North America we have likely reached peak Facebook engagement
I disagree. There are signs that US teens are decreasing engagement or maybe not even registering: http://istrategylabs.com/2014/01/3-million-teens-leave-faceb.... However, other age groups are apparently doing well, and there's no sign that 80% of users will leave by 2017.
On the whole, they continue to grow. The question is whether the teens are an early indicator that will apply to all demos/geos and that they will all decline, or if there's maturation in the product and that it better fits an older demo and the teens will grow into FB in later years. Or maybe something else.
Wait did Facebook just start likewhoring themselves too?
I though it was quite generous of Facebook to resort to gentle mockery rather than shred its numerous stupidities.