This list is a reminder that the work done by researchers and the work done in industry have profoundly little overlap, outside of a handful of high-profile examples.
If the giants would like to do us all a massive favour: pay the IEEE and ACM for worldwide, perpetual, ongoing rights to their libraries. Because right now that vast ocean of literature is basically invisible to practitioners.
Most of those papers’ first authors were in industry shortly afterwards. Most grad students go directly to industry and take their knowledge with them.
I don't see how "I can't learn more because they work for ExampleCo on proprietary technology" is an improvement on "I can't learn more because it's behind a paywall, along with everything it refers to and everything that refers to it".
Google was founded, quite literally, out of studying the graph of literature. Here's one way they can move that original vision a little closer to reality.
Industrial research has a long and influential history -- IBM, AT&T's Bell Labs, MS Research -- all doing pure research with generous budgets.
But if their work is published by the IEEE CS or the ACM, you may not be able to read it. This situation has improved a lot in recent years as many authors will post copies on websites they control. But that doesn't account for the decades of work otherwise unavailable for direct reference.
IIRC, VMWare was started by a Stanford professor and directly based on his research in virtualization. I can't find the full story of the transition from academia to industry, but SimOS looks related for sure:
- Pixar had a pretty long road to industry, but it has deep academic roots. Co-founder Ed Catmull published significant original research in computer graphics.
- Yahoo was also started by 2 Stanford grad students, although it wasn't based on any research that I'm aware of.
- Werner Vogels was one of my professors at Cornell, doing work on distributed computing, e.g. on gossip protocols, and he is now CTO of Amazon.
- Guido van Rossum worked in Andrew Tanenbaum's lab (a professor whose OS book you might have), and Python was at least partly inspired by that experience. Python now runs much of the industry :)
It would indeed be nice to see a list of "papers that companies are based on".
The PageRank paper is well known (although Google search no longer resembles it). I suspect you can do the same thing for VMWare and Pixar, at the very least. Those two companies relied on significant technical advances.
This goes all the way back to Hewlett Packard, although I'm not sure if they're closer to Yahoo's case (same people) or Google's case (same subject area):
Is that really how research in CS works? Certainly we hear about grad students doing lots of the work in laboratory sciences like biology, but in more analytic subjects like mathematics or theoretical physics its not true at all. Grad students making significant contributions is extremely rare; basically all of the real research is done by tenured professors.
Can those of you who did postgrad CS tell us how much of the research is actually done by grad students vs professors?
In my experience it's often (but definitely not always) the case that for new post-grad students the idea or general research direction often comes from the professor (especially if the research is funded by a grant proposal). Most of the heavy lifting is then done by the grad student with guidance and feedback from the professor. As the student matures they start to operate more independently.
> Grad students making significant contributions is extremely rare; basically all of the real research is done by tenured professors.
...but most of these first authors (the people formally credited with doing the real work) are graduate students, so how does that fit in with your theory?
Well i think it just depends on how 'significant' you need the phd to be. As an example of what I mean, consider the first issue of this years Annals of mathematics [1] and note that there is not a single grad student on any of the papers, first author or otherwise.
In my experience (new professor), 90-95% of the work is done by grad students.
A good prof in my mind acts a mentor, providing frequent feedback on the research and contributes to writing of the paper. The feedback is often invaluable, but still just a small part in the overall amount of work compared to what the student puts in.
Depends a bit on the professor. Young ones are more involved than older ones for a variety of reasons. In either case, the bulk of the work is done by grad students. Professors are more like managers that provide direction, feedback, etc.
> This list is a reminder that the work done by researchers and the work done in industry have profoundly little overlap, outside of a handful of high-profile examples.
I'm not sure what you mean - this list seems to contain a healthy mix of academic and industrial work. Big names like Google, Facebook, Microsoft, Oracle, are all represented.
And why is it invisible to practitioners? Most tech companies will have a membership of these libraries anyway.
Most tech companies will have a membership of these libraries anyway.
That does not jibe with my experience at all. In fact, I don't think that, in 20 years, I've ever worked at a company that had company-wide access to IEEE / ACM / etc. libraries. Or if any of them did, they didn't try very hard to make it known.
Isn't this what hiring talent is for? I feel like companies who are interested in the best research going on right now, are either going to do it themselves or hire someone who is familiar with the topic. I don't know though.
Another option is posting them to tech forums more often and/or voting them up for visibility. I was posting CompSci here as submissions and comments. Comments got attention with submissions mostly invisible. We had quite a few of these papers in main article on Lobsters since they upvote CompSci a lot. My default spot for CompSci papers now. There might be sub-Reddits with some of them, too.
Gotta get the work out there ourselves instead of hoping the big companies will do it for us. Probably best to straight-up build stuff with it rather than telling people about it. Also, they'll listen more when someone brings data that improves their operations.
most articles from after about 2000 have a version on arxiv, if the conferences and venues aren't already open access. in CS this is almost always substantially the same work as gets published in a conference or journal.
Google Scholar the title in quotes and you will almost always find either the arXiv PDF or a version from the author website, conveniently linked along with well-formatted bibliographic information.
also re ACM, an unlimited-access subscription to every ACM publication runs $198/year with an ACM professional membership (cheaper for students), if you have a moral objection to just using sci-hub.
don't know if there's something similar for IEEE.
things are way better in CS than basically any other field. not to say there aren't huge obstacles to connecting researchers and practitioners. but access to journals doesn't have to be one, once you know the same tricks grad students use when not on campus wifi.
And studies show there is no correlation between best paper awards and citation count (actual research impact). Given that program committees are typically hijacked by faculty from elite schools and awards mostly go to faculty from elite schools it is evidence there is something wrong with the CS research community as it rewards elitism and not actual research impact.
CS Education venues never show up on these lists. In some cases, it's intentional and in others it's accidental. Either way, I wonder if we'll ever see the same respect as our peers in CS. It's a weird spot to be in.
Please don't link to google scholar. Link to the conference websites directly if you can, or otherwise to DBLP [1].
Most of these conferences are open access and make a good effort to get the papers they publish to the widest audience possible. Google scholar typically hot-links the PDFs, and keeps them invisible link-wise. This is slowly eroding the conference's "web reputation" (based on links primarily) and keeps funneling users to the big corps, something we really don't need these days ;)
doesn't look too good for german institutions (or europe in general) :( I thought it we are just not really that important in machine learning, but other disciplines are similiar. Any physicist here? Are german institutions more visible in physics? They always appear to have quite big physics faculties.
A lot of these people are European, or working in European locations such as MSR's Cambridge lab, but they do tend to be working for American institutions. The failure is our institutions, not our people.
yeah, i agree. there's a crazy brain drain. Just look at TUM. If they could hold onto the talents they developed they would count Schmidhuber, Hochreiter, Schaal, (Marcus) Hutter etc. to their faculty and would be a serious competitor in the global ML landscape.
Some of those are obvious if you've been in the field a while, but I think it's notable that big names like Intel, Harvard and NASA are so lowly ranked when compared to others on the list.
Also Notable, none of the big names in the field won any of these: Jeff Dean, Geoff Hinton, Yann Lecun, Andrew Ng, Yoshua Bengio, Jurgen Schmidhuber etc...
49 comments
[ 2.1 ms ] story [ 114 ms ] threadIf the giants would like to do us all a massive favour: pay the IEEE and ACM for worldwide, perpetual, ongoing rights to their libraries. Because right now that vast ocean of literature is basically invisible to practitioners.
Google was founded, quite literally, out of studying the graph of literature. Here's one way they can move that original vision a little closer to reality.
But if their work is published by the IEEE CS or the ACM, you may not be able to read it. This situation has improved a lot in recent years as many authors will post copies on websites they control. But that doesn't account for the decades of work otherwise unavailable for direct reference.
https://en.wikipedia.org/wiki/Mendel_Rosenblum
(If anyone has details, please chime in.)
- Pixar had a pretty long road to industry, but it has deep academic roots. Co-founder Ed Catmull published significant original research in computer graphics.
- Yahoo was also started by 2 Stanford grad students, although it wasn't based on any research that I'm aware of.
- Werner Vogels was one of my professors at Cornell, doing work on distributed computing, e.g. on gossip protocols, and he is now CTO of Amazon.
- Guido van Rossum worked in Andrew Tanenbaum's lab (a professor whose OS book you might have), and Python was at least partly inspired by that experience. Python now runs much of the industry :)
It would indeed be nice to see a list of "papers that companies are based on".
The PageRank paper is well known (although Google search no longer resembles it). I suspect you can do the same thing for VMWare and Pixar, at the very least. Those two companies relied on significant technical advances.
EDIT: Also Cisco and Stanford:
https://www.siliconvalleyhistorical.org/cisco-systems/
This goes all the way back to Hewlett Packard, although I'm not sure if they're closer to Yahoo's case (same people) or Google's case (same subject area):
https://en.wikipedia.org/wiki/Hewlett-Packard#History
Can those of you who did postgrad CS tell us how much of the research is actually done by grad students vs professors?
...but most of these first authors (the people formally credited with doing the real work) are graduate students, so how does that fit in with your theory?
[1] http://annals.math.princeton.edu/2018/188-1
A good prof in my mind acts a mentor, providing frequent feedback on the research and contributes to writing of the paper. The feedback is often invaluable, but still just a small part in the overall amount of work compared to what the student puts in.
Like 95% of research is done by grad students.
I'm not sure what you mean - this list seems to contain a healthy mix of academic and industrial work. Big names like Google, Facebook, Microsoft, Oracle, are all represented.
And why is it invisible to practitioners? Most tech companies will have a membership of these libraries anyway.
I'm focusing here on access to the publications, not on the email domain of the authors.
> And why is it invisible to practitioners? Most tech companies will have a membership of these libraries anyway.
Not everyone in tech works for a tech company.
That does not jibe with my experience at all. In fact, I don't think that, in 20 years, I've ever worked at a company that had company-wide access to IEEE / ACM / etc. libraries. Or if any of them did, they didn't try very hard to make it known.
Gotta get the work out there ourselves instead of hoping the big companies will do it for us. Probably best to straight-up build stuff with it rather than telling people about it. Also, they'll listen more when someone brings data that improves their operations.
Google Scholar the title in quotes and you will almost always find either the arXiv PDF or a version from the author website, conveniently linked along with well-formatted bibliographic information.
also re ACM, an unlimited-access subscription to every ACM publication runs $198/year with an ACM professional membership (cheaper for students), if you have a moral objection to just using sci-hub.
don't know if there's something similar for IEEE.
things are way better in CS than basically any other field. not to say there aren't huge obstacles to connecting researchers and practitioners. but access to journals doesn't have to be one, once you know the same tricks grad students use when not on campus wifi.
Quick suggestion:
Please consider including best paper awards from NeurIPS (e.g., https://nips.cc/Conferences/2017/Awards).
Quite a few faculty job postings have even been for CS Ed researchers :)
Related, the most cited CS papers (although very dated at this point ...):
http://citeseer.ist.psu.edu/stats/articles
Most of these conferences are open access and make a good effort to get the papers they publish to the widest audience possible. Google scholar typically hot-links the PDFs, and keeps them invisible link-wise. This is slowly eroding the conference's "web reputation" (based on links primarily) and keeps funneling users to the big corps, something we really don't need these days ;)
[1] https://dblp.uni-trier.de/
As a note, the first paper listed when it's 1. 2. 3. was the "best" paper (the others were runners up).
Some of those are obvious if you've been in the field a while, but I think it's notable that big names like Intel, Harvard and NASA are so lowly ranked when compared to others on the list.
Also Notable, none of the big names in the field won any of these: Jeff Dean, Geoff Hinton, Yann Lecun, Andrew Ng, Yoshua Bengio, Jurgen Schmidhuber etc...
Andrew Ng has too: http://www.robotics.stanford.edu/~ang/papers.php
This list just contains a very small number of conferences.