Curious about HN's rules on titles. How is the language relevant in any way to this particular post? That sure seems to be editorializing in an attempt to promote something other than the project itself (and obviously to get upvotes). Given the explicit rules against both editorializing and soliciting upvotes, I don't see why this title is allowed.
> Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize.
> Don't solicit upvotes, comments, or submissions.
It does matter in the context of the author addressing the /r/rust community. If you had posted the reddit link instead this would make sense.
But I don't think the language is relevant for the direct inciteful.xyz site itself. Better to submit both links separately than trying to combine them as they have different audiences.
There may or may not be some interesting reasons for choosing rust, but also the original article does not even include Rust in the title. It seems like Rust is mentioned solely for attention in this case. Would "A better way to search through academic literature" have been popular? Probably. How about "A better way to search through academic literature written in Java"? I assume not.
Regarding moderation, its a thankless task which I don't envy and its hard to draw a line over nitpicky article titles when one has been voted in already.
From an HN point of view Rust is so frequently discussed that it's probably better not to orient a submission like this in that direction—it will probably get sucked in the generic direction, and the diff here (new academic search engine) is more interesting.
I'm a big fan of trying new search engines for academic research; I hopefully am more likely to break out of whatever search bubble I'm unaware I'm in.
This one in particular had some very nice features, some of which are present in Semantic Scholar (my current favourite) but some which are certainly not.
Recommending papers based on citation graphs is a good way to very quickly get up to speed with fields I'm not to familiar with, but I'm always wary that I'll end up back in the feedback loop of very few popular papers rising to the top while perfectly good papers go unseen because they weren't well cited in the year they were written.
So I'll certainly keep an eye on this and give it a try, but I'm certainly still in the market for a "serendipity" slider on such recommendation engines.
1. Find a paper you like in a field you want to learn.
2. Use the keyword filters to filter down to papers that match your criteria.
3. Add a bunch of the interesting ones to a new graph using the purple "+" buttons.
4. On the "new" graph page, check out the similar papers section. If any of them are interesting, add those to the graph.
5. Repeat until you don't find anything else that is interesting.
The similar papers section uses a link prediction algorithm that basically says, if two papers cite a bunch of the same papers, rank them higher BUT if the paper they cite, is cited a bunch of times, don't give that connection much weight. The net effect of this is that it doesn't really matter if the paper was highly cited, only that it cites the same niche of papers as the ones you just chose. Also, because of the temporal nature of academic literature, the papers it brings up tend to be the newer and harder to discover papers.
The results are pretty great and it's as close to the "serendipity" slider that you'll get right now.
this seems very nice -- very polished and gives good results.
I would like it if the bibtex entries had meaningful cite keys as opposed to long numbers. as is, it would be pretty difficult to actually write a paper using these bibtex files.
I am planning on making the info in the BibTex files a bit more robust. Right now I'm just adding what I have readily available. But in general, if you are using Zotero or Mendely, the functionality to enrich the metadata on the entries does a good job filling in the missing info.
First impression is positive; relevant results and a reasonably straight-forward UI. I appreciate the warning about the slow-loading graph (which did load after a minute or two).
Two things that might be tweaked:
* The search didn't behave in an ergonomic way: I typed a query ("graph neural networks") and great relevant stuff came up immediately in the dropdown. When I hit enter, however, I got an error that read "Invalid search: Check your spelling, enter a DOI, or another paper identifier
or." I would have expected my action to take me to a search results page that listed what I saw in the dropdown (which I regard as a preview of the top hits) so that I could peruse the selection carefully.
* I wanted to load a paper to take a look at it and it took me a while to realize that I could click the "Yes" above "Open Access" to download it. Since one of the big use cases for a site like this is the eventual consumption of these papers, I suggest making a "read/download paper" call to action more explicit.
Much appreciated! I'll update it. I really hadn't anticipated much traffic. I was planning on doing an open beta but things have kind of taken on a life of their own (in a good way).
Creator here. It's similar in that it uses citations to make paper recommendations. But different in that I give you access to the entire paper graph rather try to distill it down to just a few. You can even write your own queries by clicking on the "SQL" button at the bottom of each table. I kind of view it as a Connected Papers for power users.
The database is about a month out of date right now. I am going to be updating it soon. You should be able to either put in the url or do arxiv:XXXX.XXXXX
Creator here! Wasn't expecting this to happen :) The site is definitely still in Beta so I appreciate any and all feedback. I just launched it a few days ago. It's been my COVID project and I finally got to the point where I felt comfortable having others use it.
The biggest hurdle was the speed of the graph creation. Basically taking a 250,000,000 paper/2,500,000 citaiton db and creating graphs that could be up to 200k papers and 3-4mm citations. For that I ended up learning/using Rust (which was a great experience).
The plan is to keep it totally free and hopefully get some institutional support once I get a better handle on demand and costs.
Ask me anything!
EDIT:
As you are going through the site, be sure to use the purple "+" buttons to create your own graphs centered on the topic of your choice. That combined with the in-graph keyword filters are probably the most powerful ways to quickly zero in on the most relevant literature.
But long story short, I end up doing most of the graph analysis by passing in the citations, using PyO3, to graph-tool in python then returning the data I need about each paper. I am planning on moving that over to Rust. But not being an academic I wanted to get feedback on the quality of the results before making it difficult to quickly test different types of algorithms.
Eventually I'd like to move the site to open source, but right now the repo isn't in a place where I can do that. As for specific parts, it's pretty purpose built and this is my first Rust project and so I'm not sure which parts would be helpful to the community. And I doubt they would meet the communities standards just yet :)
It’s better to open source sooner rather than later even if it’s not in a place you’d want it to be. Like some of the work you have to do might be done by the community.
Very nice work. I especially liked the ability to build up a collection of papers, that the response time was good, and that the SQL could be edited directly.
Do you have any plans to add a graphical visualization of top/central papers?
That is the most requested feature and something I'm working on. It's a fun (and hard) design/data problem. Which of the 5k-150k papers do you show in the graph? And then how do you render them in a way that is both visually appealing but also conveys the most import information?
they do some clever hiding of edges so graph is not overwhelming, but still only O(100) nodes.... for O(100k) nodes you'll need to do some selection for sure ;)
Just curious, what source are you using for the citation graph? I seem to remember looking for an API to something like ACM digital library at one point and not really finding what I wanted, but maybe I just didn’t know how to look.
I love this idea btw, I’m going to use it to find some holiday reading!
I played a bit with the site and liked it. The one killer feature I might pay a few dollars per year for is to create a profile, with a few graphs attached and get a daily or weekly email when a new paper is published that fits well into one of my graphs. Feel free to add a monthly "the most important old paper that you have not read yet (or not read in the last 5 years)" email too.
8 mentions of the 'graph' on the page. Zero renderings of the graph :(
It is a nice user interface and the reference material is useful and well presented. But when you get down to it, it's linear lists about the characteristics of a semantic graph. As I've said many times, this is like describing a tree with a tabular catalog of its leaves. Graph navigation needs a graphical representation, because things like branchiness (node out-degree) and other factors are more easily shown than described.
The basic problem with graph representation/ navigation/ traversal is that there are many valid ways of looking at the graph and it's hard to render them all. Maybe try using a gutter to allow users to temporarily pin certain graph characteristics and render accordingly. In this context, sometimes I might be interested in the latest research that cites a paper, other times I might be looking to see who picked it up first, or to apply some sort of windowing function to a large spectrum of citations.
But I want to see the graph, even if I am looking for a particular leaf.
Good feedback. Visualizations are definitely the most requested feature and something I'm working on. It's a hard problem to distill so many papers (5-150k) down into a usable graph. I've made big ones that look amazing but which are not that useful.
I like the idea of interactive filters to allow the person to explore it visually. I'm hoping to have something people can play with in the next few weeks. I hadn't really expected the site to take off the way it has and so it's not really feature complete yet :)
It's outside of my technical wheelhouse, and most of what I know about graph theory is from trying to get a better handle on my domain problems, and as I get older I'm increasingly reluctant to get stalled on toolmaking.
But what you're discussing is a frequent problem with graph visualization - it's very easy to end up with 'hairball charts' that may be meaningful to the person who generated them but only because of familiarity with all the steps it took to get there, and the more inclusive the graph, the more time eaten by clustering algorithms, the more clusters produced, and the more of a cluster...well you get the idea.
As you're at an early stage of development, perhaps this technique, which trades away completeness for clarity and is relatively novel, might let you leapfrog some of those problems: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8896846
I've no idea about Rust, but in Python land you might want to check out Graph Tool and similar tools. As you note, layout of large graphs is not trivial.
Can you tell me how you're managing copyright? In my understanding, you're effectively "republishing" abstracts, possibly altering or remixing them too. Presumably the site isn't a commercial concern yet, so there's probably little issue. If the site starts to generate revenue, even if it's just to cover costs, or if it becomes a registered business, does this change the copyright situation for you?
I see that you're using several sources of open data, so perhaps all the data you're using is free from copyright, or has highly permissive copyright e.g. CC0.
For those wondering about open-access, in my understanding that's about _reading_ papers. Putting papers on a website might make the website provider subject to copyright. This applies even to abstracts.
As a minimum, it might be necessary to have a button on your site, weishuhn, to report any paper that has restrictive copyright. You've already noted one example of misclassification, and I know that Crossref and other sources provide their data with a big caveat that it might have inaccuracies. Responsibility cascades to you, unfortunately.
This inevitably leads to a trade-off between completeness of your data vs usefulness of your service. It's a problem I'm wrestling with too!
None of this is criticism, just thoughts from someone working in a similar area. As others have said, it's always good to see innovation like this in literature searching. Good work!
48 comments
[ 3.5 ms ] story [ 51.9 ms ] threadhttps://inciteful.xyz/p/186039733?&keywords=hello&maxDistanc...
> Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize.
> Don't solicit upvotes, comments, or submissions.
But I don't think the language is relevant for the direct inciteful.xyz site itself. Better to submit both links separately than trying to combine them as they have different audiences.
Regarding moderation, its a thankless task which I don't envy and its hard to draw a line over nitpicky article titles when one has been voted in already.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
This one in particular had some very nice features, some of which are present in Semantic Scholar (my current favourite) but some which are certainly not.
Recommending papers based on citation graphs is a good way to very quickly get up to speed with fields I'm not to familiar with, but I'm always wary that I'll end up back in the feedback loop of very few popular papers rising to the top while perfectly good papers go unseen because they weren't well cited in the year they were written.
So I'll certainly keep an eye on this and give it a try, but I'm certainly still in the market for a "serendipity" slider on such recommendation engines.
1. Find a paper you like in a field you want to learn.
2. Use the keyword filters to filter down to papers that match your criteria.
3. Add a bunch of the interesting ones to a new graph using the purple "+" buttons.
4. On the "new" graph page, check out the similar papers section. If any of them are interesting, add those to the graph.
5. Repeat until you don't find anything else that is interesting.
The similar papers section uses a link prediction algorithm that basically says, if two papers cite a bunch of the same papers, rank them higher BUT if the paper they cite, is cited a bunch of times, don't give that connection much weight. The net effect of this is that it doesn't really matter if the paper was highly cited, only that it cites the same niche of papers as the ones you just chose. Also, because of the temporal nature of academic literature, the papers it brings up tend to be the newer and harder to discover papers.
The results are pretty great and it's as close to the "serendipity" slider that you'll get right now.
EDIT: Formatting
I would like it if the bibtex entries had meaningful cite keys as opposed to long numbers. as is, it would be pretty difficult to actually write a paper using these bibtex files.
Two things that might be tweaked:
* The search didn't behave in an ergonomic way: I typed a query ("graph neural networks") and great relevant stuff came up immediately in the dropdown. When I hit enter, however, I got an error that read "Invalid search: Check your spelling, enter a DOI, or another paper identifier or." I would have expected my action to take me to a search results page that listed what I saw in the dropdown (which I regard as a preview of the top hits) so that I could peruse the selection carefully.
* I wanted to load a paper to take a look at it and it took me a while to realize that I could click the "Yes" above "Open Access" to download it. Since one of the big use cases for a site like this is the eventual consumption of these papers, I suggest making a "read/download paper" call to action more explicit.
“on it's head” should be “on its head”.
“not only with” should be “with not only”.
“analysis'” should not have an apostrophe and should possibly be “analyses”.
The biggest hurdle was the speed of the graph creation. Basically taking a 250,000,000 paper/2,500,000 citaiton db and creating graphs that could be up to 200k papers and 3-4mm citations. For that I ended up learning/using Rust (which was a great experience).
The plan is to keep it totally free and hopefully get some institutional support once I get a better handle on demand and costs.
Ask me anything!
EDIT: As you are going through the site, be sure to use the purple "+" buttons to create your own graphs centered on the topic of your choice. That combined with the in-graph keyword filters are probably the most powerful ways to quickly zero in on the most relevant literature.
But long story short, I end up doing most of the graph analysis by passing in the citations, using PyO3, to graph-tool in python then returning the data I need about each paper. I am planning on moving that over to Rust. But not being an academic I wanted to get feedback on the quality of the results before making it difficult to quickly test different types of algorithms.
Do you have any plans to add a graphical visualization of top/central papers?
https://metacademy.org/graphs/concepts/bayesian_logistic_reg...
and src code for the graph view is here: https://github.com/metacademy/metacademy-application/blob/ma...
they do some clever hiding of edges so graph is not overwhelming, but still only O(100) nodes.... for O(100k) nodes you'll need to do some selection for sure ;)
I love this idea btw, I’m going to use it to find some holiday reading!
It is a nice user interface and the reference material is useful and well presented. But when you get down to it, it's linear lists about the characteristics of a semantic graph. As I've said many times, this is like describing a tree with a tabular catalog of its leaves. Graph navigation needs a graphical representation, because things like branchiness (node out-degree) and other factors are more easily shown than described.
The basic problem with graph representation/ navigation/ traversal is that there are many valid ways of looking at the graph and it's hard to render them all. Maybe try using a gutter to allow users to temporarily pin certain graph characteristics and render accordingly. In this context, sometimes I might be interested in the latest research that cites a paper, other times I might be looking to see who picked it up first, or to apply some sort of windowing function to a large spectrum of citations.
But I want to see the graph, even if I am looking for a particular leaf.
I like the idea of interactive filters to allow the person to explore it visually. I'm hoping to have something people can play with in the next few weeks. I hadn't really expected the site to take off the way it has and so it's not really feature complete yet :)
But what you're discussing is a frequent problem with graph visualization - it's very easy to end up with 'hairball charts' that may be meaningful to the person who generated them but only because of familiarity with all the steps it took to get there, and the more inclusive the graph, the more time eaten by clustering algorithms, the more clusters produced, and the more of a cluster...well you get the idea.
As you're at an early stage of development, perhaps this technique, which trades away completeness for clarity and is relatively novel, might let you leapfrog some of those problems: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8896846
I checked a paper, 10.1097/RHU.0000000000000563 for example; which I know for a fact is open-source marked as non open source.
Otherwise, very nice tool. Will be using this regularly. Awesome stuff.
https://api.unpaywall.org/v2/10.1097/RHU.0000000000000563?em...
According to them it's closed but I definitely see it as open here:
https://journals.lww.com/jclinrheum/fulltext/2017/09000/omeg...
I'll look to see if I can find another data source for OA papers to add more coverage.
Can you tell me how you're managing copyright? In my understanding, you're effectively "republishing" abstracts, possibly altering or remixing them too. Presumably the site isn't a commercial concern yet, so there's probably little issue. If the site starts to generate revenue, even if it's just to cover costs, or if it becomes a registered business, does this change the copyright situation for you?
I see that you're using several sources of open data, so perhaps all the data you're using is free from copyright, or has highly permissive copyright e.g. CC0.
For those wondering about open-access, in my understanding that's about _reading_ papers. Putting papers on a website might make the website provider subject to copyright. This applies even to abstracts.
As a minimum, it might be necessary to have a button on your site, weishuhn, to report any paper that has restrictive copyright. You've already noted one example of misclassification, and I know that Crossref and other sources provide their data with a big caveat that it might have inaccuracies. Responsibility cascades to you, unfortunately.
This inevitably leads to a trade-off between completeness of your data vs usefulness of your service. It's a problem I'm wrestling with too!
None of this is criticism, just thoughts from someone working in a similar area. As others have said, it's always good to see innovation like this in literature searching. Good work!