This is pretty clever, thanks to the author for releasing it.
Personally, I might rather just add the script tag to my site's layout rather than introduce another gem dependency. But maybe if you have lots of sites to maintain, this could be preferred?
Definitely a good point. Interestingly, I didn't set out intending to use JS to solve this problem. I wanted to use Rack to intercept the request and check the headers and proceed from there. However, when I was playing around with it in Request Bin, I noticed that Genius actually modifies the URL so that there is no reference to them anywhere in the headers. This is generally poor etiquette and forced me to modify my approach.
Folks blocking, curious your motivations? (I can think of a few but am curious about the specific issues with Genius vs. the platonic ideal of a universal web annotation layer)
Motivations for disabling comments vary widely, for one thing.
For another, Genius is a company with a history of controversy. I can imagine some sites being ok with universal annotations but averse to Genius. It'd be useful to know how Genius is going about things wrong, because I dream of someday having a universal annotation layer that doesn't provoke widespread blocking.
You mean like how CNN disables comments so that people are unable to call them out on their bullshit? So they can post fearmongering articles with little basis in reality, backed up by quotes from unnamed anonymous "intelligence sources?" So they can continue to spread misinformation on behalf of the US government without anyone questioning it?
The mainstream media is a complete poison to society. It needs to be dismantled and ripped apart. It's a dark, dark villain that needs to be exposed to the sunlight and shown for the monster that it is.
That would be easier if comments were allowed on "publications" like CNN, but unfortunately they're disabled.
Personally, I am fully in support of federated "overlay" commenting systems like news genius, because they rightfully remove control/moderation over the comments from the publisher, and place it into the hands of a "trusted 3rd party" (genius).
They probably don't want hostile/parasitic services profiting off their content for free, even though that seems to be the business plan of every other startup. If Genius wants to act in good faith they should let people opt-out of annotations on their sites / pages.
Also, it's just another walled garden for content from a company with past ethically questionable business practices.
What is the motivation behind this? It strikes me as something like an anti-ad-blocker tool, except instead of serving to protect your sites ad revenue it's just trying to suppress free discourse about your content through a third party tool that happens to be exposed to this kind of suppression, but isn't actually invasive, and is only used by people who opt-in to it anyway. Basically it seems to me like a tool for people to protect their ability to successfully disseminate misinformation. Which, I guess is your prerogative and all, but I personally find disheartening. Having an effective universal annotation platform in wide use across the web could go a long way towards improving the SNR for all users.
You'd be able to make a blog post without annotations accusing you of disseminating misinformation and suppressing free thought, for one.
I think you're severely overrating the quality of Internet comments too, and with these annotations there isn't even a report feature available to help clean them up. Genius.
> I think you're severely overrating the quality of Internet comments
Maybe so, but I think perhaps you're underrating the desire by state and corporate actors for tools to suppress free discourse.
> there isn't even a report feature available
No, there definitely is a "Report Abuse" button. But, there clearly isn't much in the way of a global reputation/meta-moderation system, except what's carried over from the original Genius site, which doesn't exactly share the same editorial values as an (my) ideal annotation platform. The managing editor being from Gawker and going around posting snarky/superficially critical comments on serious pieces of writing is pretty emblematic of this. Which is a shame.
Since it's related, and I'm a fan of their work, here's another annotation platform, https://hypothes.is/, whose developers seem much more serious about creating a robust reputation system.
I'll take your word for it; it's pretty shocking that it wasn't there from day one. As I've written elsewhere, it seems like Genius's vision of universal annotations is not really lined up with the/my/some tech folks platonic ideal of it - Genius seems to want to extend RapGenius to the rest of the web to enable a mix of both factual and editorial commentary, whereas the aforementioned ideal would be much more focused on fact checking, not just providing a universal commenting layer.
It's clearly a grayer issue than I first comprehended when writing my initial response to your work; I apologize for my strongly negative tone.
>Having an effective universal annotation platform in wide use across the web
...funded by venture capital with an expectation of hyper growth and liquidity event would not necessarily be beneficial for all users. To put it mildly.
But anyway, reading Ms. Dawson's post does make me seriously reconsider the ethical motivations for making this tool. I suppose my initial negative reaction was more based on the 'platonic ideal of a universal web annotation layer' as gkop puts it, rather than an actual understanding of how the Genius platform operates.
First, there's been tools like this for a while. MS had an annotations tool built into IE in the 90s. (Though I think it required some sort of Office server or something.)
Second, her reasoning seems to revolve around not having comments on her site. Isn't this literally what every link aggregator from HN to reddit does?
She mentions reddit and others, but doesn't explain how they're different. Apparently it's because people can "scribble" "on her website". This makes News Genius "worse than Twitter". I guess if reddit created a browser plugin she'd hate it too?
>pasted in links to my tweets—which theoretically they should not have had access to, as I had blocked them both on Twitter
How can you block anyone on Twitter and think it'll work? Just sign out and block's gone?
>No one likes being questioned
I think that sums up her attitude. Note I think her work is fine, but she's literally complaining about people discussing her work on other sites.
>she's literally complaining about people discussing her work on other sites
Things like this don't become a problem until it is so easily organized. A bunch of tweets, FB posts, HN, reddit or other social comments are so scattered they don't have as much impact. The comments aren't visible on the the same page as the content.
I'm going to have to think more about this. I've had this thought a couple of times now but I don't have a full picture of it yet.
They are not visible on the same page as the content unless you have news genius installed or have navigated to the news genius proxy for that page. You need to opt in to see the comments.
God forbid people can post organized opinions you disagree with...
Thanks for the link. The protestations are a little silly from what I gather.
So this is the crux of it: Genius allows people to comment on stuff that people write - like an improved comment section. Occasionally these comments suck and are offensive.
From the blog post: "News Genius adds one more way for people on the Internet to be made unsafe." and "A tool that allows my abusive ex-boyfriend to interact with me and my content is a tool that should not exist."
The solipsism is laughable. The Internet will forever provide an infinite supply of trolls but when you publish on a public platform as opposed to behind a password protected blog or via email, people will talk about what you write (if you're lucky. many people publish in obscurity) and it just so happens that you may not always agree with what they say.
The reflex to shut down rather than, say, ignore dissenting opinion is an odd but increasingly more common one.
Yea, I agree that her reasoning isn't entirely well founded. But I'm nonetheless empathetic, because she's an embattled blogger fighting trolls to justify her work all the time, and the gap between that experience and the idealistic motivations for a universal annotation platform must be enormous.
From our tech-centric perspectives, the value of a universal annotation platform seems self evident, but for a layman's perspective, it just seems like another toxic commenting layer that content creators can't moderate (which, in the ideal, it is). The fact that it's then collocated with their content, instead of, say, a click away on HN or twitter, makes it feel more violating.
The truly ideal platform would have massive adoption with benevolent power users that both prevented abuse and generated value adding content by verifying factual claims, adding appropriate context, etc. This ideal use case is pretty far from Genius's original platform (explaining rap lyrics in an entertaining fashion), and that mismatch of intents seems to be the source of the problem.
What a crock of shit. People installing an addon to comment on your content "silences writers" by making their readers actually think about the crap they write? God forbid people actually think. I'm so sick of this anti-commenting drivel promoting mindless belief. If you can't stand being questioned, please stop posting on the internet.
I suppose it's the annotations being presented on your site to give it an undeserved sense of legitimacy; instead of a detractor otherwise of no consequence complaining on their own blog there is a detractor getting real estate alongside your content.
It's a blurry line for sure, but I think the motivation of silencing the 'riff-raff' is understandable.
Thanks for your comment. News Genius' site is a bit misleading, as there are a few different ways one could wind up with site annotations.
The first, as you mention, is by opting in. I agree that such a tool could be a useful way to foster discourse on your content. This tool is not meant for people who opt in to Genius annotations.
Further down on Genius' page, it states that any page can be annotated simply by prepending `genius.it/` to a site URL, regardless of whether the content publisher has granted permission. Unfortunately, the comments section of any Internet page tends to showcase some extremely uncivilized discourse. This is why many site authors choose to disable comments. Genius circumvents these people's wishes. Further, unlike a comments section, Genius does not actually notify the owner of a website if his/her content is annotated. Site owners have the option to moderate comments; it took Genius until today to even add a Report Abuse button.
Hopefully this clarifies some of the motivation. Thanks for sharing your perspective!
The readers of the comments need to opt-in to reading the newsgenius annotations. So the argument of "some people might find these comments offensive" does not make sense. If you find the comments on news genius offensive, don't read them.
If you ask me, site owners should NOT have a right to moderate comments. That just encourages silencing dissent and disagreement and publishing misinformation, as others have noted.
Should site owners also have the right to issue takedown notices to reddit or hacker news when someone links to their site? Or to delete comments on those websites?
I haven't seen many people arguing that News Genius shouldn't be permitted to exist (except perhaps on copyright/derived work grounds, which given their implementation is quite arguable), just that it's not a good idea as implemented. And, of course, people are well within their rights to use javascript/etc to prevent the embedding that News Genius does, or to e.g. block referrals from reddit or HN.
Haha, I get it. I was too quick to defend Genius as a proxy for the universal annotation platform of my dreams. But the idea is that, in wide adoption, such a platform would be much more resilient to comments like your example. Sort of like HN's commenting atmosphere!
I originally intended to leverage Rack all the way through by inspecting headers. However, much to my chagrin, I discovered that Genius does not reveal its identity in the headers, so I had to resort to JS instead to accomplish my goal.
In this situation, genius will probably try to circumvent a bunch of these techniques. You could imagine this library adds more techniques to fight genius, genius adds work arounds, the library adds different techniques, and it goes back and forth. For users of the library, they just have to update the gem to stay ahead.
IANAL, but I absolutely believe there's the possibility to raise a copyright violation suit. There have been cases brought against Google in the past for caching webpages. This is why google/bing web crawlers abide by robots.txt.
NewsGenius needs to at least 1) identify their crawler, and 2) provide a way to opt-out. Otherwise, it doesn't look good for them. They're not being a very good citizen of the Web.
Unlike the others (which prevent hosting on genius.it) this one actually prevents annotations via plugin etc by injecting invisible characters at random. Its a WP plugin but the method could be adapted.
Going a bit meta here: Is there a standard library to use for monitoring 3rd-party interactions users bring to a website?
Typically this would mean browser add-ons and extensions, but I wouldn't be surprised to discover other things (like headers injected by cellular companies, or who knows what from malware infections).
Such a tool would be useful when troubleshooting various 'works on my machine' scenarios.
I guess Genius doesn't care about the rampant copyright violation involved with the "web annotator"? Using genius.it/ creates derived works, and it does so almost certainly without approval of the copyright owner.
I guess they will argue fair use, but hostile 'annotations' definitely reduce the value of the work, they reuse the entire work, and they use the work for commercial gain. I wouldn't consider this a slam dunk.
I actually really dislike this concept of "we're going to wrap your website and mark it up with other stuff to make it look like you're hosting this content here". Sure, people should be able to talk wherever they want, but keep that in a popup, or at worst a frame - don't highlight text to suggest that the author has done that, and don't load your own elements in their page.
Thanks for the interesting, non-inflammatory comment. You raise a good point.
The fact that they proxy the content on their servers definitely puts them in a more vulnerable position than if this were, for example, a strictly client-side chrome extension that fetched annotation metadata and re-rendered the page accordingly.
In the purely client-side case, the full article would not pass through their servers, only bits and pieces of it. It would be easier to argue fair-use if you're only storing parts of the articles, and comments/highlights, on a remote server, instead of proxying the entire content.
(What if the article is behind a paywall, for example? Then genius.it effectively becomes a circumvention proxy.)
Didn't even know this was a thing but after taking a gander at the first example of this on the News Genius (http://news.genius.com/) website I'm not surprised people want to get rid of it. I read the 2nd article, a buzz feed production about Trump's campaign adviser, and honestly I'm highly disappointed at the quality of the annotations. Even more so when I found out that the person doing said annotations is the Managing News Editor at Genius. Buzz feed is by no means a bastion of journalistic integrity but the superficial comments this editor left are a joke. If she was actually the editor for this article it might make sense to complain about wording choices but as I understand it Genius Annotations are meant to explain and expand upon things in the article. Here, the person Genius felt most qualified to head this whole effort, uses annotations like a permanent twitter stream of her personal opinion of the article. She complains about not expanding upon allegations but the article does. Not always but in a lot of cases they can't. No one is going on the record against a man working for the likely republican nominee. They can't afford to burn bridges because they still need access to Trump. All in all if that is the quality of annotations created by Genius' Managing New Editor this is a feature that needs to be taken out back.
Thanks for the code review - it's always good to see suggestions on how to improve things. If you have ideas on how to make the code better, please feel free to submit a pull request. Thanks for sharing your feedback!
If Genius were actually acting in good faith, they'd allow some sort of 'robots.txt' like functionality, which would solve this whole thing in a nice way.
It would be nice if they made it even easier for users on hosted services who can't necessarily change robots.txt so you could opt out on a per-page basis. But I don't think anyone expects Genius to act in good faith.
Based on what I read about it before I actually saw it in action, I had guessed that they were showing the original site in a frame served from the original site with their annotations outside that frame.
(That may or may not be legal. I'm not sure if it has been settled whether or not displaying someone's page served from their site in a frame served from your site might be a violation of the public display copyright right).
However, after looking at the genius.it page that was linked in another comment for part of this very thread, and also looking at the genius.it page for my own website, that's not the case. They appear to have made a copy of the base page itself and are serving that themselves. They don't seem to copy images and stylesheets--those are still being served from the original server.
In the copy of the page they serve, they modify the links contained therein to be genius.it links. That's making a derivative work.
I don't think Genius could argue successfully that this use is fair use. They are a commercial site, and they are copying the entire page from the original site.
Wouldn't it be extremely easy for this to happen in the users browser wherein you have no right nor power and just fetch annotations based on the current url?
If it doesn't already work that way it would be trivial to do so.
51 comments
[ 3.3 ms ] story [ 90.8 ms ] threadPersonally, I might rather just add the script tag to my site's layout rather than introduce another gem dependency. But maybe if you have lots of sites to maintain, this could be preferred?
Definitely a good point. Interestingly, I didn't set out intending to use JS to solve this problem. I wanted to use Rack to intercept the request and check the headers and proceed from there. However, when I was playing around with it in Request Bin, I noticed that Genius actually modifies the URL so that there is no reference to them anywhere in the headers. This is generally poor etiquette and forced me to modify my approach.
Thanks for your comment!
For another, Genius is a company with a history of controversy. I can imagine some sites being ok with universal annotations but averse to Genius. It'd be useful to know how Genius is going about things wrong, because I dream of someday having a universal annotation layer that doesn't provoke widespread blocking.
The mainstream media is a complete poison to society. It needs to be dismantled and ripped apart. It's a dark, dark villain that needs to be exposed to the sunlight and shown for the monster that it is.
That would be easier if comments were allowed on "publications" like CNN, but unfortunately they're disabled.
Personally, I am fully in support of federated "overlay" commenting systems like news genius, because they rightfully remove control/moderation over the comments from the publisher, and place it into the hands of a "trusted 3rd party" (genius).
Also, it's just another walled garden for content from a company with past ethically questionable business practices.
I think you're severely overrating the quality of Internet comments too, and with these annotations there isn't even a report feature available to help clean them up. Genius.
Maybe so, but I think perhaps you're underrating the desire by state and corporate actors for tools to suppress free discourse.
> there isn't even a report feature available
No, there definitely is a "Report Abuse" button. But, there clearly isn't much in the way of a global reputation/meta-moderation system, except what's carried over from the original Genius site, which doesn't exactly share the same editorial values as an (my) ideal annotation platform. The managing editor being from Gawker and going around posting snarky/superficially critical comments on serious pieces of writing is pretty emblematic of this. Which is a shame.
Since it's related, and I'm a fan of their work, here's another annotation platform, https://hypothes.is/, whose developers seem much more serious about creating a robust reputation system.
It's clearly a grayer issue than I first comprehended when writing my initial response to your work; I apologize for my strongly negative tone.
...funded by venture capital with an expectation of hyper growth and liquidity event would not necessarily be beneficial for all users. To put it mildly.
The README links to the explanation: https://ellacydawson.wordpress.com/2016/03/25/how-news-geniu...
But anyway, reading Ms. Dawson's post does make me seriously reconsider the ethical motivations for making this tool. I suppose my initial negative reaction was more based on the 'platonic ideal of a universal web annotation layer' as gkop puts it, rather than an actual understanding of how the Genius platform operates.
Here's another article with additional context: http://www.vox.com/2016/3/31/11336852/genius-annotation-cont...
Second, her reasoning seems to revolve around not having comments on her site. Isn't this literally what every link aggregator from HN to reddit does?
She mentions reddit and others, but doesn't explain how they're different. Apparently it's because people can "scribble" "on her website". This makes News Genius "worse than Twitter". I guess if reddit created a browser plugin she'd hate it too?
>pasted in links to my tweets—which theoretically they should not have had access to, as I had blocked them both on Twitter
How can you block anyone on Twitter and think it'll work? Just sign out and block's gone?
>No one likes being questioned
I think that sums up her attitude. Note I think her work is fine, but she's literally complaining about people discussing her work on other sites.
Things like this don't become a problem until it is so easily organized. A bunch of tweets, FB posts, HN, reddit or other social comments are so scattered they don't have as much impact. The comments aren't visible on the the same page as the content.
I'm going to have to think more about this. I've had this thought a couple of times now but I don't have a full picture of it yet.
God forbid people can post organized opinions you disagree with...
http://genius.it/robots.txt
So this is the crux of it: Genius allows people to comment on stuff that people write - like an improved comment section. Occasionally these comments suck and are offensive.
From the blog post: "News Genius adds one more way for people on the Internet to be made unsafe." and "A tool that allows my abusive ex-boyfriend to interact with me and my content is a tool that should not exist."
The solipsism is laughable. The Internet will forever provide an infinite supply of trolls but when you publish on a public platform as opposed to behind a password protected blog or via email, people will talk about what you write (if you're lucky. many people publish in obscurity) and it just so happens that you may not always agree with what they say.
The reflex to shut down rather than, say, ignore dissenting opinion is an odd but increasingly more common one.
From our tech-centric perspectives, the value of a universal annotation platform seems self evident, but for a layman's perspective, it just seems like another toxic commenting layer that content creators can't moderate (which, in the ideal, it is). The fact that it's then collocated with their content, instead of, say, a click away on HN or twitter, makes it feel more violating.
The truly ideal platform would have massive adoption with benevolent power users that both prevented abuse and generated value adding content by verifying factual claims, adding appropriate context, etc. This ideal use case is pretty far from Genius's original platform (explaining rap lyrics in an entertaining fashion), and that mismatch of intents seems to be the source of the problem.
It's a blurry line for sure, but I think the motivation of silencing the 'riff-raff' is understandable.
Thanks for your comment. News Genius' site is a bit misleading, as there are a few different ways one could wind up with site annotations.
The first, as you mention, is by opting in. I agree that such a tool could be a useful way to foster discourse on your content. This tool is not meant for people who opt in to Genius annotations.
Further down on Genius' page, it states that any page can be annotated simply by prepending `genius.it/` to a site URL, regardless of whether the content publisher has granted permission. Unfortunately, the comments section of any Internet page tends to showcase some extremely uncivilized discourse. This is why many site authors choose to disable comments. Genius circumvents these people's wishes. Further, unlike a comments section, Genius does not actually notify the owner of a website if his/her content is annotated. Site owners have the option to moderate comments; it took Genius until today to even add a Report Abuse button.
Hopefully this clarifies some of the motivation. Thanks for sharing your perspective!
If you ask me, site owners should NOT have a right to moderate comments. That just encourages silencing dissent and disagreement and publishing misinformation, as others have noted.
Should site owners also have the right to issue takedown notices to reddit or hacker news when someone links to their site? Or to delete comments on those websites?
https://github.com/toddself/genius-blocker
Thanks for sharing. The gem works in a similar way, with a small JS script.
I originally intended to leverage Rack all the way through by inspecting headers. However, much to my chagrin, I discovered that Genius does not reveal its identity in the headers, so I had to resort to JS instead to accomplish my goal.
NewsGenius needs to at least 1) identify their crawler, and 2) provide a way to opt-out. Otherwise, it doesn't look good for them. They're not being a very good citizen of the Web.
https://github.com/vijithassar/genius-defender
Unlike the others (which prevent hosting on genius.it) this one actually prevents annotations via plugin etc by injecting invisible characters at random. Its a WP plugin but the method could be adapted.
Unsure if there are SEO/other implications.
More info: http://www.vijithassar.com/2641/how-to-block-genius-annotati...
Typically this would mean browser add-ons and extensions, but I wouldn't be surprised to discover other things (like headers injected by cellular companies, or who knows what from malware infections).
Such a tool would be useful when troubleshooting various 'works on my machine' scenarios.
I guess they will argue fair use, but hostile 'annotations' definitely reduce the value of the work, they reuse the entire work, and they use the work for commercial gain. I wouldn't consider this a slam dunk.
I actually really dislike this concept of "we're going to wrap your website and mark it up with other stuff to make it look like you're hosting this content here". Sure, people should be able to talk wherever they want, but keep that in a popup, or at worst a frame - don't highlight text to suggest that the author has done that, and don't load your own elements in their page.
The fact that they proxy the content on their servers definitely puts them in a more vulnerable position than if this were, for example, a strictly client-side chrome extension that fetched annotation metadata and re-rendered the page accordingly.
In the purely client-side case, the full article would not pass through their servers, only bits and pieces of it. It would be easier to argue fair-use if you're only storing parts of the articles, and comments/highlights, on a remote server, instead of proxying the entire content.
(What if the article is behind a paywall, for example? Then genius.it effectively becomes a circumvention proxy.)
For example it'd match "https://example.com/blog/how-to-block-genius.it" and redirect the user to a (probably) non-existent link with that string removed.
[1]: I just play one on HN.
Thanks for the code review - it's always good to see suggestions on how to improve things. If you have ideas on how to make the code better, please feel free to submit a pull request. Thanks for sharing your feedback!
But that's unlikely.
Based on what I read about it before I actually saw it in action, I had guessed that they were showing the original site in a frame served from the original site with their annotations outside that frame.
(That may or may not be legal. I'm not sure if it has been settled whether or not displaying someone's page served from their site in a frame served from your site might be a violation of the public display copyright right).
However, after looking at the genius.it page that was linked in another comment for part of this very thread, and also looking at the genius.it page for my own website, that's not the case. They appear to have made a copy of the base page itself and are serving that themselves. They don't seem to copy images and stylesheets--those are still being served from the original server.
In the copy of the page they serve, they modify the links contained therein to be genius.it links. That's making a derivative work.
I don't think Genius could argue successfully that this use is fair use. They are a commercial site, and they are copying the entire page from the original site.
If it doesn't already work that way it would be trivial to do so.