They are going to have a problem with standing for the exact reason you suggest. This case was one company who was scraping other people’s copyrighted works suing another company for doing the same.
Isn’t their main argument unfair competition? Google, the starting point of the internet, decided to undermine their business by taking their collated content and publishing it at the top of results?
Google appears to do this for other things, asking questions often shows answers without needing to visit the website. Perhaps these are all licensed and there is a kick back for these sites...
Google appear to be serving ads on content other people have collated while eliminating the source of traffic to the original site.. If that isn’t unfair business practice and taking advantage of their monopoly on search I don’t know what is.
But companies are absolutely allowed to do things that cause other companies to go out of business. It happens all the time. Think of the buggy whip manufacturers.
I agree that this Google practice looks dodgy to me. But the question is, what law specifically is being broken? This looks like a copyright case, and if that's the issue, then the copyright holder is the generally the one who has to bring the case in. (I believe there are exceptions such as when exclusive rights are granted, but I haven't seen a justification that they apply here.) That's what the law requires; otherwise the courts would be even more swamped.
If Google can scrape my site, am I allowed to scrape Google results? Could I create a Google clone by scraping?
If I scraped the most common search results from Google, front page only, and removed all the ads what would Google's argument against that be?
On one hand, so many sites make finding information difficult, on the other it feels pretty scuzzy that Google prevents searchers from clicking through to the site that put the work into generating content.
Yeah, the tricky thing for those scraped by google is that given google's search monopoly, the sites can't block their scraping entirely, since they need to be shown in search results.
The problem with the robots.txt "standard", e.g., ones like Google's with no "crawl-delay" directives, is that it does not define what is a "robot". The query above is obviously not a "robot", but Google, with all it resources, still treats as such.
Google probably does more (abusive) scraping than any other entity. Web scraping is in their DNA. It is in their web pages, too.
The amount of Google captchas that you needeed to solve when searching on Google from Microsoft office made me think that it was some kind of psychological warfare.
> Genius isn’t the copyright holder for these lyrics, it just licenses them itself.
Both Google and Genius are licensing the lyrics. Ironically, Genius ended up having to settle a case years ago because they were using lyrics without the appropriate licensing [1].
I think the legal argument services like serpapi make is that as long as you don't create a google account and/or accept google's terms then you are free to scrape and clone what is publicly accessible (at least in the US). I have no idea though.
"Scraping" was the wrong word. I was remembering incorrectly. Microsoft did not have to scrape. They were apparently using search data captured through Internet Explorer. There was a time before Google had control of the browser. This illustrates how companies with browsers can gather data about users' web activity and how far they can go. Today, even Firefox is gathering data about users' activity with "telemetry", and users typing things into the browser's search box are by default sending this data to Google.
Genius claims that Google’s actions caused a decline in traffic to its site. The lawsuit was probably a way to assuage nervous investors (who have poured >70M into the company)
So genius scraped all the other lyric sites that came before it, and now they are mad that Google is stealing the lyrics genius stole from other sites... ok.
Clearly if you come up with a b.s. site like genius you'll have to deal with that type of stuff at some point. Can't get mad if Google is profit off stealing from you while you steal from other sites.
Techcrunch article left too many questions for me so I looked up and found https://www.rollingstone.com/music/music-features/genius-law... which seems to be a much more detailed and nuanced explanation of what is going on here, with backgrounds on who, what, etc.
They license lyrics however and, critically, often times lyrics and their copyright are not a package deal. many songs don't have lyrics provided from the publisher. Genius allows users and artists to upload the lyrics. Google has the copyright but blatantly steals the content. This obviously is not protected in US law but it is elsewhere (see news sites and such in some countries requiring google to pay). Google then uses their search results to leverage their position.
> Which means they are only not OK with scraping when Google uses the scrapped for purpose they deemed reduced their traffic
Obviously. The way the profit model for the internet works right now, for sites to coexist with Google, they must actually receive some of the traffic that is generated from searches matching their content. Who would be okay with having all of their content scraped with the result being that they get none of the traffic and thus the monetary benefit from the work they do?
Because they'll block you. You can prevent Google from indexing your content using robots.txt (Google has a robots.txt on its site as well).
You don't have a right to access their service as many times as you want to, eg by automated means, although you can attempt it. Flip a coin on whether they sue to stop you if you become too annoying.
The Genius complaint is essentially that they want to be represented in Google search without having Google take lyrics from their service and use them in their own served-up content snippets (making a sizable part of the value of genius.com void). Genius knows Google can get lyrics elsewhere if they have to, the lawsuit is probably out of spite due to past conflict with Google and their annoyance at Google competing with them in a shady way (Google was de facto using Genius's service to reduce the value of Genius).
I wonder if it's even possible to fix Google search in the framework of a for-profit company. It seems like the trajectory of any ad-supported service eventually lands it in a "don't let the user out no matter the cost" phase. Perhaps such a service really does need to operate as a non-profit foundation of some sorts.
There was a post about regulating Google like a public utility recently, but perhaps we should also consider looking at other less conventional internet "public utilities" - things like the Internet Archive, Wikipedia or essential open source projects like Debian. I think a search engine that's transparent both in terms of its logic and how it's maintained and managed might be the only way.
It's not Genius' content. It's the musicians', who actually wrote the lyrics. If it was their content, they could sue for copyright infringement. Google has been sued on those grounds, and forced to change how they do their business (e.g. provide ContentID to handle copyright infringement on YouTube).
But I suppose we could say "don't scrape and present content outside of a regular search result". But then again, Google claims they haven't done so - that they got those lyrics from LyricFind, a lyric licensing platform - and Genius didn't present any evidence that this wasn't the case. So I'm not clear on how could any laws help here.
Finally, the question was not about Genius, it was about allowing competitors to Google to emerge. I don't see how would this help.
It’s not just copyright. Because of Google’s monopoly what they are doing is stealing a revenue stream from Genius. This is another case of Google leveraging their search monopoly.
They need to be declared a monopoly and put on a very short leash here. I hope something comes of the hearings in Washington late last month.
I am not following your line of reasoning correctly.
The watermarking is done using characters that look different from each other. Printing the book will not change those characters, and therefore it is still possible to extract the original information. Harder, but not impossible.
For this information to be lost, Evince would have to print the book in a crappy font that uses a single symbol for both. Which is possible, but I think it's not what you were going for.
The hash won't be the same, which is what matters the most.
They won't have people manually reviewing the punctuation to catch infringement, and it's highly probable that Evince printing to PDF will mess the whole internal structure, so it's hard to automate it.
If you just print it then that defeats the point. They don't care about individual copies, they want to find who is stripping DRM and posting books on torrent trackers.
How likely are you to notice if your resulting PDF had a specific unique pattern of zero-width spaces embedded with it? Or if a few curly quotes aren't curly? Let alone scenarios like the cover JPEG having stenographic info in it...
Absolutely. It usually tends to be either more visible, or less visible though - some pdf files have a literal watermark on the pages, while other formats like epub contain a guid or other watermark content in the source (epub is zipped xhtml)
Amusingly, I believe if you use calibre's ebook conversion to replace stylesheets and add toc, it may also actually remove those markers that have no actual content and only exist to provide a unique ID.
If I was going to scrape this data and re-purpose it, I would've absolutely cleaned up those apostrophes. The pivoting between straight and curly would certainly be a pet peeve. Unless there's a semantic difference between the two I'm unaware of.
The song was released almost a year before UTF-8 was first presented at USENIX, so I think it is reasonable for the lyrics to be expressed using the commonly available technology available of the time.
I queried for this song's lyrics, and I found only a single quoting symbol (') used entirely throughout.
Taking a layperson's reading of the lyrics, I didn't find anything off due to quoting issues. If there was any disruption, it was minimal and unnoticed.
Yeah, makes sense, but this is still a pretty good approach. Inserting invisible or unusual Unicode symbols would prompt the scraper to carefully cleanup the read files (maybe even fixing these apostrophes as a result). Unusual whitespace is also likely to be removed and cleaned up.
On the other hand, these alternating apostrophes have a chance to stay unnoticed (or neglected), falling through the cracks.
On this quantity of data, you wouldn't be able to do this manually.
If you hope to avoid being caught this way, I'm going to assume you noticed this without the benefit of hindsight and plan to correct all out-of-place Unicode characters automatically. How will you avoid over-correcting?
There's also no reason to believe this is the only fingerprinting Genius has done (they only need to publish the most obvious fail). For example, I can use the same fingerprinting technique but switch between American and British spellings.
I think if you had multiple corpuses of lyrics, you could cross-check for anomalies of any variety (odd quoting, switching between american/british english), etc.
The fingerprinting isn't likely applied to every song, to prevent obvious detection. If you went through multiple databases, you might see N prevailing copies of a song's lyrics, and 1 that seemed different. The one that's different has the anomaly.
There is a semantic difference between the two. The straight quote is a superset of the curly one.
So "rock 'n' roll" is correct. And "rock ’n’ roll" is correct. But "rock ‘n’ roll" is not correct, since the wrong apostrophe is used. We're not quoting the letter n, we're showing that the letter a was removed.
(Software patents should be abolished. I just like to point out their absurdity and how it's easy to independently develop a technique (steganography in a search engine result) that someone has already grubbed a "patent" on.)
That is just so bizarre to me too. That one would be able to copy the exact core of the idea, but write it in a different language, and suddenly it's not infringing, even though it causes the computer to do the same thing.
How these patents even pass the sniff test, I'll never understand.
> while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. _Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims._
If I'm reading this correctly, this patent is claiming things that are "apparent" (obvious?) to those "in the know". Computer or not, how did this get granted?
I posted about this the other day and someone who used to work at Microsoft made a really interesting/helpful response - worth checking out: https://news.ycombinator.com/item?id=24112418
There's no "benefit", they were just looking for a nice unique way of watermarking textual content, to prove that what shows up on a Google search is indeed sources from them and not some other transcription of the lyrics.
And they definitely do it with maps. There is a tiny little village I visit in rural Roscommon each year. Each year a new major retailer appears to have opened in this 500 population village, well according to Google Maps that is. At the moment there is a branch of New Look situated on a farm down a single track country lane.
I recall coming across this in my travels. There was a named "town" at the intersection of two streets - upon passing through there, nothing. Later wondering where the town went I found that it was not ever there and was just present to identify people copying that map.
I'm not disputing that they proved their point, but this is triggering one of my pet peeves about common misunderstandings of Morse code.
Timing is critical in Morse code. You can't just write out a bunch of dashes and dots to transcribe it without clearly transcribing the rests between dots and dashes as well. They haven't given us the rests at all, so all the info they end up having is:
And that can be interpreted in any number of different possible ways besides "REDHANDED". E.g. it could also be "AU5EWRFE", or any of thousands of different interpretations (actually probably a lot more than that; this would be a fun programming problem). They should have used a binary encoding; 22 bits (all they have given us) is not enough information to uniquely encode the string "REDHANDED". Once you include the short rests that are needed, we're talking 44 binary bits or 22 ternary bits. And if you want the long rests to distinguish properly the spaces between words, then 22 ternary bits won't do it; you need the full 44 binary bits.
>They should have used a binary encoding; 22 bits (all they have given us) is not enough information to uniquely encode the string "REDHANDED".
The fact that the sequence can be interpreted as REDHANDED with a particular way of grouping the input is just being cute. Regardless of the grouping, it is a binary encoding of a 22-bit number, and so would have a one-in-2^22 chance of being reproduced at random.
Edit: To clarify: You're saying they should've mentioned 22-bits in the context of binary digits without mentioning Morse code, and if they did want to bring up Morse code they should've used trits or more bits to encode the stops. I'm saying that the fact that their 22-bit sequence can be interpreted in Morse code as a relevant word is just dressing, and does not detract from the point that the sequence was likely copied. Put another way, if someone tried to counter by saying their sequence could've been generated independently because "AU5EWRFE" and many other strings also encode to the same sequence, it would not affect the facts at all.
There is a misunderstanding because anyone who properly understood Morse code would never represent it in this manner where it's indistinguishable from a large number of other possible strings.
There is no misunderstanding because it was never the intention to use the bitstring to represent a unique word in Morse code in the first place. Instead, Morse code was merely used as a convenient method to devise a bitstring from a relevant word.
Just a thought experiment: If Google releases paid version of search where it won't show ads, personalize based on your click-through history and do not copy content from source for presentation, won't share your data with anyone, would you opt for it?
Root cause of most of the issues seem to be monetization model of Google where they are optimising people to stay within their ecosystem.
I'd like to be able to remove the ads, but I don't care about the rest of it. Besides removing ads, the other things you mention would make the service less valuable for me.
It's an interesting question. I don't trust Google, and a fast pivot to another product/offering wouldn't change that overnight, so I would lean towards no.
I paid for Youtube Red, even though I never signed into the service on Youtube and even though I already got all of the Youtube-specific benefit in the form of adblocking and NewPipe. I did that purely to try and signal to Google, "I will pay for content, I want you to have revenue sources outside advertising."
But ultimately Youtube Red seems to have been a failure, my signal hasn't changed the course of the company, and the parts of Youtube Red I did get value from (say, Music) have gotten noticeably worse over the years. I'm planning to drop my subscription in September. I think it would be tough for me to buy into another product like that from Google, my experience trying to buy products from Google to get around advertising/tracking has been both a practical and moral failure. Certainly I won't sign up for a paid GSuite account now.
But I do already pay for Email (Fastmail), bookmarking (Wallabag), and a few other "free" internet services and apps today. So I might pay for search if a company like DuckDuckGo offered it instead of Google. But I would need to see what their offer actually was.
I mean, heck, I'd be giving recurring donations to DuckDuckGo today if they accepted them, they're on my list of companies I want to exist. So paying 5$ a month for a "premium" search experience wouldn't really be that different, even if I never signed into it.
> I mean, heck, I'd be giving recurring donations to DuckDuckGo today if they accepted them, they're on my list of companies I want to exist. So paying 5$ a month for a "premium" search experience wouldn't really be that different, even if I never signed into it.
Different people have different reasons for disabling ads. Some people only care about egregious ads and privacy violations, that's fine.
I am against ads in general, I noticed a sharp quality of life increase when I started blocking ads universally everywhere I could, regardless of whether or not I was on the web. That's a longer conversation, I'm not going to get into it now. I respect that other people have different opinions, but I also feel reasonably strongly about my own.
I do spread DuckDuckGo to other people, but I also spread uBlock Origin, so I'm not sure I'm a net positive there either. :)
It's again, an interesting question. I've been pushing pretty hard in my personal life to financially support Open Source software that I use, projects that I really care about. DuckDuckGo is one of the few companies that I really like that I haven't ever really supported in any tangible way.
I do feel guilty about that, but not guilty enough to allow myself to be turned into a product. There is likely no company where I would ever feel guilty enough to turn on ads. But given that DuckDuckGo isn't going to allow donations any time soon, I could see a lightweight 'premium' DuckDuckGo product effectively being a way for me to just give them money without it feeling to them like it's a donation.
Actually, what I should do is to look into some of their merch and see what they offer and if they actually make a profit on it. I thought at one point DuckDuckGo sold shirts or something, but I don't know if they still do. Again though, this all kind of ends up being a messy proxy for donations. I'm not super-jazzed about walking around as a living billboard either, even for a company I like.
The difference in amounts between what Google makes per 1000 searches and what a profitable/sustainable search engine needs to make are quite vast.
Can't remember the source but G make around ~$70 per 1000 searches. Bing make less than half than this, DDG less again.
G commands higher rates because of higher competition for ads and also because of its retargeting and all the other methods that raise that per 1000 figure higher.
To answer your question (sort of), Google's non-ad version would need to be expensive to break even with its current model.
Google has a history of scraping content that they want, their business is built on the back of scraping other peoples content. The story I read just recently of what happened to Celebrity Net Worth was an interesting read where Google asked for an API, they refused and Google just scraped the content anyway. There was no lawsuit, but CNW put up fake content and sure enough, it made its way to Google.
It is all ironic given how aggressive Google are in blocking any attempts to scrape its content.
We live in a society of laws. Even soldiers. Google have shown they have no respect for the law not equality before it and will cheat while using the law as a cudgel. Recall law exists that the strongest might not always get their way. "Ironic" is the pole way of pointing this out.
Without law, Google cease to exist immediately. They are incapable of enforcing property rights without it.
Pardons aside, soldiers go to jail for taking an attitude like Google's.
Just like Genius, Google licensed the lyrics. If they didn't, the publishers definitely would have sued.
Ironically, it is Genius that seems to have no respect for copyright law. Genius ended up having to settle a case years ago because they were using lyrics without the appropriate licensing [1].
Which law did google break? Scraping in and of itself isn't illegal last time i checked, and usa doesn't have database copyrights unlike some juridsictions.
> somewhat, per things like the Americans with Disabilities Act
This is just not right at all. There is nothing in the Americans with Disabilities Act that make blocking scrapers illegal.
I think you mean you don't like the power imbalance of the large company taking away from smaller companies while using technological means to stop the same thing happening to them.
I don't like it either, but that doesn't magically make it is illegal. I'm not even sure it should be.
> There is nothing in the Americans with Disabilities Act that make blocking scrapers illegal.
Retrieving, processing, and displaying information in a manner contrary to the wishes of the provider of that information is necessary for accessibility to disabled users. As a specific example, any attempt to block use of wget for scraping also blocks use of wget as part of a `wget | filter | text-to-speech` pipeline[0], and is thus a discrimination against blind or otherwise visually impaired users. The ADA is, as mentioned, only somewhat effective in prohibiting such things, though.
> it's not illegal
> that doesn't magically make it is illegal.
I don't think anyone is claiming that scraping itself actually is legally protected - I interpreted DigitalSea and harry8 as implying that it should be.
0: in either the shell sense or the workflow sense
Retrieving, processing, and displaying information in a manner contrary to the wishes of the provider of that information is necessary for accessibility to disabled users. As a specific example, any attempt to block use of wget for scraping also blocks use of wget as part of a `wget | filter | text-to-speech` pipeline[0], and is thus a discrimination against blind or otherwise visually impaired users. The ADA is, as mentioned, only somewhat effective in prohibiting such things, though.
This is not the case. Unfortunately (?) the ADA doesn't allows the disabled person to specify their own technology. If Google can reasonably say that speech to text works via a standard screenreader (which it does) then they are ok.
> The ADA is, as mentioned, only somewhat effective in prohibiting such things, though
Well that's not the intent of the ADA, so not really surprising.
I’d say most of Genius’ visitors comes from the “song x lyrics” so hiding those with robots would ultimately make them lose almost all of their traffic.
robots.txt is designed to keep garbage off search results. It has absolutely no power to prevent a bot to do anything. Also if the site added robots.txt they might as well shut down because their entire userbase comes from people searching lyrics on google.
To be fair to TC, if you disable JavaScript you get a pretty good experience - just the full article, legible. Not like those sites that require JS to load the text and/or images.
The problem is that Google is stealing content and placing it on search so the user never goes to the source, By blocking it with robots they block themselves from google results AND Google may already keep scraping the content.
> The Unicorn tier is for large companies or companies that would like to have a reciprocal relationship with our foundation. If you need special guarantees, indemnities or require us to sign your contract for a data license, please select this tier. If you have another creative idea you would like to propose, please also select the unicorn tier.
> For any of these cases, please detail your request in the company information field and we will work with you to fit your company's mythical situation. We will also find an appropriate monthly support amount to our non-profit foundation of $1500 or more per month. Please always consider enabling the growth of our non-profit foundation and the continuous growth of our metadata!
The crux of the legal decision here is that Genius doesn't own the copyright on the lyrics either. Hard to steal from Genius what they don't own in the first place.
Genius does give the tools to crowdsource lyrics, meanings, and comments. Google brought the rights to display lyrics but not lyrics themselves..they just steal that from other companies and rehost it with their own ads. Copyright law was not designed to handle this, which is something that is obviously morally wrong and will eventually harm consumers. Google has stated: 'if you don't want to be crawled, use Robots.txt' which because they have 90% market share, is clearly impossible for Genius. It's downright evil.
Downright evil seems like a stretch, especially given that Google doesn’t seem to have gotten the lyrics from scraping Genius and rather bought them from a third party who themselves scraped them from Genius. Whether or not Genius blocks googlebot in their robots.txt doesn’t actually seem to be relevant. But this does seem like a good case for the US to introduce “database rights” into copyright law to reward entities that assemble collections of otherwise-disparate information that they don’t own the copyright for, either because someone else does it because it’s something that cannot be copyrighted. Then Genius would have legitimate grounds to sue the third-party that scraped their lyrics. On the other hand, this would also had it existed have allowed Google to sue Bing back when it was first starting for piggybacking on Google’s search results. It’s not obvious to me that database rights are a good idea.
From genius.com: "Genius Media Group, Inc. (GMG) is fully licensed to display lyrics across all of its properties. In 2013, GMG entered into licenses with every major music publisher: Sony/ATV Music Publishing, EMI Music Publishing, Universal Music Publishing Group, and Warner/Chappell Music. In addition, GMG developed a form license with the National Music Publishers' Association (NMPA) which today covers more than 96% of the independent publisher market."
Original copyright holder could give someone else authorisation to sue on their behalf, e.g., through an assignment. Doubtful Genius got an assignment in the agreements they have with publishers.
Also, Google claimed it is sub-licensed to re-publish through a third party, LyricFind, which has licenses with "over 4000" music publishers.
>Copyright holder could give someone else authorisation to sue on their behalf, e.g., through a license.
They can't assign the bare right to sue. To have standing the plaintiff will need to hold at least one of the exclusive rights in 17 U.S. Code § 106 aiui. Cf Righthaven cases, Silvers v Sony Pictures
I am almost certain you are correct, I would be shocked if genius was granted an exclusive right to lyrics. If nothing else, how could you sell the songs without rights to the lyrics?
So I don't think genius ever had standing to pursue this case.
Again, not a lawyer. But you'd think the actual lawyers would have checked this more carefully.
> Defendants made unauthorized reproductions of Plaintiff’s lyric transcriptions and profited off of those unauthorized reproductions, which is behavior that falls under federal copyright law.
Does this mean Genius still has grounds to sue, as the copyright protections are on their “work” which is the transcription?
It also seems to mean a ToS is not useful for protecting content, only determining legal users interaction with it (Excluding loading the page). A webpage is a work rendered through a browser, kinda makes sense I guess.
I think that’s their case to make, but from what I could find the answer seems to be probably not.
Derivative works are copyrightable (Translations, adapting a book to a movie), but it might be hard to argue an exact copy of the song lyrics are “derivative” as they aren’t an original creation, but simply a subset of the old work.
Rich snippets are all stolen pageviews. To make matters worse, they now don't let the site which is featured in the snippet have a secondary ranking further down the page.
They keep adding schema, which SEOs eat up because it provides a secondary level of content optimization but it's not as effective as it used to be for increasing CTR on your serp results.
The one positive of this is I no longer have to goto recipe sites which overload my screen with ads and put the ingredients 50% down the page. Google just scrapes the content and displays it in the snippet box.
You very well can, but that doesn't mean Google can't block you (CFAA protection). Genius here was reliant on Google for a large portion of their regular traffic so they couldn't just block Google without suffering revenue losses.
> LyricFind. LyricFind is a Google licensing partner, and may be the source of the Genius content appearing in Google’s search results. LyricFind published an explanation on its web site Monday, saying, “Some time ago, Ben Gross from Genius notified LyricFind that they believed they were seeing Genius lyrics in LyricFind’s database. As a courtesy to Genius, our content team was instructed not to consult Genius as a source. Recently, Genius raised the issue again and provided a few examples. All of those examples were also available on many other lyric sites and services, raising the possibility that our team unknowingly sourced Genius lyrics from another location. As a result, LyricFind offered to remove any lyrics Genius felt had originated from them, even though we did not source them from Genius’ site. Genius declined to respond to that offer. Despite that, our team is currently investigating the content in our database and removing any lyrics that seem to have originated from Genius.”
If that flies, it's a great tactic. You can't just use the data from site A, so you build anonymous sites B, C and D who use the data from site A, and use the data from those sites instead. "We didn't source from A".
It's not a great tactic if the owner of site A decides to sue you and subpoenas the hosting providers for B, C and D. You can only get away with it as long as you aren't successful enough to draw the attention of anyone with money to burn on legal fees.
Last I checked, ignorance wasn't usually a defense but I'm not a lawyer. I just know not to pretend the Keurig I bought off the back of a truck was a good deal for everyone involved.
But physical analogies to IP fall apart quickly so I'm not going to encourage people to read into that too deeply
So there's two different concepts. One is original creative output, which is copyrightable. The other is information, which is not copyrightable.
If you find something verbatim identical in a bunch of different places, you've got a strong case that it's just information, because if it were original creative output it wouldn't show up identically in multiple places.
If it turns out everyone was plagiarizing a single source, but you were unaware and took down the offending content when asked, you won't have much in the way of legal liability.
Sounds like everyone and their mother is scraping stuff off Genius, not just Google; they went after Google specifically because they knew they couldn't just disappear and they had the financial means to pay for compensation, unlike the thousands of crappy lyrics websites.
That said, it would've been just if Google would pay for access to Genius' particular, well-curated, "source" database of lyrics, especially given that they're basically stealing traffic.
But it sounds like the issue is that Google really wasn't using Genius's data directly. The problem is that Google is sourcing from "The Internet," and everybody and their grandmother is 'stealing' from Genius.
Here's an interesting question: if Genius closed up shop tomorrow, how long would it take Google to become the primary source of song lyrics online (by rebuilding Genius's dataset from general Internet harvesting)?
This was confusing to me as well. I saw EDNY and was confused about why it was booted due to federal preemption. Considering the importance of this procedural aspect, it seems like a pretty big deal to get this wrong.
The real story here is left completely untold: why didn't they bring a copyright claim? Could they bring a copyright claim in the future? Could one of the owners of the copyright bring a claim? These are the questions that matter.
300 comments
[ 6.2 ms ] story [ 299 ms ] threadGoogle appears to do this for other things, asking questions often shows answers without needing to visit the website. Perhaps these are all licensed and there is a kick back for these sites...
Google appear to be serving ads on content other people have collated while eliminating the source of traffic to the original site.. If that isn’t unfair business practice and taking advantage of their monopoly on search I don’t know what is.
"Unfair competetion" has a specific legal meaning. As I understand primarily in the US it's under state law. Here's a summary: https://www.law.cornell.edu/wex/unfair_competition
I agree that this Google practice looks dodgy to me. But the question is, what law specifically is being broken? This looks like a copyright case, and if that's the issue, then the copyright holder is the generally the one who has to bring the case in. (I believe there are exceptions such as when exclusive rights are granted, but I haven't seen a justification that they apply here.) That's what the law requires; otherwise the courts would be even more swamped.
Again, I'm not a lawyer.
If I scraped the most common search results from Google, front page only, and removed all the ads what would Google's argument against that be?
On one hand, so many sites make finding information difficult, on the other it feels pretty scuzzy that Google prevents searchers from clicking through to the site that put the work into generating content.
Your idea to scrape Google's search results is pithy, ironic counter-innovation at its dastardly best.
All you need to pull this off is funding for a top legal team, and deep reserves of emotional energy.
Go for it!
For example, if you include a User-Agent header and put certain strings in it, e.g., "curl/7.47", you will be blocked.
The problem with the robots.txt "standard", e.g., ones like Google's with no "crawl-delay" directives, is that it does not define what is a "robot". The query above is obviously not a "robot", but Google, with all it resources, still treats as such.Google probably does more (abusive) scraping than any other entity. Web scraping is in their DNA. It is in their web pages, too.
https://www.wired.com/2011/02/bing-copies-google/
> Genius isn’t the copyright holder for these lyrics, it just licenses them itself.
Both Google and Genius are licensing the lyrics. Ironically, Genius ended up having to settle a case years ago because they were using lyrics without the appropriate licensing [1].
[1] https://www.nytimes.com/2014/05/07/business/media/rap-genius...
You are alllowed. Google would not likely try to sue you. They will try to block you however.
Bing was created by copying Google results. Google did not sue Microsoft, but they did try to expose the copying.
Do you have a source? Just curious about the back story.
https://searchengineland.com/google-bing-is-cheating-copying...
https://www.wired.com/2011/02/bing-copies-google
https://news.ycombinator.com/item?id=24112418
Not excusing MS, but it seems they were not wholesale copying Google results.
This is incorrect. Firefox will prompt you whether to enable search suggestions on first use, and will not fetch suggestions until you say yes.
Still, Google is being very fucking evil here. It's as if they stole that $70M for themselves.
Your abuse of words is a heinous crime against humanity.
In Genius's case, does it disallow Google for scraping?
From their robots.txt, I can't tell:
https://genius.com/robots.txt
Genius is given a way out to prevent itself from being scrapped.
But it doesn't. Mean they probably value traffic from Google.
Which means they are only not OK with scraping when Google uses the scrapped for purpose they deemed reduced their traffic
Obviously. The way the profit model for the internet works right now, for sites to coexist with Google, they must actually receive some of the traffic that is generated from searches matching their content. Who would be okay with having all of their content scraped with the result being that they get none of the traffic and thus the monetary benefit from the work they do?
https://genius.com/mecha
You don't have a right to access their service as many times as you want to, eg by automated means, although you can attempt it. Flip a coin on whether they sue to stop you if you become too annoying.
The Genius complaint is essentially that they want to be represented in Google search without having Google take lyrics from their service and use them in their own served-up content snippets (making a sizable part of the value of genius.com void). Genius knows Google can get lyrics elsewhere if they have to, the lawsuit is probably out of spite due to past conflict with Google and their annoyance at Google competing with them in a shady way (Google was de facto using Genius's service to reduce the value of Genius).
There was a post about regulating Google like a public utility recently, but perhaps we should also consider looking at other less conventional internet "public utilities" - things like the Internet Archive, Wikipedia or essential open source projects like Debian. I think a search engine that's transparent both in terms of its logic and how it's maintained and managed might be the only way.
Google shouldn't steal Genius or Yelp content. Facebook shouldn't steal YouTuber's content, etc. etc.
It's not hard to see how if you were in the little guys shoes you'd feel screwed over. We need to innovate our laws to reflect that.
But I suppose we could say "don't scrape and present content outside of a regular search result". But then again, Google claims they haven't done so - that they got those lyrics from LyricFind, a lyric licensing platform - and Genius didn't present any evidence that this wasn't the case. So I'm not clear on how could any laws help here.
Finally, the question was not about Genius, it was about allowing competitors to Google to emerge. I don't see how would this help.
They need to be declared a monopoly and put on a very short leash here. I hope something comes of the hearings in Washington late last month.
https://imgur.com/IGs0sg7
The way to check this is pretty easy though, get 2 users to bit for bit compare their books to work out if they are identical.
The watermarking is done using characters that look different from each other. Printing the book will not change those characters, and therefore it is still possible to extract the original information. Harder, but not impossible.
For this information to be lost, Evince would have to print the book in a crappy font that uses a single symbol for both. Which is possible, but I think it's not what you were going for.
They won't have people manually reviewing the punctuation to catch infringement, and it's highly probable that Evince printing to PDF will mess the whole internal structure, so it's hard to automate it.
https://gizmodo.com/harpercollins-is-now-using-digital-water...
https://goodereader.com/blog/e-book-news/everything-you-need...
Amusingly, I believe if you use calibre's ebook conversion to replace stylesheets and add toc, it may also actually remove those markers that have no actual content and only exist to provide a unique ID.
Taking a layperson's reading of the lyrics, I didn't find anything off due to quoting issues. If there was any disruption, it was minimal and unnoticed.
If you hope to avoid being caught this way, I'm going to assume you noticed this without the benefit of hindsight and plan to correct all out-of-place Unicode characters automatically. How will you avoid over-correcting?
There's also no reason to believe this is the only fingerprinting Genius has done (they only need to publish the most obvious fail). For example, I can use the same fingerprinting technique but switch between American and British spellings.
This is not a straightforward problem.
The fingerprinting isn't likely applied to every song, to prevent obvious detection. If you went through multiple databases, you might see N prevailing copies of a song's lyrics, and 1 that seemed different. The one that's different has the anomaly.
So "rock 'n' roll" is correct. And "rock ’n’ roll" is correct. But "rock ‘n’ roll" is not correct, since the wrong apostrophe is used. We're not quoting the letter n, we're showing that the letter a was removed.
[1]: https://patents.google.com/patent/US9881516B1/en
(Software patents should be abolished. I just like to point out their absurdity and how it's easy to independently develop a technique (steganography in a search engine result) that someone has already grubbed a "patent" on.)
The patent claim requires the program to "load available PHP server header information"
How these patents even pass the sniff test, I'll never understand.
> while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. _Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims._
If I'm reading this correctly, this patent is claiming things that are "apparent" (obvious?) to those "in the know". Computer or not, how did this get granted?
https://www.urbandictionary.com/define.php?term=hiybbprqag
https://www.cbsnews.com/news/hiybbprqag-how-google-tripped-u...
E.G. When they caught Bing copying them... https://www.wired.com/2011/02/bing-copies-google/
And they definitely do it with maps. There is a tiny little village I visit in rural Roscommon each year. Each year a new major retailer appears to have opened in this 500 population village, well according to Google Maps that is. At the moment there is a branch of New Look situated on a farm down a single track country lane.
I recall coming across this in my travels. There was a named "town" at the intersection of two streets - upon passing through there, nothing. Later wondering where the town went I found that it was not ever there and was just present to identify people copying that map.
Timing is critical in Morse code. You can't just write out a bunch of dashes and dots to transcribe it without clearly transcribing the rests between dots and dashes as well. They haven't given us the rests at all, so all the info they end up having is:
dot dash dot dot dash dot dot dot dot dot dot dot dash dash dot dash dot dot dot dash dot dot
And that can be interpreted in any number of different possible ways besides "REDHANDED". E.g. it could also be "AU5EWRFE", or any of thousands of different interpretations (actually probably a lot more than that; this would be a fun programming problem). They should have used a binary encoding; 22 bits (all they have given us) is not enough information to uniquely encode the string "REDHANDED". Once you include the short rests that are needed, we're talking 44 binary bits or 22 ternary bits. And if you want the long rests to distinguish properly the spaces between words, then 22 ternary bits won't do it; you need the full 44 binary bits.
The fact that the sequence can be interpreted as REDHANDED with a particular way of grouping the input is just being cute. Regardless of the grouping, it is a binary encoding of a 22-bit number, and so would have a one-in-2^22 chance of being reproduced at random.
Edit: To clarify: You're saying they should've mentioned 22-bits in the context of binary digits without mentioning Morse code, and if they did want to bring up Morse code they should've used trits or more bits to encode the stops. I'm saying that the fact that their 22-bit sequence can be interpreted in Morse code as a relevant word is just dressing, and does not detract from the point that the sequence was likely copied. Put another way, if someone tried to counter by saying their sequence could've been generated independently because "AU5EWRFE" and many other strings also encode to the same sequence, it would not affect the facts at all.
>but this is triggering one of my pet peeves about common misunderstandings of Morse code.
and my post is about how there is no misunderstanding.
Root cause of most of the issues seem to be monetization model of Google where they are optimising people to stay within their ecosystem.
I paid for Youtube Red, even though I never signed into the service on Youtube and even though I already got all of the Youtube-specific benefit in the form of adblocking and NewPipe. I did that purely to try and signal to Google, "I will pay for content, I want you to have revenue sources outside advertising."
But ultimately Youtube Red seems to have been a failure, my signal hasn't changed the course of the company, and the parts of Youtube Red I did get value from (say, Music) have gotten noticeably worse over the years. I'm planning to drop my subscription in September. I think it would be tough for me to buy into another product like that from Google, my experience trying to buy products from Google to get around advertising/tracking has been both a practical and moral failure. Certainly I won't sign up for a paid GSuite account now.
But I do already pay for Email (Fastmail), bookmarking (Wallabag), and a few other "free" internet services and apps today. So I might pay for search if a company like DuckDuckGo offered it instead of Google. But I would need to see what their offer actually was.
I mean, heck, I'd be giving recurring donations to DuckDuckGo today if they accepted them, they're on my list of companies I want to exist. So paying 5$ a month for a "premium" search experience wouldn't really be that different, even if I never signed into it.
https://help.duckduckgo.com/duckduckgo-help-pages/company/do...
They seem to want you to enable their ads. Is that something you would ever do?
Different people have different reasons for disabling ads. Some people only care about egregious ads and privacy violations, that's fine.
I am against ads in general, I noticed a sharp quality of life increase when I started blocking ads universally everywhere I could, regardless of whether or not I was on the web. That's a longer conversation, I'm not going to get into it now. I respect that other people have different opinions, but I also feel reasonably strongly about my own.
I do spread DuckDuckGo to other people, but I also spread uBlock Origin, so I'm not sure I'm a net positive there either. :)
It's again, an interesting question. I've been pushing pretty hard in my personal life to financially support Open Source software that I use, projects that I really care about. DuckDuckGo is one of the few companies that I really like that I haven't ever really supported in any tangible way.
I do feel guilty about that, but not guilty enough to allow myself to be turned into a product. There is likely no company where I would ever feel guilty enough to turn on ads. But given that DuckDuckGo isn't going to allow donations any time soon, I could see a lightweight 'premium' DuckDuckGo product effectively being a way for me to just give them money without it feeling to them like it's a donation.
Actually, what I should do is to look into some of their merch and see what they offer and if they actually make a profit on it. I thought at one point DuckDuckGo sold shirts or something, but I don't know if they still do. Again though, this all kind of ends up being a messy proxy for donations. I'm not super-jazzed about walking around as a living billboard either, even for a company I like.
I'd happily pay 10€/m for better results though.
Can't remember the source but G make around ~$70 per 1000 searches. Bing make less than half than this, DDG less again.
G commands higher rates because of higher competition for ads and also because of its retargeting and all the other methods that raise that per 1000 figure higher.
To answer your question (sort of), Google's non-ad version would need to be expensive to break even with its current model.
That alone makes too different an experience.
It is all ironic given how aggressive Google are in blocking any attempts to scrape its content.
It's just the war that is being fought, not some sort of hypocrisy or irony.
We live in a society of laws. Even soldiers. Google have shown they have no respect for the law not equality before it and will cheat while using the law as a cudgel. Recall law exists that the strongest might not always get their way. "Ironic" is the pole way of pointing this out.
Without law, Google cease to exist immediately. They are incapable of enforcing property rights without it.
Pardons aside, soldiers go to jail for taking an attitude like Google's.
Ironically, it is Genius that seems to have no respect for copyright law. Genius ended up having to settle a case years ago because they were using lyrics without the appropriate licensing [1].
https://www.nytimes.com/2014/05/07/business/media/rap-genius...
(And to head off the obvious: rate-limiting is orthogonal to whether the high-request-rate querient is scraping.)
But.. it's not illegal?
> somewhat, per things like the Americans with Disabilities Act
This is just not right at all. There is nothing in the Americans with Disabilities Act that make blocking scrapers illegal.
I think you mean you don't like the power imbalance of the large company taking away from smaller companies while using technological means to stop the same thing happening to them.
I don't like it either, but that doesn't magically make it is illegal. I'm not even sure it should be.
Retrieving, processing, and displaying information in a manner contrary to the wishes of the provider of that information is necessary for accessibility to disabled users. As a specific example, any attempt to block use of wget for scraping also blocks use of wget as part of a `wget | filter | text-to-speech` pipeline[0], and is thus a discrimination against blind or otherwise visually impaired users. The ADA is, as mentioned, only somewhat effective in prohibiting such things, though.
> it's not illegal
> that doesn't magically make it is illegal.
I don't think anyone is claiming that scraping itself actually is legally protected - I interpreted DigitalSea and harry8 as implying that it should be.
0: in either the shell sense or the workflow sense
This is not the case. Unfortunately (?) the ADA doesn't allows the disabled person to specify their own technology. If Google can reasonably say that speech to text works via a standard screenreader (which it does) then they are ok.
> The ADA is, as mentioned, only somewhat effective in prohibiting such things, though
Well that's not the intent of the ADA, so not really surprising.
https://techcrunch.com/2013/12/25/google-rap-genius/
https://archive.ph/9eUkv
> The Unicorn tier is for large companies or companies that would like to have a reciprocal relationship with our foundation. If you need special guarantees, indemnities or require us to sign your contract for a data license, please select this tier. If you have another creative idea you would like to propose, please also select the unicorn tier.
> For any of these cases, please detail your request in the company information field and we will work with you to fit your company's mythical situation. We will also find an appropriate monthly support amount to our non-profit foundation of $1500 or more per month. Please always consider enabling the growth of our non-profit foundation and the continuous growth of our metadata!
It perfectly fits their mission "to organize the world's information and make it universally accessible and useful".
Genius with it's annotations is much more interesting than just the lyrics in plain text.
Also Google makes you click the big down arrow before showing the complete lyrics, which makes me irrationally annoyed.
s/organize/steal
Google is blatantly ripping off companies and putting them out of business.
Original copyright holder could give someone else authorisation to sue on their behalf, e.g., through an assignment. Doubtful Genius got an assignment in the agreements they have with publishers.
Also, Google claimed it is sub-licensed to re-publish through a third party, LyricFind, which has licenses with "over 4000" music publishers.
They can't assign the bare right to sue. To have standing the plaintiff will need to hold at least one of the exclusive rights in 17 U.S. Code § 106 aiui. Cf Righthaven cases, Silvers v Sony Pictures
So I don't think genius ever had standing to pursue this case.
Again, not a lawyer. But you'd think the actual lawyers would have checked this more carefully.
https://news.ycombinator.com/item?id=6963365
https://searchengineland.com/google-has-officially-penalized...
Pretty funny that Google depends on them for content now.
Does this mean Genius still has grounds to sue, as the copyright protections are on their “work” which is the transcription?
It also seems to mean a ToS is not useful for protecting content, only determining legal users interaction with it (Excluding loading the page). A webpage is a work rendered through a browser, kinda makes sense I guess.
Compare to the whole word perfect clip art lawsuit
Derivative works are copyrightable (Translations, adapting a book to a movie), but it might be hard to argue an exact copy of the song lyrics are “derivative” as they aren’t an original creation, but simply a subset of the old work.
Rich snippets are all stolen pageviews. To make matters worse, they now don't let the site which is featured in the snippet have a secondary ranking further down the page.
They keep adding schema, which SEOs eat up because it provides a secondary level of content optimization but it's not as effective as it used to be for increasing CTR on your serp results.
The one positive of this is I no longer have to goto recipe sites which overload my screen with ads and put the ingredients 50% down the page. Google just scrapes the content and displays it in the snippet box.
https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...
https://searchengineland.com/google-to-add-attribution-to-li...
The dismissal seems logical to me
But physical analogies to IP fall apart quickly so I'm not going to encourage people to read into that too deeply
If you find something verbatim identical in a bunch of different places, you've got a strong case that it's just information, because if it were original creative output it wouldn't show up identically in multiple places.
If it turns out everyone was plagiarizing a single source, but you were unaware and took down the offending content when asked, you won't have much in the way of legal liability.
That said, it would've been just if Google would pay for access to Genius' particular, well-curated, "source" database of lyrics, especially given that they're basically stealing traffic.
Here's an interesting question: if Genius closed up shop tomorrow, how long would it take Google to become the primary source of song lyrics online (by rebuilding Genius's dataset from general Internet harvesting)?
Even if Google scraped it on purpose in order to steal traffic, it would likely be legal.
8 months ago: https://news.ycombinator.com/item?id=21781668
and https://news.ycombinator.com/item?id=21700866
June 2019: https://news.ycombinator.com/item?id=20194952
https://news.ycombinator.com/item?id=20201139
https://news.ycombinator.com/item?id=20204625
>Eastern District of New York
Ugh. Basic legal literacy can no longer be expected in the media?
EDNY is a federal court, not a state one.
The real story here is left completely untold: why didn't they bring a copyright claim? Could they bring a copyright claim in the future? Could one of the owners of the copyright bring a claim? These are the questions that matter.