If they somehow could turn those books accessible by a "Google All Book Access" type of service, that would in turn enhance your searching to include all their scanned books it would be amazing. They would however have to somehow figure out how to even make such a service affordable while still keeping publishers 'happy'.
I think the what's missing from this article is the intransigence of publishers to give up anything, and the politicians will usually side with them too with copyright law. There's a world of information out there which could benefit mankind but it's all locked up for profit.
Which might not be so bad if the copyright owners were actually profiting from it, but for the most part we're talking about books that are out of print and not earning their copyright owners anything -- it's just sheer intransigence.
Problem is, many rightsholders either no longer exist (orphaned works), or refuse to profit (keeping books out of further print).
What we really need in the US is a copyright system that is "use it or lose it". For a mass produced work such as books, music, or 2D art (anything that is not strictly a limited edition run, or a one off, really), if you refuse to sell it to someone who wishes to buy it, you lose the copyright.
A lot of things pirated are pirated simply because people cannot legitimately buy them. The reason Popcorn Time works so well? Because rightsholders refuse to do business with Netflix and Amazon Prime Video. Rightsholders do not have the right to make that choice in today's America, and need to be forced to understand that they must make the sale if someone offers them an acceptable amount of money.
Before computers and the Internet, yes, they had a right to say no, since publishing books and tapes/CDs and whatever cost money... now with the advent of modern technology, publishing works electronically now has a cost that is not relevant to this discussion. Literally, they are throwing money away by telling people no, and that actually MIGHT be illegal depending on how you interpret US law (given how most of these companies are publicly held).
> Because of the strange distortions of copyright protection, there are twice as many newly published books available on Amazon from 1850 as there are from 1950.
Additionally, The Walt Disney Company is sure to get legislation passed before 2024 to extend copyright once more.
"If Google was, in truth, motivated by the highest ideals of service to the public, then it should have declared the project a non-profit from the beginning, thereby extinguishing any fears that the company wanted to somehow make a profit from other people’s work."
I think Google might win over some critics if they resumed the project; set it up as a non-profit, but not some slick non-profit that really doesn't help anyone other than Google?
The bylaws would be lawyer proof, and BOD proof. The out of print(out of copyright) books would be available to anyone for free.
I was very excited about this project, and it did seem to just die?
I used to like and defend Google. As of the last few years, with the tracking, plethora of Ads, and the way they ruined YouTube, at least for me.(Yea, I didn't like the way they took it over. I don't like all the advertisements. Plus, I still have embarrassing videos up there that I literally can't get off. Some kind of password screwup that is beyond the helpful customers at the "Help Boards". See Google employees can't be bothered with trivial stuff like my videos. (I asked, and was told to figure it out.)
So Google, if you are listening, go back to your roots. Some people, including myself, hold no loyality to your company anymore. My sister uses Bing. I used to tell her, you might like Google better. Those days are long gone. I'd tell her about Duckduckgo, but it's just not quite their yet.
Google's an advertising company. Of course they were going to add adverts. Even if the original founders still ran the site it'd probably eventually have gotten adverts. Bandwidth isn't free, especially at that scale. You expect Google to run it out of the love of their hearts after paying a huge sum to buy it?
If you don't like the adverts, just use Adblock Plus/uBlock Origin and call it a day.
The tone you have taken to answer, together with the last comment, make no sense together. So you support the ad model, including the silly "out of the love of their hearts" rhetoric, but conclude with installing an ad blocker.
Well, it does. OP said it makes sense from youtube's point of view to put in ads, and was to be expected of them. He didn't support the ad model himself, just claimed it made sense.
Well ben, if you don't like comments criticising Google, just stop reading them and call it a day.
Google is not just showing ads, they're tracking their users and building profiles which include contacts, location, e-mail content, what you search for, which sites you visit, photos, videos and so on and so forth.
No one really expects them to give their stuff away for free, but many consider this data-for-product deal much worse than simply paying for a product, and they shouldn't be silenced with trite recommendations such as "use adblock".
Yes Google is just showing ads. They collect data about you to target their ads well, but that's basically all they do. They don't send your info to the NSA, they don't send you SPAM, they don't sell your info to private investigators. Their end goal is just to show you targeted ads. That's it.
Your whole post is just nonsense. First you say they just show ads, then in the second sentence you say yourself that they do much more then that. Then comes the strawman that they don't send spam and sell the info.
You're kidding--right? I imagine the NSA has complete control over Google's head honchos? ("Give me the data, or we will audit every stock trade, every dime that you put into a bank? Every aspect of your private life will be scrutinized, if you don't go along with our polite requests?"). Hell--I wouldn't be surprised if the NSA has a pipeline directly attached to Google servers?
In my world, SPAM is any ad, including targeted ads? I actually miss the phalus enlargement ads?
They don't sell your information to Investigators(government, private, etc.), I imagine they give it away, if their lives, money will be affected?
I do feel their end goal is to sell you products, and services, but just the amount of information they have on us is staggering? Why do I feel they can predict the future? I wouldn't be surprised if they know right now who will become the final two, or three presidential candidates?
I have no evidence for these outlandish claims. I'm just a guy who doesn't believe in what these companies are telling us? Conspiracy theory nut--maybe I am? Or, maybe I watched Three Days of the Condor too many times? I just think they do a lot with that data? I think the government does a lot with that data?
I'm in no mood for a debate, I need to remove seven rivets in order to change my ball joints(horrid manufacturing/engineering), and the neighbors don't like noise. Need to complete the job before 7:00 p.m., or I might get a visit from the Blue boys? Love my kind neighbors?
I think you don't understand the economics of the situation, of course everyone agrees it would be lovely if all companies hired a personal concierge for each user that attended to their every whim. But for that to happen, that concierge has to be paid a salary.
Now extrapolate how many users each concierge has to manage to make it viable if your margin per user is microscopic, like fractions of a cent microscopic? Your only option to make it work is to make it a paid service...then those users are suddenly not interested, and move to the platform with crappy customer service but no subscription fee.
The reminds me of people's attitudes towards reCAPTCHA - started by researchers at Carnegie Mellon University. "Stop spam, read books."
Everyone was supportive of it when it was used towards the non profit digitization of out of copyright books. So, started externally, Google ran with it and continued and expanded the range of books. It remained good. Apparently loads of books were digitized.
Then, possibly along with the change in this article - or along with the perceptible change within Google where everything had to be business accountable a few years back, reCAPTCHA started being used to digitize address numbers from StreetView to improve Googles online mapping, geocoding offering. Nothing to do with books, nothing to do with improving the world.
Now reCAPTCHA is being used for image recognition and training (identify the images with salads). Nothing to do with books, information or improving the world - everything to do with Googles own offerings.
What's even sadder is that http://captcha.net/ still states that it is being used to "help digitize books" but all the links go to https://www.google.com/recaptcha/intro/index.html which have all but removed any public benefit wording. " "Stop spam, read books." was removed from Googles site in 2014.
They are improving your life in exchange for information about you and the places you visit. Their product is closed, the proprietary data is not accessible outside their apps, or if they wish to license it.
"the proprietary data is not accessible outside their apps"
This is not true. Many APIs are available, specifically to allow building 3rd-party apps: Directions API, Distance Matrix API, Elevation API, Geocoding API, and so on. See https://developers.google.com/maps/pricing-and-plans/
Not exactly free, but the places you visit can be observed with the naked eye from public property, and a network of cameras feeding into ANPR and facial recognition would be a hell of a lot cheaper (and more reliable) than a mapping product.
Not to mention that realtime traffic data and intelligent routing around congestion is only workable by tracking individual vehicles on the road and recording how long it takes them to get from A to B.
> Not to mention that realtime traffic data and intelligent routing around congestion is only workable by tracking individual vehicles on the road and recording how long it takes them to get from A to B.
Wrong. There is a better solution, used for traffic lights that adapt to traffic:
Use an inductive loop to count every vehicle and its speed at ever lane of every intersection and exit.
Far lower cost yet, and provides far better data. (Also is totally anonymous)
Um, what? Lower cost to dig up a street and install infrastructure inside it, than to use the cheap and ubiquitous internet and GPS connected computers that are already in every car?
The sum total cost paid by users of maps in data charges and electricity alone outweigh the cost of implementing a public system. Just because the cost is distributed does not mean it's not being paid.
If you want to compare induction loops against power and data charges then you need to magically make your induction loops capable of giving turn-by-turn navigation without a device in the car.
You need to dig up every street every winter for repairs anyway, and you need the inductive loops for adapted traffic lights anyway.
Effectively, you need the infrastructure, most cities on the world have it in some way – either as inductive loops, or by using RFID toll systems, or similar.
So, you have it already, and you can get the same data out of it without the cost of breaking everyone's privacy.
I live in a very heavy snow area in the North East US (Buffalo region) and we definitely don't "dig up every street every winter for repairs". What is usually done is that when the snow thaws and the temperature warms a crew will come out and fill necessary holes in the pavement, not dig up streets. This process is very fast, it only takes a few minutes per hole and would definitely not include digging up the streets.
Here in Germany we fill the holes every few weeks in winter, and then rebuild the street completely every few years when more than 50% of the street are those filled holes.
Feeding data into a proprietary system involves the risk that your access to that system may be revoked for any reason by the owner of that system at any point in the future. The threat of revocation, in turn, can be used as leverage to force you to accept damaged version of the system you like, with "damage" ranging from charging money to unwanted surveillance to unwanted advertising to software failure.
Some people think that this risk is unbearable. Others weigh the economic realities and make a case-by-case judgement. The simple truth is that it costs money to host a good tool that's always on, doesn't lose people's data, that keeps up with changes in technology and requirements.
You still haven't demonstrated why they are not 'improving the world'.
They may not be doing it to the extent of your liking, but they are certainly are improving it (if only for the people who choose to use their service).
The main OpenStreetMap.org site is really intended mainly for those contributing to the map. They don't make it clear enough that end users should probably be using one of the many other sites based on their data.
It's only a problem if you think the OSM project is trying to build a replacement for maps.google.com. Since they're not, it's not a problem. Most users of their free location database are not using it directly on the OSM.org site, but through other apps and services. It is rightly the job of these 3rd parties to deliver excellent user interfaces for the data.
Commercial proprietary products have positive and negative externalities. These can improve the world even if they're not free.
We have am mixed capitalist system of free and publicly funded advancements as well as privately funded advancements.
The internet as it exists today is a demonstrate of that fact, as is the mobile smartphone market. This has obviously improved the lives of billions of people and would not have existed without both private (proprietary) and public contributions.
They're both "improving the world" by some standard, but not quite in the same way. There's nothing wrong with for-profit services, and plenty of "for profit stuff" absolutely makes the world better.
BUT... as technologists / hackers and people with (presumably) a broader vision, I think the onus is on us to prefer and to advocate for services which are "open" or "free" / "libre" (without getting too deeply into the definitions of "open" and/or "free" here).
I would say that it would be wrong to stay that Google Maps doesn't make the world a better place, but I'd also agree with your general sentiment and say let's do all we can to promote OpenStreetMap and the like.
In my country (Romania) it's hit or miss. Not all addresses can be located by Google Maps. Many of them yield just the street, which might be kilometers long. It's a shame that Google Maps is inferior to my iGo tablet running Windows Mobile from 10 years ago. And it didn't need to have continuous wireless data access. Even routes that have been looked up are not saved, so there is no way to recover the route if the app crashes or I need to interrupt it midway and I have no data signal. I am very frustrated with Google Maps as a car GPS. It's an inferior product in my country. How was iGo capable to deliver superior maps so many years ago?
A bit further on the west (in Bosnia & Herzegovina) Google's street view is still not available (not even in our capital). As for the names of the streets, OpenStreetMap is far more detailed (and precise). Even Bing's Maps are more usable.
> It's a shame that Google Maps is inferior to my iGo tablet running Windows Mobile from 10 years ago. And it didn't need to have continuous wireless data access. ... How was iGo capable to deliver superior maps so many years ago?
They purchased data from a mapping company. Ten years ago, the two largest mapping companies in the world were Navteq and Tele Atlas. Google used to buy data from these companies as well, but they stopped paying and decided to build up their own maps data from scratch.
Navteq was bought by Nokia, got renamed HERE Maps, and has just been sold to a consortium of German automakers.
Microsoft currently has a multi-year contract with HERE Maps as part of their deal with Nokia. Also, Helmut Panke is on the Microsoft Board of Directors. He used to be the Chairman of the Management Board of BMW.
> A bit further on the west (in Bosnia & Herzegovina) Google's street view is still not available
Could it be because of the maze of laws that governs public photography? In the US, it's somewhat simple: what's visible from the street can be photographed. But in other countries, you have murky laws, and Google, with deep pockets, can get sued easily.
In Germany, where Google famously lost, the law is also simple: From eyelevel, you can photograph everything. Above eyelevel, you need permission, or it has to be only imagery that could have been seen from eye level, too.
One reason for this is that people might build a 2m tall fence around their frontyard and walk around naked, but Google with a 3m tall StreetView camera could obviously photograph that.
Google ignored the law, and hoped it would be easier to ask for forgiveness than for permission, but, as Germans are, everyone asking for forgiveness will be punished even harder to prevent such business policies.
In my region, their data is from 2005, with no MapMaker available, so people always end up in my suburb and ask where the freeway is.
StreetView is only available in some areas, after Google put their cameras at a height of 3.5m, far above eye level, and needing written permission from everyone (cause people built hedges and fences with eyelevel in mind, so people might be in their frontyard, behind a hedge, naked, and now Google photographs that – big no-go). Just another thing where Google thought violating reasonable laws would be possible for them.
In the end, their quality is hit and miss, and the cost is too high. No public transit data, not the ability to download larger areas, etc.
If I worked on making the big mac tastier would you call that improving the world? I mean, plenty of people eat them and improving the flavour improves their lives.
Ultimately the specific terminology is unimportant. Do we agree that there's a distinction between digitizing books in a way that's accessible to everyone, and working to improve Google products that exist primarily to increase Google's revenue, and can you see why people would want to contribute to one and not the other?
First, reCAPTCHA's purpose was to digitize books, not digitize them for public domain. If the work was copyrighted, it was returned to rightful owners.
Second, if Big Mac got 2000% tastier every year, and while at that, 2 billion people got to eat dinner for free everyday, it sure will make the world better.
Data mining books was the lowest hanging fruit. Being a library was a side effect. First and foremost was training their search engine with a massive amount of information.
When it is able to offer search suggestions or corrections to your search, I think some of that was based on machine learning on a massive corpus of text.
I don't think so. I think it simply performs a spellcheck vs existing database of terms and offers a likely correction.
Ie when you put in a word 'Mroscow' it sees that this term has 500 results while a similar term 'Moscow' has 100 million results. Therefore it assumes that you simply misspelled the word and offers the alternative as suggestion.
It does not know what the actual underlying word means.
The world is not a place where events can be sandboxed to one instance like this.
Luis von Ahn did digitize the entire archive of NYT before flipping reCAPTCHA to Google, and I'm sure it's still helping. But because he did that, he had time to start monolingo project (now duolingo) which is helping translate the Web.
So if reCAPTCHA was not being turned into a tool for profit, it would be sitting in dust at some old room in Wean Hall. Or, Luis would be still with it, and Duolingo would never have existed.
Besides, all work that Luis did on CAPTCHA at Berkeley and reCAPTCHA at CMU is published. Nobody is being prevented from making another one just for books. Amazon, maybe?
reCAPTCHA is being utilized for other things because digitizing books in that manner is not useful anymore. Most publishers now have ebook versions of most of their books, which are already digitized.
> because digitizing books in that manner is not useful anymore. Most publishers now have ebook versions of most of their books, which are already digitized.
Perhaps for kids looking for new books that's wonderful, but those of us that use books older than your mom, there are no ebooks. Google Books was literally the only place to access that information outside of a few select world-class university libraries.
I was referring to digitizing books as Luis defines it - actually typing in the scanned pages. Not the scanned pages themselves, as Google Books had.
My comment was regarding reCAPTCHA not being used to convert scanned book pages to text.
For books older than my mom, most of which must be public domain, there are a lot online archives. In fact, some of the more common archives like Project Gutenberg, are available as free ebooks on Kindle and iBooks. For more obscure titles, there are field-specific archives from where Google accessed them.
They only got information from publishers and archives. So information you're looking for is out there somewhere in digital form.
The difficult period is approximately between the 1920s and 1990s, where books are under copyright and may have become orphan works (rightsholders cannot be found).
> For books older than my mom, most of which must be public domain
How old is your mom that a book written when she was alive is almost certain to have been written by someone who has now been dead for more than 70 years?
> Perhaps for kids looking for new books that's wonderful, but those of us that use books older than your mom, there are no ebooks.
Project Gutenberg was a thing before Google Books, and those two things aren't the only source of electronic versions of books "older than your mom", or even the only source of free electronic versions, where such isn't legally prohibited by a combination of copyright law and preference of the copyright holder.
I failed the "salad test" too, I think because I identified various plates of mixed meat and non-green veg as salads. Probably too much time spent going to hip London restaurants.
IIRC the image recognition doesn't even show up if you just move your mouse randomly across the frame. I have been requested to identify pictures like two or three times in total.
reCAPTCHA switched away from digitizing books simply because it was no longer a good test for human vs robot -- the robots were better at it than humans.
"[Google's research shows that] today’s Artificial Intelligence technology can solve even the most difficult variant of distorted text at 99.8% accuracy. Thus distorted text, on its own, is no longer a dependable test"
But lots of sites depend on reCAPTCHA, it would have been unreasonable to just shut it down and break a huge swath of the internet because it had achieved the "we could digitize just about any book written in latin script" goal.
- Google stopped using book scans for reCAPTCHA, because it stopped preventing spam
- How can you claim Google's maps product hasn't improved the world? That's a bold statement. I have a data point: It was used quite a bit during katrina and fukashima.
- The training data that's now being used for ML is helping the world of image and audio recognition immensely. They do publish a lot of this stuff, ya know: https://research.google.com/
I was an early and enthusiastic fan of Google Books. I often do research that relies heavily on 17th-19th century English academic works, which is right in the Google Books public domain sweet spot.
But something went awry—I'm not sure what—and the project was allowed to languish by Google. The interface has been in maintenance mode for ages, with no development going on. This leads to a lot of frustration: for instance, you cannot share all of your saved and tagged books with another user, and sharing a shelf or series of shelves is awkward and clunky. In terms of UX, it appears to be abandonware.
On top of that, there's no way to know what's going on or who to talk to, because users can't actually contact anyone at Google Books.
A related complaint is that, even for public domain books, downloading them is painful: sometimes you need to fill a CAPTCHA, recently you often have to create a Google Account just to download a free book <http://uzy.me/qc>, and hunt for download options (the system tries to make you read the book within Google Play instead). At some point, also, the books' first page included boilerplate with requests for attribution and non-commercial use (which I think is bogus for public domain material, even if Google digitized it).
If Google Books really were about improving public access to books, downloading public domain books would be frictionless, and bulk downloading and mirroring would be encouraged.
What went awry is what's described in the article -- the rights holders killed the possibility of fulfilling the goals of the grand idea with lawsuits. I wouldn't be eager to continue on with a project after getting that reception either.
> I wouldn't be eager to continue on with a project after getting that reception either.
So google's feelings got hurt? Give me a break. Obviously they were primarily motivated by attracting the almighty user attention, and positive pr to their brand, all part of a long term strategy for money, and that didn't work out so well, and they never cared much about doing a public good, or else they would continue to support the over 1 million public domain books they have. Or, if they really cared, they would have put code of their books site out as free software, and helped share their entire collection of public domain books with other organizations.
The problem is that they cannot do that easily. It's no longer a little company, it's a giant with it's own clauses and rules. Maybe founders want to do good, but now, they are stuck with this big behemoth of a company they created and unfortunately, they live in denial that they can do good, despite the fact that the primary goals have shifted towards making money at all costs.
It's not about feelings getting hurt, it's about trying to do something that you consider good and actively being fought against for it. Of course it's more likely you're going to give up in that situation because it's a lot more difficult to succeed if people are fighting you every step of the way than if you have support and help.
To use an analogy, if someone gets their car stuck on a snowy road I'm going to go help dig them out. If they start shoveling show to re-bury their tires I'm not going to continue helping them.
> I have a simpler suggestion, nicknamed the Big Bang license. Congress should allow anyone with a scanned library to pay some price—say, a hundred and twenty-five million dollars—to gain a license, subject to any opt-outs, allowing them to make those scanned prints available to institutional or individual subscribers.
Wouldn't this be great? Many of these materials are not indexed and chances to discover them are decreasing every day. Second, getting access to these materials is for many almost impossible (out of print, not available in the libraries, etc.)
Shout out to the Internet Archive, a non-profit close to what the author describes. Unfortunately it turns out book scanning is expensive and rights-holders still don't like it.
Note that the Internet Archive’s founder Brewster Kahle was one of the strongest critics of the creative class-action settlement which would have allowed Google as a one-off to publish the out-of-print orphan works that it scanned. If the lawsuit had gone to trial and Google prevailed on fair-use, it would have set a precedent that would benefit the Internet Archive. But the Internet Archive was vehemently opposed to any settlement that didn’t help them.
I can say pretty certainly that all the text they've gathered through the Google Books project is in use in their language models and other AI models for their search engine, speech recognition, etc.
They got what they wanted. I can't see what incentive they have as a business to grant access to the books that justifies paying employees for it.
Gather data on search queries and highlighted phrases for books? There is value in knowing which subset of the corpus is more valuable.
Apple acquired a "Pandora for Books" recommendation startup which had permission from publishers, who provided text for indexing, http://www.businessinsider.com/apple-buys-booklamp-2014-7 . Their machine classification made it possible to search books for topics whose words were not present in the book.
Just my personal opinion, but when you have an indexed copy of the whole web, a few million OCRed-but-not-corrected books from previous centuries added to your LM are not going to improve 2015 speech recognition quality.
It would illustrate how language and ideas evolve over time. It would illustrate how language and ideas that are from different geographical sources might differ or be similar, especially during pre-Internet periods. It would provide source the material which is being referenced in contemporary works. It would provide many, many other benefits.
Way way more than a corpus of a few million published books, that's for sure. Hell, there are individual message boards that have higher word count than millions of books. Wikipedia arbitration cases (these aren't articles, but rather, an esoteric back channel for handling disputes between users) frequently reach novel-length.
The average quality is going to be lower, of course.
The least interesting thing about Mexican American War is what type of dash you use between Mexican and American. There are over twenty thousand words about that dash on wiki meta.
15,000 words would be okay if at the end of it there was some kind of consensus, or something that could be tramsfered to different articles.
The future people are going to have a skewed image of us if they think meta wiki is representative.
In June 2015, the US Copyright Office issued their report with draft legislation, "Orphan Works and Mass Digitization", after consultation with creators, libraries and tech companies. This guidance can be used by the US Congress to create laws permitting Google Books and other efforts to digitize orphaned works.
"How is this scheme, which provides for no damages (in some circumstances at least), compliant with the TPP requirement that the US must provide for either “pre-established damages,” or “additional damages”? No damages = no damages, no? A court would (as I read the new statute) NOT be permitted to award punitive, or exemplary, damages in orphan works cases – but the TPP seems to require that."
>The thrilling thing about Google Books, it seemed to me, was not just the opportunity to read a line here or there; it was the possibility of exploring the full text of millions of out-of-print books and periodicals that had no real commercial value but nonetheless represented a treasure trove for the public.
I had the same excitement as the author when Google Books came out. The service has stagnated over the years. Reading snippets is such a frustrating experience (you cannot even cut and paste the text). Even the books where one can buy an ebook is not available for some countries. Many times it is quicker to search on archive.org to find related books digitized by Microsoft.
We still have a long way to go where knowledge can be distributed at low cost and in abundance ...
Indeed Project Gutenberg[1] has for a number of years been putting out-of-copyright books online. They take great pains[2] to make sure the books are no longer in copyright.
Another very important difference is that PG books are proofread by human beings and have a much, much lower rate of OCR errors than the bulk-scanned Google books which are, often, abysmally, disgustingly full of blatant scan errors, sometimes to the point of being unreadable.
Attempting to force Google to change course on this would at best result in an empty gesture on Google's part with likely no follow up regarding the bigger project. They may open up what is already scanned, but they're not likely to resume the project under benevolent terms.
Employment used to last as lifetime. One could retire from Sears with decent benefits if they were loyal. Today, many of us are contract employees or Uber style "partners". This is the same phenomenon we see happening here, just on the product side. We've been going down this road for a while now. You often don't buy products that last a lifetime anymore. It's often necessary to buy a whole new one rather than fix the one you already have because the replacement parts aren't made available. So, it's happened to our workforce and our products, now it is also happening to our companies. Always in the name of profit. Fair weather friends.
Sometimes this is good. Sometimes it is bad. Almost always it is a false choice.
The logic of capitalism insists on competition and specialization which are often at odds with cooperation and leisure. Why should Google need to choose between the bottom line and benefiting its community by sharing the work they've already accomplished. In a word, competition.
To me, this is the greatest strength of open-source. It allows for cooperation and facilitates transparency, the very foundations of community. Open-source has done nothing to stymie specialization, one of the main thrusts of "The Wealth of Nations". The other, self interest serving the whole, I think, has been shown by history and OSS to be one possible, but not the sole, idea regarding what motivations might facilitate a productive community.
So, back to Google Books. What happened to Google Books is exactly what one should expect to have happened in a political-economy such as ours. A different outcome would not necessarily have resulted from different decisions by Google, but by different incentives and structures of a different economic framework. Let's not be distracted by the red herring of this anecdote, framing it as a one off, but instead look to this anecdote as a case study in a much larger domain.
They did sue the non-profit HathiTrust (a coalition started by the U of Michigan) -- HathiTrust won, after many years.
Now, HathiTrust was not generally making _full text_ of in-copyright works available, but only search results. HathiTrust doesn't even provide snippets of your search results, just page numbers, unfortunately -- mostly out of fear of lawsuits.
But HathiTrust's court victory helped establish at least some fair use rights for scanning of books -- more than Google itself did, or was interested in -- if the settlement had been accepted, no legal rights would have been established for anyone, only Google would have had permission from the Author's Guild, endorsed by a court to give them -- but nobody else-- freedom from lawsuits from anyone else too.
So these books that the fight has been about are out of print and essentially do not exist anymore. They do not make money for anyone. They do not contribute to anyone or anything. For all intents and purposes, they might as well not have existed at all. Google tries to make a library of these nonexistent works so that they can once again benefit humanity and the copyright holders (which is pretty much never the authors when it comes to books) are upset because they're losing out on their $0 of profit. Yeah, copyright law really works well in this country.
Why? If anything it should be split between copyright holders. Whoever they are. Legally, author / publisher, are meaningless.
But really if we can't get it together as a culture to eliminate perpetual copyright we should at least make a rule that if a work is not available (print or online) for 5 years then it is deemed abandoned and no longer under copyright. Available doesn't mean free.
Our cultural heritage is a shared resource. It is not right for it to be locked away.
Google making the project non-profit would not have saved them from the lawsuit. The Author's Guild, separately, sued a non-profit partner in the Google Books scanning project -- HathiTrust. [1]
That lawsuit was not resolved until 2012 -- when, without a settlement, HathiTrust won on fair use.
The court decided that scanning books for searching was fair use. While the court did not say the same for displaying full text -- what the OP wants -- it is notable that the court's opinion was not primarily based on non-profit status of the organization (as is common in U.S. fair use case law; the non-profit factor has generally dwindled in court decision-making), but on transformativeness.
The OP mentions "Others argued that the settlement could create a monopoly in online, out-of-print books," but gives that opinion rather short shrift. This was a very real concern -- what if Google's use really would be fair use? If the court decided that, the opinion would apply as precedent to everyone. But a settlement really does apply only to Google -- no one else even had access to the terms of the settlement. Anyone else trying to do the same would risk being subject to a decade-long lawsuit of their own.
The OP should ask, why didn't Google go to trial?
"If Google was, in truth, motivated by the highest ideals of service to the public...." they should have gone to trial to establish the right for all. As HathiTrust did.
The Google Books project still exists, they did not take it down because of legal worries, even in the absence of the settlement. But it has indeed been allowed to languish. While I'm sure the multi-year lawsuit contributed to this -- Google starting ambitious projects and then allowing them to languish, without improvement, without fulfilling their original promise, slowly degrading and withering away -- is a pretty common Google practice even without multi-year lawsuits.
> Google making the project non-profit would not have saved them from the lawsuit. The Author's Guild, separately, sued a non-profit partner in the Google Books scanning project -- HathiTrust.
> That lawsuit was not resolved until 2012 -- when, without a settlement, HathiTrust won on fair use.
I think you're mixed up on some things. HathiTrust wasn't sued until after the settlement was rejected, meaning Google was getting sued simultaneously.
It is good the original settlement was rejected, though I agree with the author that a modification to the settlement, similar to what the Internet Archive was asking for, was a much better solution than another decade of court battles and no orphan works for another century.
> The OP should ask, why didn't Google go to trial?
Google did go and the district court ruled that scanning, searching and snippets were fair use in 2013[1]. The case is still ongoing, the Second Circuit heard arguments last December[2]
>But, of course, leaving things to Congress has become a synonym for doing nothing, and, predictably, a full seven years after the court decision was first announced, we’re still waiting.
I've made $96,000 so far this year working online and I'm a full time student. Im using an online business opportunity I heard about and I've made such great money. It's really user friendly and I'm just so happy that I found out about it.
Heres what I do...www.Buzz34.Com
The law is probably similar, but the technological ability to restrict access is not. Youtube allows you to stream only, not download. That makes it easier to get license holders on board. (Yes, there are workarounds for that, but most users won't bother to go to the trouble, and the quality of the download would be degraded from a purchased copy.) With books, there's no easy way to limit access like that while providing a usable service.
My biggest pet peeve with Google Books is that too many books which are presumably in the public domain have access to them restricted. Not sure if this is oversight, or on purpose.
123 comments
[ 58.1 ms ] story [ 1032 ms ] threadWhat we really need in the US is a copyright system that is "use it or lose it". For a mass produced work such as books, music, or 2D art (anything that is not strictly a limited edition run, or a one off, really), if you refuse to sell it to someone who wishes to buy it, you lose the copyright.
A lot of things pirated are pirated simply because people cannot legitimately buy them. The reason Popcorn Time works so well? Because rightsholders refuse to do business with Netflix and Amazon Prime Video. Rightsholders do not have the right to make that choice in today's America, and need to be forced to understand that they must make the sale if someone offers them an acceptable amount of money.
Before computers and the Internet, yes, they had a right to say no, since publishing books and tapes/CDs and whatever cost money... now with the advent of modern technology, publishing works electronically now has a cost that is not relevant to this discussion. Literally, they are throwing money away by telling people no, and that actually MIGHT be illegal depending on how you interpret US law (given how most of these companies are publicly held).
> Because of the strange distortions of copyright protection, there are twice as many newly published books available on Amazon from 1850 as there are from 1950.
Additionally, The Walt Disney Company is sure to get legislation passed before 2024 to extend copyright once more.
I think Google might win over some critics if they resumed the project; set it up as a non-profit, but not some slick non-profit that really doesn't help anyone other than Google? The bylaws would be lawyer proof, and BOD proof. The out of print(out of copyright) books would be available to anyone for free.
I was very excited about this project, and it did seem to just die?
I used to like and defend Google. As of the last few years, with the tracking, plethora of Ads, and the way they ruined YouTube, at least for me.(Yea, I didn't like the way they took it over. I don't like all the advertisements. Plus, I still have embarrassing videos up there that I literally can't get off. Some kind of password screwup that is beyond the helpful customers at the "Help Boards". See Google employees can't be bothered with trivial stuff like my videos. (I asked, and was told to figure it out.)
So Google, if you are listening, go back to your roots. Some people, including myself, hold no loyality to your company anymore. My sister uses Bing. I used to tell her, you might like Google better. Those days are long gone. I'd tell her about Duckduckgo, but it's just not quite their yet.
Google's an advertising company. Of course they were going to add adverts. Even if the original founders still ran the site it'd probably eventually have gotten adverts. Bandwidth isn't free, especially at that scale. You expect Google to run it out of the love of their hearts after paying a huge sum to buy it?
If you don't like the adverts, just use Adblock Plus/uBlock Origin and call it a day.
Google is not just showing ads, they're tracking their users and building profiles which include contacts, location, e-mail content, what you search for, which sites you visit, photos, videos and so on and so forth.
No one really expects them to give their stuff away for free, but many consider this data-for-product deal much worse than simply paying for a product, and they shouldn't be silenced with trite recommendations such as "use adblock".
In my world, SPAM is any ad, including targeted ads? I actually miss the phalus enlargement ads?
They don't sell your information to Investigators(government, private, etc.), I imagine they give it away, if their lives, money will be affected?
I do feel their end goal is to sell you products, and services, but just the amount of information they have on us is staggering? Why do I feel they can predict the future? I wouldn't be surprised if they know right now who will become the final two, or three presidential candidates?
I have no evidence for these outlandish claims. I'm just a guy who doesn't believe in what these companies are telling us? Conspiracy theory nut--maybe I am? Or, maybe I watched Three Days of the Condor too many times? I just think they do a lot with that data? I think the government does a lot with that data?
I'm in no mood for a debate, I need to remove seven rivets in order to change my ball joints(horrid manufacturing/engineering), and the neighbors don't like noise. Need to complete the job before 7:00 p.m., or I might get a visit from the Blue boys? Love my kind neighbors?
Now extrapolate how many users each concierge has to manage to make it viable if your margin per user is microscopic, like fractions of a cent microscopic? Your only option to make it work is to make it a paid service...then those users are suddenly not interested, and move to the platform with crappy customer service but no subscription fee.
Everyone was supportive of it when it was used towards the non profit digitization of out of copyright books. So, started externally, Google ran with it and continued and expanded the range of books. It remained good. Apparently loads of books were digitized.
Then, possibly along with the change in this article - or along with the perceptible change within Google where everything had to be business accountable a few years back, reCAPTCHA started being used to digitize address numbers from StreetView to improve Googles online mapping, geocoding offering. Nothing to do with books, nothing to do with improving the world.
Now reCAPTCHA is being used for image recognition and training (identify the images with salads). Nothing to do with books, information or improving the world - everything to do with Googles own offerings.
What's even sadder is that http://captcha.net/ still states that it is being used to "help digitize books" but all the links go to https://www.google.com/recaptcha/intro/index.html which have all but removed any public benefit wording. " "Stop spam, read books." was removed from Googles site in 2014.
OpenStreetMap is improving the world.
This is not true. Many APIs are available, specifically to allow building 3rd-party apps: Directions API, Distance Matrix API, Elevation API, Geocoding API, and so on. See https://developers.google.com/maps/pricing-and-plans/
So they are improving the world and for free. I'm glad we all agree
Not to mention that realtime traffic data and intelligent routing around congestion is only workable by tracking individual vehicles on the road and recording how long it takes them to get from A to B.
Wrong. There is a better solution, used for traffic lights that adapt to traffic:
Use an inductive loop to count every vehicle and its speed at ever lane of every intersection and exit.
Far lower cost yet, and provides far better data. (Also is totally anonymous)
Um, what? Lower cost to dig up a street and install infrastructure inside it, than to use the cheap and ubiquitous internet and GPS connected computers that are already in every car?
Effectively, you need the infrastructure, most cities on the world have it in some way – either as inductive loops, or by using RFID toll systems, or similar.
So, you have it already, and you can get the same data out of it without the cost of breaking everyone's privacy.
Some people think that this risk is unbearable. Others weigh the economic realities and make a case-by-case judgement. The simple truth is that it costs money to host a good tool that's always on, doesn't lose people's data, that keeps up with changes in technology and requirements.
They may not be doing it to the extent of your liking, but they are certainly are improving it (if only for the people who choose to use their service).
But then their site and UX are so bad, I always cry when I use it for the lost effort of so many people.
We have am mixed capitalist system of free and publicly funded advancements as well as privately funded advancements.
The internet as it exists today is a demonstrate of that fact, as is the mobile smartphone market. This has obviously improved the lives of billions of people and would not have existed without both private (proprietary) and public contributions.
BUT... as technologists / hackers and people with (presumably) a broader vision, I think the onus is on us to prefer and to advocate for services which are "open" or "free" / "libre" (without getting too deeply into the definitions of "open" and/or "free" here).
I would say that it would be wrong to stay that Google Maps doesn't make the world a better place, but I'd also agree with your general sentiment and say let's do all we can to promote OpenStreetMap and the like.
They purchased data from a mapping company. Ten years ago, the two largest mapping companies in the world were Navteq and Tele Atlas. Google used to buy data from these companies as well, but they stopped paying and decided to build up their own maps data from scratch.
> Even Bing's Maps are more usable.
If you zoom to Sarajevo and look at the bottom-right of the Bing Maps window, it says: "© 2015 HERE. © 2015 Microsoft Corporation."
Navteq was bought by Nokia, got renamed HERE Maps, and has just been sold to a consortium of German automakers.
Microsoft currently has a multi-year contract with HERE Maps as part of their deal with Nokia. Also, Helmut Panke is on the Microsoft Board of Directors. He used to be the Chairman of the Management Board of BMW.
Could it be because of the maze of laws that governs public photography? In the US, it's somewhat simple: what's visible from the street can be photographed. But in other countries, you have murky laws, and Google, with deep pockets, can get sued easily.
One reason for this is that people might build a 2m tall fence around their frontyard and walk around naked, but Google with a 3m tall StreetView camera could obviously photograph that.
Google ignored the law, and hoped it would be easier to ask for forgiveness than for permission, but, as Germans are, everyone asking for forgiveness will be punished even harder to prevent such business policies.
StreetView is only available in some areas, after Google put their cameras at a height of 3.5m, far above eye level, and needing written permission from everyone (cause people built hedges and fences with eyelevel in mind, so people might be in their frontyard, behind a hedge, naked, and now Google photographs that – big no-go). Just another thing where Google thought violating reasonable laws would be possible for them.
In the end, their quality is hit and miss, and the cost is too high. No public transit data, not the ability to download larger areas, etc.
Ultimately the specific terminology is unimportant. Do we agree that there's a distinction between digitizing books in a way that's accessible to everyone, and working to improve Google products that exist primarily to increase Google's revenue, and can you see why people would want to contribute to one and not the other?
Second, if Big Mac got 2000% tastier every year, and while at that, 2 billion people got to eat dinner for free everyday, it sure will make the world better.
Ie when you put in a word 'Mroscow' it sees that this term has 500 results while a similar term 'Moscow' has 100 million results. Therefore it assumes that you simply misspelled the word and offers the alternative as suggestion.
It does not know what the actual underlying word means.
PageRank is one of the 200 metrics Google uses nowadays. It's weight is not as great before.
Luis von Ahn did digitize the entire archive of NYT before flipping reCAPTCHA to Google, and I'm sure it's still helping. But because he did that, he had time to start monolingo project (now duolingo) which is helping translate the Web.
So if reCAPTCHA was not being turned into a tool for profit, it would be sitting in dust at some old room in Wean Hall. Or, Luis would be still with it, and Duolingo would never have existed.
Besides, all work that Luis did on CAPTCHA at Berkeley and reCAPTCHA at CMU is published. Nobody is being prevented from making another one just for books. Amazon, maybe?
reCAPTCHA is being utilized for other things because digitizing books in that manner is not useful anymore. Most publishers now have ebook versions of most of their books, which are already digitized.
Perhaps for kids looking for new books that's wonderful, but those of us that use books older than your mom, there are no ebooks. Google Books was literally the only place to access that information outside of a few select world-class university libraries.
My comment was regarding reCAPTCHA not being used to convert scanned book pages to text.
For books older than my mom, most of which must be public domain, there are a lot online archives. In fact, some of the more common archives like Project Gutenberg, are available as free ebooks on Kindle and iBooks. For more obscure titles, there are field-specific archives from where Google accessed them.
They only got information from publishers and archives. So information you're looking for is out there somewhere in digital form.
Were that true, you'd think Google would be able to find it.
> Were that true, you'd think Google would be able to find it.
Or not - because of copyright laws. :-(
The difficult period is approximately between the 1920s and 1990s, where books are under copyright and may have become orphan works (rightsholders cannot be found).
How old is your mom that a book written when she was alive is almost certain to have been written by someone who has now been dead for more than 70 years?
Project Gutenberg was a thing before Google Books, and those two things aren't the only source of electronic versions of books "older than your mom", or even the only source of free electronic versions, where such isn't legally prohibited by a combination of copyright law and preference of the copyright holder.
I tried many times and failed the salad test. It's so frustrating that I gave up.
then
> nothing to do with improving the world
seems contradictory to me.
"[Google's research shows that] today’s Artificial Intelligence technology can solve even the most difficult variant of distorted text at 99.8% accuracy. Thus distorted text, on its own, is no longer a dependable test"
http://googleonlinesecurity.blogspot.com/2014/12/are-you-rob...
But lots of sites depend on reCAPTCHA, it would have been unreasonable to just shut it down and break a huge swath of the internet because it had achieved the "we could digitize just about any book written in latin script" goal.
- Google stopped using book scans for reCAPTCHA, because it stopped preventing spam
- How can you claim Google's maps product hasn't improved the world? That's a bold statement. I have a data point: It was used quite a bit during katrina and fukashima.
- The training data that's now being used for ML is helping the world of image and audio recognition immensely. They do publish a lot of this stuff, ya know: https://research.google.com/
But something went awry—I'm not sure what—and the project was allowed to languish by Google. The interface has been in maintenance mode for ages, with no development going on. This leads to a lot of frustration: for instance, you cannot share all of your saved and tagged books with another user, and sharing a shelf or series of shelves is awkward and clunky. In terms of UX, it appears to be abandonware.
On top of that, there's no way to know what's going on or who to talk to, because users can't actually contact anyone at Google Books.
If Google Books really were about improving public access to books, downloading public domain books would be frictionless, and bulk downloading and mirroring would be encouraged.
But you are right, often they are horrible to access.
So google's feelings got hurt? Give me a break. Obviously they were primarily motivated by attracting the almighty user attention, and positive pr to their brand, all part of a long term strategy for money, and that didn't work out so well, and they never cared much about doing a public good, or else they would continue to support the over 1 million public domain books they have. Or, if they really cared, they would have put code of their books site out as free software, and helped share their entire collection of public domain books with other organizations.
To use an analogy, if someone gets their car stuck on a snowy road I'm going to go help dig them out. If they start shoveling show to re-bury their tires I'm not going to continue helping them.
Wouldn't this be great? Many of these materials are not indexed and chances to discover them are decreasing every day. Second, getting access to these materials is for many almost impossible (out of print, not available in the libraries, etc.)
https://archive.org/details/texts&tab=about
Reference: Brewster’s op-ed http://www.washingtonpost.com/wp-dyn/content/article/2009/05... and filings by Internet Archive and Open Book Alliance https://dockets.justia.com/docket/new-york/nysdce/1:2005cv08...
They got what they wanted. I can't see what incentive they have as a business to grant access to the books that justifies paying employees for it.
Apple acquired a "Pandora for Books" recommendation startup which had permission from publishers, who provided text for indexing, http://www.businessinsider.com/apple-buys-booklamp-2014-7 . Their machine classification made it possible to search books for topics whose words were not present in the book.
The average quality is going to be lower, of course.
Here's one discussion over over ten thousand words: https://en.m.wikipedia.org/wiki/Wikipedia:Village_pump_(poli...
The least interesting thing about Mexican American War is what type of dash you use between Mexican and American. There are over twenty thousand words about that dash on wiki meta.
15,000 words would be okay if at the end of it there was some kind of consensus, or something that could be tramsfered to different articles.
The future people are going to have a skewed image of us if they think meta wiki is representative.
http://copyright.gov/orphan/
"How is this scheme, which provides for no damages (in some circumstances at least), compliant with the TPP requirement that the US must provide for either “pre-established damages,” or “additional damages”? No damages = no damages, no? A court would (as I read the new statute) NOT be permitted to award punitive, or exemplary, damages in orphan works cases – but the TPP seems to require that."
I had the same excitement as the author when Google Books came out. The service has stagnated over the years. Reading snippets is such a frustrating experience (you cannot even cut and paste the text). Even the books where one can buy an ebook is not available for some countries. Many times it is quicker to search on archive.org to find related books digitized by Microsoft.
We still have a long way to go where knowledge can be distributed at low cost and in abundance ...
Another very important difference is that PG books are proofread by human beings and have a much, much lower rate of OCR errors than the bulk-scanned Google books which are, often, abysmally, disgustingly full of blatant scan errors, sometimes to the point of being unreadable.
[1] http://www.gutenberg.org/wiki/Main_Page [2] http://www.gutenberg.org/wiki/Gutenberg:Copyright_How-To
Employment used to last as lifetime. One could retire from Sears with decent benefits if they were loyal. Today, many of us are contract employees or Uber style "partners". This is the same phenomenon we see happening here, just on the product side. We've been going down this road for a while now. You often don't buy products that last a lifetime anymore. It's often necessary to buy a whole new one rather than fix the one you already have because the replacement parts aren't made available. So, it's happened to our workforce and our products, now it is also happening to our companies. Always in the name of profit. Fair weather friends.
Sometimes this is good. Sometimes it is bad. Almost always it is a false choice.
The logic of capitalism insists on competition and specialization which are often at odds with cooperation and leisure. Why should Google need to choose between the bottom line and benefiting its community by sharing the work they've already accomplished. In a word, competition.
To me, this is the greatest strength of open-source. It allows for cooperation and facilitates transparency, the very foundations of community. Open-source has done nothing to stymie specialization, one of the main thrusts of "The Wealth of Nations". The other, self interest serving the whole, I think, has been shown by history and OSS to be one possible, but not the sole, idea regarding what motivations might facilitate a productive community.
So, back to Google Books. What happened to Google Books is exactly what one should expect to have happened in a political-economy such as ours. A different outcome would not necessarily have resulted from different decisions by Google, but by different incentives and structures of a different economic framework. Let's not be distracted by the red herring of this anecdote, framing it as a one off, but instead look to this anecdote as a case study in a much larger domain.
Now, HathiTrust was not generally making _full text_ of in-copyright works available, but only search results. HathiTrust doesn't even provide snippets of your search results, just page numbers, unfortunately -- mostly out of fear of lawsuits.
But HathiTrust's court victory helped establish at least some fair use rights for scanning of books -- more than Google itself did, or was interested in -- if the settlement had been accepted, no legal rights would have been established for anyone, only Google would have had permission from the Author's Guild, endorsed by a court to give them -- but nobody else-- freedom from lawsuits from anyone else too.
er... https://www.eff.org/cases/authors-guild-v-google-part-ii-fai...
Why? If anything it should be split between copyright holders. Whoever they are. Legally, author / publisher, are meaningless.
But really if we can't get it together as a culture to eliminate perpetual copyright we should at least make a rule that if a work is not available (print or online) for 5 years then it is deemed abandoned and no longer under copyright. Available doesn't mean free.
Our cultural heritage is a shared resource. It is not right for it to be locked away.
Google making the project non-profit would not have saved them from the lawsuit. The Author's Guild, separately, sued a non-profit partner in the Google Books scanning project -- HathiTrust. [1]
That lawsuit was not resolved until 2012 -- when, without a settlement, HathiTrust won on fair use.
The court decided that scanning books for searching was fair use. While the court did not say the same for displaying full text -- what the OP wants -- it is notable that the court's opinion was not primarily based on non-profit status of the organization (as is common in U.S. fair use case law; the non-profit factor has generally dwindled in court decision-making), but on transformativeness.
The OP mentions "Others argued that the settlement could create a monopoly in online, out-of-print books," but gives that opinion rather short shrift. This was a very real concern -- what if Google's use really would be fair use? If the court decided that, the opinion would apply as precedent to everyone. But a settlement really does apply only to Google -- no one else even had access to the terms of the settlement. Anyone else trying to do the same would risk being subject to a decade-long lawsuit of their own.
The OP should ask, why didn't Google go to trial?
"If Google was, in truth, motivated by the highest ideals of service to the public...." they should have gone to trial to establish the right for all. As HathiTrust did.
The Google Books project still exists, they did not take it down because of legal worries, even in the absence of the settlement. But it has indeed been allowed to languish. While I'm sure the multi-year lawsuit contributed to this -- Google starting ambitious projects and then allowing them to languish, without improvement, without fulfilling their original promise, slowly degrading and withering away -- is a pretty common Google practice even without multi-year lawsuits.
[1] https://en.wikipedia.org/wiki/Authors_Guild,_Inc._v._HathiTr...
> That lawsuit was not resolved until 2012 -- when, without a settlement, HathiTrust won on fair use.
I think you're mixed up on some things. HathiTrust wasn't sued until after the settlement was rejected, meaning Google was getting sued simultaneously.
It is good the original settlement was rejected, though I agree with the author that a modification to the settlement, similar to what the Internet Archive was asking for, was a much better solution than another decade of court battles and no orphan works for another century.
> The OP should ask, why didn't Google go to trial?
Google did go and the district court ruled that scanning, searching and snippets were fair use in 2013[1]. The case is still ongoing, the Second Circuit heard arguments last December[2]
[1] https://www.publicknowledge.org/files/google%20summary%20jud...
[2] https://www.eff.org/cases/authors-guild-v-google-part-ii-fai...
Ha, never thought about that 7 years ago. :)
Surely the copyright law is quite similar.