Hmmm, should I use a document scanner application, made by a company in China, after the recent discovery of numerous applications on iOS and OSX sending data to China.....
Great idea. Someone please build a public database of tables of contents and indexes, keyed by ISBN, so we can skip the scanning and OCR steps. And dear publishers: when this database emerges please add to it as you publish new works.
Eat Your Books [0] is amazing for this, but is limited to the cookery world.
You tell it what books you own. It knows what’s in them but does not show you the contents. You search for whatever you have in the fridge. It tells you to go to book x, page y, and there’s the recipe.
It's not his business to figure out how they pay their expenses. It's his business to make value judgements for himself. He decided this service wasn't worth the cost.
As long as the pricing is right I appreciate that alternative.
We complain a lot here (and to a certain degree, righly so) here on HN about annoying ads and creepy tracking so I will try to e positive when someone actually honestly makes a great product and asks for money instead of trying to extract it from profiling me and selling the profile.
But: if something don't have an obvious way to monetize I fear one of three:
- I'm not the user but the product
- They're too idealistic and will disappear
- They're looking to sell out to a bigger actor, -and then my data will be abused
With an open source self hosted solution I can be somewhat safe.
With a paid product there is a bigger chance they will be smart enough not to to kill the goose that lays the golden eggs by selling out their paying customers.
Great idea. Even if it's not fair use (as amelius suggests below), I can imagine a lot of publishers would get on board, especially for technical books. It would increase the value of books that you buy, and if you search books you didn't own, point you to books you might want to buy.
The problem with Google Books is search capability within "your library" really has no way to constrain itself to tables of contents and/or indices; even advanced search falls flat on its face by this measure and generally sucks hind tit by every other. I'm just imagining running a search query for "recursion" only to have Hofstadter and Wolfram comprise 90%+ of the return.
Even if full search capability is what you really want, you're limited to ebooks for which digital rights are established via Google Play purchase. For everything else that isn't acknowledged public domain, you'll get a tease preview without even a hint of completeness...if you get anything at all. Invested in a Kindle digital library? I'm liable to suspect Google gave users vested in Amazon DRM a gratuitous GTFO-not-in-my-backyard finger.
Then there's the issue of specific revisions, e.g. it doesn't matter that Google Books shows skant preview of 4e Sedra/Smith when 5e is on my bookshelf--which, oh by the way, doesn't have preview.
Then there's the issue of copyright lockdown, e.g. only one of Carroll Smith's popular ...to Win series has any preview; the others are on lockdown. Same with Milliken on vehicle dynamics, Katz on aerodynamics, and many other titles published by SAE.
(Observe how the thematic high mark we've been striving for is a mere preview.)
Which brings up the issue of availability, e.g. good lucking trying to find the Institute of Navigation's canonical GPS red books in a Google Books query, or specific translations of the Bhagavad Gita, or other rare publications that an archivist would generally not hesitate to shoot you dead if caught attempting to adulterate in a scanner.
Surely I'm not the only person on HN who maintains a tangible library that's 1000+ large and growing. Like the author, I care about what's in my library, not what Google Books superficially pretends to offer as a front to getting me to purchase books in the digital (when available) that I've already paid handsomely for. Amazon and eBay combined see roughly 70% of my bookshelves. Even if Google Books did a fair job by some objective measure, I neither need nor desire their service.
You are taking the comment too literally. Taking it to the extreme 'whole internet' would just be a list of IP addresses. (Because the 'world wide web' is all of the sites).
There's a marginal difference between searching the contents of all the books on earth and the indexes/glossaries of the books you have on your shelf. The latter is a great productivity boost, while the prior is most likely infuriatingly pointless.
"Oh great, I found this exact algorithm I need in this obscure book which has been out of print since 1986."
The fact that we have to scan any technical book published after 1980 is a distortion of capitalism. The obvious most efficient solution for everyone would be to have the darn fully searchable digital version of the book + the source code.
It's plausible that the fact we have so many amazing technical books is also a "distortion of capitalism". The obvious most efficient course of action is to not write a book.
I’d rather scan the whole book and replace my library with an iPad. In fact, I’ve already done that. With over 1,000 books on my iPad it’s the only reason I’m still married.
Does anyone have any experience with book scanning in general? I've been eyeballing a unit from "CZUR" but am a bit skeptical of the product in general. I would prefer to buy something more generic/high end HW wise like a V-shaped scanner where you bring your own DSLRs but can't find out if there is a serious open source software platform for them.
I plan to build a [linear book scanner](https://linearbookscanner.org/) someday. I admit it doesn't satify OP's concern though, at it uses two upward-facing flatbed scanners, rather than consumer cameras.
I made a hardware DIY Book scanner and run it off a Rpi using https://github.com/Tenrec-Builders/pi-scan with two Nikon J1 cameras. If you're not an experienced handy-person be prepared for frustration and spending more than you think on tools.
Otherwise very happy with the result and experience. I can scan about 800 pages per hour currently. Once scanned I use scan tailor to adjust the pages then a small script to tesseract OCR them before creating a PDF.
I typically do all the scanning while watching Netflix as it doesn't require a lot of attention once you get in the flow.
What a shame that people have to use homebrew methods to duplicate the effort that someone (Google) has already done with vastly greater resources.
Google Books, a.k.a., Google Book Search, has already scanned and OCR'ed about 20% of all books in existence -- i.e., 25 million of an estimated 130 million books. So, "somewhere at Google there is a database containing 25 million books and nobody is allowed to read them."[1]
I understand the legal issues, authors not getting compensated, etc., but it's still a shame that it actually exists but it's inaccessible (except for snippets).
Well, at least Google has all the books indexed, even if we can't see them, so that you know what to go to your local library and look for even though you can't see it there on the internet.
Ha! Me too. I was expecting some kind of analysis of the perfect shelf material and optimum depth and height of each shelf based on the number of books by genre and their dimensions.
Ha! I am making some shelves for myself and spent the weekend measuring the dimensions and weight of all of my books to later divide into different bins based on height, depth and weight (density).
I ended up with four different shelve heights based on my measurements (paperbacks, hardcovers, misc., and art-books), and used the classical technique of bird's beak shelving supports to make all shelves configurable.
Not at all. The shelves are 18 mm thick multiplex, but they all have a thin 20 mm strip of MDF covering their front edge (overlapping the shelves by 2 mm downwards), so the effective shelve thickness is 20 mm.
The bird's beaks vertically divide each case in increments of 53½ mm. The four practical shelve heights this allows are:
A: 53½ mm × 4 - 20 mm: 194 mm
B: 53½ mm × 5 - 20 mm: 247½ mm
C: 53½ mm × 6 - 20 mm: 301 mm
D: 53½ mm × 7 - 20 mm: 354½ mm
Other configurations don't make much sense, although I do use (53½ mm × 2 - 20 mm = 87 mm) for the special atlas shelves at the bottom.
You may have to adjust these sizes depending on the books you own (particularly paperbacks; they can have regional variations in height). If you go for the bird's beak approach (looks really classy, but a lot more work) you could increase or decrease that basic unit size of 53½ mm.
Test it all in a mock-up before you build it. :)
I used a custom jig and a router to cut the bird's beaks from 18mm multiplex (went through three router bits; gruelling work that), but you can also buy them pre-made.
Feel free to contact me on Twitter if you need more details. I'm only an amateur woodworker though. I really should write up the process of building this beast some day.
Do you have a photo of the end of a shelf? I can't tell if you've got beaks at the back and front of the shelving, or if you're doing some kind of cantilevered setup.
I've wanted a app that I can take a digital picture of my 12' x 8' wall of uncatorgized books, and the program will tell he roughly where the book looking for is physically.
For instance, the "Book you are requesting is located top right, second shelf from the top."
I've thought about building one, but I don't see a market. Maybe, Librarians, and what's left of book stores? For instance, "Hay the computer says we have that book, but someone probably put it in the wrong section? Go use that app?"
App: take picture, hard part (would pull titles on spines of books, and put into database. Don't have a clue if that's even possible, without huge financial resources? Yes-it would be OCR, but what open source program could I fool with?)
(I'm also considering getting rid of most of my books. It's kinda tough. I've spend a lifetime collecting reference books, and 1st editions. To get a idea of the used book market, I have had over 100 pretty current computer/programming books on CL for $500, and one person was interested, and that was a scam.)
This reminds me of Bret Victor’s Bookcase which displays the sections “highlighted” by projecting them on a wall and navigating to that page on an iPad.
Offtopic: huydotnet, would you consider adding RSS to your blog? I'd like to subscribe, but couldn't find a link anywhere, including the source code of the page.
AFAIK, OCR is only available in OneNote 2016 (i.e. through a licence or Office 365), but not in the OneNote UWP app, which you get for free in the Windows Store.
Edit:
Though, if you imported the Image/Print in OneNote 2016 you can search the OCR result in OneNote UWP and access it using Right-Click -> Picture -> Alt Text
Any chance you can tell me the steps you've used in the past to get OCR out of it? I just took like 30 photos of a book to test this idea out and can't OCR it.
65 comments
[ 2.8 ms ] story [ 124 ms ] thread[1] https://www.camscanner.com/
Sure, why not....
/sarcasm
Thanks.
But I guess making available only the index would fall under fair use. IANAL though.
You tell it what books you own. It knows what’s in them but does not show you the contents. You search for whatever you have in the fridge. It tells you to go to book x, page y, and there’s the recipe.
Oh, and it includes magazines.
[0]: https://www.eatyourbooks.com/
I liked the idea, but really, not nearly enough to pay to index my cookbooks.
It's not his business to figure out how they pay their expenses. It's his business to make value judgements for himself. He decided this service wasn't worth the cost.
We complain a lot here (and to a certain degree, righly so) here on HN about annoying ads and creepy tracking so I will try to e positive when someone actually honestly makes a great product and asks for money instead of trying to extract it from profiling me and selling the profile.
https://news.ycombinator.com/item?id=17940617
But: if something don't have an obvious way to monetize I fear one of three:
- I'm not the user but the product
- They're too idealistic and will disappear
- They're looking to sell out to a bigger actor, -and then my data will be abused
With an open source self hosted solution I can be somewhat safe.
With a paid product there is a bigger chance they will be smart enough not to to kill the goose that lays the golden eggs by selling out their paying customers.
Example where this worked: Basecamp
Example where this failed: WhatsApp
And the search result is limited to just my books, not the whole internet, that's the point of scanning instead of Googling.
I’ve found books.google.com be alright for this as well.
Google Books doesn't index the internet. It indexes books. Including using OCR.
Even if full search capability is what you really want, you're limited to ebooks for which digital rights are established via Google Play purchase. For everything else that isn't acknowledged public domain, you'll get a tease preview without even a hint of completeness...if you get anything at all. Invested in a Kindle digital library? I'm liable to suspect Google gave users vested in Amazon DRM a gratuitous GTFO-not-in-my-backyard finger.
Then there's the issue of specific revisions, e.g. it doesn't matter that Google Books shows skant preview of 4e Sedra/Smith when 5e is on my bookshelf--which, oh by the way, doesn't have preview.
Then there's the issue of copyright lockdown, e.g. only one of Carroll Smith's popular ...to Win series has any preview; the others are on lockdown. Same with Milliken on vehicle dynamics, Katz on aerodynamics, and many other titles published by SAE.
(Observe how the thematic high mark we've been striving for is a mere preview.)
Which brings up the issue of availability, e.g. good lucking trying to find the Institute of Navigation's canonical GPS red books in a Google Books query, or specific translations of the Bhagavad Gita, or other rare publications that an archivist would generally not hesitate to shoot you dead if caught attempting to adulterate in a scanner.
Surely I'm not the only person on HN who maintains a tangible library that's 1000+ large and growing. Like the author, I care about what's in my library, not what Google Books superficially pretends to offer as a front to getting me to purchase books in the digital (when available) that I've already paid handsomely for. Amazon and eBay combined see roughly 70% of my bookshelves. Even if Google Books did a fair job by some objective measure, I neither need nor desire their service.
"Oh great, I found this exact algorithm I need in this obscure book which has been out of print since 1986."
Otherwise very happy with the result and experience. I can scan about 800 pages per hour currently. Once scanned I use scan tailor to adjust the pages then a small script to tesseract OCR them before creating a PDF.
I typically do all the scanning while watching Netflix as it doesn't require a lot of attention once you get in the flow.
Google Books, a.k.a., Google Book Search, has already scanned and OCR'ed about 20% of all books in existence -- i.e., 25 million of an estimated 130 million books. So, "somewhere at Google there is a database containing 25 million books and nobody is allowed to read them."[1]
I understand the legal issues, authors not getting compensated, etc., but it's still a shame that it actually exists but it's inaccessible (except for snippets).
[1] https://www.edsurge.com/news/2017-08-10-what-happened-to-goo...
Finished bookcase:
https://twitter.com/jdhoek/status/885499829052485632
The four common shelve-heights I chose are 4, 5, 6, or 7 bird's beak 'units' spaced apart vertically.
You can see the configurable parts here:
https://twitter.com/jdhoek/status/876126132734656514
The bird's beaks vertically divide each case in increments of 53½ mm. The four practical shelve heights this allows are:
Other configurations don't make much sense, although I do use (53½ mm × 2 - 20 mm = 87 mm) for the special atlas shelves at the bottom.You may have to adjust these sizes depending on the books you own (particularly paperbacks; they can have regional variations in height). If you go for the bird's beak approach (looks really classy, but a lot more work) you could increase or decrease that basic unit size of 53½ mm.
Test it all in a mock-up before you build it. :)
I used a custom jig and a router to cut the bird's beaks from 18mm multiplex (went through three router bits; gruelling work that), but you can also buy them pre-made.
Feel free to contact me on Twitter if you need more details. I'm only an amateur woodworker though. I really should write up the process of building this beast some day.
https://twitter.com/jdhoek/status/881243775288389632
And the second photo here:
https://twitter.com/jdhoek/status/876126132734656514
So beaks at the front and back, with trapezoidal blocks that fit in between. The shelves rest on those blocks.
I've wanted a app that I can take a digital picture of my 12' x 8' wall of uncatorgized books, and the program will tell he roughly where the book looking for is physically.
For instance, the "Book you are requesting is located top right, second shelf from the top."
I've thought about building one, but I don't see a market. Maybe, Librarians, and what's left of book stores? For instance, "Hay the computer says we have that book, but someone probably put it in the wrong section? Go use that app?"
App: take picture, hard part (would pull titles on spines of books, and put into database. Don't have a clue if that's even possible, without huge financial resources? Yes-it would be OCR, but what open source program could I fool with?)
(I'm also considering getting rid of most of my books. It's kinda tough. I've spend a lifetime collecting reference books, and 1st editions. To get a idea of the used book market, I have had over 100 pretty current computer/programming books on CL for $500, and one person was interested, and that was a scam.)
It seems like you're generating pages from Org mode. I've recently discovered ox-hugo, maybe it'll be of interest to you too. I wrote about my setup here https://rakhim.org/2018/09/moved-from-jekyll-to-hugo-and-ox-...
Edit: Though, if you imported the Image/Print in OneNote 2016 you can search the OCR result in OneNote UWP and access it using Right-Click -> Picture -> Alt Text