Another site that breaks the browser's back navigation. Why do so many sites do this? Do they imagine they retain user attention for longer if they break navigation? It's pretty trivial to long-press the back button or just close the tab and not come back again to your site...
I'm curious, how would fiddling with navigation impede text extraction?
The site renders perfectly fine without javascript, and the markup looks straight forward enough
Good to know! I don't believe PM is possible on hacker news so I hope you don't mind that I describe some details right here?
My browser is the latest (v73.0.1) Firefox on the latest build of Windows10. I confirmed the issue with all addons disbled so it is not an addon issue. I think I know what may be responsbile. When initially I load the page the back button works as intended for about a second. After that delay the page seems to load some resources from static.parastorage.com and www.mymobileapp.online. Once those resources are finished loading the back button does not navigate back to the HN article on the first press. Have to press once more. So I presume a script from one of those domains is responsible. Hope this helps!
Broken back buttons are often due to a site where a placeholder loads and it turns around and loads the real thing. Back takes you to the placeholder which promptly takes you back where you were.
There are sites that explicitly mess with the back button but I haven't seen one in ages.
An order of magnitude increase of time is very significant. If you're just processing a few documents with a lot of human oversight you may be right, but it's definitely not a generalised best approach, at least going by the article.
The best technique for having a PDF with extractable data is to include the data within the PDF itself. That is what LibreOffice can do, it can slip in the entire original document within a PDF. Since a compressed file is quite small, the resulting files are not that much larger, and then you don't need to fuss with OCR or anything else.
If you're worried about malicious differences, "regular" PDFs are worse.
As noted in the article, it is extremely difficult to figure out the original text given only a "normal" PDF, so you end up using a lot of heuristics that sometimes guess correctly. There's no guarantee that you'll be able to extract the "original text" when you start with an arbitrary PDF without embedded data. So if you're extracting text, neither way guarantees that you'll get "original text" that exactly matches the displayed PDF if an attacker created the PDF.
That said, there's more you can do if you have an embedded OpenDocument file. For example, you could OCR the displayed PDF, and then show the differences with the embedded file. In some cases you could even regenerate the displayed PDF & do a comparison. There are lots of advantages when you have the embedded data.
Yes to embedding. In Canada, folks have always been able to e-file tax returns, but the CRA (Canada Revenue Agency) also has fillable PDF form for folks who insist on mailing in their returns (with their receipts and stuff so they don't have to store them and risk losing them).
When you're done filling the form, the PDF runs form validity checks and generates a 2D barcode [1] -- which stores your all field entry data -- on the first page. This 2D barcode can then be digitally extracted on the receiving end with either a 2D barcode scanner or a computer algorithm. No loss of fidelity.
Looks like Acrobat supports generation of QR, PDF417 and Data Matrix 2D barcodes.[2]
Just the receipts relevant to the tax return. If you e-file you're responsible for storing receipts up to 6 years in case of audit. (or something like that)
I almost always have to resort to a dedicated parser for that specific pdf. I use it, for example, to injest invoice data from suppliers that won't send me plain text. Always end up with a parser per supplier. And copious amounts of sanity checking to notify me when they break/change the format.
The relatively small company I work for makes me fill out some forms by hand, because they receive them from vendors as a PDF. So I print it out, sign it, and return it to my company by hand.
If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great.
It's not a billion dollar idea but there are a lot of little companies that would save a lot of time using it.
Something like that would work for signing, but the hard part is "turn this pdf into an online form". That way after a user finishes a form, you can perform some basic error checking like, did they fill out everything, is this field a valid format, etc. After 100 employees turn in a multi-page printed out form, someone has to go through it and make sure they signed everywhere, filled out all the fields, etc.
Again, not sexy, but it is so stupid I have to fill out a direct deposit form by hand and turn it into my company, who checks it, then hands it off to the payroll vendor, who has to check it, just to enter the damn data into a form on their end.
Coming from a documents format world (publishing), there are a lot of cases like this.
In theory it sounds like it should be straightforward but it hinges so much on how well the document is structured underneath the surface.
Being that these tools were primarily designed for non-technical users first the priority is in the visual and printed outcome and not the underlying structure.
One document can look much the same as another in form—uses black borders to outline fields, similar or same field names, etc, but may be structured entirely differently and that can be a madhouse of frustrating problems.
It can be complex enough to write a solution for one specific document source. Writing a universal tool that could take in any form like that would probably be a pretty decent moneymaker.
My first intuition, though, would be it may be more successful (though no less simple) to develop a model that can read from the visual of the document rather than parsing it successfully.
Out of curiosity, what exactly are non-technical people doing with PDF's, and why does there need to be a universal tool in the space? What would the tool do with the extracted data?
All kinds of things. PDF is the unifying data exchange format for a lot of businesses who use computers at some end to manage things and need to exchange documents of any kind without relying on the old "can you open Word files?" type problems.
There is a wide world outside of consumers of SaaS products for every little niche problem.
Sometimes they are baked in processes that still use PDF's to share information, sometimes they're old forms of any kind, sometimes even old scanned docs that are still in use but shared digitally. A lot of the businesses that carry on that way are of the mind that "if it's not broke, don't fix it" which is quite rational for their problem areas and existing knowledge base. They might be a potential market at some point for a new solution, but good luck selling them on a web-based subscription SaaS solution when a simple form has been serving their needs for 30+ years.
OP's problem of the PDF being the go-between to digital endpoints is more common than you might think.
The universality I was referring to was the wide range of possibilities for how a given form might be laid out. And old documents contain a lot of noise when they've been added to or manipulated. Look inside an old PDF form from some small-medium sized business sometime. Now imagine 1000 variations of that form one standard problem. Then multiple that by the number of potential problem areas the forms are managing.
Also like OP said—it's not sexy, but it's very real and having an intelligent PDF form reader and consumer would be a time-saver for those businesses who aren't geared to completely alter their workflow.
The tool could do anything with the extracted data. If it allowed you to connect to any of your in house services (like payroll or accounting) either with a quick config/API or a custom patch, or Google Drive, or whatever without complications like online-required and web accounts especially. No whole solution like that exists to my knowledge. At least nothing accessible to the wider market.
Thanks for the comment, this is really interesting. I guess i'm still confused what people actually do with these PDF's though. Are people looking at a PDF sent to them and manually entering that data somewhere else (like payroll or accounting), so this tool would take that data from the PDF and pump it in there automatically?
Thanks again, I just want to make sure I understand.
I assume you mean a drawn form as opposed to a true PDF form. The former would be difficult to parse automatically into inputs.
OTOH, a PDF form works exactly they way you’d like. Maybe there’s a small market in helping convert one to the other for collecting input from old paper-ish forms.
I used Xournal for a couple years in college. It was perfect in how simple it was to mix handwritten and typed notes or markup documents. The only thing is that I wish it had some sort of notebook organization feature. It would have been nice keeping all of my course notes in one file, broken down by chapter or daily pages. Instead, I ended up with a bunch of individual xojs that did the job but made searching for material take longer.
There are quite a few services that should be able to solve this problem (turning a PDF into a web form and collecting signatures.) Here's a few of the services I'm aware of:
(I know about all these because I'm working on a PDF generation service for developers called DocSpring [1]. I'm also working on e-signature support [2], but that's still under development, and still won't be a perfect fit for your use-case.)
As a meta point, it's really nice to see such a well-written, well-researched article that is obviously used as a form of lead generation for the company, and yet no in your face "call to actions" which try to stop you reading the article you came for and get out your wallet instead.
i mean except for the banner at the top and bottom! but yeah, an SEO article with actual substance, well formatted, not grey-on-grey[1], no trackers[2], is rare these days.
[1] recently read an SEO post on okta's site. who can read that garbage?
it doesn't correlate across sites by default -- the reasonable definition of a 3rd party tracker. by your definition, everything not complete self-hosted is a 3rd-party tracker. eg, netlify, which uses server logs to "self"-analyze would be a 3rd party tracker. it is not self-hosted and the data is stored elsewhere.
some might add: for the purpose of resale of the data, but I don't think that's a requirement to be classified as 3rd party tracker. the mere act of correlation, no matter what you then do with the data, makes you a 3rd party tracker. in case you think that's just semantics, this is important for GDPR and the new california law.
you can turn on the "doubleclick" option, which does do said correlation and tracks you. but that's up to the site to decide. GA doesn't do it by default.
On a personal project, I had a good experience extracting PDF text using Tabula[1]. You specify the bounding boxes where desired data is, and it spits out the content it finds.
It still hits the issues mentioned in this article (surprise spaces appearing in middle of words, etc)
There's also camelot in Python [1]. Discovered it on HN [2]. Still a decent amount of manual work afterwards though but it's probably unreasonable to expect otherwise.
So all through this I’m thinking “just OCR it and be done”, and we get to:
> Why not OCR all the time?
> Running OCR on a PDF scan usually takes at least an order of magnitude longer than extracting the text directly from the PDF.
... so? Google can OCR video and translate it in something that feels like real-time; what PDF processing are they doing that is so performance bound?
> Difficulties with non-standard characters and glyphs
OCR algorithms have a hard time dealing with novel characters, such as smiley faces, stars/circles/squares (used in bullet point lists), superscripts, complex mathematical symbols etc.
Sure, but more than the random shit you find in PDFs anyway?
> Extracting text from images offers no such hints
Finding an algorithm that approximates how a human approaches a page layout doesn’t feel like it would be all that hard.
Obviously it’s very easy to stand on the sidelines and throw stones, but parsing PDFs using anything other than OCR + some machine learning models to work out what the type of a piece of text feels like pretending we are still constrained by the processing costs of 5 years ago
1) Do you have trillion or so dollars at your beck and call? If not, you're not Google.
> Finding an algorithm that approximates how a human...
2) ...is generally nigh impossible even for someone with Google's resources (e.g. Waymo, although when it comes to reading, it's somewhat usable). Also, look at 1)
Unless by approximate you mean toddler level. In that case:
Without having heard of or tested the solution, I'll bet anyone $1M that I can produce an image that produces an incorrect answer. Which would mean it's not "solved".
If I can produce an image that you incorrectly label as Bird or No Bird, does that mean it's accurate to say you cannot tell me if pictures have birds in them? Or is that needlessly pedantic beyond any practical use case and clearly the intended context?
"Doing better than me", or any other human, wasn't the problem proposed. Anything other than 100% accuracy means the problem isn't solved as there will always be room for a better solution.
I worked on PDF generating software for years. It's a horrible format that should never have been approved as an ISO standard.
When in doubt, use plain text. It's a million times better in every way that counts.
I wish my bank statements and such could be downloaded as plain text files, instead of massive PDF files that embed another copy of a bunch of typefaces in each file.
I think that root problem here is that most people still have trouble separating the data from the presentation. We have to understand that in the end, substance always beats form.
Ugh, this. I still fail to understand how a device from 2019, even a phone, could show any rendering delay when scrolling to page 200 of a 400 page static document. I thought PDF was less programmable than PostScript, but there's still got to be some kind of non-local semantics in there.
My bank lets me download bank statements in several formats, CSV among them - not entirely plain text, although embedded in it; seems like the best choice for the usecase.
Since gdpr, businesses are "required" to make your data available to you for transfer in a machine-readable format and you could argue that pdf is not exactly machine-readable in the sense of the law. Practically I have seen cases where you do get csv or something similar, but especially smaller firms will probably give you word documents, excel files or pdfs.
Interesting! Halifax Bank in the UK changed their generation library for PDFs the other year such that new statements rendered incorrectly on the Mac. Old statements were fine. New ones were garbage text.
Chrome displayed them fine, Preview on Mac did not.
Trying to communicate this to them was like talking to a tree, or an alien, or a room of catatonic individuals.
I’m a contractor. One of my gigs involved writing parsers for 20-something different kinds of pdf bank statements. It’s a dark art. Once you’ve done it 20 times it becomes a lot easier. Now we simply POST a pdf to my service and it gets parsed and the data it contains gets chucked into a database. You can go extremely far with naive parsers. That is, regex combined with positionally-aware fixed-length formatting rules. I’m available for hire re. structured extraction from PDFs. I’ve also got a few OCR tricks up my sleeve (eg for when OCR thinks 0 and 6 are the same)
I build such a service, but it is impossible to guarantee any reliable result. I ended up shutting it down.
The PDF standard is a mess, and the number of 'tricks' I've seen done is astonishing.
Example: to add shade or border effect to text, most PDF generators simple add the text twice with a subtle offset and different colors. Result: your SaaS service returns every sentence twice.
Off course there were workarounds, but at some point it became unmaintanable.
In a way it has. In my experience, there have been multiple times where a "generate PDF" requirement has come up, with the best viable solution being "develop it in HTML using standard tech" followed by "and then convert it to PDF".
I'd say exactly the opposite. PDF makes it easy to create a document that looks exactly the way you want it to, which seems to be all that most web designers want (witness all the sites that force a narrow column on a large screen and won't reflow their text properly on a small screen).
The demand for automating text extraction is still very high — or at least it feels like it when you’re working around the clock to cater to 3 of your customers, only to wake up to 10 more the next day. We’re small but growing extremely quickly.
Everything. Insurance companies to fledgling AI startups.
It’s definitely harder to get government business because the sales process is so long and compliance is so stringent. That said, we are GDPR compliant.
Any tricks for decimal points versus noise? Its a terrifying outcome and all I've got is doing statistical analysis on the data you've already got and highlighting "outliers".
(Novice speaking) Maybe there's something about looking for the spacing / kerning that is taken up by a decimal point? (Not sure if OCR tools have any way to look for this)
I've done this a bit. I define ranges per numeric field and if it exceeds or is below that range, I send it to another queue for manual review. Sometimes I'll write rules where if it's a dollar amount that usually ends ".00" and I don't read a decimal but I do have "00", then I'll just fix that automatically if it's outside my range.
For something like bank statements, I'd use the rigidly-defined formatting (both number formatting and field position) to inform how to interpret OCR misfires. My larger concern then would be missing a leading 1 (1500.00 v 500.00), but checking for dark pixels immediately preceding the number will flag those errors. And I suppose looking for dark pixels between numbers could help with missed decimals too.
Id also be interested in a blog or any basic tips/examples! I totally understand you don't want to give too much away, but I'm sure HN would love to see it!
Many years ago, I regularly had to parse specifications of protocols from various electronic exchanges. The general approach I used was to do a first pass using a Linux tool to convert it to text: pdftotext. Something like:
After that, it was a matter of writing and tweaking custom text parsers (in python or java) until the output was acceptable, generally an XML file consumed by the build (mainly to generate code).
A frequent need was to parse tables describing fields (name, id, description, possible values etc.). Unfortunately, sometimes tables spanned several pages and the column width was different on every page, which made column splitting difficult. So I annotated page jumps with markers (e.g. some 'X' characters indicating where to cut).
As someone else said, this is like black magic, but kind of fun :)
Oh my goodness, this whole thread is deja vu from some code I wrote to parse my bank statements. I arrived at exactly the same solution of "pdftotext -layout" followed by a custom parser in Python. And ran into the same difficulty with tables: I wrote a custom table parser that uses heuristics to decide where column breaks are.
I worked for an epub firm that used a similar approach a while ago - we took PDFs and produced Flash (yes, that old) versions for online, and created iOS and Android apps for the publisher.
I've come across most of the problems in this post but the most memorable thing was when we were asked to support Arabic, when suddenly all your previous assumptions are backwards!
Well I am putting the finishing touches on a front end that allows extracting PDF text visually. It's also able to adjust when the PDF page size vary for a given document type. Once you build the extractor for a document type, it can run on a batch of PDFs and store to Excel or Database (or any other format).
I sense this tool facilitates and automates a lot of the 'dark art' you mention. Of course there are always difficult documents that don't fit exactly in the initial extraction paradigm, for those I use the big guns ...
OCR tricks? Assuming post processing dev stuff - may I know your OCR engine. We are supported with Kofax and openText along with cloud engines like GVision as a backup.
I've written similar code for investment banks, to extract financial reporting data from PDFs. It's shocking to think how much of the financial world runs on this kind of tin-cans-on-a-piece-of-string solution.
I work in the print industry and some clients have the naive idea they'll save money by formatting their own documents (naive because usually this just means a lot more work for us, which they end up paying for).
We need some metadata to rearrange and sort PDF pages for mailing and delivery (such as name, address, and start/end page for that customer).
Our general rule is you provide metadata in an external file to make it easy for us. Otherwise, we run pdftotext and hope there's a consistent formatting for the output (e.g. every first page has "Issue Date:", "Dear XYZ,", or something written on it).
If that doesn't work then we're re-negotiating. It is not too difficult usually to build a parser for one family of PDF files based on a common setup as you've said and you get to learn various tricks. It is very difficult though to write a general parser.
Personally, I found parsing postscript easier since usually it was presented linearly.
I can cosign on this methodology. I used to work in an organization that used to build pdfs for accounting and licensing documentation. I used a proprietary tool (Planetpress :( ) to generate the documents using metadata from a separate input file (csv or xml) to determine what column maps to what field.
Good thing about this was as you have already outlined: It allowed for some flexibility in what was acceptable input data. For specific address formats or names we could accept multiple formats as long as they were consistent and in the proper position in the input file.
Regarding renegotiating: We didn't get that far. However, if a customer within our organization was enlisting our expertise and could not produce an acceptable input file, then we would go back to them and explain the format that we require in order to generate the necessary documents. Of course, creating our document through our data pipelines is obviously the better choice, but this was not an option in some cases at the time.
As far as doing the work of creating these documents in a tool like Planetpress is concerned, well, don't use Planetpress. You are better of doing it in your favorite language of choice's libraries tbh. Nothing worse than having to use proprietary code (Presstalk/Postscript.) that you have to learn and never be able to use anywhere.
By re-negotiating I mean in terms of quoting billable hours. A rule of thumb for a typical Postscript scraper was around 20 hours end to end (dev, testing, and integration into our workflow system).
The problem we have with a lot of client files is that they look fine but printers don't care about "look fine", they crash hard when they run out of virtual memory due to poor structure. And usually without a helpful error message, so that's more billable hours to diagnose. The most common culprit is workflows that develop single document PDFs then merge them resulting in thousands of similar and highly redundant subset fonts.
I remember writing one of my first parsers was for a pdf and I had to employ a similar methodology where I had to rely on regex and "positionally-aware fixed-length" formatting rules. I would literally chunk specific groups by the number of spaces they contained lol. I had to do very little manual intervention but, damn it all, it worked :D .
My first internship was at a small company that did PDF parsing and building for EU government agencies and it was really painful work but paid an absolute shitton.
Are you me? Wish that I had known the insertion order trick, though it isn't straightforward to implement with the stack I was using at a previous gig. (Tabula + Naive parsing + Pandas Data Munging). I can expand on a few issues challenges I've run into when parsing PDFs:
# Parser drift and maintenance hell
Let's say that you receive 100 invoices a month from a company over the course of 3 months. You look over a handful of examples, pick features that appear to be invariant, and determine your parsing approach. You build your parser. You're associating charges from tables with the sections their declared in, and possibly making some kind of classification to make sure everything is adding up right. It works for the example or two pdfs you were building against. It goes live.
You get the a call or bug report: it's not working. You try the new pdf they send you. It looks similar, but won't parse because it is--in fact--subtly different. It has a slightly different formatting of the phone-number on the cover page, but identical everywhere else. You change things to account for that. You retest your examples, they break. Ok, two different formats same month, same supplier. You fix it. Chekhov's Gun has been planted.
A month passes, it breaks. You inspect the offending pdf. Someone racked up enough charges they no longer fit on a page. You alter the parser to check the next page. Sometimes their name appears again, sometimes not, sometimes their next page is 300 pages away. It works again.
A few more months later, a sense of deja-vu starts to set it. Didn't I fix this already? You start tracking three pdfs across 3 months:
pdf 1 : a -> b -> c (Starts with format a, change to be same as pdf 2, then changes again)
pdf 2 : b -> b -> c (Starts with one format, stays the same, changes the same way as pdf 1)
pdf 3 : b -> a -> b (Starts same as pdf 2, changes to same as pdf 1 first month, same as pdf 3)
What's the common factor between these version changes? The return address is determining the version.
PDFs are slightly different from office to office, with templates drifting slightly each month in diverging directions. You have to start reevaluating parsing choices and splitting up parsers. It's difficult to account for incurring linear maintenance cost for each new supplier and amortize that over a sizeable period of time. My arch nemesis is an intern who got put to work fixing the invoices at one office of one foreign supplier.
# PDFs that aren't standards compliant
In this case, most pdf processing libraries will bail out. Pdf viewers on the other hand will silently ignore some corrupted or malformed data. I remember seeing one that would be consistently off by a single bit. Something like `\setfont !2` needed to have '!' swapped out for another syntactically valid character that would leave byte offsets for the pdf unchanged.
TLDR: If you can push back, push back. Take your data in any format other than PDF if there is any way that is possible.
You'd hope so, but some printers run some very finicky software with less horsepower than your desktop machine so can fall over on complex PDF structures. I preferred Postscript!
Not sure if I'm misunderstanding but PCL is another page description language like PS, some printers can use both depending on the driver.
Most of our Xerox printers spoke Postscript natively, these days more printers can use PDF. We generally used a tool to convert PCL to PS to suit our workflow if that was the only option for the file, because being able to manipulate the file (reordering and applying barcodes or minor text modifications) was important. Likewise for AFP and other formats. PCL jobs were rare so I never worked on them personally.
Unfortunately, when you need the output of program A as the input to B sometimes you have to jump through such hoops. I've never done it with .pdf but I've fought similar battles with .xps and never fully conquered them. (And the parser was unstable as hell, besides--it would break with every version and sometimes for far lesser reasons.)
> By looking at the content, understanding what it is talking about and knowing that vegetables are washed before chopping, we can determine that A C B D is the correct order. Determining this algorithmically is a difficult problem.
Sorry, this is a bit off-topic regarding PDF extraction, but it distracted me greatly while reading...
I'm pretty sure the intention was A B C D (cut then wash). Not sure why the author would not use alphabet order for the recipe...
[edit] Sorry, I made it read to a colleague and he mentioned the A B C D annotations were probably not in the original document. This was not clear at all for me while reading, and if they are not included it's indeed hard to find the correct paragraph order.
Even if the ABCD was in the original document, how would the computer figure out it's supposed to indicate the order?
And of course, even if the letters were there in the original document, it would be clear to a human that they're incorrect because it doesn't make sense to wash vegetables after cutting.
PDF is, without a doubt, one of the worst file formats ever produced and should really be destroyed with fire... That said, as long as you think of PDF as an image format it's less soul destroying to deal with.
PDF is good at what it's supposed to be good.
Parsing pdf to extract data is like using a rock as a hammer and a screw as a nail, if you try hard enough it'll eventually work but it was never intended to be used that way.
Actually, parsing text data from a pdf is more like using the rock to unscrew a screw, in that it was not meant to be done that way at all. But yeah, the pdf was designed to provide a fixed-format document that could be displayed or printed with the same output regardless of the device used.
I'm not sure (I haven't thought about it a lot) that you could come up with a format that duplicates that function and is also easier to parse or edit.
I think my fastener analogy would probably involve something more like trying to remove a screw that's been epoxied in. Or perhaps trying to do your own repairs on a Samsung phone.
It's not that the thing you're trying to do is stupid. It's probably entirely legitimate, and driven by a real need. It's just that the original designers of the thing you're trying to work on didn't give a damn about your ability to work on it.
It's pretty silly when you think about it. There's an underlying assumptions that you'll work with the data in the original format that you used to make the PDF.
QFT. PDF should really have been called “Print Description Format”. At heart it’s really just a long list of non-linear drawing instructions for plotting font glyphs; a sort of cut-down PostScript.
(And, yes, I have done automated text extraction on raw PDF, via Python’s pdfminer. Even with library support, it is super nasty and brittle, and very document specific. Makes DOCX/XLSX parsing seem a walk in the park.)
What’s really annoying is that the PDF format is also extensible, which allows additional capabilities such as user-editable forms (XFDF) and Accessibility support.
Accessibility makes text content available as honest-to-goodness actual text, which is precisely what you want when doing text extraction. What’s good for disabled humans is good for machines too; who knew?
i.e. PDF format already offers the solution you seek. Yet you could probably count on the fingers of one hand the PDF generators that write Accessible PDF as standard.
(As for who’s to blame for that, I leave others to join up the dots.)
... and is much more difficult to extract text from than PDF, given that it's turing complete (hello halting problem) and doesn't even restrict your output to a particular bounding box.
PDF is great what it meant to be, a digital printed paper, with its pros (It will look exactly the same anywhere) and cons (Can't easily extract data from it or modify it).
Currently, there is no viable alternative if you want the pros but not the cons
> Currently, there is no viable alternative if you want the pros but not the cons
I remember OpenXPS being much easier to work with. That might be due to cultural rather than structural differences, mind - fewer applications generate OpenXPS, so there's fewer applications to generate them in their own special snowflake ways.
This is the first time I heard of it. When I search for it I only find the Wikipedia article and 99 links to how to convert it to pdf.
The problem with this is that from an average person perspective it doesn't have the pros. There is no built-in or first-party app that can open this format on Mac and Linux.
More than 99% of the users only want to read or print it. It's hard to convince them to use an alternative format when it's way more difficult to do the only thing they want to do.
It's a Windows-thing, since W7, IIRC. It's ok now, but it has been buggy for years, and yes, who eats xps-files, so better it is, but it's not more useful.
For me, the biggest con of PDFs is that like physical books, the font family and size cannot be changed. This means you can't blow the text up without having to scroll horizontally to read each line or change the font to one you prefer for whatever reason. It boggles my mind that we accept throwing away the raw underlying text that forms a PDF. PDF is one step above a JPEG containing the same contents.
We have to fill existing PDFs from a wide range of vendors and clients. Our approach is to raster all PDFs to 300DPI PNG images before doing anything with them.
Once you have something as a PNG (or any other format you can get into a Bitmap), throwing it against something like System.Drawing in .NET(core) is trivial. Once you are in this domain, you can do literally anything you want with that PDF. Barcodes, images, sideways text, html, OpenGL-rendered scenes, etc. It's the least stressful way I can imagine dealing with PDFs. For final delivery, we recombine the images into a PDF that simply has these as scaled 1:1 to the document. No one can tell the difference between source and destination PDF unless they look at the file size on disk.
This approach is non-ideal if minimal document size is a concern and you can't deal with the PNG bloat compared to native PDF. It is also problematic if you would like to perform text extraction. We use this technique for documents that are ultimately printed, emailed to customers, or submitted to long-term storage systems (which currently get populated with scanned content anyways).
You could probably reduce file size by generating your additions as a single PDF, and then combining that with the original 'form', using something like
There's itext7 (also for java). Not sure how it compares with other libraries, but it will parse text along with coordinates. You just need to write your own execution strategy to parse how you want.
From my experience, it seems to grab text just fine, the tricky part is identifying & grabbing what you want, and ignoring what you don't want... (for reasons mentioned in the article)
I read ebooks on my Nintendo DSi for several years when I was in college; The low-resolution screen combined with my need for glasses (and dislike of wearing them) made reading PDF files unbearable. Later on I got a cheap android tablet and reading PDF files was easier, but still required constant panning and zooming. Today I use a more modern device (2013 Nexus 7 or 2014 NVidia Shield), and I still don't like PDF files. I usually open the PDF in word if possible, save it in another format, then convert to epub with calibre, and dump the other formats.
Epubs in comparison are easy, as all it takes is a single tap or button press to continue. When there's no DRM on the file (thanks HB, Baen) I read in FBReader with custom fonts, colors, and text size. It doesn't hurt any that the epub files I get are usually smaller than the PDF version of the same book.
Personally, I think the fact that Calibre's format converter has so many device templates for PDF conversion says a lot.
Is there a tool that works for the limited subset of PDFs generated by Latex? Do those documents have more structure than the average PDF? Less? It'd be nice to extract text from scientific articles at least.
I spent some time extracting abstracts from NLP papers (ACL conferences) and it was mostly straightforward. Using pdfquery to extract PDF -> XML gave each character as an element, and they were mostly ordered sensibly and grouped into paragraphs.
However... this didn't work in some cases, mainly with formatted text but sometimes with PDFs that looked like they were compiled in some nonstandard way. As a result I ended up chucking the XML structure entirely and recompiling the text from character-level coordinates. Formatted text was also an issue, with slightly offset y coordinates from regular characters on the same line.
I'm not sure I could take this experience and say that extracting _all text_ would be straightforward. Hopefully for most documents the XML is nicely structured, but I imagine there are many more opportunities for inconsistencies in how the PDF is generated when thinking about diagrams, tables etc. rather than just abstracts.
Considered writing up a blog post about my experiences with the above but imagined that it was far too niche. Code's here [1] if it's of interest.
I've had remarkably good results in general (for reading) using the Poppler library's "pdftotext" utility. Since it defaults to writing output to file, I wrap that in a bash function to arrive at a less-like pager, with page breaks noted:
The key is the "-layout" argument, which preserves original layout of the document. This ... may not be what you want visually, but makes backing out the original text somewhat easier.
Of course, requesting the LaTeX sources would be preferred.
Not a general tool but arxiv-vanity - which produces webpages of articles submitted to arxiv - works by parsing the source code that's submitted along with the PDF. You can probably use this data to train a model that converts between pdf, tex, and html.
Around couple of years ago I am working on a home project and utilised Tesseract and Laptonica for OCR. Storage and search HDFS, HBase and SolrCloud on extracted text. You can find the details here on my website. I was very impressed with conversion of hand written pdf docs with 90% readable accuracy. I have named it as Content Data Store(CDS) http://ammozon.co.in/headtohead/?p=153 . Source code is open and you may find steps on installation and how to run here.
http://ammozon.co.in/headtohead/?p=129http://ammozon.co.in/headtohead/?p=126
A short demo
http://ammozon.co.in/gif/ocr.gif
I didnot get time to enhance it further but planning to containerize the whole application. See if you find it useful in its current form.
I had a similar problem and ended up using AWS' Textract tool to return the text as well as bounding box data for each letter, then overlayed that on a UI with an SVG of the original page, allowing the user to highlight handwritten and typed text. I plan to open source it so if anyone's interested let me know.
Not a fan of the potential vendor lock in though, so it's only really suitable for those in an already AWS environment not worried about them harvesting your data.
The open source project I work on [0] returns the letters, their positions and other associated information.
We provide support for retrieving words as well as a bunch of different algorithms for document layout analysis [1]. But like the other commenters here mention, it's an extremely difficult problem which doesn't have an easy or general solution.
I was trying to build a custom library on top of the open-source library that did a bit more processing, multi-column analysis, statistical analysis of whitespace size, etc. But building something that works for the general case is difficult enough to be functionally impossible.
Despite that I think the PDF format is well suited to what it is for and there are very few "implementation mistakes" in the spec itself (no up-front length for inline image data is the main one, plus accessibility obviously). It's ultimately become too successful and as a result developers are stuck handling cases where it's being used for entirely the wrong purpose but I can't see a way to another format gaining purchase for the correct purpose (perhaps it's like JavaScript in that way, it has huge adoption because it was first, not because it does all jobs well).
Perhaps a content-first format which also handles presentation well could gain a foothold if it came with a shim for PDF viewers and software to use but I dread to think how much effort that would be.
I've also done a lot of work in this space and one thing I don't understand is why more extraction libraries don't support images as input. If your PDF isn't layered or OCR'd, it might as well be an image. I've lost count of the number of times I've downloaded some PDF extraction tool and then had to hack it into accepting an image.
On the other hand... OCR is meanwhile so good that it can be used for many PDF text extraction projects. So often there is no longer the need to bother with PDF internals, just screenshot the PDF document and parse it. A free pdf ocr service is for example ocr.space.
I'm an ML engineer, worked as a part time data engineer consultant for a medical lines/claims extraction company, for 3 years, which majorly involved in extracting the tabular data from the PDFs and Images. Developer rules or parsers as such is JUST no help. You end up creating a new rule every time you miss the data extraction.
With that in consideration, and the existing resources are little help especially on skewed, blurry, handwritten and 2 different table structure in the input, I ended up creating an API service to extract tabular data from Images and PDFs - hosted as https://extracttable.com . We cared it to be robust, average extraction time on images is under 5 seconds. On top of maintaining accuracy, A bad extraction is eligible for credit usage refund, which literally not any service offer it.
i Invite HN users to give it a try and feel free to email saradhi@extracttable.com for extra API credits for the trail.
I chuckled at your "the worst image" sample. Which still looked quite decent all things considered.
You're "handwritten" example looks a bit "too decent" as well. I can see how that works. You first look for the edges of the table, and then you evaluate the symbol in each cell as something that matches unicode.
So, how well does this cope with increasing degradation? i.e. pencil written notes that bleed outside cell borders, curve around borders, etc.? Stamps and symbols (watermarks) across tables?
"pencil written notes that bleed outside cell borders, curve around borders, etc.? Stamps and symbols (watermarks) across tables?"
"The Worst Image" is a close match to that, except it is a print.
Regarding increasing degradation - as stated above, the OCR engine is not proprietary - we confined ourselves to detect the structure at this moment, and started with the most common problems.
Hi, author and maintainer of Tabula (https://github.com/tabulapdf/tabula). We've been trying to contact you about the "Tabula Pro" version that you are offering.
Am I reading the repos correctly? It looks like Extractable copied Tabula (MIT) to its own repo rather than forking it, removed the attribution, and then tried to re-license it as Apache 2.0. If so, that would be pretty fucked up.
Not really. They import tabula_py, which is a Python wrapper around tabula-java (the library of which I'm a maintainer).
Still, I would have loved at least a heads up from the team that sells Tabula Pro. I know they're not required to do so, but hey, they're kinda piggybacking on Tabula's "reputation".
If you control the Tabula trademark (which doesn't necessarily require a formal registration), you may be able to prohibit them from using the TabulaPro name. That's exactly what trademark law is for.
William, the intention of "TabulaPro" is to give the developers a chance to use a single library instead of switching ExtractTable for images and tabula-py for text PDFs.
What do you recommend us to do, to not make you feel we made a dick move.
"commercialize the original author's work with the author"
- No, but let me highlight this, any extraction with tabula-py is not commercialized - you can look into the wrapper too :) or even compare the results with tabula-py vs tabulaPro.
Copying the TabulaPro description here, "TabulaPro is a layer on tabula-py library to extract tables from Scan PDFs and Images." - we respect every effort of the contributors & author, never intended to plagiarize.
I understand the misinterpretation here is that we are charging for the open-sourced library because of the name. We already informed author in the email about unpublishing the library, this morning, I just deleted the project and came here to mention it is deleted :)
Sorry, Saradhi, I don't think you can reasonably claim there was no intention to plagiarize. Adding a "pro" to something is clearly meant to suggest it's the paid version of something. And it's equally clear that "TabulaPro" is derived from "Tabula".
It may be that you didn't realize that people would see your appropriation as wrong, although I have a hard time believing that as well given that the author tried to contact you and was ignored. As they say, "The wicked flee when no man pursueth."
So what I see here is somebody knowingly doing something dodgy and then panicking when getting caught. If you'd really like to make amends, I'd start with some serious introspection on what you actually did, and an honest conversation with the original author that hopefully includes a proper [1] apology.
And I'm going add it's really weird that your answer ("No, No, Zero") is exactly the same as what the library author said [1] two hours before you posted. But you do that again without acknowledging the author, and with just enough format difference that it's not a copy-paste. It's extremely hard for me to imagine you didn't read what he said before writing that; it's just too similar.
If you upload a pdf to google drive and download it 10 minutes later it will magically have BY FAR the best OCR results in the pdf. Note my pdf tests were fairly clean so your experience may not be the same.
I have used Google's fine OCR results to simulate a hacker.
- Download a youtube video that shows how to attack a server on the website hackthebox.eu
2 minutes is probably long enough. I did notice that google drive doesn't seem to like it if you upload a lot of files. I have had files sit and never get OCR, but I forgot about them so they may have OCR on them now.
The open-source Ghostscript [1] can convert simple PDFs to text, while keeping the layout. I doubt it will handle some of the more complicated cases outlined in the article though.
I use it quite successfully to turn my bank statements into text, which can then be further processed.
xpdf seems to have started to respect the not-copyable-flags, while in days yonder, it didn't. So now, even something like a manual of some command-line tools or a text book on C++ or Rust, you still have to re-type the text (wtf). Time to remove and search for something better, something that does not need a 0.5GB update every 3 days (on Windows). (yes, exaggerating slightly)
Maybe it's the new QT version, OpenBSD still has the Motif one, and it works great. For Windows you have SumatraPDF which is pretty good and it's libre.
I've recently done this. Have scanned over 5,000 documents to PDF, then batch converted those from PDF to TIFF using Ghostscript, and then Tesseract to OCR the TIFF and combine both back into a searchable PDF. Tesseract may not be the worlds best OCR software but it's free and both it and Ghostscript are easy to automate.
Now all I need is a good front end search system for my document archive.
I have a Brother ADS-2700w[1] as my scanner which is network connected. It scans directly to a network share (SMB, but also supports FTP, nfs etc.) and outputs as PDF. The PDFs are basically 'dumb' PDFs in that each page of the PDF is an image all wrapped up inside the PDF container.
So that's where Ghostscript comes in. On a schedule I have a script that picks up new PDFs in the share, runs them through Ghostscript to create a multipage TIFF, that TIFF is then given to Tesseract (as it can't handle PDFs natively) which does the OCR and outputs a nice PDF with searchable text. All very simple.
The scanning of the pages is very fast, but the scanner takes an age sending the PDFs over the network - it's ethernet port is only 100mbit/s but to be honest I just think the CPU inside the scanner is slow. It also doesn't have enough internal buffer which means you can't scan the next document until the previous one has completed being sent to the share.
If I hooked the scanner up to USB, then the PC could run the Brother software which does use OCR - but it's not automatic, all it does is display the PDF inside Paperport once the scan is complete. For bulk scanning, it's not workable.
Regarding indexing - I've started looking at Solr, and it might suit my needs. I was hoping for a visual type search system, where you could see thumbnails of the PDFs in the results.
It's nice to note how several of these problems already exist in much more structured document types, such as HTML.
Using white-on-white dark-hat SEO techniques for keyword boosting? Check. Custom fonts with random glyphs? Check. I didn't see custom encodings (yet).
We try to keep HTML semantic, but google has been interpreting pages to a much higher level in order to spot issues such as these. If you ever tried to work on a scraper, you know how it's very hard to get far nowdays without using a full-blown browser as a backend.
What worries me is that it's going to get massively worse. Despite me hating HTML/web interfaces, one big advantage for me is that everything which looks like text is normally selectable, as opposed to a standard native widget which isn't. It's just much more "usable", as a user, because everything you see can be manipulated.
We've seen already asm.js-based dynamic text layout inspired by tex with canvas rendering that has no selectable content and/or suffers from all the OP issues! Now, make it fast and popular with WASM...
Hiding page content unless rendered via JS is the darkest dark pattern in HTML I've noted.
Though absolute-positioning of all text elements via CSS at some arbitrary level (I've seen it by paragraph), such that source order has no relationship to display order, is quite close.
349 comments
[ 5.0 ms ] story [ 299 ms ] threadWe've taken no intentional action to change the way the back button works - in fact, I too hate it when websites do that.
Can you PM me with some details about what you're seeing? I'm having issues reporducing it with my particular setup.
My browser is the latest (v73.0.1) Firefox on the latest build of Windows10. I confirmed the issue with all addons disbled so it is not an addon issue. I think I know what may be responsbile. When initially I load the page the back button works as intended for about a second. After that delay the page seems to load some resources from static.parastorage.com and www.mymobileapp.online. Once those resources are finished loading the back button does not navigate back to the HN article on the first press. Have to press once more. So I presume a script from one of those domains is responsible. Hope this helps!
There are sites that explicitly mess with the back button but I haven't seen one in ages.
As noted in the article, it is extremely difficult to figure out the original text given only a "normal" PDF, so you end up using a lot of heuristics that sometimes guess correctly. There's no guarantee that you'll be able to extract the "original text" when you start with an arbitrary PDF without embedded data. So if you're extracting text, neither way guarantees that you'll get "original text" that exactly matches the displayed PDF if an attacker created the PDF.
That said, there's more you can do if you have an embedded OpenDocument file. For example, you could OCR the displayed PDF, and then show the differences with the embedded file. In some cases you could even regenerate the displayed PDF & do a comparison. There are lots of advantages when you have the embedded data.
When you're done filling the form, the PDF runs form validity checks and generates a 2D barcode [1] -- which stores your all field entry data -- on the first page. This 2D barcode can then be digitally extracted on the receiving end with either a 2D barcode scanner or a computer algorithm. No loss of fidelity.
Looks like Acrobat supports generation of QR, PDF417 and Data Matrix 2D barcodes.[2]
[1] https://www.canada.ca/en/revenue-agency/services/tax/busines...
[2] https://helpx.adobe.com/acrobat/using/pdf-barcode-form-field...
The Canadian tax agency offers free storage for whatever receipts you mail them? Sounds nifty. Does the IRS (or any other tax agency) do this?
For various values of "always"
Is there a special "data" section of the PDF that includes this? Can you point me to any documentation regarding this? It sounds quite good TBH.
If someone could make a service that lets you upload a PDF that contains a form, and then let users fill out that form and e-sign it and collect the results, and then print them out all at once, it would be great.
It's not a billion dollar idea but there are a lot of little companies that would save a lot of time using it.
Again, not sexy, but it is so stupid I have to fill out a direct deposit form by hand and turn it into my company, who checks it, then hands it off to the payroll vendor, who has to check it, just to enter the damn data into a form on their end.
In theory it sounds like it should be straightforward but it hinges so much on how well the document is structured underneath the surface.
Being that these tools were primarily designed for non-technical users first the priority is in the visual and printed outcome and not the underlying structure.
One document can look much the same as another in form—uses black borders to outline fields, similar or same field names, etc, but may be structured entirely differently and that can be a madhouse of frustrating problems.
It can be complex enough to write a solution for one specific document source. Writing a universal tool that could take in any form like that would probably be a pretty decent moneymaker.
My first intuition, though, would be it may be more successful (though no less simple) to develop a model that can read from the visual of the document rather than parsing it successfully.
Open to learning something here, though!
There is a wide world outside of consumers of SaaS products for every little niche problem.
Sometimes they are baked in processes that still use PDF's to share information, sometimes they're old forms of any kind, sometimes even old scanned docs that are still in use but shared digitally. A lot of the businesses that carry on that way are of the mind that "if it's not broke, don't fix it" which is quite rational for their problem areas and existing knowledge base. They might be a potential market at some point for a new solution, but good luck selling them on a web-based subscription SaaS solution when a simple form has been serving their needs for 30+ years.
OP's problem of the PDF being the go-between to digital endpoints is more common than you might think.
The universality I was referring to was the wide range of possibilities for how a given form might be laid out. And old documents contain a lot of noise when they've been added to or manipulated. Look inside an old PDF form from some small-medium sized business sometime. Now imagine 1000 variations of that form one standard problem. Then multiple that by the number of potential problem areas the forms are managing.
Also like OP said—it's not sexy, but it's very real and having an intelligent PDF form reader and consumer would be a time-saver for those businesses who aren't geared to completely alter their workflow.
The tool could do anything with the extracted data. If it allowed you to connect to any of your in house services (like payroll or accounting) either with a quick config/API or a custom patch, or Google Drive, or whatever without complications like online-required and web accounts especially. No whole solution like that exists to my knowledge. At least nothing accessible to the wider market.
Thanks again, I just want to make sure I understand.
OTOH, a PDF form works exactly they way you’d like. Maybe there’s a small market in helping convert one to the other for collecting input from old paper-ish forms.
It lets me type in to forms - or draw text over them if necessary. Then I paste in a scan of my signature. Then save as a PDF an email across.
I've been doing this for years. Job applications, mortgages, medical questionnaires. No one has every queried it.
If you're hand delivering a printed PDF, it's just going to be copy-typed by a human into a computer. No need to make it too fancy.
* https://www.hellosign.com/products/helloworks
* https://www.useanvil.com
* https://www.pandadoc.com
* https://www.pdffiller.com
* https://www.platoforms.com
* JotForm (https://www.jotform.com/help/433-How-to-Add-an-E-Signature-t...)
* https://www.webmerge.me
(I know about all these because I'm working on a PDF generation service for developers called DocSpring [1]. I'm also working on e-signature support [2], but that's still under development, and still won't be a perfect fit for your use-case.)
[1] https://docspring.com
[2] https://docspring.com/docs/data_requests.html
[1] recently read an SEO post on okta's site. who can read that garbage?
[2] only GA ... which isn't a 3rd-party tracker.
Why not? It's not self-hosted and results are stored elsewhere.
some might add: for the purpose of resale of the data, but I don't think that's a requirement to be classified as 3rd party tracker. the mere act of correlation, no matter what you then do with the data, makes you a 3rd party tracker. in case you think that's just semantics, this is important for GDPR and the new california law.
you can turn on the "doubleclick" option, which does do said correlation and tracks you. but that's up to the site to decide. GA doesn't do it by default.
It’s still a pretty manual process, but it does the most difficult part good enough.
http://tabula.technology/
It still hits the issues mentioned in this article (surprise spaces appearing in middle of words, etc)
[1] https://tabula.technology/
[1] https://camelot-py.readthedocs.io/en/master/
[2] https://news.ycombinator.com/item?id=18199708
> Why not OCR all the time? > Running OCR on a PDF scan usually takes at least an order of magnitude longer than extracting the text directly from the PDF.
... so? Google can OCR video and translate it in something that feels like real-time; what PDF processing are they doing that is so performance bound?
> Difficulties with non-standard characters and glyphs OCR algorithms have a hard time dealing with novel characters, such as smiley faces, stars/circles/squares (used in bullet point lists), superscripts, complex mathematical symbols etc.
Sure, but more than the random shit you find in PDFs anyway?
> Extracting text from images offers no such hints
Finding an algorithm that approximates how a human approaches a page layout doesn’t feel like it would be all that hard.
Obviously it’s very easy to stand on the sidelines and throw stones, but parsing PDFs using anything other than OCR + some machine learning models to work out what the type of a piece of text feels like pretending we are still constrained by the processing costs of 5 years ago
1) Do you have trillion or so dollars at your beck and call? If not, you're not Google.
> Finding an algorithm that approximates how a human...
2) ...is generally nigh impossible even for someone with Google's resources (e.g. Waymo, although when it comes to reading, it's somewhat usable). Also, look at 1)
Unless by approximate you mean toddler level. In that case:
3) The approximation is probably useless
"In CS, it can be hard to explain the difference between the easy and the virtually impossible."
https://xkcd.com/1425/
You italicized the word 'solved' to emphasize that under technical scrutiny it is not true in response to a claim about being pedantic.
100% accuracy on identifying birds is, epistemologically, impossible.
When in doubt, use plain text. It's a million times better in every way that counts.
I wish my bank statements and such could be downloaded as plain text files, instead of massive PDF files that embed another copy of a bunch of typefaces in each file.
It's not.
It used to be long ago, but now it has full programmability with JavaScript.
Chrome displayed them fine, Preview on Mac did not.
Trying to communicate this to them was like talking to a tree, or an alien, or a room of catatonic individuals.
Thankfully I think they've fixed it now.
The PDF standard is a mess, and the number of 'tricks' I've seen done is astonishing.
Example: to add shade or border effect to text, most PDF generators simple add the text twice with a subtle offset and different colors. Result: your SaaS service returns every sentence twice.
Off course there were workarounds, but at some point it became unmaintanable.
[1] https://en.wikipedia.org/wiki/EPUB
The demand for automating text extraction is still very high — or at least it feels like it when you’re working around the clock to cater to 3 of your customers, only to wake up to 10 more the next day. We’re small but growing extremely quickly.
It’s definitely harder to get government business because the sales process is so long and compliance is so stringent. That said, we are GDPR compliant.
I've bookmarked your site for future research... but the aviation part has me curious!
Also, how do you manage things when one of those banks decides to change the layout/format?
I managed to find your email address from your GitHub profile. Going to send you an old fashioned email.
A frequent need was to parse tables describing fields (name, id, description, possible values etc.). Unfortunately, sometimes tables spanned several pages and the column width was different on every page, which made column splitting difficult. So I annotated page jumps with markers (e.g. some 'X' characters indicating where to cut).
As someone else said, this is like black magic, but kind of fun :)
Edit: grammar
I've come across most of the problems in this post but the most memorable thing was when we were asked to support Arabic, when suddenly all your previous assumptions are backwards!
See:
https://news.ycombinator.com/item?id=22156456
In the GNU Awk User's Guide:
https://www.gnu.org/software/gawk/manual/html_node/Multiple-...
Tracking column and field widths across page breaks is ... interesting, but more tractable.
Parsing statement PDFs from every bank is pretty hellish.
In the past we did purposely make it more difficult to parse our PDFs
The experience helped me to roll out an API, as https://extracttable.com, for developers.
OCR tricks? Assuming post processing dev stuff - may I know your OCR engine. We are supported with Kofax and openText along with cloud engines like GVision as a backup.
We need some metadata to rearrange and sort PDF pages for mailing and delivery (such as name, address, and start/end page for that customer).
Our general rule is you provide metadata in an external file to make it easy for us. Otherwise, we run pdftotext and hope there's a consistent formatting for the output (e.g. every first page has "Issue Date:", "Dear XYZ,", or something written on it).
If that doesn't work then we're re-negotiating. It is not too difficult usually to build a parser for one family of PDF files based on a common setup as you've said and you get to learn various tricks. It is very difficult though to write a general parser.
Personally, I found parsing postscript easier since usually it was presented linearly.
Good thing about this was as you have already outlined: It allowed for some flexibility in what was acceptable input data. For specific address formats or names we could accept multiple formats as long as they were consistent and in the proper position in the input file.
Regarding renegotiating: We didn't get that far. However, if a customer within our organization was enlisting our expertise and could not produce an acceptable input file, then we would go back to them and explain the format that we require in order to generate the necessary documents. Of course, creating our document through our data pipelines is obviously the better choice, but this was not an option in some cases at the time.
As far as doing the work of creating these documents in a tool like Planetpress is concerned, well, don't use Planetpress. You are better of doing it in your favorite language of choice's libraries tbh. Nothing worse than having to use proprietary code (Presstalk/Postscript.) that you have to learn and never be able to use anywhere.
The problem we have with a lot of client files is that they look fine but printers don't care about "look fine", they crash hard when they run out of virtual memory due to poor structure. And usually without a helpful error message, so that's more billable hours to diagnose. The most common culprit is workflows that develop single document PDFs then merge them resulting in thousands of similar and highly redundant subset fonts.
# Parser drift and maintenance hell
Let's say that you receive 100 invoices a month from a company over the course of 3 months. You look over a handful of examples, pick features that appear to be invariant, and determine your parsing approach. You build your parser. You're associating charges from tables with the sections their declared in, and possibly making some kind of classification to make sure everything is adding up right. It works for the example or two pdfs you were building against. It goes live.
You get the a call or bug report: it's not working. You try the new pdf they send you. It looks similar, but won't parse because it is--in fact--subtly different. It has a slightly different formatting of the phone-number on the cover page, but identical everywhere else. You change things to account for that. You retest your examples, they break. Ok, two different formats same month, same supplier. You fix it. Chekhov's Gun has been planted.
A month passes, it breaks. You inspect the offending pdf. Someone racked up enough charges they no longer fit on a page. You alter the parser to check the next page. Sometimes their name appears again, sometimes not, sometimes their next page is 300 pages away. It works again.
A few more months later, a sense of deja-vu starts to set it. Didn't I fix this already? You start tracking three pdfs across 3 months:
pdf 1 : a -> b -> c (Starts with format a, change to be same as pdf 2, then changes again)
pdf 2 : b -> b -> c (Starts with one format, stays the same, changes the same way as pdf 1)
pdf 3 : b -> a -> b (Starts same as pdf 2, changes to same as pdf 1 first month, same as pdf 3)
What's the common factor between these version changes? The return address is determining the version.
PDFs are slightly different from office to office, with templates drifting slightly each month in diverging directions. You have to start reevaluating parsing choices and splitting up parsers. It's difficult to account for incurring linear maintenance cost for each new supplier and amortize that over a sizeable period of time. My arch nemesis is an intern who got put to work fixing the invoices at one office of one foreign supplier.
# PDFs that aren't standards compliant
In this case, most pdf processing libraries will bail out. Pdf viewers on the other hand will silently ignore some corrupted or malformed data. I remember seeing one that would be consistently off by a single bit. Something like `\setfont !2` needed to have '!' swapped out for another syntactically valid character that would leave byte offsets for the pdf unchanged.
TLDR: If you can push back, push back. Take your data in any format other than PDF if there is any way that is possible.
Most of our Xerox printers spoke Postscript natively, these days more printers can use PDF. We generally used a tool to convert PCL to PS to suit our workflow if that was the only option for the file, because being able to manipulate the file (reordering and applying barcodes or minor text modifications) was important. Likewise for AFP and other formats. PCL jobs were rare so I never worked on them personally.
Sorry, this is a bit off-topic regarding PDF extraction, but it distracted me greatly while reading...
I'm pretty sure the intention was A B C D (cut then wash). Not sure why the author would not use alphabet order for the recipe...
[edit] Sorry, I made it read to a colleague and he mentioned the A B C D annotations were probably not in the original document. This was not clear at all for me while reading, and if they are not included it's indeed hard to find the correct paragraph order.
And of course, even if the letters were there in the original document, it would be clear to a human that they're incorrect because it doesn't make sense to wash vegetables after cutting.
I'm not sure (I haven't thought about it a lot) that you could come up with a format that duplicates that function and is also easier to parse or edit.
It's not that the thing you're trying to do is stupid. It's probably entirely legitimate, and driven by a real need. It's just that the original designers of the thing you're trying to work on didn't give a damn about your ability to work on it.
QFT. PDF should really have been called “Print Description Format”. At heart it’s really just a long list of non-linear drawing instructions for plotting font glyphs; a sort of cut-down PostScript.
https://en.wikipedia.org/wiki/PostScript
(And, yes, I have done automated text extraction on raw PDF, via Python’s pdfminer. Even with library support, it is super nasty and brittle, and very document specific. Makes DOCX/XLSX parsing seem a walk in the park.)
What’s really annoying is that the PDF format is also extensible, which allows additional capabilities such as user-editable forms (XFDF) and Accessibility support.
https://www.adobe.com/accessibility/pdf/pdf-accessibility-ov...
Accessibility makes text content available as honest-to-goodness actual text, which is precisely what you want when doing text extraction. What’s good for disabled humans is good for machines too; who knew?
i.e. PDF format already offers the solution you seek. Yet you could probably count on the fingers of one hand the PDF generators that write Accessible PDF as standard.
(As for who’s to blame for that, I leave others to join up the dots.)
Currently, there is no viable alternative if you want the pros but not the cons
I remember OpenXPS being much easier to work with. That might be due to cultural rather than structural differences, mind - fewer applications generate OpenXPS, so there's fewer applications to generate them in their own special snowflake ways.
The problem with this is that from an average person perspective it doesn't have the pros. There is no built-in or first-party app that can open this format on Mac and Linux. More than 99% of the users only want to read or print it. It's hard to convince them to use an alternative format when it's way more difficult to do the only thing they want to do.
Once you have something as a PNG (or any other format you can get into a Bitmap), throwing it against something like System.Drawing in .NET(core) is trivial. Once you are in this domain, you can do literally anything you want with that PDF. Barcodes, images, sideways text, html, OpenGL-rendered scenes, etc. It's the least stressful way I can imagine dealing with PDFs. For final delivery, we recombine the images into a PDF that simply has these as scaled 1:1 to the document. No one can tell the difference between source and destination PDF unless they look at the file size on disk.
This approach is non-ideal if minimal document size is a concern and you can't deal with the PNG bloat compared to native PDF. It is also problematic if you would like to perform text extraction. We use this technique for documents that are ultimately printed, emailed to customers, or submitted to long-term storage systems (which currently get populated with scanned content anyways).
pdftk form.pdf multibackground additions.pdf output output.pdf
Sounds like you are in need of OCR if you want to be able to use arbitrary screen coords as a lookup constraint.
From my experience, it seems to grab text just fine, the tricky part is identifying & grabbing what you want, and ignoring what you don't want... (for reasons mentioned in the article)
https://github.com/itext/itext7-dotnet
https://itextpdf.com/en/resources/examples/itext-7/parsing-p...
Not even when they try to select and copy text?
Epubs in comparison are easy, as all it takes is a single tap or button press to continue. When there's no DRM on the file (thanks HB, Baen) I read in FBReader with custom fonts, colors, and text size. It doesn't hurt any that the epub files I get are usually smaller than the PDF version of the same book.
Personally, I think the fact that Calibre's format converter has so many device templates for PDF conversion says a lot.
However... this didn't work in some cases, mainly with formatted text but sometimes with PDFs that looked like they were compiled in some nonstandard way. As a result I ended up chucking the XML structure entirely and recompiling the text from character-level coordinates. Formatted text was also an issue, with slightly offset y coordinates from regular characters on the same line.
I'm not sure I could take this experience and say that extracting _all text_ would be straightforward. Hopefully for most documents the XML is nicely structured, but I imagine there are many more opportunities for inconsistencies in how the PDF is generated when thinking about diagrams, tables etc. rather than just abstracts.
Considered writing up a blog post about my experiences with the above but imagined that it was far too niche. Code's here [1] if it's of interest.
[1] https://gist.github.com/GuyAglionby/4b55d00803710f2e2e9877fd...
Of course, requesting the LaTeX sources would be preferred.
I didnot get time to enhance it further but planning to containerize the whole application. See if you find it useful in its current form.
Not a fan of the potential vendor lock in though, so it's only really suitable for those in an already AWS environment not worried about them harvesting your data.
We provide support for retrieving words as well as a bunch of different algorithms for document layout analysis [1]. But like the other commenters here mention, it's an extremely difficult problem which doesn't have an easy or general solution.
I was trying to build a custom library on top of the open-source library that did a bit more processing, multi-column analysis, statistical analysis of whitespace size, etc. But building something that works for the general case is difficult enough to be functionally impossible.
Despite that I think the PDF format is well suited to what it is for and there are very few "implementation mistakes" in the spec itself (no up-front length for inline image data is the main one, plus accessibility obviously). It's ultimately become too successful and as a result developers are stuck handling cases where it's being used for entirely the wrong purpose but I can't see a way to another format gaining purchase for the correct purpose (perhaps it's like JavaScript in that way, it has huge adoption because it was first, not because it does all jobs well).
Perhaps a content-first format which also handles presentation well could gain a foothold if it came with a shim for PDF viewers and software to use but I dread to think how much effort that would be.
[0]:https://github.com/UglyToad/PdfPig
[1]:https://github.com/UglyToad/PdfPig/wiki/Document-Layout-Anal...
With that in consideration, and the existing resources are little help especially on skewed, blurry, handwritten and 2 different table structure in the input, I ended up creating an API service to extract tabular data from Images and PDFs - hosted as https://extracttable.com . We cared it to be robust, average extraction time on images is under 5 seconds. On top of maintaining accuracy, A bad extraction is eligible for credit usage refund, which literally not any service offer it.
i Invite HN users to give it a try and feel free to email saradhi@extracttable.com for extra API credits for the trail.
You're "handwritten" example looks a bit "too decent" as well. I can see how that works. You first look for the edges of the table, and then you evaluate the symbol in each cell as something that matches unicode.
So, how well does this cope with increasing degradation? i.e. pencil written notes that bleed outside cell borders, curve around borders, etc.? Stamps and symbols (watermarks) across tables?
"The Worst Image" is a close match to that, except it is a print.
Regarding increasing degradation - as stated above, the OCR engine is not proprietary - we confined ourselves to detect the structure at this moment, and started with the most common problems.
Feel free to reachme at manuel at jazzido dot com
Am I reading the repos correctly? It looks like Extractable copied Tabula (MIT) to its own repo rather than forking it, removed the attribution, and then tried to re-license it as Apache 2.0. If so, that would be pretty fucked up.
https://github.com/tabulapdf
https://github.com/ExtractTable/tabulapro
Still, I would have loved at least a heads up from the team that sells Tabula Pro. I know they're not required to do so, but hey, they're kinda piggybacking on Tabula's "reputation".
(IANAL)
What do you recommend us to do, to not make you feel we made a dick move.
TIA
Did you ask permission of the original author to use a derived name?
Did you discuss your plan to commercialize the original author's work with the author? Before starting out?
Since starting a commercial project, how much money have you given to the original author?
1) No. 2) No. 3) Zero.
"commercialize the original author's work with the author" - No, but let me highlight this, any extraction with tabula-py is not commercialized - you can look into the wrapper too :) or even compare the results with tabula-py vs tabulaPro.
Copying the TabulaPro description here, "TabulaPro is a layer on tabula-py library to extract tables from Scan PDFs and Images." - we respect every effort of the contributors & author, never intended to plagiarize.
I understand the misinterpretation here is that we are charging for the open-sourced library because of the name. We already informed author in the email about unpublishing the library, this morning, I just deleted the project and came here to mention it is deleted :)
It may be that you didn't realize that people would see your appropriation as wrong, although I have a hard time believing that as well given that the author tried to contact you and was ignored. As they say, "The wicked flee when no man pursueth."
So what I see here is somebody knowingly doing something dodgy and then panicking when getting caught. If you'd really like to make amends, I'd start with some serious introspection on what you actually did, and an honest conversation with the original author that hopefully includes a proper [1] apology.
[1] Meaning it includes an explicit recognition of your error and the harms done, a clear expression of regret, and a sincere offer to make amends. E.g., https://greatergood.berkeley.edu/article/item/the_three_part...
[1] https://news.ycombinator.com/item?id=22483334
I would like to get the author's comment on "tried to contact you and was ignored", as I was the one who emailed yesterday.
I have used Google's fine OCR results to simulate a hacker.
- Download a youtube video that shows how to attack a server on the website hackthebox.eu
- Run ffmpeg to convert the video to images.
- Run a jpeg to pdf tool.
- Upload the pdf to google drive.
- Download the pdf from google drive.
- Grep for the command line identifiers "$" "#".
- Connect to hackthebox.eu vpn.
- Attack the same machine in the video.
ImageMagick. convert *.jpg out.pdf
By the way, why do you wait 10 minutes? Is there a signal that the PDF is done processing?
Or is there just some kind of voodoo magic that seems to happen that just takes 10 minutes to do?
Also, I am not aware of a signal when it is done.
Now I think about it, I don't know what you mean by "upload a pdf to google drive and download it 10 minutes later".
Uploading and downloading a file shouldn't change it at all, at bit level.
I use it quite successfully to turn my bank statements into text, which can then be further processed.
[1]: https://www.ghostscript.com/
Also, muPDF for Windows: https://www.mupdf.com/downloads/archive/mupdf-1.16.0-windows... unzip in a folder and run mupdf.exe
Now all I need is a good front end search system for my document archive.
Does the scanning system you use not do OCR?
I use a Mac and Spotlight does a good job of indexing the files. I think alternatives for other OSes might be something like Apache Solr?
So that's where Ghostscript comes in. On a schedule I have a script that picks up new PDFs in the share, runs them through Ghostscript to create a multipage TIFF, that TIFF is then given to Tesseract (as it can't handle PDFs natively) which does the OCR and outputs a nice PDF with searchable text. All very simple.
The scanning of the pages is very fast, but the scanner takes an age sending the PDFs over the network - it's ethernet port is only 100mbit/s but to be honest I just think the CPU inside the scanner is slow. It also doesn't have enough internal buffer which means you can't scan the next document until the previous one has completed being sent to the share.
If I hooked the scanner up to USB, then the PC could run the Brother software which does use OCR - but it's not automatic, all it does is display the PDF inside Paperport once the scan is complete. For bulk scanning, it's not workable.
Regarding indexing - I've started looking at Solr, and it might suit my needs. I was hoping for a visual type search system, where you could see thumbnails of the PDFs in the results.
---
[1] https://www.brother.co.uk/scanners/ads-2700w
Using white-on-white dark-hat SEO techniques for keyword boosting? Check. Custom fonts with random glyphs? Check. I didn't see custom encodings (yet).
We try to keep HTML semantic, but google has been interpreting pages to a much higher level in order to spot issues such as these. If you ever tried to work on a scraper, you know how it's very hard to get far nowdays without using a full-blown browser as a backend.
What worries me is that it's going to get massively worse. Despite me hating HTML/web interfaces, one big advantage for me is that everything which looks like text is normally selectable, as opposed to a standard native widget which isn't. It's just much more "usable", as a user, because everything you see can be manipulated.
We've seen already asm.js-based dynamic text layout inspired by tex with canvas rendering that has no selectable content and/or suffers from all the OP issues! Now, make it fast and popular with WASM...
"yay"
Though absolute-positioning of all text elements via CSS at some arbitrary level (I've seen it by paragraph), such that source order has no relationship to display order, is quite close.