These folks must really have their hands full with the 3M+ pages that were recently released. Hoping for an update once they expand this work to those new files.
Re the OCR, I'm currently running allenai/olmocr-2-7b against all the PDFs with text in them, comparing with the OCR DOJ provided, and a lot it doesn't match, and surprisingly olmocr-2-7b is quite good at this. However, after extracing the pages from the PDFs, I'm currently sitting on ~500K images to OCR, so this is currently taking quite a while to run through.
> DoJ explicitly avoids JPEG images in the PDFs probably because they appreciate that JPEGs often contain identifiable information, such as EXIF, IPTC, or XMP metadata
Maybe I'm underestimating the issue at full, but isn't this a very lightweight problem to solve? Is converting the images to lower DPI formats/versions really any easier than just stripping the metadata? Surely the DOJ and similar justice agencies have been aware of and doing this for decades at this point, right?
Any guesses why some of the newest files seem to have random ”=” characters in the text? My first thought was OCR, but it seemed to not be linked to characters like ”E” that could be mistakenly interpreted by an OCR tool. My second guess is just making it more difficult to produce reliable text searches, but probably 90% of HN readers could find a way to make a search tool that does not fall apart in case a ”=” character is found (although making this work for long search queries would make the search slower).
Has anyone analysed JE's writing style and looked for matches in archived 4chan posts or content from similar platforms? Same with Ghislaine, there should be enough data to identify them atp right? I don't buy the MaxwellHill claims for various reasons but it doesn't mean there's nothing to find.
There was a post on here about a project in stylometry that analyzed HN users comment history. The tool helped find accounts that had an extremely similar writing style to a given account. The site was soon removed due to privacy concerns but many users with multiple account attested to its accuracy
It turns out stylometry is actually a pretty well-developed field. It makes me wanna write an AI browser assistant that can take my comments and stylize them randomly to make it harder to use these sorts of forensics against me
On the one side it's a shame this tool was removed because it's very interesting, but on the other hand, the main use case would likely abuse and (cyber)stalking.
That said, best to assume that the various government agencies have tools like this, and better - if you're trying to hide your identity online, don't just change users or go through VPNS/proxies/TOR but change your writing style too.
(Also I'm convinced most VPNs/ proxies / TOR nodes / public access points are honeypots)
The writing style is rather interesting. Epstein seems borderline dyslexic, but almost none of the emails I've seen are written in a coherent way, regardless of the sender.
Either people on that level rarely write anything on their own and have completely forgotten how to construct proper sentences or maybe that just how they communicate. Sort of language internal to the group.
What is the legal basis for releasing the someone's private files and communications? If they can do it to Epstein, they can do it to you, to the Washington Post journalist, to former President Clinton, etc.
Is the scope at least limited somehow? Generally I favor transparency, but of course probably the most important parts are withheld.
There are also other documents that appear to simulate a scanned document but completely lack the “real-world noise” expected with physical paper-based workflows. The much crisper images appear almost perfect without random artifacts or background noise, and with the exact same amount of image skew across multiple pages. Thanks to the borders around each page of text, page skew can easily be measured, such as with VOL00007\IMAGES\0001\EFTA00009229.pdf. It is highly likely these PDFs were created by rendering original content (from a digital document) to an image (e.g., via print to image or save to image functionality) and then applying image processing such as skew, downscaling, and color reduction.
Another explanation is that it's simply one form of lazy ineffective obfuscation performed by inexperienced relative luddites in an attempt to walk the fine line between complying with the supreme court directive & not releasing anything useful.
Other investigations into the files have found oddities like redaction of the word "don't" indicating a haphazard find-&-replace approach to redaction, possibly LLM-aided.
The DOJ/Akamai online hosted search feature is also incomplete - potentially due to some of these "digitally scanned" files not being subject to OCR.
> to pass off fraudulent or AI generated images as real.
Possibly but I don't find it compelling, if only because a significant portion of the media reportage on the files has made claims that are entirely baseless - if there were a narrative to be sold one would expect such reportage to be actively leveraging such fraudulent images.
Apparently he paid Peter Mandelson for UK government information of significant financial significance, which is resulting in him being disgraced for, what, third or fourth time? This time he's even been reported to the police.
Some of the gathered data is shown here, right? Probably not all.
Now ... that's static information though. That's not really an analysis,
most definitely not an independent (open ended) analysis. And it will only
show a very incomplete part of the full picture.
This is why I think the "release the files" movement, as good as they are,
seems incomplete. I'd rather know a lot more about how they operate their
networks, getting away involving underage women. How about secret services
of other countries? Should that not also be highly important? So why is there
not really a larger investigation as well as independent analysis? Those .pdf
files alone can not tell the whole picture. That can just be the tip of the
iceberg; and it evidently involves other countries too, with Prince Andrew
being the most famous here (aka, the UK, but we already saw that other countries
also have similar issues where people suddenly had to step away from politics
when it was found out they visited the party-locations of Mr. Epstein).
Stylometry works. I've seen it used it cases where the individual was identified from a group.
One thing that is telling about the Epstein case study is how long it has stayed in public view. Pizzagate, which involved more powerful people, was shut down faster than I've ever seen for anything else. I still remember and have archived the more extreme content it's sick.
DOJ are technically breakng the law by releasing a heavily moddified "reproduction" of the original files, not the "actual" files. The software they used "OmniPage CSDK 21.1" removes all usefull metadata and any encrypted files if any where stored.
Just on the redaction point, I did notice one email that looked correctly redacted but when zoomed in you could see some pixels from a few letters had escaped a little. It might be possible to reverse engineer the email just from that.
40 comments
[ 2.7 ms ] story [ 67.9 ms ] thread(But seriously, great work here!)
hopefully someone is independently archiving all documents
my understanding is that some are being removed
Maybe I'm underestimating the issue at full, but isn't this a very lightweight problem to solve? Is converting the images to lower DPI formats/versions really any easier than just stripping the metadata? Surely the DOJ and similar justice agencies have been aware of and doing this for decades at this point, right?
https://news.ycombinator.com/item?id=33755016
It turns out stylometry is actually a pretty well-developed field. It makes me wanna write an AI browser assistant that can take my comments and stylize them randomly to make it harder to use these sorts of forensics against me
That said, best to assume that the various government agencies have tools like this, and better - if you're trying to hide your identity online, don't just change users or go through VPNS/proxies/TOR but change your writing style too.
(Also I'm convinced most VPNs/ proxies / TOR nodes / public access points are honeypots)
Either people on that level rarely write anything on their own and have completely forgotten how to construct proper sentences or maybe that just how they communicate. Sort of language internal to the group.
Why not? Clear motive, matching timeline, mentions of that reddit account in the released FBI documents of her case
Is the scope at least limited somehow? Generally I favor transparency, but of course probably the most important parts are withheld.
There are also other documents that appear to simulate a scanned document but completely lack the “real-world noise” expected with physical paper-based workflows. The much crisper images appear almost perfect without random artifacts or background noise, and with the exact same amount of image skew across multiple pages. Thanks to the borders around each page of text, page skew can easily be measured, such as with VOL00007\IMAGES\0001\EFTA00009229.pdf. It is highly likely these PDFs were created by rendering original content (from a digital document) to an image (e.g., via print to image or save to image functionality) and then applying image processing such as skew, downscaling, and color reduction.
Sign a blank paper, scan it, paste the original doc on it. Then keep the scan for future docs.
The only reason I can think of for why someone would want to do this is to pass off fraudulent or AI generated images as real.
Other investigations into the files have found oddities like redaction of the word "don't" indicating a haphazard find-&-replace approach to redaction, possibly LLM-aided.
The DOJ/Akamai online hosted search feature is also incomplete - potentially due to some of these "digitally scanned" files not being subject to OCR.
Possibly but I don't find it compelling, if only because a significant portion of the media reportage on the files has made claims that are entirely baseless - if there were a narrative to be sold one would expect such reportage to be actively leveraging such fraudulent images.
Who paid him?
Who did get paid?
Some of the gathered data is shown here, right? Probably not all.
Now ... that's static information though. That's not really an analysis, most definitely not an independent (open ended) analysis. And it will only show a very incomplete part of the full picture.
This is why I think the "release the files" movement, as good as they are, seems incomplete. I'd rather know a lot more about how they operate their networks, getting away involving underage women. How about secret services of other countries? Should that not also be highly important? So why is there not really a larger investigation as well as independent analysis? Those .pdf files alone can not tell the whole picture. That can just be the tip of the iceberg; and it evidently involves other countries too, with Prince Andrew being the most famous here (aka, the UK, but we already saw that other countries also have similar issues where people suddenly had to step away from politics when it was found out they visited the party-locations of Mr. Epstein).
OCR is so bad of course that decoding the Base64 seems futile without a lot of effort.
Example: https://www.justice.gov/epstein/files/DataSet%2011/EFTA02609...
(More mentioned here: https://old.reddit.com/r/Epstein/comments/1qu9az2/theres_unr...)
One thing that is telling about the Epstein case study is how long it has stayed in public view. Pizzagate, which involved more powerful people, was shut down faster than I've ever seen for anything else. I still remember and have archived the more extreme content it's sick.