67 comments

[ 3.4 ms ] story [ 139 ms ] thread
The thread on 4chan:

https://boards.4chan.org/g/thread/100843783

Magnet link:

https://files.catbox.moe/jsowk8.txt

(Archived: https://archive.is/N3M8F)

--

It looks like most peers are at 86%, so should have confirmation pretty soon.

> The New York Times leak does include sensitive information - I was able to identify a database of ~1k users (email, name and surname, hashed passwords)

https://x.com/stackdiary/status/1799180195892535441

From the linked article:

> those attempting to download the data via torrent are reportedly stuck at 85% and unable to complete the download... A few hours later, the person responsible for the leak provided another torrent link, which separated the files as opposed to making them a single archive. This new torrent has three folders called nytimes, nytm, and TheAthletic. The two files inside the TheAthelic folder are iOS.tar and android.tar; we verified that they reflect the source code of The Athletics mobile apps.

How can a news outlet even have 270GB of source code to begin with? That’s like 2’000 times the entire Linux kernel…
Images/Video maybe?

Though I would not want to use that git repo!

Apparently its 5000 repos, somehow.
Multiple forks, huge blobs of data that someone stuck into a repo for some reason (had issues where someone would git add -A without a good .gitignore and wind up with several ~50-100 MB generated files in a repo)...
One node_modules directory?
(comment deleted)
Easy. It’s not code.

NYT regularly creates interactive stories and those creative developers have no idea how to deal with assets. I deal with that category often and our little projects reach half a gig unless I catch it in time. No PRs, just commit all, merge from remote and push.

(comment deleted)
It doesn't have to be their source code. They mught just have local fork of every single public repo they have used at least once in a project. I don't know a single company that does cleanup of repos that are unused so it can grow quickly.
(comment deleted)
They don't.

The person who tweeted/xed/whatever doesn't know what he's talking about.

270 gigs of data? Entirely possible.

270 gigs of source code? YeahNo.

God bless 4chan.
Yeah in particular that second comment. Stay classy 4chan.
That comment made me laugh out loud, solely because of how incredibly quick it took for it to make it's way into the discussion. That being comment #2 is insane, any screenshots of the post for articles are going to have to crop it out lmao
The one reply to that comment is also great:

> ripgrep is faster

What do people expect to find here? The smoking gun for mainstream media's space laser Covid microchip woke conspiracies?
[flagged]
I'm really curious to see how that translates to TypeScript.
[flagged]
Not sure why this is flagged when it's a literal piece of source code.
[flagged]
[flagged]
>stupid and bigoted

What was stupid and bigoted? I provided a factual instance of this happening in the not distance past.

>as a human I contributed to down voting it

I never made the claim that the flagging was automatic or non-human.

I'll repost what got flagged without snark:

This is from Twitch's source leak:

  OR replace($1,'_','') SIMILAR TO '%. (hate|kill|keel|hang|burn|gasthe)%(black|bl4ck|black|jew|trans|gay|african|afrikan|minorit|asian|nig|n1g)%'
Notice how White is missing?
Are you being like this on purpose? You've already been flagged twice.

Also, what's up with White being capitalized?

(comment deleted)
>Are you being like this on purpose?

Yes? Fact based and logical. I call it being a reality driven adult.

>Also, what's up with White being capitalized?

English may be a second language for you (or possibly you're just anti-White). Racial and ethnic groups are designated by proper nouns and are capitalized.

> What do people expect to find here?

Source code for Wordle.

IIRC Wordle was mostly JavaScript and available on the site. At least before NYT bought it. For example you could just look at the word list if you wanted to know the answer every day (it’s just, like, cheating at Wordle would be so incredibly lame and pointless).
It needs work.

The dedication to homely, friendly, inoffensive, kind, folksy words needs to stop.

(comment deleted)
And what will they think if they can't find it?
"They must have hid the incriminating stuff because they knew we were on to them."

It's 4chan. Where conspiracy theories fall flat, they'll just spin more conspiracies about a cover-up. The flow-chart process of conspiracies is designed to never disappoint the believers.

Please don't equate "conspiracy theories" and "conspiracies".
"spin more conspiracies".

So... Explicitly fabricated conspiracies shall not be conflated with "conspiracy theories"? Whatever attachment you have to either would make you a conspiracy theorist.

People "spin conspiracies" every day while prosecuting crime in court.

Conspiracy theories are when you make that kind of thing up.

A conspiracy is just one or more people working together. Putting a weird tone on that word is...weird.

Whatever attachment you have to using words wrong - would make you a fool.

That's exactly what I came here to find out. Most leaks, my imagination goes off the rails. This one just seems so god damn boring. Who tf wants to download 270GB of website source code? What are ya gonna do with it? Fix bugs for them?
I expect there's some good stuff in there. The creator of D3.js used to work for the NYTimes doing amazing visualizations. As did some other well-known developers.

My guess is that there will be also be some interesting tooling in there that they used for data-mining massive numbers of scans/pdf's snagged from disinterested/hostile bureaucracies.

this only looks like a leak from the IT/infra/"website" org, not the news org which is composed of reporters. it makes sense that they'd have different internal github instances since in media companies those are largely separate entities, and the news org likely handles much more sensitive code/data/assets with stricter access controls.
NYT uses WordPress, so that goes some way to the size...
The main site is based on WordPress? That is wild.
(comment deleted)
full list of repos in the leak: https://files.catbox.moe/jx7ksm.txt
I'm shocked that there are so many repos. To be sure, are these all really separate repos and not just directories?

I've only worked at monorepo companies, and when I see the "monorepo vs. multiple repo" debates, I always picture in my mind that we're arguing about 1 vs. maybe 5 or 6 repos--like a repo for each major project. But thousands of repos, one for every little nugget??? That is totally wild. Is this an actual industry practice?

This list seems long but closer to typical than 5-6 would be, imo.
(comment deleted)
Repos are cheap to create
I don’t think this could hury NYT in any way except bringing questions on data safety concerns.

They would have better open sourced their code in the first place.

How much of it is still Perl?
List of GPL violations... 3...2...1..!
Wouldn't it need to be agpl generally for nytimes to be in violation?
Are you implying that using any GPL licensed software forces you to publish everything you create that touches it must also be published under GPL? That's not how it works.