292 comments

[ 3642 ms ] story [ 3096 ms ] thread
Looks like fractal encoding going bad.
bit rot
That can be ruled out. Bit rot does not appear suddenly in a widespread manner and does not cause very similar visual effects in a variety of images. Also the original images are still available, just new modified versions were shown. To me it looks like a new compression algo gone wrong.
(comment deleted)
A bad bit wouldn’t even be noticeable in an image. Changing one bit would make literally no difference. Artifacts look like something the compression algorithm has messed up. Also since the images are actually intact according to Google this is likely just the compression algorithm used to browse the images.

Update: my limited knowledge on this is proof I’m wrong.

A bad bit might have no visible effect at all (e.g. Affecting part of EXIF metadata, or a totally black pixel turning again very close to total black) or can corrupt the whole image (flipping a bit in the file header or cosine quantization table).

It totally depends.

As a total layman when it comes to storage: Don't they use forward correcting codes to make sure a single bad bit can reliably be detected and flipped without affecting the data at all? Or are these used only for data transmission?
The vast majority of file formats do not do such things. Such a concern is (typically) out of scope for that layer.

Even most data transmission components don't use forward correcting codes. They typically just have error detection.

Correction: "bit rot as depicted in a movie"
(comment deleted)
Last post suggests Google has already started fixing the issue. Another suggests the original isn't lost and is available via takeout
Even the original isn’t in takeout. They separate the photo from the exif data.
These artifacts aren't present when I download the original image. This gave me quite the scare though.
I had noticed one of the images in my Google Photos account seemingly not showing up. It was very noticeable because it was one of the "People" images in my Photos account. It's a bit concerning that a photo storage service seemingly can't maintain the integrity of their data. Makes me wonder about what other data issues there are, lurking in the depths of Google Drive...
I personally found a corrupted file stored in online storage only, though it wasn't with Google, but with Microsoft's OneDrive.

I tried to open a Word document that I hadn't viewed for several years, but got an error that the file was damaged. It looks like this happens sometimes, after searching for the error online. Luckily I was able to recover an earlier version of the file (via version history), but it was alarming that it happened in the first place (also I'm not sure if important edits were made in the latest corrupted version).

It's unfortunate how many cloud services default to storing on the cloud only instead of also keeping a local copy, and don't even provide the option to opt out. Even if you choose the "keep offline" option for several services, for unknown reasons, this doesn't seem reliable (Maybe I'm 'using it wrong'? But in practice, I've found myself having to download files that I was sure I set to keep offline.)

Google is pretty proud of their record of not losing user data. As far as I know, they have never permanently lost even one byte of consumer data.

The same isn't true for GCP - there they lost some writes to customers persistent disks - but only ~50 megabytes worldwide, which is still rather good when you consider they store millions of terabytes.

It really depends what you consider lost. Thousands of users (at least) have been permanently and eternally cut off from their Google data after some AI system decided to axe someone's access to their own account. The loss is significantly over a byte. Or 50 MB.
> As far as I know, they have never permanently lost even one byte of consumer data.

It seems that every other day you learn about people being permanently locked out of of their Google account. Every byte of data stored in there is permanently lost to the person losing their account.

Perhaps I should add... "except by policy".
What if the policy is to lock accounts when data integrity errors are detected...

"We have always had better storage durability than Europa."

Note, I am not arguing that this is necessarily what happens. The thread full of contrary views on cloud storage just tickled this cynical take loose from wherever it was lodged.

They definitely had permanent data loss in App Engine around a decade ago, along with leaving any app active during the outage with copies of every database table to manually merge
> Google is pretty proud of their record of not losing user data.

As far as I know, they just haven't admitted to losing consumer data. Until they define that and put even a little bit of effort into checking whether it might hold true, they don't really have anything to be proud of.

There have been multiple mass-deletion events and Google has admitted to at least one of them - the infamous "only 0.02% of our users" incident, for example - but only because so many people were affected, the press picked up on it and google's PR firms couldn't keep it quiet.
Looks like someone is doing some unauthorized time travelling.
Seems like the back to the future theory of time travel. Photos start fading.
For context, I know this is a display issue and not a storage issue and you can get your originals out via Takeout.

That being said my photos are the only digital files I really care about not losing. My photos and videos from my phone get backed up to iCloud, Google Photos, OneDrive and Amazon’s photo storage that comes with Prime.

My videos get backed up to all of the above except Amazon’s storage.

You should back them up locally as well even though that has its own failure points. I consider anything other than a dedicated local and/or cloud file backup system a nice supplementary place to store photos but not primary.
At one point I had them synced to my Windows computer via iTunes and that was backed up to Backblaze.

But, I don’t have a personal computer anymore and soon won’t have a personal residence. My wife and I are going to be digital nomads traveling across the country for a few years.

1TB thumb drives are tiny and can hold a TON of pictures.
Same here. I lost some photos years back and decided to never make that same mistake.

My backup flow is: phone, iCloud, local ZFS array served by PhotoPrism (synced using PhotoSync), Backblaze.

Thanks for sharing. I would add a par2 in there somewhere. Not that my system is as good as yours yet.
The local ZFS array is the “source of truth”. ZFS has built-in end-to-end checksumming. I also have a monthly scrub that should correct any untouched corrupt blocks.

Details on ZFS checksums: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Chec...

Thanks, that is one way to do it. But if you don’t know about par2 (sorry, I can’t tell from what you’ve said) it’s worth chasing down. It is way, way beyond checksums in what it can do.

It’s an old project and information on the internet is getting thin but the math is sound, the tool works, and there is a wikipedia page:

https://en.m.wikipedia.org/wiki/Parchive

Edit: btw use of par2 files can be added on as an extra step with minimal storage overhead, without replacing anything you have now.

I’ve heard of par2 through Usenet, but I’ll definitely do some more reading. Thanks for the pointer!
Once a year I select about 100 of my favourite pictures from my phone and have them printed and bound to a book. I suspect this is likely to be the only way my descendants will be able to see them.
I've started picking my favorites and printing on metal/canvas.

My home backup is going to be it's own problem after my death.

That's why I, and every other enthusiast photographer I know, keep my archive locally, meaning on my laptops hardrive and an external one as backup. You cannot trust any cloud provider to properly maintain your data, and external storage is becoming cheaper every year.
> You cannot trust any cloud provider to properly maintain your data, and external storage is becoming cheaper every year.

Exactly, and setting up a NAS has never been easier.

For 99% of people, a NAS is overkill. Plug-and-play USB drives get the job done at a fraction of the complexity.
I love having a NAS that replaces Time Machine, Dropbox, iCloud, Netflix, Spotify, Disney plus, Hulu, runs home automation, has my security system on it, etc etc.

People use so many cloud services and having it all private and self-hosted is absolutely amazing. It’s really not that hard to set up either, and there’s little to maintenance outside replacing a failed drive.

I vastly prefer content be cloud based. Ever since Spotify came out, I've never downloaded music illegally again -- it's just so much better and more convenient. Can't imagine going back to what.cd days and having to download and maintain a music collection manually.
Sure, but then Netflix removed stargate on a whim and one of my favorite shows went poof.

That started me down the path to owning all my own media.

It also didn’t help that iCloud Photos is a privacy nightmare.

People here likely know how to do all that safely, but lots of less technically inclined people end up with their data encrypted and have to choose between starting over or paying a ransom. I have a QNAP device so I follow their subreddit and there's a steady stream of people who have lost everything to ransomware because their NAS was visible from the internet. I follow the standard advice from that subreddit - keep it behind a firewall with UPnP turned off and no ports forwarded to the NAS and never enable MyQNAPCloud (or whatever it's called).
Mind sharing your setup, I’d love to replicate it. Thanks
8 bay synology with 60TB usable, multiple docker containers running for tools as part of my setup.
A toaster box NAS with hotswap drives, RAID, and support for Android and iOS is a reality.
One benefit of a NAS: my data's robust against any single drive dying, which is something that's happened to me, friends, and family.
What will you do if there is a fire and you lose your laptop and the external hard disk?
Law of propability: I had zero dataloses due to fire, some data loss due to loosing a thumb drive backup and some data losses due to cloud storage shennenigans.
Hmm. I highly recommend a three way redundancy as others have mentioned. Put a copy in the cloud or store your harddisks somewhere else.
You are right, a cloud storage would make things a tad saver. Personally, I made a point in not putting anything in the cloud, especially none of the big providers, since I value my privacy and just don't trust any o those companies.
Create a new backup from the cloud provider one. Vice-versa if the cloud provider backup gets lost. Chances of loosing both at the same are minuscule.
(comment deleted)
I do both, personally. Have everything on my HD and on the Google cloud. That seems like an appropriate level of redundancy.
For the really important digital stuff (almost 30 years of family digital photography and video and some paperwork) I have a rotating backup schedule. Two of the drives in the rotation stay at my mother's place. Nothing fancy and it's mostly when I can bother to remember. About once a year or so. Still, some peace of mind.
> You cannot trust any cloud provider to properly maintain your data

I trust any cloud provider a lot more than my lazy self.

> and external storage is becoming cheaper every year

Sure, but electricity isn't. At the current price in the UK it seems like running a small NAS would be 10 GBP per month, more than most online storage solutions.

That is without counting the initial investment of buying the NAS, the cost of maintaining it (if I have to spend an hour a month on maintenance, that's $1000+ a year of time wasted), the footprint of the NAS, the noise, the heat, etc.

Why not both.

A backup in one place isn't a backup. The 3, 2, 1 rule and all.

Of course, but then my solution will be an external drive, not a NAS.
Yeah. I have a NAS for other purposes but redundant external USB drives is the easiest solution. You can even periodically make a clone and store it somewhere else.
It sounds like you know... very little on the topic. My NAS requires no maintenance, it makes no sound while running, and costs maybe pennies to run.

Meanwhile, there hasn't been a cloud provider without massive reliability or data loss issues. My NAS has better uptime than Amazon, Microsoft, or Google.

Their estimate of £10/month seems reasonable to me – that's basically 1 or 2 kWh of electricity per day (can't find current UK electricity rates). Is your NAS under 40W power draw?

> costs maybe pennies to run.

Pennies per hour, per day, or per month?

My RPi 4b runs on a 18W power supply, which would put it at 432Wh per day if running at max load (It doesn't, it idles at <3W most of the time). Any arm NAS is going to have a similar power draw, and you really should pay attention to the power draw when choosing a NAS.
What about the storage devices (the S in NAS)? My external HDD (just for reference) has an AC adaptor that outputs 12V at 1.5A, so that's another 18W. And most people who care enough about storage to have a NAS have multiple disks...

I'm not saying it isn't under 40W - as you note, if it's idling properly, it's pretty easy to have a NAS including disks under 10W idle. But the electricity cost is not negligible when you are running something 24/7. Even 10W24hr30day is 7.2kWh per month, and most places cant count on 8c/kWh electricity anymore.

> My RPi 4b runs on a 18W power supply, which would put it at 432Wh per day

Which would be 13 kWh per month. If we're comparing to monthly fees for cloud storage, it's important to look at the same time frame.

> What about the storage devices (the S in NAS)?

I have two SSDs on it that are powered by the RPi itself, so the whole package is < 18W. Also, SSDs use way less power than HDDs.

> Which would be 13 kWh per month. If we're comparing to monthly fees for cloud storage, it's important to look at the same time frame.

Yes, running at max load. A more realistic load will be 3W/hr -> 72W/day -> 2kWh/month. That would put it at ~0.3€/month at a realistic load and ~1€/month at max load (0.15€/kWh where I live).

You also have to take into account there's other stuff running on such "NASes", like the PiHole, a git server, a plex server, which are going to cost you separately if you need them. On the other hand, my setup cost me ~300€ (RPi, charger, drives, USB-SATA adapters), which buys a whole lot of VPS and a whole lot of online storage.

Yes, SSDs would certainly help cut your power load down significantly! My system has 3 HDDs and it's a fairly small NAS compared to many - SSDs are prohibitively expensive once your storage needs get above a couple TBs.
> My NAS requires no maintenance

So you took it out of the shipping box, plugged it to power, and pressed the power button, and everything was up and running?

And since then you haven't had to update any software on it, reboot it, unplug and replug an ethernet cable, shut it down and opened it to add storage, etc.?

> it makes no sound while running

Looking at a basic NAS on Synology shows that it's around 20 dB, which is admittedly very little, but is not 0.

> and costs maybe pennies to run

Again, looking at the same NAS and using my actual price per kWH, it's around $10 a month.

> there hasn't been a cloud provider without massive reliability or data loss issues

My searches haven't turned up any article about a data loss from any of the 3 big cloud providers. Even the story here, while scary, is just about compressed files, not originals.

---

I'm not denying that a NAS has advantages (the fact that it's a lot more than just a data backup), but in the context of backing up files, it's not at all convenient compared to using online solutions.

(comment deleted)
As an enthusiast photographer with a lot of inner circle friends: we encrypt and backup to cloud. 0 degradation.
Can you share some specifics on the setup? Trying to do something similar. Haven’t tried it yet, but MacOS can create encrypted “sparse images”, which can be mounted and contain files. I’d like find to a better way, especially something that could stand the test of some time (10 years seems reasonable).
Now imagine you are old like me and got your first camera in 1972, and still have all the film.
There are services that will take your film, and get you a pile of images back. I did that some years ago, as my folks had hundreds of slides.
what happens when your house floods, burns down etc? you need to keep another backup off site somewhere, i keep it at my parents house just in case.
I like that. Convenience of phone camera + curation, quality, and resiliency of physical media.
After a few hundred years, most of your descendants may live in a computer.
Even after a few millennia, I doubt it. The human experience is inseparable from the human body.
That’s a pretty bold statement. The null hypothesis is that there’s nothing in the body that cannot in principle be emulated in silico to whatever precision necessary.
(comment deleted)
Why is that the null hypothesis? For one thing, the only experience we have is existing as bodies in a real world, so that should be the default with any other claims being supported by evidence. For another thing, simulating a bridge falling down is not a bridge falling down, and it's not proven that simulated thinking is thinking (or even, possible).
Sad news for people hoping for a life after death, though! (ie nearly everyone)

I think it’ll be possible to create inorganic ‘brains’ that have human-like subjective experience, because to think otherwise reduces to some form of vitalism, but I doubt we’ll be able to move specific individual consciousness across technologies.

> (ie nearly everyone)

Are you sure about that? That does not sound right to me.

Well, I’m assuming the majority of people currently alive (and even more of those who have ever lived) are religious, and the majority of religions followed hold with the Heaven & Hell / Cycle of Reincarnation until Enlightenment model - and that experience in the afterlife was still recognisably ’human’.

Though, saying that, OP’s point about needing a human body to have human experiences may be correct if souls get a new human body in the afterlife - I’d just assumed liberated souls, like ghosts, had no physical form.

There’s a vast 49% gulf between nearly everyone and the majority.
What makes you say that? You could literally be a brain in a jar or be hooked up to god knows what "deep diving" / "full VR" experience humanity will come up with.
> The human experience is inseparable from the human body.

I'd say that the human body is just one very sophisticated I/O device.

The fact that we can't replicate it now doesn't mean that it wouldn't be possible after a millenia of engineering, even though this replacement device might as well be a partially organic one in the end.

That said, there are no guarantees that we don't fall into a sort of "dark age" of no more growth/research into any new directions, or don't wipe ourselves out altogether.

After a few hundred thousand years, sure.

Nowadays it's only available inside the world of Black Mirror and similar fiction.

They're going to be p!ssed at degradation issues like this one.
> printed and bound to a book. I suspect this is likely to be the only way my descendants will be able to see them.

My experience is that when someone dies, all their paperwork gets left in a drawer for years until the next house move and then it's all thrown out.

My family has/had a handful of generations worth of records. The problem is finding out who last had what. Some of it has been backed up digitally, but finding a thumb drive is a lot harder than spotting a man sized pile of boxes.
But at least people will notice the photo book and at least have one look through. And it might even be a great moment when family members together can look at it and discuss memories. And quite likely it will be kept by a descendant. And it can be found one day by their descendants.

The digital archive or cloud storage will likely in many cases not even be discovered. And if it is discovered, it will probably be such a load of images, good and bad, that no-one will have the time to really look it through and appreciate it.

But for sure, both options combined is probably best.

Don't worry, the images will be used to train an AI which will feed derivations of them into a replicant to keep them sane.
My gmail account has a deadman’s switch to grant full access to my next of kin.

If I had a gigabytes of photos of my grandparents or great grandparents I would spend hours pouring over them looking at what their life was like before I knew them. That would be such a neat keepsake.

I imagine someone may one day do the same for me, but sadly most of my photos are clogged with mundane screenshots and repeint pictures and other junk.
I hate to tell you this but many people have been locked out of their Google Accounts despite knowing the correct username/password combination. Their security policies and ML algo locks accounts and requires 2FA to be used from time to time. How confident are you that in your passing, your next of kin will be able to get into that account if 2FA or backup codes are inaccessible?

Given their is no real customer service team that handles their free tier accounts, I am personally reluctant to assume my Gdrive data will be salvageable.

According to the Google Inactive Account support page, they aren't actually assuming the account, they're just getting access to the data. So, presumably, they'd just download the data and that would be it.

https://support.google.com/accounts/answer/3036546?hl=en

(That's not to say there aren't other concerning reasons why that data may not be accessible.)

That is lovely to know about. I was only aware of the dead mans switch feature which deletes data after a period of time.
My experience is that when someone dies, all their paperwork gets left in a drawer for years until the next house move and then it's all thrown out.

My experience has been the opposite, with relatives eagerly snatching up the old photographs.

My boss is young and one of those people for whom digital is the only way. Then one day she got a box of old photographs and was absolutely amazed that she could turn them over and read where they were taken, on what occasion, and who was in the photos.

She now prints out all of her photos and writes notes on the back of them and keeps them in a box, just like people have done for the last century.

I was on the subway in Toronto in 2017, two college age girls had an inch thick stack of 4x6's that they were going through. It was a strange throwback to the pre-digital era.
The point is for someone to discover them years later and post them on Reddit.
Nope. It will be passed to the children. But likely not the grandchildren. Even if they do, the knowledge behind the pictures will be lost and grandchildren wont know who's who in most of the pictures. (grand uncles aunts etc). And grand-grandchildren may not even know grand-grandfathers anymore...
I'd like to do this myself. What service do you use for it?
rsync.net is my off-site/cloud of choice. And they keep your bits on ZFS.
Check your local independent print shop. They're not expensive.
What service do you use?
My wife does a photo book every year, combining the best of both our photos. It’s a bit of a chore, but I’m so glad she does it, they are so much nicer to browse through. The kids love looking at them all from time to time.
Photos probably worn off from all that AI training /s
My first thought was that Google as recompressed everyone's images to save space and corrupted them in the the process. Hopefully everyone gets their images restored.
I would have rather thought it would be some sort of image enhancement algo gone wrong.
Yeah it's definitely some overflowing brightness enhancement thing. You can see that in the before/after photo that someone posted.
I was thinking something similar: it has a distinct posterization look that I tend to get when I mess around with images and reduce the bit-width of my input a little too much and my integers overflow. But those tend to result in "zebra stripes" like patterns, so you're probably right that the issue is more one of not using saturating integers while amplifying a value above the bit-width of its container.
(comment deleted)
You can boost contrast to make low bitrate images look better, so it could be a combination of both.
In the thread they say that the original files are available in Google Takeout, so it'll probably get fixed eventually.
Every time Google tells me my 15 GB of storage is almost full, I download every photo and delete it from their servers.
Old problem. If you have only one copy, you have none...
This is an indication that Google underestimates the value of data stored by users.

E.g. if this was about storing monetary data, this would never have happened.

Do you have any suggestions for a secure/private option to store photos? I'm not sure what would be best. (Dang, I really want to move away from Goo now)
Your own devices, with a proper backup solution (which you should already have with or without your photos in it). Nobody can mess with your data then.

Personally, I use Syncthing to sync between devices, including a NAS as a target, which regularly backs up to Backblaze.

This sounds a bit like what I do. I r-sync two different libraries to NAS then upload them to Backblaze B2. Post r-sync success result to a muted Slack channel and failure to an unmuted alert channel.

rsync -av /Source /Destination rsync -h

iCloud
iCloud may be secure, but private? No end-to-end encryption and Apple scans the user's content. It's no better than Google Photos in that regard.
I'm using Nextcloud. Works great.
Do you self-host it? Do you access it with Android and iOS?
Amazon photos is pretty good as a like for like alternative
As a starter, look for a "dumb" service that doesn't treat photos differently from any other type of binary blob. No thumbnails. No grouping by date or location. No tagging other people. The storage service should only concern itself with safekeeping the binaries you upload, not analyzing and processing them. Use your own machine to do the latter.
I stood up a pair of QNAP NAS for both my parents and I, and encrypt-then-backup their photos/documents to my NAS, and vice-versa. RAID 5 or 6 keeps hardware failure at bay, and keeping them tucked away in our respective homes makes fire or flood irrelevent (for this).

QNAP or Asustor has standalone apps my parents can use as well. Without getting your parents involved, you could either pay for remote hosting for your off-site, or trade RAID space with a friend.

Definitely use client side encryption, if only because it defeats attempts to "repair" image data or rewrite exit headers, etc.

Another good test: upload your photo collection, then download it and binary diff the two sets of files.

Managed Nextcloud hosting in the EU is an option.
> E.g. if this was about storing monetary data, this would never have happened.

Let’s hope none of the images were NFT “assets” then haha

But with nfts don't you legally own them?
If you subscribe to the cypherpunk mantra that ”code is law“, then yes. Otherwise, if you stick to the law that lawyers and judges apply, then mostly no. NFT ownership does not come with any rights to the underlying asset, unless specified in an accompanying contract.
It's naive and/or biased to think that. Everyone makes mistakes, everywhere, all the time.
That's a fairly serious accusation to level. Google's bad behavior here is not that their compression algorithm ruined users photos. Google's bad behavior is running a compression algorithm at all.
Every photo sharing and storing site compresses photos for view. The originals are still there (read further down in the issue).
That's not the point. I doubt that the engineers who worked on the compression algorithms would store their own precious photos using the same techniques.

Also, when was the last time you heard a banker say that everyone makes mistakes all the time.

Hi, I work on the processing algorithms at Google like the one here (not sure about this particular issue) and I use Google photos for all my old photos. People at Google read Hacker News and fix things, they just don't comment on specifics for things like this.
> This is an indication that Google underestimates the value of data stored by users.

That becomes apparent if you ever had any issue with a google product. There's no way to resolve issues outside of canned answers from "AI" systems and public forums.

Or maybe you overestimate the data of users?
No doubt the average value of data is not very high. That doesn't mean it also holds for the most valuable data stored.
most users aren't paying for Google Photos. sounds to me like they're accurately estimating the value.
>Google underestimates the value of data stored by users.

Of course they do. How can they possibly put any value on endless PB of holiday snaps? They only care that you are now reliant on them to store your memories.

Could it be some sort of clipping/banding issue when decompressing? The way it follows curves would be unusual for photos corruption.
Yikes, this is kind of scary. Google Photos is the only place to find some of my older photos. What’s the easiest way to download my entire library for backup on a physical drive?
Google Photos used to have the option to sync all photos into Google Drive but that feature was removed. I’m guessing the raw data is still available via Google Takeout
I do takeout once a year for such an eventually.

https://takeout.google.com/

I have been having real problems trying to do Google takeout with my Google Photos. I keep getting an error emailed to me saying "However, we were unable to create a copy of all your files. You can still download your files until September 25, 2022, but please be aware that there will be data missing."

Apparently I am not the only one[1][2]. Other people have a mix of errors[3]. It is not the reliable data extraction I was expecting for something as valuable as my family photos. Google is losing a lot of trust from me based on this.

[1]: https://news.ycombinator.com/item?id=25591630 [2]: https://sysc.org/what-is-google-takeout/ [3]: https://www.reddit.com/r/googlephotos/comments/khited/google...

I used this not too long ago: https://github.com/gilesknap/gphotos-sync and it stores photos and videos in a YYYY/MM directory structure.

It works extremely well, and you can re-run it any time to sync new files locally too.

I used it like that for a few months before I finally installed syncthing on my phone and stopped using Google Photos altogether.

Now what I do is take photos on my phone, have them sync to a NAS. And on the NAS I used a modified version of this https://forum.syncthing.net/t/android-photo-sync-with-exifto... to build up a YYYY/MM folder organisation and move files older than 30 days from the syncthing folder into my archive. My archive is then in my Plex so it's still accessible to me.

In essence: 1) Take photo (implicit sync to NAS), 2) wait 30 days, 3) archive photo into long term directory naming convention, whilst making available to Plex and deleting the version from my phone (by deleting the syncthing version it will delete the one on the phone after 30 days too).

Do you know if gphotos-sync downloads the photos in their original quality/version?
It does.

The only thing that it gets wrong is that it cannot restore the file created date to video. But in the grand scheme of things, that's a nit.

Unfortunately it looks like it's subject to the same limitation of the Google Photos API: Photo download compresses the photos even if you ask for the original file https://gilesknap.github.io/gphotos-sync/main/explanations/g...

I've been putting off a Google photos back up for years, and this was my excuse - waiting for this issue to be resolved. But I'm being a fool, 60% compression is better than losing everything.

Ah you are correct.

I have just compared originals I had which were uploaded to Google Photos vs the ones downloaded.

Visually there is a barely perceptible difference in a side-by-side comparison on a high quality and calibrated monitor. I wouldn't have noticed if it weren't side-by-side and zoomed in.

On the file size, it's gone from a 6MB jpeg down to about a 1.5MB jpeg.

On the smaller files that difference is far less, but jpegs above 3MB seem to see the biggest reduction in file size.

Thankfully I have all of my originals that weren't taken on the phone camera so I don't mind this so much as running my exiftool script over the original archive produces the same naming convention and I can merge those folders and still have the originals I care about.

Sheesh though... such a thing for Google to screw up on.

Not your server not your photos?
Impressive compression glitch. Bitrot evolved.
Nothing a little Stable Diffusion can't fix. Soon, digital memories will be just like real ones — slightly different every time we look back, but with no way of knowing what's changed
> with no way of knowing what's changed

You can just hash them locally or use AI to identify modifications.

Exactly..what is proof of existence for $400 Alex
The promise of the cloud was that you don't need local storage, because Someone Else will remember things for you, and you just ask them what you remember. This is also the problem of the cloud.
"The promise of the cloud was that you don't need local storage, because Someone Else will remember things for you ..."

Agreed. And that promise has been fulfilled - although it costs more than nothing and you need to do your homework.

You know these providers are proven bad actors. You know their business model(s) are antagonistic to your goals. You also know there are stable, time-tested alternatives.

What did you think was going to happen here ?

I would say at least partially aligned with your goals otherwise why would you use them in the first place?
They’re unaligned with rsync.net’s goal of getting people to pay for services.

Edit: to pay a minimum of $12/month??

ok and then what? :D you still can't recover original
That's already what browsing most websites feels like.
> slightly different every time we look back, but with no way of knowing what's changed

This is already the case with different image compression algorithms improving on one another. Users couldn't and shouldn't tell the difference between JPG, or slightly smaller / resized JPG, or WebP, or whatever. I personally don't think image hosts should have to alert users when making undetectable changes to their images like this.

I'm sorry, but JPEG compression is extremely noticable especially with non-photographic material such as web graphics. The compression is lossy by nature, and predictably introduces noticeable artifacts. WebP on the other hand has support for lossless encoding.

It's simply not the same thing. If I opened my file browser one day and saw that all of my FLACs had been converted to MP3, I would go on a murder spree.

"Soon, digital memories will be just like real ones — slightly different every time we look back, but with no way of knowing what's changed ..."

Well, let's see:

ssh user@rsync.net sha256 photos/IMG_1234.HEIC

Yep, checks out.

Sadly, you stored only the MD5 back then.
We don't store any checksums - we just store your files.

The checksum commands:

(md5, sha1, sha224, sha256, sha384, sha512, sha512t256, rmd160, skein256, skein512, skein1024, cksum)

... are available to run as you see fit.

I think you misinterpreted the comment you're responding to; they meant that "sadly, you [the user] never thought to store a high-quality hash (e.g. SHA256) for integrity comparison with your stored data; and the hash you did decide to archive for integrity-comparison was a low-quality one (e.g. MD5) that can be trivially preimage-attacked such that a cloud provider could silently replace your data with a different one — with the same low-quality hash — without your knowledge."

Nothing about rsync.net per se, other than the general idea that any data you put on "somebody else's server" can't be trusted to stay the same if you don't have a high-quality integrity-comparison content-hash of that data kept somewhere.

MD5 still has no known preimage attacks. Neither does MD4, for which collisions are absolutely trivial to generate.
To be fair, it may yet turn out that MD5 is (and has always been, we just didn't know it yet) a low-quality hash that can be trivially preimage-attacked. I think that's unlikely[0], but I can easily imagine someone reasonable being a bit more pessimistic. I certainly wouldn't willingly rely on MD5 preimage resistance for anything important.

0: I'd be willing to bet money at 1:1 odds that MD5 preimages still cost at least 2^96 bit operations (out of a nominal 2^127 hash invocations) 5, 20, or 100 years from now.

ZFS defaults to fletcher4 for checksum. It has never had MD5 support.
MD5 is weak against malice but still much stronger than something like CRC if your goal is to detect random corruption.
No, it's not. MD5 can still protect you from someone modifying the file without changing the hash.
Expecting general users to use checksums is exactly the answer I expect from HN.
A solution around archive fidelity isn’t far from this idea though. And how hard is it to sling a find command to collect all your photo hashes?
Extraordinarily difficult, judging by GP's cynicism. Personally it is appalling just how little people are motivated to be nomically tech-literate in this extremely tech-driven society.
Tech is the only industry that asks this of it’s users, it’s just laziness on the developers part. People needing to know about hashing in order to save photos would be like people needing to know what refrigerant their A/C uses in order to keep cool.

Can you? Sure. Are their weird edge cases where it’s mildly helpful to know as opposed to having to ask someone? Debatably. Should you need to in order to not have things break for no reason? Absolutely not.

I kind of agree with you both. We do ask a lot of our users. But also, there is no A/C system out there nearly as complex as software. It is hard. For everyone. We can do better, but what does that even look like? Consumer level technology consumption is not in a great spot, but it also isn’t horrible. Power users who can get more out of computing will be a thing for a long time to come.
Lots of things are complex and confront people to deal with it directly, we just also have legions of people that we can pay to step in for us and a culture where laziness and convenience are core principles to our everyday lives. Not to mention a political establishment that touts the "virtues" of a simple, ignorant life.

Maintaining a house or yard, for example, or taxes. Driving also has a pretty high learning curve -- high enough to mandate driving lessons -- but once we learn it we're set for life. (Tech is the same way here....)

why do you assume GP is talking about general users? as you point out, this is HN: it’s reasonable to think that most comments here are written for that mostly-tech-literate HN audience, unless they state otherwise, no?
Yeah. And somebody may put up an Open Source repo for doing that 'easier'. But of course, nothing that any average user on the net could even know to exist, leave aside actually use...
(comment deleted)
> Nothing a little Stable Diffusion can't fix.

As someone who works in generative modeling, that's probably what is creating these issues in the first place. Kinda like how you can see certain GAN type of patterns on Netflix when there is upscaling or if you don't turn off the fancy features on your TVs. Of course, finding how these images are corrupted leads to better models so they will be solved. This isn't the first time Google has edited photos and made them worse and it won't be the last. But the trend is that the photos get better.

At least one user on that thread is reporting it’s fixed. It’ll be interesting to see if all do.
I've been long considering switching my Google Photos library fully over to ente [0], of course alongside my existing monthly backups to local HDDs and an annual backup to a cloud server via borgbackup.

With every passing week, every Google fuck up like this and with ente's great pace of development, I draw myself nearer and nearer to the precipice of full switchover. I will soon be having my last Google Takeout.

[0]: https://ente.io/

I remember trying ente, but the mobile app had a notoriously low amount of features and low FPS (felt like 30fps on my modern, 120Hz phone - stuck out like a sore thumb). Has there been any progress on that front?
I think flutter is pretty bad, some animations feel off, but core functionality like scrolling through hundreds of pictures functions just fine.

I switched to the service completely.

Hey! One of the founders of ente here. We have shipped a bunch of performance improvements and features[1] over the last few months. But we are far from where we want to be.

If you happen to revisit and find something lacking, please write to vishnu[at]ente.io, we'd love to make ente work better for you.

[1]: https://roadmap.ente.io/top-released/

Unrelated question: Is the name meaning ”duck“ in German intentional? I had a chuckle.
"duck" means "mine" in Malayalam (my native language), and I have a weird obsession with rubber ducks, so it just fit, and perhaps most importantly, the domain was available :)
That's why I use Dropbox. They added a lot of new Photos features.
I use Dropbox and Google Photos. Photos for sharing and Dropbox + local backups of the Dropbox folder for a more permanent storage solution.
Seems unlikely Dropbox would be immune from some sort of issue like this. Probably best to have your photos in storage somewhere else (ideally both locally and off-site).
You know how the mantra in the crypto community is "not your wallet, not your coins?" This applies to this situation. Once you trust Google with your bits, don't be surprised when they modify them without telling you.
Us this the beginning of some Back To The Future moment in this timeline?
I keep intending to launch a service that downloads cloud data and sends it to you on an SD card.

I hope to fix up my prototype and launch it next year.

https://www.clonecamel.com/

If anyone knows of a quality source of bulk SD cards or flash drives in high capacity please let me know.

The susceptibility of the SD card to data corruption is a big showstopper. The only digital files I've had corrupted were stored on my phone's external SD cards (after 3 to 7 years). When I looked into it, it seemed to be an issue related to lack of radiation hardening. So if you do make this service, please include some error detection/error correction metadata.
That is good feedback

The idea would be to subscribe to have this data shipped once or more per year so hopefully some of the data would survive.

Any advice on error correction metadata and tools?

Only tangentially related but I just checked the Google Photos iOS app to see if any of mine are corrupted, and it won't even allow me to view my cloud photos without giving it full permission to all current photos on my device?! Isn't this against the app store policy? You can't deny me access to one feature just because I refuse to grant permissions to a different feature.

But because it's Google I'm sure they can break whatever rules they want as long as it doesn't involve trying to avoid Apple's 30% fees.

And it will spend its spare time analyzing every single photo in your library.
Google doesn't scan photos that don't exist on their servers.
... currently.

Trusting a dubious central authority up until you get burned is a bad heuristic.

I agree, which is why I don't pay for cloud storage.
Especially when Google has said "we won't do thing X", then been caught already doing it, or just silently starting to do it.

Just location tracking. The lies. Over, and over, and over, and over again the lies.

Institutions of that nature are incapable of following through on nearly any promise made to the public at large; every promise is a lie, whether they break it or not, because they can't be said to have the intention to uphold it.

I see no reason not to believe that Page and Brin sincerely desired Google not to be evil; but it's the institution decides to do evil, not any individual, so they did not have (or could not maintain, as the organization scaled) the power to guarantee this. No one does.

What's the difference? Potentially apple itself can fecide to scan all your photos on the phone....
I don't use an iPhone, and I use an aftermarket Android ROM.
Fun fact, Xiaomi does by default on their stock OS.
Who would allow the CCP to scan their device?
What difference does it make anyway at this point? Everybody does it.
There wasn't anything to deny. There's no way to opt out.
How do you know? With full access, it's basically impossible to tell.
> water stain with massive discoloration

Maybe they store physical photos and the albums got soaked in water. Sadly it happened to all our 90s photo albums during a flood. Some of the photos were stained really bad.

Which is why it’s obnoxious to ask permissions for all photos.
It immediately presented me with a screen with the choice to "save originals to cloud or save a compressed version to the cloud" (I don't recall the exact words) with no option to not upload to the cloud, and then immediately started uploading my library.

I scrolled back and checked some pre-2015 photos, they look fine, and then deleted the app.

A typical feature of these cloud drives is automatic uploading of pictures from the camera roll, that’s the legitimate use case for demanding access to all pictures. Who knows when Google whoops! accidentally makes that the default. You know, it’s optional, you can just turn it off!
I'm not sure I believe this _at all_.

I uploaded a photo to google maps. A week later it found some random picture of a local park and offered up a notification suggesting that I upload this photo to the destination it had located in the maps app.

How on earth are they doing this without "scanning" the photo? Seriously, please tell me.

This is an engagement hack. Get the user to give max permissions at the start so that any future call to action will have less friction and/or not require a modal.

It’s less about Google wanting to scan your data today and more about Google’s stingy strategy for photos plus Google’s lazy engineering (too many other interesting games to solve). Google Photos was born out of Google Plus, and is under heavy revenue efficiency pressure.

Each passing month I am more and more convinced that I should urgently get a) my mails out of my personal (non paying) Gmail account and b) get my associated photos out of Google Photos, mainly because of baby pictures which I cannot fathom to lose.

But then I think about the convenience of my 2TB Google Drive and I can't do it.

And I think, too, of the Polaroid pictures my parents took and gave me in a box once I got married and are now fainted. Mayhaps algorithms deciding that it's time to let go of your own data is just the modern form of yellowing.

Mirror them to a external 2TB hard drive?
> I should urgently get a) my mails out of my personal (non paying) Gmail account and b) get my associated photos out of Google Photos, mainly because of baby pictures which I cannot fathom to lose.

I highly recommend it. It feels like pure freedom, putting a price tag on my personal data. The migration is something done in two steps: set up your alternative service, but also keep the old one (gmail/gdrive/calendar/...) running. Then start to use the new email service, while keeping the old one.

There are enough affordable options and you do not have to migrate everything at once. I started with my emails - gmail is still running and forwards all incoming emails, so I wont miss anything. I already updated all my accounts/logins to the new email address.

The next step was copying all my files from Dropbox, GDrive, and so on to {service I am using}. The data in the other clouds are still there, but I am going to delete it all soon.

There are things that are not perfect - like accepting appointments, which I still want to show up in my Google calendar. To those mails I still have to respond in google mail. But I am getting there, one step after another. The win so far is much bigger than the loss/pain.

I take periodic backups of my Google data using Takeout, primarily for the GMail backup. When I took one a few months ago, I noticed that the Drive part of the backup had some mangled filenames that the backup from a year ago didn't have. I don't remember exactly what it was but it was something like long-filename.pdf got truncated to long-file~01.pdf, like back in the 8.3 DOS days. Luckily I already have a local copy of the Drive contents so I didn't need to rely on it, but anyone else should be worried.
Smugmug pro my friend. Not even paid to shill, but it's reasonably priced for complete photo -freedom for me
You can put those on a service like AWS Glacier for next to nothing, the only challenge is keeping your billing details current.
a) was very freeing. I didn't realize there were services so much better than Gmail. Even ones I pay a small amount for and get as many custom domains as I want. b) did this a long time ago. Glad I did. I can sync any file I want too, not just photos. Really been liking Seafile + AutoSync (android) for this.
(comment deleted)
I'm going through the process of switching off of Gmail and onto a Fastmail account under my own domain. I'm loving the freedom of having my own domain that I can switch to another email provider whenever I want. Also, Fastmail with a custom domain has some nice features that were unavailable with a Gmail account (aliases, masked emails, automatically snoozing certain emails so I get a bunch all at once, etc.)

As for pictures, I do use Google Photos, but it's not my primary photo video store. It's more for the app that lets me lookup pictures.

For storing original versions of pictures/videos, I sync from my phone to a home computer using Syncthing, and store backups of my files on AWS S3 using Restic for encrypted backups.

If I lose Google Photos, no big deal. I still have local and cloud copies of my data.

This is definitely not a plug, but I'm actually working on an alternative to email, cloud storage, authentication/authorization/sso/etc. solutions from Google, Microsoft, etc. and one of my specific focuses is to allow free personal use and free smaller company use. It is refreshing to see I'm not building this in a vacuum. I started the project for no other reason except Google yanking out their free accounts. I absolutely do plan to commercialize it, however, I would still be happy with the result even if it failed, since under Google's pricing, I'd pay more than $1,000 a year vs. less than $10/year with my own solution.
How does it work exactly? This sounds pretty cool
Please share more info about how we can follow your current or future project!
>But then I think about the convenience of my 2TB Google Drive and I can't do it.

Why not sync them to a cheap external drive on a regular basis?

Or use any of the hundreds of alternatives.. it’s not like a cloud drive is a distinguishing feature.
Pcloud is pretty great, lots of festures and lifetime plans.
Second this. Several years of use with nary an issue, plus Linux support.

Speaking of which, as an aside there's no excuse for big tech to not support the Linux desktop (I also use macOS and Windows 11 so not a die-hard fanboy). It doesn't matter if the userbase is tiny. Linux enables massive profits in the cloud so any cloud provider that supports multiple platforms but not Linux, whilst also generating huge profits from it, is doing a very bad thing. And that includes MS, Apple, and Google - full support regardless of the (comparatively small) cost as thanks for the profits they generate from it.

Burn everything important onto some old CDs or DVDs, that should be good enough. Filecoin seems cool too, but you have to pay in filecoin.
Most writable optical media use organic dyes that aren’t stable for more than a couple years—if you can still read the contents 10 years later, consider yourself lucky.

M-DISC use a more stable media, and should last much longer, but are (much) more expensive and require more recent burners. There are dual-layer M-DISC with higher capacities, too.

I did a bunch of research on long-term data archival and wrote it up here: https://photostructure.com/faq/how-do-i-safely-store-files/

As mancerayder says below, why can't you back them up to an external USB drive? I rsync my 'archive' partition (photos, videos, music, programs, etc) daily to several external 4TB USB drives. Those are dirt-cheap these days. The process takes maybe 5 or 10 minutes depending on whether or not there have been any new files to transfer in the last 24 hours.

I refuse to relinquish my files to some insouciant third-party. And what recourse would you have if they lost them?? Very probably them saying "Sorry. Not our problem."

Yes, I ran into this issue after I switched to iOS a couple years ago. Google demanded full permissions to even open the app. I promptly downloaded my image library on the web, and also, related to OP's post, some of my images had artifacts. Some were even corrupt. I moved everything over to onedrive, and restored the corrupted ones that I had local copies of (gradually, it took a lot of work) with local copies I have. I now mirror in "the cloud" with both iCloud and OneDrive. I also have a local copy and a couple of USB sticks with a backup handy on my keychain (solely for if a fire breaks out and I remember to grab my keys).