- Synology NAS is accessible on LAN & phones sync photos when at home
- Nightly rsync from internal NAS storage to 1 external (non-raid) hard drive
The nightly rsync does not delete files, so the external hard drive is pretty close to an off-site backup even though it's still attached. The only better thing I've thought of doing is mounting it ro after backups + re-mounting before backups.
At the end of the day we get 3 copies of data: the photo on your phone, the photo on the shared NAS, and the photo on an external hard drive that spends 99% of its life spun down.
IMO the biggest remaining risk to this is a house fire at night while you are at home. You wake up to alarms and smoke and run out the door without your phone. All three copies of your photos will burn at once. This is where an additional off-site would be valuable. That might be a second external hard drive that you store off-site and re-sync every few months, or maybe an encrypted cloud backup using restic.
I'm hoping that it's just a stale cache or microservice that hit an issue building its view of the metadata or something. They use S3 to store your data (according to this blog[0]) so my fingers are crossed that the data is still there, they just need to get an engineer onto it.
I don't trust to open my network to the internet though. So I've created a VPN to connect with my ubiquity network from mobile. Sadly, I can't share photos with other folks or allow my family to participate easily.
I'm a layman in terms of networking. I always wonder how people can sleep if they open their home network. In this case either by using QuickConnect or forward ports and access the machine through a custom domain secured by TLS. I mean that would be perfect but I just can't convince me to take that burden of maintaining that open door into my network.
I've read about creating sub networks with the power of ubiquiti (or any other router), but I will never trust anything I configure there either because I can't confirm it's secure.
So I'm stuck with Google Photos additionally to Synology Photos because of the convenience of sharing.
My solution is to not open it. Use the cloud for browsing, use the home machine to copy the files down. You rarely need to browse on the home machine, it's really just there if the cloud provider ditches you.
I own a Synology NAS, and now believe that it would have been easier to just set up a computer for the tasks I use it for. There are just too many cases where the solution is to use a container (and the clumsy web interfaces for reverse proxying &c), and if I end up SSHing into it too much, I prefer to have a more controllable environment.
It does fit some workflows very well though, you just have to assess if your needs over the next few years fit their business intentions.
Synology has an app which will automatically back up your photos to the NAS/server (only when on WiFi). I haven't been able to find an alternative to this on Android, have you? (photosync looks good but seems to only work for iOS last I checked).
+1'ing this too. Very user friendly with its web interface. And I think it is the only self-hosted solution that can backup live photos from iOS devices.
There is only one significant issue that I found: Backing up large videos fails If I put the Syno behind Cloudflare. Because the photo backup app uploads it as one large HTTP form data and CF does not like it. So now I need to make do with basic firewalling at my place.
How fast are Backblaze transfer speeds these days? I wasn't a huge fan of the client, but running something third party like Arq[1] felt like a bad workaround.
restic backing up to B2 works very well for me, and it’s really cheap. The restores I’ve tried so far have been acceptably fast, albeit in the gigabyte rather than terabyte range.
I don't know if it's due to one of those weird GDPR blocking rules (I'm in Europe) but amazonforum.com doesn't let me connect. It just drops my connection:
http https://www.amazonforum.com
http: LogLevel.ERROR: ConnectionError: HTTPSConnectionPool(host='www.amazonforum.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10b551e10>: Failed to establish a new connection: [Errno 61] Connection refused')) while doing a GET request to URL: https://www.amazonforum.com/
Those blocks are tiresome. Like the eu is going to go after the eg alabama times or other small sites. I just vpn to usa and send them an email. Google etc...easy meat for gdpr, but for small site - why are you tracking me anyway?
As a slight aside - I was surprised to learn that Amazon Photos (if you have Prime) includes RAW camera formats as valid photos for the purposes of their unlimited storage offer.
I've taken to essentially backing up all my RAWs to Amazon Photos - even the ones I would typically discard.
I also. I don't know why they would offer that as part of their base Prime service when no competitors seem to. The upsell for printing doesn't seem appealing to the type of person who would be storing RAWs.
I thought I had received an email from Amazon a few months ago telling me that my storage on Amazon Drive was going away. Pretty frustrating any time a service like this just goes poof.
EDIT: I misremembered the details, actually the email says:
> Beginning December 31, 2023, files stored on Amazon Drive will no longer be available to customers. We will continue to support Amazon Photos and ensure customers can safely back up, share, and organize photos and videos with the Amazon Photos service.
> Local backups shall be stored in different locations (depends on importance) not reachable by network. The cloud is the server of someone else and not a backup.
That's common knowledge among tech people but blaming the victim is not the solution here. These services are marketed as safe, reliable, and not needing backups (in fact sometimes they make it difficult to take backups, even more so for a non-technical user).
Maybe the regulation should be changed that any consumer-grade storage service either needs to provide adequate SLAs & compensation, or have to advertise in bold and prominent font (at least as prominent as their selling points) "WE MAY EAT YOUR DATA AT ANY TIME, PLAN ACCORDINGLY".
Nobody is blaming the victim, they're saying that you should have a backup outside of your cloud provider.
I don't think people care enough about SLAs to pay a premium, until something happens to their data. But by then it's too late. The bold warning is cute, but have you seen cigarette packaging in Europe lately?
> The bold warning is cute, but have you seen cigarette packaging in Europe lately?
The warning served its purpose quite well though - the majority of people including smokers concede that smoking is harmful. If they are still smoking despite that, it's on them.
The same cannot be said of cloud services. Very few people outside of tech think that the cloud can fail (whether technical failure or just randomly ban you for potential unspecified violation of ToS), so the same kinds of warnings would be useful to spread awareness.
> If they are still smoking despite that, it's on them.
That's my point: the warnings don't lead to reasonable decisions.
"If you don't pay for it, you are the product" has to be one of the most echoed sentiments of the last few years, and people still demand everything for free, and prefer to pay with their data instead of paying for the content they consume (e.g. on newspapers).
If Amazon gave them a choice of "you're data is most most most likely fine" and "your data is most most most most likely fine, and we'll pay you $1000 if it's not but you pay twice the money each month", I doubt they'll see a lot of people switch over. They'll have some, but not many, because ultimately, people don't care enough.
> the warnings don't lead to reasonable decisions.
The warnings lead to informed decisions. They may not always be reasonable, but at least the person knows what they're getting themselves into.
My intention is not to outlaw "best effort" cloud storage or legally compel the providers to offer an SLA, but merely to make the limitations clear so that customers, including non-technical ones, can make an informed decision.
> I don't think people care enough about SLAs to pay a premium
Huh? People specifically pay for cloud to not lose their data. Before that my parents used to copy photos from laptop to laptop, and lose them when a hard disk failed.
Now if cloud loses your data, they might ask wtf am I paying for.
There's "cloud" and there's "guaranteed cloud with SLA and compensation if it fails", and the latter will be more expensive. The former is somewhat "best effort", and that's also why it's cheap.
Is it really cheap? For 3 years of cloud subscribtion, I can buy 2 Nas drives, drop them at the homes of two of my friends, sync my PC to NAS 1, and backup NAS 1 to NAS 2.
They are like 2 -click setup if you use Synology/Qnap/etc.
You can do that, the average person can't, so you should probably consider the time you'd spend on it, and how much it would cost the average person to hire someone to set it up for them. I don't think it'll be too cheap.
In the end, not dealing with that is most likely worth it for most people (including most people who have the necessary skill set).
Man, if I smoked, I'd buy a cigarette case and transfer those bad boys over. I don't wanna look at gross lungs every time I go for a cig. Of course that's the point, but yuck.
I‘m sorry if you interpret here a blame. I consider repeating common knowledge actually a necessity!
To keep tech people informed and aware and also non-tech people.
This is just a reminder to take a backup of your cloud photos and store it locally (or other cloud provider). For example, with Google Takeout you can get a ZIP file with all your photos and videos from Google Photos.
Local photo library is a folder, disguised as a package. One of the folders store all the originals on the device. Copy that folder, and you're good to go. You can also get the full folder hierarchy, if you wish.
If you enable iCloud for the photo library, and even if you select store originals, I have found that Photos still offloads some pictures. It’s just not guaranteed.
I really wish Apple had a complete export button, similar to google takeout. Or at least a good CLI tool to get the original images one by one.
I rsync images out of Apple’s weird file structure that’s in the bundle. It’s horrible, but with ‘download originals’ enabled it works well. It’s particularly frustrating with large libraries as they occupy nearly all an SSD. For this reason I do it all on a VM that’s on a Synology.
I have no idea why you're getting downvoted. The icloud-photos-downloader is an amazing tool. I run it from a monthly cronjob to have a local backup in case icloud iexplodes.
You can also say: cloud servers are on fire and the house is on fire, have another backup someplace else. It's all probability.
The probability that your house will be on fire these days is low, and the probability that your computer/hard drive will get on fire is also low (since you will probably take it out, no one will leave his phone in a burning house right?).
We can say that the probability above is that low that it doesn't really affect anything. There are many other different parameters to add, and I use a bit of math here which doesn't always reflect on life, but you get the point.
I had a pretty big incident mid last year, which will turn into a dramatic blog post eventually.
I didn't lose anything I've yet noticed. Very happy with how well the backup strategy held up, although potentially quite lucky the second on-site backup was unaffected. Didn't have to dip into the complexities of the off-site chaos (which reminds me...).
One of the nice things about having an off-site backup is that, in addition to being offsite, it's presumably a completely different backup process so if the local backup wasn't working for some reason (and you didn't notice) you're probably still covered.
Having a spare hard drive that you occasionally sync up and drop off at a family member's house in addition to other backup strategies is indeed a decent idea.
However, I'm guessing that--if you're anything like me--you get lazy about refreshing it and, if something happens, you realize it's been a year since you did a fresh backup.
A cloud provider backup (as opposed to sync) is a good belt-and-suspenders cheap insurance backstop to local backups that you hopefully never need.
I have local Time Machine and Synology NAS but I also pay Backblaze a nominal amount. Companies were paying Iron Mountain large sums of money before there was a cloud.
I hate how Google Photos seems to no longer have the ability to auto sync to your computer. When it was integrated more with Google Drive you could do that, and then my daily automated backups would pull them. That way I had them in Google Photos, my computer's hard drive, and my off-site backups, all automatically. Now my only option is to periodically do a manual Takeout, as far as I can tell.
So... what do y'all actually do with your hundreds of thousands of photos? I just bulk delete everything older than a month when my phone is low on storage. Do you ever actually go back and look at any of the hundreds of thousands of pictures? When do you do this?
All the time actually. I have monthly folders, automatically backed up by my phone to the NAS. Each month my wife and I sift through the photos and delete all duplicates and boring stuff and then we have around 150 photos per month that we sometime just have on as a slideshow.
Yearly, we create printed photo albums that we look at regularly and show to our kids.
But all this only works after lots and lots of hours of manual work filtering the photos.
Still, over the last 15 years or so for two people we have way more than 100k photos stored on the NAS.
When you have kids, and when they turn into teenagers, you'll want to remember them as they were before.
;)
My daughter loves looking back at home videos of the family. So embarrassing, so beautiful, so many little things about their personalities that would otherwise be forgotten.
Just a few hours ago I was checking backups and stumbled into a nest of folders that had some more home videos in it, and of my son just being his quirky little kid self. Gives my life meaning, shows me I did stuff ok.
I have photos backed up on Google Photos for the last 10 years. I go back to find old photos all the time. There's different reasons, sometimes I'm just feeling nostalgic for a time in my life and want to see photos for them. Sometimes I want to find a specific photo of someone to send to them.
Here's a practical example of it coming in handy: back in university I moved to another city for an internship and took some framed photos I had on my walls. When I moved back into my parents' house when the internship was over, the nails were still in the walls and I wanted to hang them the same way I had before but couldn't remember which photo went where. So I went into my Google Photos, typed "photo frames," and was able to find pictures in my library with the framed photos in the background, and piece together which photo went where to get everything back to where it had been before.
I look at them a good bit. Having a good tool like Google Photos or similar makes it easy to jump back to the photos from that one trip a decade ago, that holiday together with the family no longer with us, the dreams I had for myself in highschool. I find it interesting to see the changes in my life objectively by looking at myself, the places, the people, and things around me at various points in my life. I find it helps me continue to keep perspective of my life in the same way some might keep a journal. It can help me jump back into the shoes of who I was 5 years ago, 10 years ago, 20 years ago, and reflect on who I am today and where I'm going.
If I don't have any real reference points, its hard to measure things. Its hard to improve things you cannot measure. And sometimes it just feels good to help remember the good times with people no longer with us.
The thing is also to delete photos you don't want to keep long term. Most of my photos are only synced when I am on my local wifi, so until then having a good hygiene of deleting badly taken photos right after taking them.
Having said that I have recently started (late december) to do a weekly printing session of my key moments of each weeks. Well there have been weeks in january and february I didn't print anything. I sometimes only have one or two photo per day/event but since I put the date on the notebook next to the photo and keep all other photos it helps me find back the whole event if I want to see them all. I am using smaller photo format than 10x15cm, mostly zink 2"x3" adhesive photos and kodak instant print 3"x3" from instant cameras that have a printer function.
Fun party trick: look up old photos of people when you meet them after a long time and show it to them. Everyone is shocked/embarrassed. Extra points if you can pull up childhood photos of them.
I liked that the X-Files reboot kept the old credits. Mulder and Scully look like babies. Then the credits end and reality intrudes. Of course, I'd aged at the same rate (maybe faster) but it was still a shock.
I've been enjoying the iOS photos widget. Every day it surfaces new photos from the past, based on different themes. A nice 2 minute diversion each morning.
But you're right, otherwise I would never have time to dig back into old photos.
1.5 Terabytes? (Assuming 6Mb/photo which is what I'm getting from an old nikon in raw format, dunno what everyone else is using - enlighten me please if necessary).
Less than USD$50 for a 2TB disk. So buy 3, post one to a family member when its full. Double the price it for SSD?
How much Amazon Photos storage do you get for $150? Ok for this guy, none, and I get connection refused from the link above so I got no details. But assuming it worked as advertised how much would you get?
Nikon RAWs are more like 50 Mb on their current cameras. Amazon offers unlimited photo storage with Prime.
I agree they should also have their own physical backup if the photos are important, even at ~10x the storage requirements you estimate, but I'd also say any complaints about Amazon losing their photos are valid.
Data point: I have 7T of "photo data". I think some proportion of that is video, but probably no more than 10%. I am quite old and it does include scans of all the film images I've taken since birth, and all of one side of my family's film media scanned, and backups of all the cell phone images from family members' devices. From memory however most of the data comes from digital-era DSLR images which can be 30-40M each.
None of this data is stored in a FAANG company's DC.
After a scare of being temporarily unable to recover my account on Flickr (later resolved), I got into the habit of printing my significant photos in a shop and storing them in a box in a cool and dry place. Of course, I also have local backups.
Just to add context - I do have backups, but they're spread across a myriad of different hard drives. I had used Amazon Photos to consolidate, and benefit from the facial recognition to be able to quickly locate pictures of family members quickly.
I think that I'll give DigiKam another go, against the NAS loaded up.
> I think that I'll give DigiKam another go, against the NAS loaded up.
I'm also looking for something FOSS that can do basic face recognition + maybe even more, but last time I checked DigiKam's detection didn't work so well, or maybe I got spoiled by the detection in Google Photos. If you do give it a go, would be nice if you reported back on your experience :)
What I found in my previous search is that either they're really good at managing large collections of photos, or they're really good at doing various AI/recognition, but none of what I've found have been good at both.
Hence my interest in DigiKam as their management features for large collections is second to none.
Isn't that OK? You should be able to find some reasonable workflow around using the recognizer to tag the photo in its EXIF/IPTC metadata, and then load that into the organizer.
It's not not OK :) Just not what I'm looking for, I spend enough time professionally with writing glue code, that I'm looking for a good out-of-the-box experience for something like this.
Not sure if it has been mentioned already but photoprism added face recognition a few releases ago and it is working well for me: https://github.com/photoprism/photoprism
The quality of this software is overall extremely good. It is a solo-developer as well so you might want to consider sponsoring them if you end up using it.
Seconding photoprism. The face recognition was pretty meh when it was first introduced, more of a gimmick than anything. But (I assume) a recent update pushed it into "wow, this is actually really good" territory.
Thirding photoprism, got it hosted on a raspberry pi and made it accessible outside of my home via Tailscale. Seems to work fairly well... though I could probably make it faster with an SSD vs an HDD over USB.
Photoprism is absolutely amazing. It was the quickest donation I’ve ever made to an OSS project.
I have a Photoprism instance running on my home server backed by a raidz1 ZFS pool (1st backup). Photos are periodically synced to a Backblaze bucket with versioning enabled (2nd backup). The source of most photos is an iOS device with iCloud enabled (main copy). I rely on PhotoSync to periodically sync from the iOS device to Photoprism.
To photoprism users: do you know how well it handles identifying photos of kids over the span of years?
Google photos absolutely blows me away with its ~1% false positive rate (that is, given a photo, correctly identifying with kid(s) are in it) identifying each of my kids, nieces, and nephews. I don’t really know what its false negative rate (a false negative would be an uploaded photo of John but that doesn’t get tagged as having John in it) is, though.
I can search for each of my kids and see photos of them going back to their birth.
There's a difference between face detection and face recognition. Last I checked (less than a year ago) it could find faces, but was abysmal at recognizing them. I spend a few days on it, tweaking this and that, taking to the devs. It was crap.
I'm away from my desktop computer, but it wasn't half bad when I last updated its DB. I don't take much portraits, so I don't use that feature much, but give it a look when I can and update you.
Latest release failed the tests (with some memory corruption yikes) even. Doesn't inspire a lot of confidence that it won't eat your photos.
But the v7 uses DNN from OpenCV for face detection that supposedly has 91% accuracy on CPLFW which is pretty good! So how come people here say it sucks?
Not that it helps you now, but i also keep all our family photos in the cloud (iCloud in my case), but at the same time i have a small ARM machine at home that keeps a mirror of the iCloud data.
That ARM machine also has the responsibility of making backups, local to a USB drive, as well as to another cloud. Not mirrors, but proper versioned backups (as in Restic, Borg, Arq, Duplicacy, Kopia, etc).
I also maintain a couple of USB drives with yearly updated mirrors of the entire photo library. The drives are stored at geographically different locations, and surface scanned, updated and rotated yearly.
And finally, as a "last ditch recovery", i maintain an archive of M-disc Blu-Ray discs that contain a complete copy of our family photo library. Every year i make an identical set of discs containing the past years photos, and these sets are stored alongside the USB drives.
I don't bother archiving documents as everything that is important is stored on government servers anyway, or exists in hardcopy. Also, if every step in my normal 3-2-1 backup scheme has failed and i need to recover from the archive, i probably have bigger issues than retrieving my budget for this years finances.
You have to fuck around with the storage format because by default scanning sofware, even when scanning documents, will basically just take poorly compressed high resolution images. Maybe there's some service which sorts that out for you but so far I have a little webapp which stores an index of paper copies. Once I figure out how to automate scanning, compressing and OCRing documents then I think it will be worth storing digital copies. But for now it works.
I do scan some paper but there's increasingly less of it and I'm likely to need future access to so little of it that it's mostly not worth the trouble.
I do occasionally want to retrieve older documents. As you say, they're smaller and the effort to put individual docs into a "I might want this someday" folder seems more trouble than it's worth. Of course, having a vast sea of basically write-only docs does make finding things harder especially as I don't put as much effort into organizing things any longer.
I still do a 3-2-1 backup of documents with 2 versioned backups, one at home and one in another cloud provider, just like with photos.
The archive however is the recovery if I’m not able to retrieve my normal cloud copy (hacked, ransomware, loss of credentials, etc), I cannot access my local mirror copy (ransomware, dead disk, etc), I cannot access my local backup (dead disk, separate from the mirror disk), and I cannot access my cloud backup either.
For all of those things to go wrong a the same time, something major has to happen. Besides, where I live, most required documents (drivers license, passport, birth certificate, tax records, etc) exists in government databases, so all I have at home will be various documents that maybe have sentimental value, but not exactly needed.
Furthermore, documents change “frequently” where photos tend to be somewhat more static, so I can archive photos, and maybe get <10% “duplicates” due to later edits, archiving documents will pretty much be a lot of duplicates each year.
That being said, I think we have like 1GB documents in total, so it would be easy to fit in the archive.
As a fellow small ARM machine owner, what's your strategy for getting the photos from iCloud? Is there a tool one shouldn't feel weird to give their iCloud credentials to?
i have been using icloudpd for years. there is a docker image that makes it simple to install on my synology. because of 2fa, i have to re-authenticate it from the terminal every couple of months, but that takes one minute.
I tried this but I think my issue is the library is too large. It can pull a few dozen or a hundred or so photos just fine, but then it will time out but not quit out, so I have to babysit the process. Maybe there's a flag I missed in the documentation to retry downloads, but it basically was a nonstarter to me in the state it was in. I think its apples fault though; I can't get a big zip file of icloud data to succesfully download with their website either, it will also time out. Likewise when I try and update the bootcamp drivers on my intel bootcamp machine, I will get 30% of the way there and then it times out. I'd blame my home connection, but from cursory glances at various forums this is apparently a widespread issue with these sort of downloads form Apple.
I was doing just this for a while but stopped because my Mac Mini was too old for the last few OS releases. I then switched to using the Windows iCloud client but a bug from ~2 years ago that consumes tremendous CPU cycles made that less than ideal. (The best you can do is lock it to a single thread, which will then use 100% 24/7)
Now I just don't backup my iCloud, though I do remove everything older than one year every new years to my home server which follows a good 3-2-1 backup strategy.
TL;DR: If you go this route, try to get a Mac Mini that can run a supported macOS for some time to come.
I've been using Windows iCloud client to backup to a VPS. It works okay for files, but for photos it pegged disk usage even when there weren't any new photos to download and my VPS provider wasn't happy. So far my solution is Windows iCloud client for files, and then OneDrive on my iPhone for photo backup, with OneDrive again on the VPS.
I like how the backup is outside my house, but I'm about to add Yubikey to my iCloud account and I'm not sure the Windows iCloud client is going to like that.
It doesn’t talk about not being able to access person tags. Not being able to programmatically access the data about who google thinks is in each of my photos has been an annoying pain point for me for years. Last I checked, the data is also not included in Google Takeout dumps.
I have recently switched to a M1 Mac Mini, and just have each family member sign in to that using Remote Desktop. It brings the added bonus of working as a content cache for anything iCloud.
My only gripe is that it downloads the shared photo album (new in iOS 16) once for each account, and when your photo library is 1.8TB, that suddenly becomes a lot of wasted space. When it comes to backing it up the backup software deduplicates the data, but not for the initial storage.
I really wish Apple would implement some kind of method for backing up photos stored in the cloud without the need for mirroring them.
Before the M1 I was using iCloud photo downloader ( https://github.com/icloud-photos-downloader/icloud_photos_do... ) on a Raspberry Pi 4 which also worked well, but in the end I got tired of iCloud credentials expiring every ~90 days, requiring each family member to login again through a console.
Considering the M1 idles at roughly 20% more than a RPi4 (M1 at 4.5W) it was an easy sell. I just got the cheapest model and added a large USB drive. Using a Mac also gives you the possibility of using something like Backblaze Personal with unlimited backup storage, if that’s your thing :-)
I use Healthchecks.IO ( https://healthchecks.io/ ) to keep an “eye” on the backup status (and other more mundane tasks like monitoring the power state of my summerhouse)
* A nextcloud+recognize (for face recog) instance running on Hetzner. $20/month (I co-host other services on the VM). This is attached to a Hetzner storage box.
* Syncthing pushes all of nextcloud to a zfs.rent (great service BTW) machine, I purchased my disks up front, so it's $cheap/month. ZFS snapshots are taken.
* Local RPi NAS. Syncthing up to the other two.
About $40/month in total. How valuable are 240k photos to you?
Uploading photos is a matter of sending them to Nextcloud. Syncthing does the rest.
Digikam is less crashy than it used to be. Its slowly going through my 650,000+ photos and finding faces. I started this past Sunday. I expect to be done in a week or two.
I'm glad you haven't really 'lost' the photos since you have copies elsewhere. It is surprising how many people these days rely on some cloud service to store and preserve their only copy of some important and unreproducible data.
You are right about file systems doing a lousy job of helping you organize and locate photos, especially when they are spread across many different HDD and SDD drives attached to various computers. They are equally bad at organizing other forms of content (documents, videos, logs, music, software, etc.)
File systems were invented decades ago when the biggest hard drives could only hold a few thousand files at most, and drives were also so expensive that most people only had one of them. Traditional file systems are antiquated and need to be replaced with something better.
Self-host and use Photoview [1], it has face recognition. I use this since 2 years with 5TB of photos (on ZFS) and it works flawlessly. It is worth it - everything under your control and you decide when to migrate. My phone's photos are also uploaded automatically to a Nextcloud folder, which is read-mounted to Photoview and scanned automatically.
I'm curious if you've asked your family members whether they're ok with having photos of themselves being used to train Amazon's facial recognition model? I know it's a common practice and Amazon isn't alone, yet I feel people should be more aware of what's happening.
I would be careful with hard drives. As I learned couple of days ago when I got my 10 year old hard drive from storage, I was unable to read most of the data as the drive would start to intermittently disconnect from the computer.
Most likely the drive is okay, but the board has developed an issue - maybe capacitors gone bad or something.
Now, I am yet to check if it is the one that has hardware encryption (I supposed to get rid of all of them and copy data to alternative drives) - there are drives that encrypt the data by default and if you don't set up the key, the encryption still takes place with some default key. If the board dies for some reason, then you won't ever be able to decrypt the data - even if technically was unencrypted, even by swapping the board from a working hard drive. This happened to me once and I lost 3TB of important data couple of years ago.
That being said - I think the best additional backup is to store important things on BluRay - producers estimate they should last 80-100 years and so far never had a bad disk, even those burned years ago.
Lots of reminders to make backups and alternatives offered, but I think it's fair to expect something from Amazon here since Photos, being a Prime benefit, is a paid service, which should come with a reasonable expectation of stability. It's a shame Amazon doesn't have an easier way to find formal support.
Does anyone know if iCloud (photos) storage is AWS S3? Curious if using both iCloud & Amazon Photos is a decent redundant backup plan in addition to a local copy. Or if I have iCloud & Prime, why not? (I also use Backblaze.)
What if a strong solar EMP wipe all storages on earth? you need redundancy on at least two other planets (I would suggest the moon but it's close to earth)
I think Apple uses GCP as well. A photo is probably replicated twice across AWS and GCP by Apple, and AWS and GCP will then replicate it at least 5 times on their own side onto different racks or even AZ.
That is why always you should synchronize your cloud with a local backup storage, always remain skeptical on cloud-backed storage as you are relying on someone else keeping it up (it can go anytime).
Cloud storage is just as volatile as your physical storage breaking or getting lost.
That super sucks. Stories like this (but about Google Photos) led to me setting up the following backup strategy.
1. First, all photos are on a fat external harddrive connected to a homelab server. When I come home with photos, I dump them onto this (TODO to find a way to automatically copy them into the proper spot off the camera upon the camera being plugged in, or a different spot if the gopro is plugged in)
2. Once a day, these are rsynced to a separate set of harddrives, which serve self hosted photo software I use (similar functionality to google photos. Looking for a replacement that lets me create arbitrary albums of photos, and then create web pages pointing to those arbitrary albums, that's FOSS. Currently evaluating photoview and librephotos)
3. Once a day, all photos are backed up to a backblaze b2 bucket using duplicati. with several previous versions also kept.
4. Once a year, a fucknormous external drive is purchased, all photos dumped onto it, and mailed to my parent's house, where they put it into a firebox.
5. Oh, and all my photos are also uploaded to google photos, because why not
I'm very much a scrub when it comes to sysadmin tasks and was able to set up all of the above with about... I don't know, no more than 100 hours of effort I'm sure, probably less. I recommend at least looking into consumer backup services like the one by backblaze!
A 'better than nothing' but very easy thing I do is use two cloud providers for photo storage. Since my photos are all phone based, backing up to any cloud provider is pretty straightforward.
It literally only has been 1 hour since OP posted that thread.
I agree with all the points about having backups and/or using better service/self host etc., but IMO it's too early to conclude anything about this incident.
I'd be freaking the fuck out too, tbh. These days you tend to get better support from the megacorps if you tweet or post on HN, it can bypass the obligatory outsourced support layer.
The stories that get helped often start with the premise that a company has destroyed the person's life/business and the company is hated by a large portion of people here.
That will get the most upvotes, visibility, and sometimes a company will reach out.
I have too, but when they have a bad policy they’ll defend it to death.
Specifically I contacted them about a product listing that, at best, was word salad right out of Blueler’s book on schizophrenia. They told me they didn’t want to hear any complaints unless I’d bought the product.
Of course, that kind of listing is endemic on AMZN, and I’d have no problem reporting several of those each time I go shopping.
Actions have consequences though and AMZN is now the last place I look when shopping for things since shopping there is somewhere between visiting Mariupol and the South Bronx in the 1970s. It is a matter of time before I pay my last recurring bill of any kind to them.
> They told me they didn’t want to hear any complaints unless I’d bought the product.
You have to think about the game-theoretic equilibrium of the alternative, given the same set of bad actors involved: if you could report listings you hadn't bought, then these idiots would all just be constantly reporting their product-category brand-name competitors' listings, as non-suspicious one-off actions from different dark-web-purchased pre-aged accounts, to the point that Amazon would just have to ignore "reported listings" as a signal anyway.
In that case, you're wrong: the Denial of Service is "jamming the human review queue", and it would be very successful. Even Amazon doesn't have infinite money to hire moderators to keep up with the number of [99.99999% automated and illegitimate] reports that would flood in.
(Actually, it's probably a supply problem more than a demand problem — there probably aren't enough people willing and able to work as moderators on the entire Earth, to manually review the number of complaints that would be generated here. It would cost them orders of magnitude less to just hire people to manually review all changes made to product listings! ...which they still won't do, because that's still apparently too high a cost to bear.)
I never see senseless product listings on Ebay. For that matter, I get Ebay shipments from Japan faster than AMZN gets me packages from a warehouse in the next state. People selling animals and farm stuff on craiglist sometimes struggle to spell words like ‘cow’ and ‘pig’ but once more the listings make sense.
Do you believe that this is because Ebay knows something that Amazon doesn't know? I personally think Ebay just doesn't drive the kind of sales volume — or have the particular expected sale+logistics lifecycle (immediate payment to enable immediate shipment of locally-warehoused goods through Prime, etc) — that makes these particular scammers interested in exploiting it.
Also, as you're describing this more, I'm increasingly confused, because I don't think I've ever seen an Amazon store listing that is truly "senseless" in the way you're describing. Maybe it's just that I've only ever visited Amazon's Canadian storefront, and these particular attackers aren't active on there; but while I see lots of AliExpress-dropshipped product listings, lots of page grooming, etc., but never a completely-senseless listing. I believe you that they exist; but can you link an example?
Yeah I usually have bad experiences calling Amazon and have stopped buying anything from them, preferring wither other online stores or driving to a physical store depending. The CSRs usually don’t speak english well enough to respond to anything other than the issues the app also lists. One CSR insisted I send back an item that fell out of the package in the delivery driver’s car.
At least for code, git being distributed means that there's always multiple copies with typical workflows.
For example, in addition to the copy on GitHub whoever pushed last should have a completely current copy. And everyone else working with the repo should have something pretty current.
Knowing Amazon retail / services support, their data is gone and they'll be lucky if they raise anyone at all.
If you pay $15k a month for AWS enterprise support, you will find someone who may or may not be able to help you within 6 months after misinterpreting the problem and having to raise your account manager and wave your hands around.
I'm surprised, Amazon is well known of having the best in class customer support for their store (I mean: "Customer: My kindle broke, because I sat on it. Amazon: no worries, let me send you a replacement, no need to send back the old kindle" level of good support) - oh and human support, not google botocracy.
Have you tried getting through to a human recently on Amazon retail support? It's quite difficult. First you have to negotiate with a bot with an IQ and service level of an angry wasp, then you get filtered through to someone who usually can't help. So you have to try 2-3 times to get stuff sorted.
The whole thing is designed to put people off and make it difficult to get problems resolved.
I'm fairly good at being annoyed enough to work around this but most people out there can't handle things like that easily and Amazon knows it.
It depends on "recently".
I contacted them (human, right away, speaking my language - I'm in EU and the store was in different country than mine) about a year or two ago where I had issue with my phone, and there was no issue to get my money back (Pixel 4 had issue with NFC not working correctly) - no "We will repair it", like any other retail store I used.
Last month here in UK. After a package was stolen from my doorstep but the delivery driver said it was handed to receptionist. We don't have a receptionist, or a reception!
I'm a "high hitter" customer. I have prime and spend a hell of a lot of money with amazon every year so they tend to do what you ask eventually. But if you're a new customer or low value one your customer experience will be very very different. I know several people buying 1-5 things a year who have been told to piss off.
I've had a similar situation, where the delivery guy said I wasn't home, so he left the package at some drop-off point. It was a small package and the house has a big ass mailbox. There wasn't a drop-off notice, either. And both my parents and I were at home, with my mom hanging around in the garden in plain view of the mailbox.
The opening hours of the place were a PITA, so after two attempts I called up Amazon and complained about this. Sure enough, they sent the package again without trying anything funny.
"The 'computer' is lying" situation is a problem generally. I didn't receive a 16-port switch and another small item a few years back even though it was noted as delivered. Neither Amazon nor UPS would do anything about it. A couple weeks later (I think it had been snowing) I discovered the package tossed in a ditch to the side of my driveway. Everything was actually fine but I had reordered and didn't have a need for a second switch.
Last November, my Logitech mouse's scroll wheel started acting funny. Tried to check on Amazon the warranty situation, the docs said: take it up with the manufacturer (note: the mouse was sold by Amazon, wasn't a marketplace item). This was like two weeks before the 2-year warranty was up.
So, I headed on over to Logitech. Man, their site sucks. My mouse was a "gaming" model, so it wasn't directly clear where I should seek help. I end up with some support guy who was obviously doing something else in parallel, judging by the time between replies. Went through the usual checklist (is it on / did you try unplugging and plugging it back again?). Never mind that before getting on the chat, I had to fill up a form with the exact same questions, up to and including the serial number, which is the first thing the chat person asked for. Half an hour later, he asked the killer question: where did you buy it? Amazon. Oh, take it up with them, then.
Went back to Amazon, opened a new "other" case. Less than two minutes later, I was on the phone with someone from Amazon, asking me point-blank whether I wanted them to try to fix it or reimburse me. Said the latter, and he asked me to be sure to send it within 30 days.
Guess where I bought my next Logitech mouse?
So yeah, Amazon's retail support experience is still top-notch in my book, at least on this side of the pond. This was for an account without prime support that doesn't do a lot of business with them.
I'm fairly confident that if I actually opened the support request with Amazon first, the ending would have been the same. At no point did Logitech say that their online support was only for items bought through their online store.
Sure, Amazon could have said "yeah, call us" instead of saying that I should see the manufacturer, and it would have saved me half an hour.
But I was quite disappointed with Logitech's reaction.
I've had an issue with a bag and when I took it up to one of their stores for repairs, they just took it in for repairs for free without any question asked whatsoever. Not even where or when I'd bought the item (I didn't buy it from their name-stores, but from some generic department store). Didn't ask to see any receipt. Just "It'll take around a week. Can you leave it now?"
Note that this wasn't some fancy boutique bag or brand, it was barely pricier than the mouse.
Since I'm complaining about Logitech, I should name the brand with great service: Le Tanneur, a French leather goods company.
Any large-scale consumer-grade service optimizes for cheapness of support.
It's just a coincidence that when it comes to retail, the cheap option (just ship another one out) also works well for the customer. At scale, shipping duplicate items, even expensive ones still works out cheaper than well-trained, competent, well-treated and well-paid humans.
This one on the other hand requires engineering time (can't be resolved by a bot or monkey pressing a "ship another item" button) which isn't cheap and unless they have legal liability, it's cheaper to just stonewall or even ban this customer forever and lose their business.
This has not been my experience. I pay 100$/mo for support and when I have had issues (had a RDS upgrade go really bad once) they have contacted me quickly to get details. Usually only 2-4 hours for an engineer to be on the issue.
The point of these services is they're supposed to be an easy-to-use, worry-free photo backup. Amazon/Google/Apple should surely have the resources available to keep my photos stored somewhere with some redundancy. I don't want to worry about how they do it, but that's the idea. In exchange they're probably training image recognition algorithms or what have you on my photos, and I certainly wouldn't keep anything too personal on those services (i.e. nudes). I may even be paying them for more than the free storage limit. The least they can do is be a reliable backup. Yahoo Mail doesn't disappear emails randomly, YouTube doesn't just disappear videos randomly (barring stupid robot moderation). You'd expect the same for your photos on these services.
335 comments
[ 2.7 ms ] story [ 246 ms ] threadStill it sucks and it shouldn’t happen, hopefully it’s just a reporting error in the app.
- Synology NAS is accessible on LAN & phones sync photos when at home
- Nightly rsync from internal NAS storage to 1 external (non-raid) hard drive
The nightly rsync does not delete files, so the external hard drive is pretty close to an off-site backup even though it's still attached. The only better thing I've thought of doing is mounting it ro after backups + re-mounting before backups.
At the end of the day we get 3 copies of data: the photo on your phone, the photo on the shared NAS, and the photo on an external hard drive that spends 99% of its life spun down.
[0]: https://aws.amazon.com/blogs/storage/how-amazon-photos-uses-...
I don't trust to open my network to the internet though. So I've created a VPN to connect with my ubiquity network from mobile. Sadly, I can't share photos with other folks or allow my family to participate easily.
I'm a layman in terms of networking. I always wonder how people can sleep if they open their home network. In this case either by using QuickConnect or forward ports and access the machine through a custom domain secured by TLS. I mean that would be perfect but I just can't convince me to take that burden of maintaining that open door into my network.
I've read about creating sub networks with the power of ubiquiti (or any other router), but I will never trust anything I configure there either because I can't confirm it's secure.
So I'm stuck with Google Photos additionally to Synology Photos because of the convenience of sharing.
It does fit some workflows very well though, you just have to assess if your needs over the next few years fit their business intentions.
Nevertheless, Nextcloud keeps offering to sync my photos, and I think it also does that on Android.
I run a Mac VM on mine but via iscsi as the actual VM host hasn’t enough storage to hold my library.
There is only one significant issue that I found: Backing up large videos fails If I put the Syno behind Cloudflare. Because the photo backup app uploads it as one large HTTP form data and CF does not like it. So now I need to make do with basic firewalling at my place.
[1]: https://www.arqbackup.com
It's not, I'm in EU as well, can connect without any issues
I've taken to essentially backing up all my RAWs to Amazon Photos - even the ones I would typically discard.
(here I was hoping that the Docker thing filled the surprised Pikachu face quota for the week’s frontpage)
EDIT: I misremembered the details, actually the email says:
> Beginning December 31, 2023, files stored on Amazon Drive will no longer be available to customers. We will continue to support Amazon Photos and ensure customers can safely back up, share, and organize photos and videos with the Amazon Photos service.
That's an interesting choice of phrasing they're using.
No local backup on a drive or memory-card? Local backups shall be stored in different locations (depends on importance) not reachable by network.
The cloud is the server of someone else and not a backup.
That's common knowledge among tech people but blaming the victim is not the solution here. These services are marketed as safe, reliable, and not needing backups (in fact sometimes they make it difficult to take backups, even more so for a non-technical user).
Maybe the regulation should be changed that any consumer-grade storage service either needs to provide adequate SLAs & compensation, or have to advertise in bold and prominent font (at least as prominent as their selling points) "WE MAY EAT YOUR DATA AT ANY TIME, PLAN ACCORDINGLY".
I don't think people care enough about SLAs to pay a premium, until something happens to their data. But by then it's too late. The bold warning is cute, but have you seen cigarette packaging in Europe lately?
The warning served its purpose quite well though - the majority of people including smokers concede that smoking is harmful. If they are still smoking despite that, it's on them.
The same cannot be said of cloud services. Very few people outside of tech think that the cloud can fail (whether technical failure or just randomly ban you for potential unspecified violation of ToS), so the same kinds of warnings would be useful to spread awareness.
That's my point: the warnings don't lead to reasonable decisions.
"If you don't pay for it, you are the product" has to be one of the most echoed sentiments of the last few years, and people still demand everything for free, and prefer to pay with their data instead of paying for the content they consume (e.g. on newspapers).
If Amazon gave them a choice of "you're data is most most most likely fine" and "your data is most most most most likely fine, and we'll pay you $1000 if it's not but you pay twice the money each month", I doubt they'll see a lot of people switch over. They'll have some, but not many, because ultimately, people don't care enough.
The warnings lead to informed decisions. They may not always be reasonable, but at least the person knows what they're getting themselves into.
My intention is not to outlaw "best effort" cloud storage or legally compel the providers to offer an SLA, but merely to make the limitations clear so that customers, including non-technical ones, can make an informed decision.
Huh? People specifically pay for cloud to not lose their data. Before that my parents used to copy photos from laptop to laptop, and lose them when a hard disk failed.
Now if cloud loses your data, they might ask wtf am I paying for.
They are like 2 -click setup if you use Synology/Qnap/etc.
And I will have more storage space.
In the end, not dealing with that is most likely worth it for most people (including most people who have the necessary skill set).
PS: I should better do another backup…
This is basically you signing that we have failed as an industry.
No other industry expects it's users to operate with tbe diligence of a bank.
I really wish Apple had a complete export button, similar to google takeout. Or at least a good CLI tool to get the original images one by one.
Fwiw my library is ~200k files and close to a TB.
I rsync images out of Apple’s weird file structure that’s in the bundle. It’s horrible, but with ‘download originals’ enabled it works well. It’s particularly frustrating with large libraries as they occupy nearly all an SSD. For this reason I do it all on a VM that’s on a Synology.
The probability that your house will be on fire these days is low, and the probability that your computer/hard drive will get on fire is also low (since you will probably take it out, no one will leave his phone in a burning house right?).
We can say that the probability above is that low that it doesn't really affect anything. There are many other different parameters to add, and I use a bit of math here which doesn't always reflect on life, but you get the point.
I didn't lose anything I've yet noticed. Very happy with how well the backup strategy held up, although potentially quite lucky the second on-site backup was unaffected. Didn't have to dip into the complexities of the off-site chaos (which reminds me...).
However, I'm guessing that--if you're anything like me--you get lazy about refreshing it and, if something happens, you realize it's been a year since you did a fresh backup.
A cloud provider backup (as opposed to sync) is a good belt-and-suspenders cheap insurance backstop to local backups that you hopefully never need.
I have local Time Machine and Synology NAS but I also pay Backblaze a nominal amount. Companies were paying Iron Mountain large sums of money before there was a cloud.
Though online backups have their own risks (ransomware).
Geographical separation is important if you're serious.
Yearly, we create printed photo albums that we look at regularly and show to our kids.
But all this only works after lots and lots of hours of manual work filtering the photos.
Still, over the last 15 years or so for two people we have way more than 100k photos stored on the NAS.
;)
My daughter loves looking back at home videos of the family. So embarrassing, so beautiful, so many little things about their personalities that would otherwise be forgotten.
Just a few hours ago I was checking backups and stumbled into a nest of folders that had some more home videos in it, and of my son just being his quirky little kid self. Gives my life meaning, shows me I did stuff ok.
Here's a practical example of it coming in handy: back in university I moved to another city for an internship and took some framed photos I had on my walls. When I moved back into my parents' house when the internship was over, the nails were still in the walls and I wanted to hang them the same way I had before but couldn't remember which photo went where. So I went into my Google Photos, typed "photo frames," and was able to find pictures in my library with the framed photos in the background, and piece together which photo went where to get everything back to where it had been before.
If I don't have any real reference points, its hard to measure things. Its hard to improve things you cannot measure. And sometimes it just feels good to help remember the good times with people no longer with us.
The thing is also to delete photos you don't want to keep long term. Most of my photos are only synced when I am on my local wifi, so until then having a good hygiene of deleting badly taken photos right after taking them.
Having said that I have recently started (late december) to do a weekly printing session of my key moments of each weeks. Well there have been weeks in january and february I didn't print anything. I sometimes only have one or two photo per day/event but since I put the date on the notebook next to the photo and keep all other photos it helps me find back the whole event if I want to see them all. I am using smaller photo format than 10x15cm, mostly zink 2"x3" adhesive photos and kodak instant print 3"x3" from instant cameras that have a printer function.
I should blog about that.
But you're right, otherwise I would never have time to dig back into old photos.
https://github.com/photoprism/photoprism
Less than USD$50 for a 2TB disk. So buy 3, post one to a family member when its full. Double the price it for SSD?
How much Amazon Photos storage do you get for $150? Ok for this guy, none, and I get connection refused from the link above so I got no details. But assuming it worked as advertised how much would you get?
I agree they should also have their own physical backup if the photos are important, even at ~10x the storage requirements you estimate, but I'd also say any complaints about Amazon losing their photos are valid.
And 15 Tb is the go. Can you encrypt them or do they require to be able to view your photos so not just bytes on disk?
>any complaints about Amazon losing their photos are valid.
Like crossing the road, being right doesn't help. :(
None of this data is stored in a FAANG company's DC.
I think that I'll give DigiKam another go, against the NAS loaded up.
Thanks for the comments!
I'm also looking for something FOSS that can do basic face recognition + maybe even more, but last time I checked DigiKam's detection didn't work so well, or maybe I got spoiled by the detection in Google Photos. If you do give it a go, would be nice if you reported back on your experience :)
https://damselfly.info
https://github.com/LibrePhotos/librephotos
https://photoview.github.io/
I may have missed some: https://github.com/awesome-selfhosted/awesome-selfhosted#pho...
Hence my interest in DigiKam as their management features for large collections is second to none.
The quality of this software is overall extremely good. It is a solo-developer as well so you might want to consider sponsoring them if you end up using it.
I am not affiliated, just a happy user :)
I have a Photoprism instance running on my home server backed by a raidz1 ZFS pool (1st backup). Photos are periodically synced to a Backblaze bucket with versioning enabled (2nd backup). The source of most photos is an iOS device with iCloud enabled (main copy). I rely on PhotoSync to periodically sync from the iOS device to Photoprism.
Google photos absolutely blows me away with its ~1% false positive rate (that is, given a photo, correctly identifying with kid(s) are in it) identifying each of my kids, nieces, and nephews. I don’t really know what its false negative rate (a false negative would be an uploaded photo of John but that doesn’t get tagged as having John in it) is, though.
I can search for each of my kids and see photos of them going back to their birth.
But the v7 uses DNN from OpenCV for face detection that supposedly has 91% accuracy on CPLFW which is pretty good! So how come people here say it sucks?
All from 1920s slang. Apart from dog's bollocks I imagine.
It was slow processing the images, so took very long after adding new pictures to them turning up in searches.
One of the main attractions, searching geotagged images by location, wasn't smooth at all and quite clunky to use.
Tried it a few times last fall but gave up.
That ARM machine also has the responsibility of making backups, local to a USB drive, as well as to another cloud. Not mirrors, but proper versioned backups (as in Restic, Borg, Arq, Duplicacy, Kopia, etc).
I also maintain a couple of USB drives with yearly updated mirrors of the entire photo library. The drives are stored at geographically different locations, and surface scanned, updated and rotated yearly.
And finally, as a "last ditch recovery", i maintain an archive of M-disc Blu-Ray discs that contain a complete copy of our family photo library. Every year i make an identical set of discs containing the past years photos, and these sets are stored alongside the USB drives.
I don't bother archiving documents as everything that is important is stored on government servers anyway, or exists in hardcopy. Also, if every step in my normal 3-2-1 backup scheme has failed and i need to recover from the archive, i probably have bigger issues than retrieving my budget for this years finances.
Isn't that 2 or 3 orders of magnitude less space though?
I do understand where you're coming from for sure.
I do scan some paper but there's increasingly less of it and I'm likely to need future access to so little of it that it's mostly not worth the trouble.
The archive however is the recovery if I’m not able to retrieve my normal cloud copy (hacked, ransomware, loss of credentials, etc), I cannot access my local mirror copy (ransomware, dead disk, etc), I cannot access my local backup (dead disk, separate from the mirror disk), and I cannot access my cloud backup either.
For all of those things to go wrong a the same time, something major has to happen. Besides, where I live, most required documents (drivers license, passport, birth certificate, tax records, etc) exists in government databases, so all I have at home will be various documents that maybe have sentimental value, but not exactly needed.
Furthermore, documents change “frequently” where photos tend to be somewhat more static, so I can archive photos, and maybe get <10% “duplicates” due to later edits, archiving documents will pretty much be a lot of duplicates each year.
That being said, I think we have like 1GB documents in total, so it would be easy to fit in the archive.
https://github.com/icloud-photos-downloader/icloud_photos_do...
Now I just don't backup my iCloud, though I do remove everything older than one year every new years to my home server which follows a good 3-2-1 backup strategy.
TL;DR: If you go this route, try to get a Mac Mini that can run a supported macOS for some time to come.
There is usually someone who’ll point out that this probably violates licensing, and it probably does if you do it on non-Apple hardware.
I like how the backup is outside my house, but I'm about to add Yubikey to my iCloud account and I'm not sure the Windows iCloud client is going to like that.
Its not perfect, but as a backup, it works well.
It doesn’t talk about not being able to access person tags. Not being able to programmatically access the data about who google thinks is in each of my photos has been an annoying pain point for me for years. Last I checked, the data is also not included in Google Takeout dumps.
My only gripe is that it downloads the shared photo album (new in iOS 16) once for each account, and when your photo library is 1.8TB, that suddenly becomes a lot of wasted space. When it comes to backing it up the backup software deduplicates the data, but not for the initial storage.
I really wish Apple would implement some kind of method for backing up photos stored in the cloud without the need for mirroring them.
Before the M1 I was using iCloud photo downloader ( https://github.com/icloud-photos-downloader/icloud_photos_do... ) on a Raspberry Pi 4 which also worked well, but in the end I got tired of iCloud credentials expiring every ~90 days, requiring each family member to login again through a console.
Considering the M1 idles at roughly 20% more than a RPi4 (M1 at 4.5W) it was an easy sell. I just got the cheapest model and added a large USB drive. Using a Mac also gives you the possibility of using something like Backblaze Personal with unlimited backup storage, if that’s your thing :-)
I use Healthchecks.IO ( https://healthchecks.io/ ) to keep an “eye” on the backup status (and other more mundane tasks like monitoring the power state of my summerhouse)
* A nextcloud+recognize (for face recog) instance running on Hetzner. $20/month (I co-host other services on the VM). This is attached to a Hetzner storage box.
* Syncthing pushes all of nextcloud to a zfs.rent (great service BTW) machine, I purchased my disks up front, so it's $cheap/month. ZFS snapshots are taken.
* Local RPi NAS. Syncthing up to the other two.
About $40/month in total. How valuable are 240k photos to you?
Uploading photos is a matter of sending them to Nextcloud. Syncthing does the rest.
I really miss Picassa... if it wasn't for a bug in the last version, where it sometimes swapped face tags, I'd still be using it.
You are right about file systems doing a lousy job of helping you organize and locate photos, especially when they are spread across many different HDD and SDD drives attached to various computers. They are equally bad at organizing other forms of content (documents, videos, logs, music, software, etc.)
File systems were invented decades ago when the biggest hard drives could only hold a few thousand files at most, and drives were also so expensive that most people only had one of them. Traditional file systems are antiquated and need to be replaced with something better.
[1]:https://github.com/photoview/photoview
That being said - I think the best additional backup is to store important things on BluRay - producers estimate they should last 80-100 years and so far never had a bad disk, even those burned years ago.
Cloud storage is just as volatile as your physical storage breaking or getting lost.
1. First, all photos are on a fat external harddrive connected to a homelab server. When I come home with photos, I dump them onto this (TODO to find a way to automatically copy them into the proper spot off the camera upon the camera being plugged in, or a different spot if the gopro is plugged in)
2. Once a day, these are rsynced to a separate set of harddrives, which serve self hosted photo software I use (similar functionality to google photos. Looking for a replacement that lets me create arbitrary albums of photos, and then create web pages pointing to those arbitrary albums, that's FOSS. Currently evaluating photoview and librephotos)
3. Once a day, all photos are backed up to a backblaze b2 bucket using duplicati. with several previous versions also kept.
4. Once a year, a fucknormous external drive is purchased, all photos dumped onto it, and mailed to my parent's house, where they put it into a firebox.
5. Oh, and all my photos are also uploaded to google photos, because why not
I'm very much a scrub when it comes to sysadmin tasks and was able to set up all of the above with about... I don't know, no more than 100 hours of effort I'm sure, probably less. I recommend at least looking into consumer backup services like the one by backblaze!
Aww, I got all excited because I read that as 3.5" FDDs, and got nostalgic for the noise... and installing Windows from like twelve of the bastards.
This may be user error? It's possible the photos are still there, or if they are gone, perhaps Amazon can restore.
OP, did you try to call / contact Amazon before you posted this?
...which is listed as the first port of call for support.[0]
> For more help, try our Amazon Drive & Amazon Photos forum
But hey, maybe I missed the prominently displayed phone number somehow, so if you can help me find it, it'd be much appreciated.
[0]: https://www.amazon.com/gp/help/customer/display.html?nodeId=...
I agree with all the points about having backups and/or using better service/self host etc., but IMO it's too early to conclude anything about this incident.
I do wish I have such luck when posting on HN though, often ended up with zero upvotes and no replies.
That will get the most upvotes, visibility, and sometimes a company will reach out.
But they were backed up to Ethereum, but Ethereum might be declared a security, so they'll be gone too!
At least I've also got them on XRP.
Specifically I contacted them about a product listing that, at best, was word salad right out of Blueler’s book on schizophrenia. They told me they didn’t want to hear any complaints unless I’d bought the product.
Of course, that kind of listing is endemic on AMZN, and I’d have no problem reporting several of those each time I go shopping.
Actions have consequences though and AMZN is now the last place I look when shopping for things since shopping there is somewhere between visiting Mariupol and the South Bronx in the 1970s. It is a matter of time before I pay my last recurring bill of any kind to them.
You have to think about the game-theoretic equilibrium of the alternative, given the same set of bad actors involved: if you could report listings you hadn't bought, then these idiots would all just be constantly reporting their product-category brand-name competitors' listings, as non-suspicious one-off actions from different dark-web-purchased pre-aged accounts, to the point that Amazon would just have to ignore "reported listings" as a signal anyway.
...if they could do that, why would you need to be able to report listings? They'd just block the bad ones automatically.
(Actually, it's probably a supply problem more than a demand problem — there probably aren't enough people willing and able to work as moderators on the entire Earth, to manually review the number of complaints that would be generated here. It would cost them orders of magnitude less to just hire people to manually review all changes made to product listings! ...which they still won't do, because that's still apparently too high a cost to bear.)
I never see senseless product listings on Ebay. For that matter, I get Ebay shipments from Japan faster than AMZN gets me packages from a warehouse in the next state. People selling animals and farm stuff on craiglist sometimes struggle to spell words like ‘cow’ and ‘pig’ but once more the listings make sense.
Do you believe that this is because Ebay knows something that Amazon doesn't know? I personally think Ebay just doesn't drive the kind of sales volume — or have the particular expected sale+logistics lifecycle (immediate payment to enable immediate shipment of locally-warehoused goods through Prime, etc) — that makes these particular scammers interested in exploiting it.
Also, as you're describing this more, I'm increasingly confused, because I don't think I've ever seen an Amazon store listing that is truly "senseless" in the way you're describing. Maybe it's just that I've only ever visited Amazon's Canadian storefront, and these particular attackers aren't active on there; but while I see lots of AliExpress-dropshipped product listings, lots of page grooming, etc., but never a completely-senseless listing. I believe you that they exist; but can you link an example?
I got told I was insane for backing up github. I clone all our github repos daily, zip them up and stuff them in S3
For example, in addition to the copy on GitHub whoever pushed last should have a completely current copy. And everyone else working with the repo should have something pretty current.
Our company has thousands of repos, some of them years old.
https://news.ycombinator.com/item?id=35181205
If you pay $15k a month for AWS enterprise support, you will find someone who may or may not be able to help you within 6 months after misinterpreting the problem and having to raise your account manager and wave your hands around.
The whole thing is designed to put people off and make it difficult to get problems resolved.
I'm fairly good at being annoyed enough to work around this but most people out there can't handle things like that easily and Amazon knows it.
I'm a "high hitter" customer. I have prime and spend a hell of a lot of money with amazon every year so they tend to do what you ask eventually. But if you're a new customer or low value one your customer experience will be very very different. I know several people buying 1-5 things a year who have been told to piss off.
The opening hours of the place were a PITA, so after two attempts I called up Amazon and complained about this. Sure enough, they sent the package again without trying anything funny.
Last November, my Logitech mouse's scroll wheel started acting funny. Tried to check on Amazon the warranty situation, the docs said: take it up with the manufacturer (note: the mouse was sold by Amazon, wasn't a marketplace item). This was like two weeks before the 2-year warranty was up.
So, I headed on over to Logitech. Man, their site sucks. My mouse was a "gaming" model, so it wasn't directly clear where I should seek help. I end up with some support guy who was obviously doing something else in parallel, judging by the time between replies. Went through the usual checklist (is it on / did you try unplugging and plugging it back again?). Never mind that before getting on the chat, I had to fill up a form with the exact same questions, up to and including the serial number, which is the first thing the chat person asked for. Half an hour later, he asked the killer question: where did you buy it? Amazon. Oh, take it up with them, then.
Went back to Amazon, opened a new "other" case. Less than two minutes later, I was on the phone with someone from Amazon, asking me point-blank whether I wanted them to try to fix it or reimburse me. Said the latter, and he asked me to be sure to send it within 30 days.
Guess where I bought my next Logitech mouse?
So yeah, Amazon's retail support experience is still top-notch in my book, at least on this side of the pond. This was for an account without prime support that doesn't do a lot of business with them.
Sure, Amazon could have said "yeah, call us" instead of saying that I should see the manufacturer, and it would have saved me half an hour.
But I was quite disappointed with Logitech's reaction.
I've had an issue with a bag and when I took it up to one of their stores for repairs, they just took it in for repairs for free without any question asked whatsoever. Not even where or when I'd bought the item (I didn't buy it from their name-stores, but from some generic department store). Didn't ask to see any receipt. Just "It'll take around a week. Can you leave it now?"
Note that this wasn't some fancy boutique bag or brand, it was barely pricier than the mouse.
Since I'm complaining about Logitech, I should name the brand with great service: Le Tanneur, a French leather goods company.
It's just a coincidence that when it comes to retail, the cheap option (just ship another one out) also works well for the customer. At scale, shipping duplicate items, even expensive ones still works out cheaper than well-trained, competent, well-treated and well-paid humans.
This one on the other hand requires engineering time (can't be resolved by a bot or monkey pressing a "ship another item" button) which isn't cheap and unless they have legal liability, it's cheaper to just stonewall or even ban this customer forever and lose their business.
240k photos is 66 photos per day, every day, over a full decade.
If you're spending time taking 66 photos a day, I think you have bigger (personal) issues to contend with than your image storage solution.
Even on vacation, relaxing, with my kids and wife, visiting all manner of picturesque spots, 66 photos would be an enormous feat of narcissism.
The author is either lying about the photo library size to garner sympathy, or has a gargantuan psychological issue to address.