HN Explorer

Ask HN: Personal photo library recommendations? Open source, browser-based

191 points by trbfred ↗ HN
I'd like to move away from Apples Photo.app to open-source, self-hosted, and browser-based application that may run on a NAS or Linux server.

There seem to be lots of alternatives out there (Nextcloud, Piwigo, ...) but I'd love to hear about recommendations and experiences.

120 comments

[ 3.1 ms ] story [ 192 ms ] thread
aosaigh
I’m looking for the exact same thing. Seems to be few and far between. I have a Synology which has two photo products but they can be a little clunky. The challenge seems to be a sensible way to add an import.
cobbzilla
rsync and then reindex worked well for me. But I’d agree that’s not really a sensible solution for non-technical users.
fundamental
I tried out a few options a couple of years back and I've stuck with Piwigo since then. It seems to work well enough for my needs and the export functionality built into digikam seems to make it simple to sync up photos between my desktop and server.
Maakuth
Me too. For me, it was important to be able to point it to my folders (well, symlinking to them) and 'syncing' so it read the metadata and generated the thumbnails without the need to upload them anywhere. Also the calendar view has proven useful. My collection of around 90k photos works well with it, even with my meek Atom-based home server.

One feature I'm missing at the moment is raw conversion support. It would be great if thumbnails and previews of raw photos could be automatically generated. I've solved this via generating previews with ImageMagick, but native raw support in Piwigo would be better.

demosthenex
I've been unable to find an open source multiuser local photo product. I'd love for my whole family to upload cell phone photos and SD cards to a central place we could share and tag from, without it belonging to Google.
pnutjam
I run a small linux server that acts as a NAS. I rsync everyone's phones to a home directory, manually, and then pull that into a shared directory.

I just went through and renamed all my pictures by the exif create date, adding any unique names to exif UserComment.

I'm working on scripting all of this, but it works for me and I can pull data from anything into the directory. Everyone in the house can access it via samba and view the pictures.

canada_dry
I'd love to find one that incorporates the functionality that Picasa (windows desktop) circa 2000 i.e. face recognition/matching.

It was way ahead of it's time, and actually worked!

Best of all, it did everything locally... not cloud based and thus retained privacy of your personal photo collection.

hazz99
Google Photos does some really handy stuff with facial recognition, which was super helpful when searching & collating photos.

Doesn't help much for privacy, however.

froindt
My biggest complaint is how long it takes for facial recognition to occur. I like searching by name, and if I do a large upload, I might have to wait a couple weeks to get the groupings. I don't get a push notification, I have to go check and suddenly I have 100 faces to tag.
alphabettsy
Privacy is a major point of concern along with the fact that friends might use this service submitting photos of people who maybe don’t want this tech used on their likeness.
dashundchen
Digikam has facial recognition and tagging using OpenCV. Can't speak to its accuracy though.
bloopernova
Picasa was so useful, so of course Google had to kill it.

It took multiple disparate photo directories and presented everything in a timeline of folders. And because everything was local, that happened quickly, rather than waiting for your browser to get the next 100 photo results from a javascript call or whatever.

Are there any photo clients for windows that present multiple folders as a single coherent timeline? And can manage tens of thousands of pictures? I've got stuff going back to the late 1990s and would love to be able to find all those old cat pictures or whatever.

bloopernova
I'm now trying Shotwell, installed in my WSL Ubuntu instance under Windows, displaying to VcXsrv. Thankfully it can import-in-place rather than trying to copy gigabytes of photos. We'll see if it copes OK with the funky permissions and file layout I've got set up.
mceachen
> Are there any photo clients for windows that present multiple folders as a single coherent timeline?

PhotoStructure does this (and I believe is the only software that does robust time zone inference, as well). (I've posted elsewhere here with more details).

jamesbr
I use and like Mylio (https://mylio.com/). Does face recognition & syncs to other devices without being in the cloud. I migrated from Aperture/Picasa to Mylio.
mceachen
I started writing PhotoStructure because I got tired of being burned by failed open and closed source media apps. I wrote more details about it in a top level post, but this blog post describes why I quit my job to do this full time, where the project is currently, and where it's going. https://blog.photostructure.com/introducing-photostructure/

I'd love to have you try out the beta and have you share feedback!

bpye
I've signed up for your beta.

One question though - if you end up feeling it's unsustainable to continue developing PhotoStructure down the line do you have a plan? Obviously we would rather not be burned by it either, you you consider at that point making it open source so existing users can continue and make improvements?

Certainly I hope it never comes to it but it's nice to have a little reassurance.

mceachen
The "photo app" market is a zombie apocalyptic wasteland of failed and undead software. I don't want to get burned again, either, so:

1) There's a corporate mandate to open-source PhotoStructure for Desktop in the event of business closure.

2) Your library consists of industry-standard files. If you choose to do so, your originals are copied into a standard YYYY/YYYY-mm-dd/ folder hierarchy. XMP sidecars are added to hold inferred or novel metadata and store nondestructive edits, like rotation. A SQLite db (with commented schema! it's pretty, honest!) holds asset-file-tag relationships, albums, and other non-file metadata.

3) It's just me, and I love open source, so if I can get to the point where my licensing stream pays for food and shelter, I can open source then.

canada_dry
Signed up, though a quick browse I wasn't able to find any mention of face matching/recognition.

It's really the killer-feature that made Picassa so great - find all the photos over several years of a family member or friend.

Sadly, I've needed this feature for funeral photo albums lately and could really use the old Picassa!

mceachen
Face detection is one of the next things I'm building, along with sharing.
canada_dry
Great!

Sorry if I missed it, but is it possible (API or directly) to control Photostructure via python?

That way we could extend the features in many interesting ways - including our own face/object detection.

demosthenex
I should explain what I've tried.

Nextcloud photos is not a photo application. It's basically a shared gallery with thumbnails. There's no metadata support or editing. No true multiuser access other than granting sharing through Nextcloud like Dropbox. The only good part is you can autoupload from your phone.

I've looked at several webapps, like Piwigo. Most of them feel like a single user application or have limited upload and metadata support.

The closest I have found is Digikam using external SQL, but this requires a local application carefully configured with a DB and a fileshare.

extra88
> I've looked at several webapps, like Piwigo. Most of them feel like a single user application or have limited upload and metadata support.

Piwigo definitely supports having multiple users. What kind of metadata do you mean? It supports tagging and reads and displays EXIF data. It has extensions/plugins for adding capabilities.

Odenwaelder
I've been looking for the same thing. An Apple Photo clone would be perfect, as I like this product very much, but I would like to have control over my photos.
ImprovedSilence
Yeah, I’m in the similar boat here too. I have an iPhone, and a Mac, but any interface between the two is needlessly completely busted unless you use iCloud. I want out of that game.
andyjohnson0
I've been using Lychee [1] on a vps with good results.

It looks pretty good, has multi-user capability, metadata editing, etc. It would be nice if it had some geotagging integration and ability to group albums into sets.

I filed a bug report and the developers/maintainers fixed it very quickly.

[1] https://github.com/LycheeOrg/Lychee

dsd
I've been helping on the laravel v4 project. The devs are pretty friendly and I haven't found anything with a better UI than lychee. Three project doesn't do everything but what it does do, it does well.
qwerty456127
Is there one that can import Mac photo libraries an remove duplicate pictures by the way?
preek
I successfully used fdupes and findimagedupes (both commandline tools) for that purpose. I moved my photos Mac library to shotwell (LINUX).
[deleted]
(comment deleted)
mceachen
I had this same issue.

PhotoStructure can read iPhoto libraries (and LR XMPs), and will coalesce both "sidecar" dupes (like JPG/RAW pairs), as well as images that were downsampled from other photo services (like Google Photos). It can leave your originals where they are, or copy them into a single time-stamped folder hierarchy (yes, you can edit the format of the timestamp if you don't like the LR standard). Files with the same SHA are not copied, but referenced in your library db.

Your library's metadata is kept both on disk (in XMP sidecars) as well as a sqlite db for fast access, and you can run queries on the db to do anything more exotic.

pjc50
Bonus question: I, like a lot of other people, did a full Flickr export due to the reduction in hosted pictures. I now have all the metadata - is there a way I could import or use this locally?

(I did take a brief look at it, it's fairly normal json)

uptown
Check out Koken. Pair it with Lightroom for editing if desired. Self hosted and quite powerful.

http://koken.me/

icebraining
Unfortunately not under an open source license, and the license can be revoked or changed "at any time, for any reason, without notice."

(EDIT: clarified that the issue is the license; the source is readable)

uptown
So basically it satisfies all of trbfred's specified requirements.
[deleted]
(comment deleted)
privong
I was excited about MediaGoblin[0] for a while, but it seems development has mostly stalled. The last release was in March 2016(!). Things were going smoothly until they seemingly started devoting their resources to implementing federated sharing instead of developing the core media hosting functionality.

[0] https://mediagoblin.org/

ebg13
This is so sad to hear. I wonder how many total hours have been wasted and projects ruined chasing federation that nobody actually wants in the first place. :|
kermitismyhero
Media Goblin is one of those projects that I really WANT to like, but I can't. The project goals are admirable. And when it's up and running it's great. But it's incredibly finicky to maintain and it's quite challenging to install.
privong
> The project goals are admirable. And when it's up and running it's great. But it's incredibly finicky to maintain and it's quite challenging to install.

Agreed. I guess I could've been more specific in my original comment, easy of installation/maintenance are the things I wish they'd have worked on before worrying about federation. At this point I'm thinking about what will e the least painful way to migrate from mediagoblin to something else.

damianmoore
Shameless plug: I'm currently working on a solution called Photonix, though it's still very much pre-1.0 at the moment.

https://photonix.org/

Installation is fairly simple with Docker, frontend is web-based (React), backend is Python with a sprinkling of Tensorflow. So far auto-tagging of photos by location, object detection and colour is fairly decent. UI is progressing and is useable on most devices, though quite minimal.

Please feel free to check out the demo site and the GitHub issues. I'd really appreciate feedback and help. Thanks.

ConfusedDog
I really like it, though I'm not photographer, so I won't be able to speak for your main user base. One thing I would like is when I open a large image, it should give me a loading status. Instead, it just blacked out for a moment and image suddenly loaded.
damianmoore
Thanks for the feedback. I agree, there should some progress display while downloading. I'll be sure to make sure that's there. There are other optimisations I want to make in this area like selecting a resized version to download depending on the current screen dimensions and pixel density. This will improve loading times.
zerkten
As a developing photographer, having the biggest image isn't as important as having a large-ish image with additional metadata.
damianmoore
At the moment I create "thumbnails" at various sizes on discovery (4k, 2k, etc). Currently the 4k version is displayed but this will adapt and download the smaller versions on smaller devices.

The UI for metadata is a bit unintuitive at the moment but you can scroll down to see it when you're viewing the fullscreen photos.

zerkten
Will you have support for handling duplicate images? People are either super organised, or the complete opposite. When you have multiple copies for backup and then want to consolidate to a library you want to hide your messiness.

I've been struggling to find a tool that handles that handles the duplicates problem within a web interface. I've been experimenting with some approaches including perceptual hashes and was wondering if that's something you'll include?

Is there any way to use metadata from Lightroom Catalogs, or enable people tagging in your current implementation?

damianmoore
I definitely hear you regarding duplication. This whole project evolved from a script I wrote to do just that. There is a concept of multiple versions (files) of an image so different edits with the same metadata timestamp will show only once and you'll be able to select the preferred one. I prefer to handle things this way to start and then suggest deletions based on same hash rather than deleting up front.

I don't have any experience of Lightroom but I can have a go at reading the files if you think it's useful.

zerkten
I like your approach. I'll play around more with your app. Thinking about the LR catalogs, I wonder how much is tied up there versus being written back to image metadata. It's probably better to see what data is there and recommended workflow before spending too much time.
mceachen
OMG, me too. @damianmoore, I've banged on this issue for > 2 years now and have a pretty solid deduping algorithm. I'd be happy to share what I've got with you.
damianmoore
Cool, share or PM me a link and I'd love to take a look.
damianmoore
Face detection and recognition is definitely planned and looks like it should be entirely doable.
barrystaes
I like this architecture and wil spin up a Photonix docker on my Unraid server in a few weeks or so. I wonder why you used graphql for the api?
damianmoore
Thank you. I'd like to hear how that goes.

The thinking behind GraphQL was to allow for advanced filtering, supporting all the attributes we store without a lot of extra API work. The GraphiQL web interface makes it quite nice to explore the data and is bundled in and accessible at /graphql . GraphQL also has "subscriptions" which allows for pushing data from the server. The Apollo JS library I used also provides built-in extras like caching.

jayniz
I have ~1TB of photos from various sources, including backups of old iPhoto libraries. Therefore I often have one photo available in many different sizes, cropped thumbnails, etc. I'd pay money for working duplicate detection :)
bloopernova
Does Photonix work with a read-only volume for /data/photos?

I'm looking at the docker-compose.yml and wanted to give it a go, but not allow it any way of deleting anything :)

damianmoore
Yep, it should work fine with a read-only photo volume as it doesn't make any changes to photo files by default - it just needs to write to the DB which is configured as another container in the Docker Compose file.
bloopernova
Unfortunately I ran into issues with the docker-compose.yml method of running Photonix. I'll give it another try later and create a github issue for you. Thank you for replying to my question!
damianmoore
Sorry that it didn't work for you. Hopefully we can solve your issue.
damianmoore
You might just want to try making a copy of a smaller selection of your photo library in a new folder and giving that to Photonix to try out. That way it doesn't matter if anything were to get removed.
alfredo_burgess
Hey mate,

I have checked this out, running the docker-compose method, and kudos for your work. Looks great.

One issue I ran into was regarding videos (tried with a couple of MOV and MP4) - it doesn't generate a thumbnail and in fact throws an error along the lines of: File "/srv/photonix/photos/models.py", line 84, in base_image_path AttributeError: 'NoneType' object has no attribute 'base_image_path'

Happy to open a gitHub issue, but thought I'd drop you a line here to see if it was your intention to support videos (which would be cool!)

Cheers,

jonahbenton
Not specifically for photos, but I have been a user of Perkeep, a long running project from one of the golang maintainers that focuses on long term storage of one's data:

https://perkeep.org/doc/

zimpenfish
Although it's no longer actively maintained (IIRC) and some of the importers are currently broken for me (Mastodon importer can't talk to Pleroma, Pinboard has a JSON error, Twitter gives random "account not found", etc.)

Which is a shame because I really like Perkeep/Camlistore as a concept.

vorpalhex
Could you try to troubleshoot some of the issues and open PRs? It doesn't take an army to resuscitate a project...

-- Edit --

Actually on checking, it doesn't seem like Perkeep is at all dead. I'm seeing several updates in the last month...

zimpenfish
I was going off https://groups.google.com/forum/#!topic/perkeep/Kqvh2dJVmoo

Which seems to me fairly definite about it no longer being actively maintained. There haven't been any updates to that posted on the @perkeeporg Twitter at least.

vorpalhex
Sounds just like their full time maintainers got busy. May well be able to find another one, or even have a few folks fill in.

I'm actually going to check out perkeep when I get a chance and see if I can't help out some.

flurdy
Many years ago I was dumping all my photos into Coppermine [1]

It was/is very good. But patching it and PHP became a chore and I eventually migrated away to a messy matrix of Dropbox, Flickr, Google Photos and Apple Photo...

I should look into consolidating them all. Always tempted to write my own, aimed at photo albums for non-photographers... but it will probably not move beyond a readme.md...

[1] https://coppermine-gallery.net/

walterbell
For those willing to use local (non-browser) apps, Photosync (iOS, Android, Windows, macOS) will transfer photos between mobile/desktop/cloud and a private central NAS. From there, you can mount the NAS or sync folders to another mobile/desktop device, for use with local apps.

https://www.photosync-app.com/home.html

decasteve
Any particular reason for the browser-based requirement?

I’ve used Lightroom and CaptureOne to manage my photo library but ended up using digiKam in the end because I can just mount a volume and use it from any desktop.

preek
I did exactly that a couple of months ago. After exporting my pictures, I ran fdupes to find duplicates and then imported all images into Shotwell.

I can still take pics on the phone which will be synced via Dropbox and Shotwell picks them up immediately. The sync is faster than I’m used to on iCloud and finally I just have files that I can tag, again.

Shotwell is also super fast, has a similar UI to photos.app (automatic events for example), but it also had hierarchical tags which it can even write to the files itself. So it’s very simple and yet portable without lock-in. Couldn’t be happier. Of course ymmv.

Good luck!

JustSomeNobody
I wish iOS photos app would allow me to sync to my server (SMB, sFTP, whatever). It is the only app allowed to sync in the background without using the location hack.
SpyKiIIer
Is there any of these that can be pointed at a photo library on a NAS or harddrive, instead of having the application copy all the images into it's own location/database (which therefore required double the harddrive space)?
jonathanp88
The static gallery generator sigal(http://sigal.saimon.org) supports this. You can configure it to include the original images in the file tree it builds, but instead of copying the images just make a symlink.
silversconfused
I wrote a shell script to generate thumbnails (including gifs for video) and a page with the thumbnails, file size, and a link to the original. Something like that?
machbio
Drawing Parallels - really would be happy if there was an application that could use Object storage as backend.. and could be hosted with AuthN and AuthZ