Tell HN: I'm working on practical implementations of tag-based file organization
Here are the core features of my MVP, addressing typical concerns with tag-based/non-hierarchical approaches, like efficient discovery, the tedium of creating/managing tags, lock-in, and portability:
- Query DSL:
A simple but expressive query language is provided to search for files/folders, based on their tags stored in a local db. Tags can be bare or categorized, and the categories can be user-defined or system generated.
- Dynamic Virtual Folders:
Guided drill-down file browsing via query-backed dynamic virtual folders as a first-class feature. Research shows spatial browsing/exploration is just as critical to file discovery as direct search.
- Heuristic auto-tagging & triage queue:
Tags are automatically generated on initial indexing using sophisticated heuristics based on file/folder metadata (path, name, os metadata, etc). Any existing or newly created files/folders that remain under-tagged go into a triage queue for quick manual bulk or individual tagging.
- Managed, flat, sharded storage:
On import, files are moved, flattened and name-normalized with their tags, into a sharded internal store, with all subsequent access routed through the tool's interfaces. While this breaks from file managers that typically leave files in place, it enforces the workflow shift needed for non-hierarchical organization to work well. Tags can be exported back to standard folder structures at any time.
- "Project" folders as atomic units:
Project directories like code repositories are not flattened, and can be tagged and retrieved as single, atomic units (in addition to their contents), which avoids flooding listings with thousands of their nested files/folders.
Portability & Privacy:
All files and project folders retain their exact contents. Since their tags are embedded in their names, and the original structure can be exported back if desired, there is no lock-in or loss of tags if the tool and its metadata db are uninstalled. Also, although file paths and metadata might be sent to a central API server to support features like the automatic tags and dynamic virtual folders generation, file contents stay strictly local.
Roadmap
As mentioned, the initial MVP is a TUI which provides a rich interface for browsing, search, basic file operations, and import, plus a CLI for some basic administrative actions. Eventually, I plan to support: - A native GUI interface - Self-hosted and Cloud-hosted servers for web UI access - Bring-Your-Own-Cloud-Storage (e.g., S3 and/or Google Drive) to replicate the internal flat shared storage in the cloud, for backup and access from other devices - Import from IMAP, mobile devices, etc. - Versioning
What are your thoughts? Do you think these approaches address the limitations of hierarchical file organization, as well as the friction involved with typical tag-based tools? Would you be comfortable with file metadata being processed or stored in the cloud? And, do you think this tool could provide you enough value to pay for it?
3 comments
[ 0.37 ms ] story [ 11.8 ms ] threadI did hold out hope that I could find an app or capability that I could simply enter text into a search box and it would provide a list of matches, but those matches often come without surrounding context, sometimes what I'm looking for is in a folder, but isn't coming up in the search results, and it's really kind of chaos. I've used Spotlight, Finder's search, some apps in the past, but I think what I need, can't speak to others, is that all incoming files go into one folder (or subfolders, but not based on tags or anything) as an 'immutable archive', and then something walks through all of that regularly building different views of the data. So for working on tax estimates and end of year tax planning, I'd like a 'view' that shows me all the bank statements, invoices, bills, etc. in a structured view like Finder gives, but without actually having all of those documents and artifacts moved around anywhere, it's just a view that knows what to include, and knows context of what I'm doing for that.
I also need to absolutely know and trust that the view is showing me all of the items relevant to that point-in-time view that I need to take care of that task. So I don't think tags and DSLs are the answer, they may support the ops, but someone has to define what to include, when, for each 'view'.
So I'm focusing now on creating a system to address the same challenge, but by creating and managing views, that the software then goes and finds all of the relevant artifacts that should be in a defined 'view', and arrange them in a visual finder-like structure that isn't actually on the file system, but is 'virtual' and points to all of those artifacts, that I can then work with, along with any context needed.
As another example, let's say I create a financial management system (personal), any entry into that 'accounting' system will have associated all receipts, bills, urls, brochures related to that expense; income will have similar associated items. Having entries and then having to go back and find all that context manually, sucks.
As for cloud-hosted, Google, etc., way too much complexity for a first pass. I'm sticking with iCloud for storage of 'artifacts', and designing a "National Archives" system for all my captures of docs, statements etc, that become immutable, that have metadata extracted, and then will look at how to implement a view-like system that I've sort of described.
Key for me is simplicity, and 'singularity', meaning I don't want sharing and all kinds of behind-the-scenes complexity, at least not until it's needed. An analogy: I start with SQLite always, first, for anything that needs a database, because I don't want to be a DB admin (and use PostgreSQL) if it's not needed. PG is awesome, and when I need it, I'll use it.
Not sure any of this is helpful, but the biggest issue I have with the list of features you've listed is what's missing: Trust. How can I be absolutely certain I'm looking at all of the relevant files and artifacts for any particular search of 'view'?