HN Explorer

Ask HN: Why is on-device search terrible?

38 points by yonz ↗ HN
My search experience across a lot of products is, to put it bluntly, garbage. Apple Mail, Gmail App on Android, Finder search, Windows Explorer search, MacOs Spotlight... rarely find what I'm looking for.

Why is it so bad? Are indexes / Io ops really that expensive?

39 comments

[ 5.7 ms ] story [ 97.1 ms ] thread
f1refly
Try `find / -iname "foo" 2>/dev/null`, works every time. Also can take quite some time.
hellfish
fd takes less time, would recommend
Sohcahtoa82
On Windows, the fastest way seems to use WizTree to scan the whole hard drive (Takes only a couple seconds to scan an SSD, and even my 2 TB spinning HD with >900,000 files takes less than 15 seconds) and then search the results for the filename.
clockwork-dev
Haven't tried this one, thanks. There's also Everything [0] which is usually my favorite search for Windows. Fast and supports regex.

[0] https://www.voidtools.com/

rococode
Everything search is incredible, for me it serves nearly instant searches for 15TB+ of data without any trouble. I think the secret sauce is a background service which continuously updates the file index.

I also like WinDirStat a lot for Windows file management, which is not really for search but gives a great view of disk usage.

delta_p_delta_x
> I think the secret sauce is a background service which continuously updates the file index

Actually, the secret sauce to both WizTree's and Everything's speed is the NTFS Master File Table (MFT) [1].

[1]: https://learn.microsoft.com/en-us/windows/win32/fileio/maste...

hackrnusr
Is there an open source analog of voidtools Everything yet?

Have never been able to find anything.

pynappo
I've seen flow launcher recommended but haven't personally tried it out yet
hackrnusr
Thanks for the suggestion.

Probably worth benchmarking against fzf/ripgrep when I can find the time.

yonz
So why do the stock implementations suck?
bravetraveler
mlocate/plocate (for locate and updatedb) are also nice

Even if the db is not up to date, it's likely faster to update/search than rely on find

I find simple searches by file name useful, but that's also because I'm fairly diligent to keep my filesystems organized

The launcher called Albert is helpful as a sort of Spotlight stand-in with Linux (maybe elsewhere?). It does math, runs things, and can have things like a dictionary added. I believe it can also search

bravetraveler
Lost my ability to edit, but here's a link to Albert: https://albertlauncher.github.io/

The demonstrations there confirm searching... and way more than I can do justice here

abdullahkhalids
Albert is pretty good.

I also catfish https://launchpad.net/catfish-search which is excellent. Also can search within text files.

yonz
Are either of you concerned about giving permissions to almost everything? I think that was why I uninstalled Albert.
abdullahkhalids
I am not sure what special permissions you are talking about that Albert gets? Just like any other software, it can do whatever the user can do. As far as I understand it doesn't "phone home" or anything with my data. And doesn't really delete or move files in any way.
bravetraveler
A bit, I'll admit I haven't audited it as much as I should -- are you aware of anything in particular?

I install it through the Fedora repositories and only use the bundled extensions, so I don't worry too much about it

runjake
Data point: Spotlight and the Gmail app searches work great for me. What issues are you seeing?

Have you tried rebuilding your Spotlight index? https://support.apple.com/en-us/HT201716

Have you tried rebuilding your Mail.app index? https://support.apple.com/guide/mail/rebuild-mailboxes-mlhlp...

yonz
Email search using the Gmail app on mobile works for you? That's a surprise to me, what is your setup? iPhone/Android, Download all emails / 90 days?

I did rebuild my index, I have a 1TB SSD drive and even excluded external drives.

MH15
iOS spotlight used to work well- now there's a three second delay at times while it fetches the internet-enabled features. All I want is it to find my apps for me!
prepend
I wish there was a way to disable internet search. I used spotlight to open apps for years and now it’s not very efficient, so I have to organize icons and stuff.
yuppie_scum
If you’ve ever ran a Mongo cluster you know that yes, search/indexing is expensive
abrookewood
On Windows, please try Everything from Void Tools. It is insanely quick: https://www.voidtools.com/en-au/support/everything/
zalebz
launch Windows Terminal -> winget install voidtools.Everything

if you're on windows and haven't discovered the newest Terminal experience nor winget, you're welcome

SOLAR_FIELDS
This is finally the Windows experience that developers wanted. I thought that OneGet (PackageManagement) was going to be the way almost a decade ago, but that fizzled out. The guy that wrote that one seems to be totally on board with this though: https://github.com/microsoft/winget-cli/discussions/186.

I think that the guy that wrote OneGet had it mostly right, but it seems (as a layman who knows nothing about Microsoft) that he simply did not have the political power to make it actually get picked up by the higher ups at Microsoft and that someone else did which is why it took an extra half-decade to actually happen.

I do wonder what would have happened in my career had OneGet actually taken off . Previously I used to work exclusively on Windows, and I now work exclusively on non-Windows machines. At the time that I switched developer experience was one of my primary frustrations - and lack of a proper Homebrew/apt/yum like experience on Windows was a non-insignificant part of that. Chocolatey was OK at the time, but paled in comparison to its MacOS and Linux equivalents.

wrldos
I don’t use search. I carefully put things where I will find them.
NoThisIsMe
It's pretty great on GNOME tbh. It worked well on Windows 7 too.
Am4TIfIsER0ppos
Its deliberate to dissuade you from keeping anything on your device. Upload it to the cloud and let google, apple, microsoft, and the government search it for you.

However yes i/o is time consuming and you can see that when programs update their indexes.

yonz
But the index update wouldn't be in the critical path (when a user searches). Unless the index size requirements are so large you can't do it on machine (doubt it)
Am4TIfIsER0ppos
No, an index wouldn't be updated the moment you want to use it. I notice when locate and man perform their updates because my computer is busy while they do. Granted I think it is low priority but I still see the result in btop (or any other system monitor program). find isn't quick unless I repeat the same search and the directories are cached.

I am not complaining about spending time on i/o and processing to make an index. I merely stated that it does take time. I didn't think Windows' one was worth it when I last used Windows so I would always disable it but I haven't done likewise on linux.

Sesse__
FWIW, man-db index updates got a lot faster in the latest versions (roughly 40x).
pjot
On the other hand, search in Photos on iPhone is _scary_ good.

Searching for “baseball” will return pictures of a baseball, a team playing baseball, photos that were taken at the location of a baseball stadium, AND photos that literally have the word baseball in text somewhere within the photo.

specialist
Overcast's local text search is also turrible. It very much regularly impedes my usage.
simne
This has same roots as why far manager (norton commander clone) does not popular on MacOs.

- For fast search need to use indexes, which tend to be few times larger than data. Far on MacOs show 4-5 versions of same file ("timemachine" version system).

I think this is just psychology question, people don't like when appear those strange files.