Search Engine with User Defined Filters

5 points by oldpond 11y ago ↗ HN

Has anyone made a search engine that allows the end user to filter sources? For example, I don't want the search engine to show me any links from, for example, newspapers, or only show North American results or don't show anything from a particular ISP network.

This is opposite of how search engines work now. They search everything and filter based on search criteria, not source criteria. Some might argue that you just add more sophisticated results filters to get the same thing, but then you have the problem of sources trying to defeat your filters all the time.

I suppose the ethical internet folks might put something together that boycotts certain segments of the net very openly, or provides the end user with the ability to tailor the input sources to suit their needs. Kind of like Search as SaaS. Does that seem viable?

6 comments

[ 3.0 ms ] story [ 24.7 ms ] thread

r721 11y ago ↗

Google Search used to have "blocked sites" feature:

http://googleblog.blogspot.com/2011/03/hide-sites-to-find-mo...

They dropped it in 2013:

http://googlesystem.blogspot.com/2013/03/google-discontinues...

oldpond 11y ago ↗

Yes, but blacklisting is not what I am thinking of. I mean the ability to exclude whole chunks from the crawler. Google would still crawl those sites, but the blacklist would exclude them from results.

richardbrevig 11y ago ↗

I'm working on something similar to this in a very defined vertical. The results will be divided into website types and the user can specify which result types they want to see.

You have a specific use case you're looking for or thinking of creating one?

oldpond 11y ago ↗

Well, say I am a programmer, and I find I can get most of what I need from the internet from Europe, North America, and Australia, for example. And, to keep the pool small and focused I turn off all media sources (news, entertainment, etc.). The crawler would use white-lists to include obvious omissions. I wonder how small you could get the pool of indexed data and still get effective search results.

rgovind 11y ago ↗

I will try to give an example. During recent Charlie Hebdo incident, I wanted to find their address. If you search for "charlie Hebdo Paris Address" the first page and many subsequent pages are all dominated by media outlets. (http://imgur.com/3nyjURA) I should go to "maps" tab to get the address

This happens very often. Media outlets dominate search results.

rgovind 11y ago ↗

I see and share your pain. No filters exactly but you can use millionshort.com to filter out most media/newspaper websites.