On mobile it asks for an email in a persistent footer that takes up 1/3 of the screen. When you type a fake email (there is no way to remove it) it brings up another footer asking if you would like to have such a footer on your own website ( also unremovable). Thank God for reader mode.
A better solution is to enter the Reading mode of your browser.
Some sites popup dialogs and even if you delete all the elements they have set some of the elements in the original page to not allow scroll. With reading mode you just bypass all that crap.
This is true, although if you do want to fix that particular issue then unticking overflow:none against body usually does the trick :) But yes, reader mode is simpler.
With my limited knowledge of javascript, I don't think it can be done as any code to change font will be lost when new page is loaded. You might want to look into greasemonkey or their official chrome extension(I haven't tested it out).
Don't worry, you're not missing out. The article is a bunch of questionable assumptions made from highly dodgy metrics written by the kind of person who calls people a "Solopreneur".
DuckDuckGo, on the other hand, is very very much worth your time, if you aren't familiar with it: https://duckduckgo.com
Recently switched over to DuckDuckGo on all my devices. Tried to do the same thing several years ago and found it didn't really work out, but this time around it is so much better, both when it comes to speed and results.
If anyone has doubts because they tried it years ago, I'd say go for it again.
> Google is really tor/proxy/anonymous user unfriendly. Requiring users to solve as many as four or five CAPTCHAs (seriously fuck you google).
They're like this for a reason. They didn't implement complex detection of proxies to annoy users. Just to keep everyone out who's not supposed to use their search.
Yeah, people like to try and crawl Google. So they're keeping robots out.
But it's actually really nice to be able to pull a result or two from ddg by crawling, or to be able to use a VPN without having to solve a zillion captcha.
I run into those captcha's occasionally and generally reconnecting to a different VPN makes them go away. I always assume they are tied to bot activity coming through the same IP or range. I've scraped google in the past (years and years ago) and from what I recall you end up hitting the captchas after a set number of results in too short a period. Back then it was a pretty standard SEO activity, but it's usefulness was minimized when google started localizing results.
"And since I regularly browse via a Digital Ocean VM"
Do you mean using a DO VM as a VPN, or via X11 forwarding to your desktop over SSH, or actually browsing on the DO VM's desktop using VNC? I've done all three in the past as experiments in private browsing, and the latter is too laggy to be comfortable.
I started feeling like I had all my eggs in the same basket. Google has my email, it knows where I want to go and when (Google Maps), it translates stuff for me, and it knows what I search for. While it doesn't really affect me that Google has all this information, I've become more and more uncomfortable with the fact that they do.
So I figured if I had a choice of two search engines, where I get satisfying results in both of them, and one of them doesn't track me, why go for the one that tracks me?
I'd gladly do the same switch when it comes to Gmail, but I really really like Gmail's web interface, haven't gotten over that hurdle yet.
EDIT: I also switched from Google Chrome for pretty much the same reasons.
I'm currently using Vivaldi, but have to say that it's not really an ideal analogue of Chrome as it doesn't carry over some of the everyday features and affordances of Chrome. That's partly understandable, because it's actually (apparently) intended to be a browser for Opera users to migrate to. It's also a bit buggy.
I'm surprised there isn't a straightforward and user-friendly Google-less Chromium browser available.
I 100% agree with your statement, but I feel that Vivaldi is currently the best browser that's close to Chrome in terms of feature parity. Hopefully though, with more support, they will become an efficient browser that's unique enough to be separate from Chrome but have the features we know and love.
Firefox on Android is vastly superior experience due to one simple thing - extensions. microBlock origin makes web actually usable again on mobile platforms.
I made the jump to Fastmail.com and couldn't be happier. The UI is very snappy (much faster than Gmail's), the actual notifications and delivery is faster, and it's a better experience overall, for me.
Plus, Google doesn't get to see my mail any more. Ditto for Firefox vs Chrome.
I'm a fan of moving away from Google where possible as well, but Fastmail would cost me $500/year with 10 accounts. My G Suite account is one of the original, so I can have up to 200 accounts for free. Granted, it's only 15GB of storage, but even after about 10 years, I only have about 1GB in my inbox, so I'm ok with the lower amount of space.
FastMail aliases are great. I've used some for sites I'm more likely to want to cut off access to me, for instance. Whereas address+site@gmail.com gives you sortability, a true alias gives you the easy ability to just shut a site out of access to you.
Another issue I had was that my main email address was already in Google and Microsoft's systems for a couple reasons, and it wouldn't let me set it properly when changing my email address everywhere. So I have google@ and microsoft@ aliases just to work around their account management quirks.
Wow, what do you do with 10 accounts? Isn't checking all of them a hassle?
I have multiple domains and aliases, and Fastmail is much better at those than Google Apps ever was (I also have the free plan but you can never change your initial domain), so I'm much happier.
I have six. Some of my clients ask me to manage their AdWords for them. Plus, I occasionally need to log in to my wife's Gmail account. (To confirm logins when paying bills, for example. Anything personal, she's smart enough to keep on her own domain.)
Several accounts are for separating services - AWS has it's own account, for example, so does Dropbox. I use a different email address for forums and such.
A few for friends and family as well. --Friends I can tell to pay for themselves if I need to, but family, not so much.
Edit: As for checking them, most don't get many emails, and every good email app can handle multiple accounts easily, so I get notifications on my phone.
Fastmail allows you to do things like stavrosk@dropbox.yourdomain.com, and then you can filter emails by that domain name. Lets you easily block out spam without having completely different email addresses.
I can only speak for my own decision-making process, but I like to pay for my email so that I have greater confidence that the provider will still be in business 10 years from now, and more confidence that my data is my own.
Paying means that the provider doesn't have to mine your email for keywords in order to display targeted ads (yeah, I know Google theoretically stopped doing this recently). So perhaps "your own" in the sense that another entity isn't accessing it.
Interesting that you bring up the UI, switching to Fastmail's UI made me start checking/managing my personal email like a responsible adult again.
I could never get over the way using GMail felt like fighting with a sluggish toy version of email compared to desktop outlook (and I'm not really a fan of outlook, either). I just assumed that GMail's interface was the best a webapp could offer: it was the best I'd seen so far and had a no-longer-deserved "who could beat gmail at webmail?" bug in my mind.
I'm kinda surprised fastmail hasn't made larger inroads among the kind of techies that will occasionally bemoan giving their lives over to google.
I am really surprised at how fast its UI loads and operates. It feels like a native app in a web where everything else feels like a slug. Major props to Fastmail for this. Hell, even Thunderbird feels slower.
Same here, trying to avoid it as much as possible Google now mainly has my email and calendar. I feel like there should be a better option for calendar but I haven't been able to find it yet (probably because I use DDG :P), but sadly, the only thing with a better interface than Gmail is Google Inbox, so...
The problem is that Google often provides the best service for the buck. But even if money is taken out of the equation some Google services are just the best. Like Google maps or even G suite (this might vary by person).
In my case it was this annoying Google's insistence on telling me what I was actually looking for instead of searching exactly what I wanted, including ignoring "" etc. It became too much hassle, I felt like fighting with windmills, then tried DDG, it did what I told it, results weren't terrible, so now I use it by default and only a few times a week I need to try Google. Maybe for regular folks Google's way is a lifesaver, but for precise search I need it feels like Altavista back when I was a kid.
> In my case it was this annoying Google's insistence on telling me what I was actually looking for instead of searching exactly what I wanted, including ignoring "" etc.
GOD YES! This gets incredibly annoying when searching for version-specific information, considering the differences there can be between version foo and version bar of $DISTRO. Nothing like searching for something specific to CentOS 7 and getting zillions of CentOS 6 and 5 hits.
Google will also helpfully ignore the "Verbatim" option, and God help you if you're trying to search for multiple specific phrases.
Google is looking more and more like Alta Vista in its waning days.
I like the suggestion functionality for typo correction, but dont like it when Google assumes something other than I wrote. This is bad. It often happens when searching technical stuff. I am not regular Joe and I don't search regular stuff.
The problem is, whenever you can't find something in DDG, you assume it's because it's DDG, and search again with !g. Meaning I end up requerying 40% or so of my searches using !g (prefixing your query with !g redirects your query from DDG to encrypted google)
Here's some example queries I ended up !g yesterday that had much better results in google than DDG:
reduce kendo javascript file size - For me, the useful result was number 2 in google. In DDG it was number 11 ("Only What You Need | Kendo UI Getting Started").
ptr overwatch - PTR is the test patch of the game overwatch. It's regularly changed. The correct result "Overwatch PTR Now Available - August 29, 2017" is number 1 in google, in DDG the blog post August 12th is not available (well, correct from my perspective)
I tend to find anything speculative DDG is fairly terrible at. For example yesterday I was googling about trying to identify the source of some weird animation CPU cycles I was seeing in Chrome Performance Profiler, I didn't really get to the bottom of it, but I ended up ditching DDG and doing all the queries in google because I would otherwise have ended up searching everything twice.
I'm also getting quite frustrated with the "instant answer" functionality, it's generally terrible. One of the most annoying ones is the SO instant answer that just utterly sucks, you can't see the code, it usually cuts anything useful in half, and takes up a HUGE amount of space meaning on a laptop you've got to scroll to start seeing the results. I just want to be able to turn it off, but because they don't track you they don't offer that functionality.
Also the maps one is really bad. I almost always want directions, but clicking the map takes me to a really nonfunctional, bare-bones map that doesn't have directions and then I have to click another button to actually get the directions. There's a drop down where you can choose your map type, but it definitely doesn't work as expected, I just want it to always embed a google maps instead of whatever they're doing.
Basically UX ain't DDG's strong point.
Ironically, if you DDG: "turn off certain instant answers duckduckgo" it comes up with terrible search results and no answer. If you "!g turn off certain instant answers duckduckgo", the top result is at least relevant.
I also find the entire bang thing to be a gimmick. Who wants to use !r when !g with "reddit" at the end will always get you better search results, reddit's own search is abysmal (as is !so).
EDIT: I'm being overly negative, after all, I haven't actually switched back, like I did last time I tried to use DDG. So it's definitely worth a go, but I don't think it's ready for your Mum to use. Also, I have Bing on my phone's Chrome to avoid AMP, and it's actually quite good.
DDG is definitely behind Google still, although it's come really far in a few years. I agree with most of what you're writing, but personally I'm okay with it not replacing 100% of my searches yet, got to start somewhere and I feel this is an acceptable point to switch at.
I have a rather better experience than you. Yes, DDG isn't as good as Google in programming queries, but it's good everywhere else, and I like knowing that most of my searches aren't tracked. That's worth having to retry a few searches once in a while.
Weird, I find myself reverting to Google (or rather Startpage, !s) maybe once or twice a month, when I don't find satisfying results on DDG. And I'd say that in many if not most cases, I don't find good answers on Google then, either.
And I quite like using bangs, e.g. !w for Wikipedia or !i for images, rather than having to go to the initial search results on the search engine and then clicking again.
Finally, I like that on DDG, you can just arrow up/down through the search results, and then open one with enter (or cmd-enter to open it in a background tab), without reverting to the mouse.
Same problem here! For development searches Google typically blows ddg out of the water but I still love ddg for general search. I didnt know about the !g :D
Ditto, for the past month or two. I'd love to get off of Gmail and Google drive, as well, but that's a longer term project.
Main reason: concerns about tracking and privacy.
Secondarily: politics.
DDG so far has been acceptable. As long as they keep their political opinions to themselves, the honeymoon will continue. My love affair with Google, on the other hand, is over. :(
Email is a tough one, but I've been using FastMail.
No advertising, strong privacy policy, based in Australia (a country with a decent privacy track record that we know of).
It's $3/month which is a decent price for liberating a lot of data from advertisers.
The spam filter is playing catch-up to Google so that's a bit of a shock at first, but you can train it up well or use second layer measures like Sanebox.
When DDG shows an ad, is there no tracking involved in that? For example, if I were to click on an ad (after turning my ad blocker off), does the destination site not get any information about me or specifically what I searched for?
And don't forget to use the bangs that can allow you to check other engines results fast if you need.
For people afraid of getting off google, you can always search something like '!g my-search', it works the same for youtube(!yt), google image(!gi), or even hackernews(!hn)
And of course the best bang is the 'I am feeling lucky' one (!), i.e.: 'hackernews !'
It's worth noting that the !g bang redirects you to encrypted.google.com, it's more secure but results are often different from a regular www.google.com search. It bugged me for a while not knowing why some queries returned unusual results. ( more here https://duck.co/forum/thread/2880/remove-the-encrypted-subdo... )
> linking to encrypted.google.com not only is not necessary but it prevents local search results from being displayed. e.g. no google.ro search results, only google.com
I can only see this as a good thing. If I want local search results, I'll add local qualifiers like "USA", "Texas", "Houston"
That was an odd one to read, being in Houston. I thought somehow you added wildcards that were auto filled with the users location, I then realized that I'm an idiot.
A lot of JavaScript things have been buggy for me with encrypted.google.com as well. For example the Google timer cards and such often just won't start at all, while they'll work fine on the regular google.com.
!ud → Urban Dictionary (what do you mean, she's 'office cute'?)
!wen → Wikipedia English
!w.. → Wikipedia (two letter language code, e.g., 'nl')
!wikt Wiktionary
Usually if I need a certain engine I just guess the bang and it's usually supported and correct. !gm for google maps, !tineye for tineye, !wayback for the wayback machine...
If that information isn't personalised in any fashion and the data are retained for a minimum period, I'd be OK with sorting out what types of queries aren't satisfied on DDG itself.
I tried switching to DDG a few years ago, but I found myself using !ge more often than not, so I wasn't really sure what the point of using DDG was. For my needs, DDG's results are pretty mediocre. Which is a shame, becuase I'd rather support them over Google.
Your needs a few years ago, or your needs now? As benbenhu says, it has gotten a lot better. Lately I've found that when DDG doesn't have the thing I want and I stick the !g in, Google doesn't necessarily do any better, which implies that it's just a hard search.
I'm tired of people mentioning !g. If you want to use google, just use google. Stop using duck duck go just to search google, and then say "but duck duck go is better" lo. no it isn't. clearly.
Yeah, if someone is just going to search Google all the time. If one is going to use DDG most of the time, but then pivots, it's worth noting. I use DDG to get to Wikipedia, Amazon, Google Images, and sometimes Google (for technical searches).
It's a way to wean yourself off of Google or quickly see results from a different perspective. There's nothing wrong with it, it's just an easy way out for people who don't trust ddg just yet.
With DDG as my default search engine (Safari, macOS & iOS), I can (in the rare cases that I need to) easily and quickly search on google or startpage or other sites (by appending !g or !s in the search box).
If you're on Google as your default search engine, it's not so convenient to see other engine's results (plus you're being tracked all the time).
It's not about using duckduckgo to search with google. I use it when duckduckgo results aren't good enough, instead of typing everything in google again. I have been using it less frequently recently.
When I tried it years ago, I would search DDG first, then use !g if I didn't get what I wanted. I ended up using that a lot, so I switched back to Google.
Sounds like they have changed a lot since then, so I'll probably give it another shot.
If you want to use google, but don't want them to connect you to your search terms and connect you to their ecosystem and track you some more, then use DDG and !g.
Now that I've learned that !a works for amazon.com and duckduckgo.com gets affiliate revenue that way, then that's how I'll search amazon from now on. Need to boost up the underdog that cares about privacy, because google surely doesn't.
But then you can’t use bangs anymore. I think the idea is that Google is there if you need it while still retaining the benefit of using bangs. So, it’s the best of both worlds.
'<Song name> !' or '<song name> youtube !' takes you right to it.
I'd love a feature that opens the first result on a search engine like wikipedia does by default. Biggest use case for me would be imdb but there would be others.
DuckDuckGo has that by using a backslash, a space and a search term (\ bed intruder youtube) or an exclamation a space and the search term (although I believe the former is the official way now). I use it all the time, it's like a superpower.
I added bangs to google by defining custom search engines with a one or two letter prefix in chrome://settings/searchEngines
Now if I type "h my-search" into the omnibar, it goes to google.com/search?q=site%3Anews.ycombinator.com+my-search which gives me only Google search results from HN.
This is using google's algorithm (not the search bar built into whichever website like DDG) which is still the best, especially if you constrain it to one domain name. I also don't have to type the "!"
The prefix messes up google's text prediction. I guess no one from the chromium team is using this feature, since it would be trivial to fix.
That also works with bookmark keywords in Firefox. The setting is in each bookmark's properties; it takes variables (I leave it to the reader to look up the syntax).
Would be interested to hear what search engines people here use. I feel the promise of DDG is the ability to match your search engine to your purpose. Maybe if you want to search for a particular code snippet you can use a search engine that is friendly to programming syntax. Or if you want more lateral results, you use a search engine with an unusual algorithm. But I haven't yet been able to find alternatives, if they exist.
I've had DDG as my primary search engine for I'd guess at least 5 years if not more. It's extremely rare that I ever need to use anything else. I do wish there was a way to tell Yummly to go to hell whenever I search for recipes, though.
Same here. Tried a while back and didn't really take. I found myself Googling most things after DDG'ing them. Tried again recently because I love the mission, and it's working much better. Usually what I want is in the top few results, although Google still does a bit better with search result quality. I see myself using it as my main search engine at this point.
I only miss google as a DDG user when DDG doesn't have a tool google has to give you an automatic result, i.e. when trying to look up some unit conversion ratio like lb to kg. But when you know that is what you want there is always !g.
$ units
Currency exchange rates from www.timegenie.com on 2017-08-24
2980 units, 109 prefixes, 96 nonlinear units
You have: 5 lbs
You want: kg
* 2.2679619
/ 0.44092452
You have: 100 furlongs
You want: meters
* 20116.8
/ 4.9709695e-05
You have: 100 miles/hour
You want: meters/second
* 44.704
/ 0.022369363
You have: 2 kiloisraelnewshekels
You want: picodollars
* 5.5271109e+14
/ 1.8092635e-15
I did the same too 2 months. If I can't it, I'll give yahoo and bing a chance first before Google. There really hasn't been much that I can't find with the first 3.
It is quite for common things but searching uncommon ones always throws it off. Searching for research publications in a specific domain or just haskell or some thing like just returns 1good result with 10 bad ones. So I use !g for those.
Also the autocorrect really throws my search terms off while Google does is right most of the time.
Same here. Two years ago DDG just wasn't cutting it. Today, I'm just fine with it.
I especially like how programming questions usually provide a top result from Stack Overflow, a lot of times I don't even need to click the link to see the answer I'm looking for.
I actually find myself going to DDG after searching on devices that default to Google. The results are much better for a lot of different things. They even have a cryptocurrency tab.
I use it as my default, and I find that at least once a day I ad a g! to my query to go to the google results. I started maybe a month and a half ago and at this point I have a decent instinct as to when I'm going to need to go to google.
The privacy aspect was the driver for me, but it wasn't enough to make the switch all this time. The privacy aspect coupled with an easy way to get google results when I need them is. While I've known they had the bang queries for a long time, it was actually a youtube search result that finally made me shift. I forget what it was, but I was searching for something for my kids and it looked like my activity had polluted the results. That has been a problem for quite some time and this particular instance was enough for me to make the change. It was a harmless, but I really don't like the idea that my activity will potentially bleed into that of my family, and frankly I get irritated when their activity bleeds into mine. DuckDuckGo doesn't replace youtube search, but it was more of the general principal. I tried working with different accounts over the years, but it's not easy to switch on all platforms and if you've ever tried entering a 60 character password on a playstation, you know why that's a non-starter.
I also tried it a few years ago but was disappointed by the speed - at a time when the google search responses came back "instantly", the latency of DDG was noticeable and not good-enough.
Fast forward to 2017 and that appears to have been solved and the results are as fast as google are from what my brain/eyes can tell anyway - I would not be surprised to find out that google was faster if you timed it. In the past 6+ months I've been using DDG, I've only ever found the need to switch back to google once (and that was image search)
With regard to Yandex and Baidu: there's the notion of threat proximity. Google is a US company, beholden to US laws. If you reside in the United States, you also reside in the jurisdiction of state actors with the capability to get at you.
If Putin himself showed up to Yandex HQ and demanded a full data dump of every search you made, he is still incapable of touching you, whereas J. Random Agent at the FBI can throw you in a 6x8 steel box.
I'm sure DDG works great if you're an American. As someone who has tried a region specific version of DDG though, I can say the results are downright terrible.
Some say the DDG bangs are a solution. What do I win by doing that? It only made me resort to !g all the time, because the results were so bad.
Now I use https://www.startpage.com/ with region set to Swedish. It's practically a proxy for Google search, so it gives me the right results but sans the filter bubble experience (yes, I want the regional bubble). If you're a non US user, I can recommend it.
I must say I'm having the reverse experience. I very seldom use its Danish regional setting, but even for Danish searches DDG is fine. It's only 4-5 times a month I need the !g bang. But knowing about Startpage, I now use the !sp bang instead of !g.
Indeed, due to the bangs, I know use DDG as my proxy to a lot of other search engines, like Wikipedia (!w/!wda/!w..) and Wiktionary (!wikt), and even obscure ones like Memory Alpha (!memoryalpha). Very handy indeed.
Note you can access startpage results from DDG with the "!s" bang.
I rarely use region specific DDG, but when I do, I find it ok. With DDG I can manually enable/disable and switch the region, while Google guesses based on my IP - very annoying, given that I frequently use VPNs. (Of course, if you're logged in to Google, you can specify region/language/etc., but if you don't want to be tracked and delete their cookies or use the browser in private/porn/incognito mode, then it'll just assume that you are looking for Dutch things in the Netherlands, because that's where the VPN exit is...)
As Finn I rarely search for Finnish specific stuff. Sometimes I check prices or if something ships here or not, but that is exceedingly rare and for that DDG works fine.
As for bangs, I love skipping clicks. Most of the time I know what I want and can use bangs accordingly. I'm guessing my most common tags are !a, !yt, !gh, !wiki, !wolf and then some game(s) specific bangs.
I never touch the DDG settings, but I often search in about a dozen different languages, and I get great results. The rare times I try to find something in another search engine, including Google, I get worse (or not better at best) results there, even for quite obscure topics, so I guess everything is user-specific.
I do agree with that. As I am from India I find myself hitting g! or actually opening google.co.in more often than I should but I am still trying DDG and will give it few weeks more.
StartPage doesn't seem to be helpful at all. It doesn't let me select language and region separately. I need to select English UK and then it sets my region as UK. Also, it's very slow (at least for me)
"Google would be an elephant while DuckDuckGo is a mosquito (this is not to emphasize; I’m actually making things better for DuckDuckGo)."
Why would this guy go out of his way to say this metaphor is actually literally informative and generous towards DuckDuckGo while apparently having exerted no mental effort and being incorrect by orders of magnitude on his own data?
If you're most generous to the writer, you get 12 million hits /day vs 13 billion hits /day according to his data from Wolfram, for a ratio of ~1/1100, which applied to a pygmy elephant of 5500 lbs yields a corresponding weight of ~5 lbs for the mosquito, or over 2,000,000 mg, versus the average mosquito weight of about 5 mg. Even if he meant a small 2000 kilogram pygmy elephant and a 20mg elephant mosquito, he's still 5 orders of magnitude too generous towards Google in mass comparison in his metaphor in which he is "actually making things better for DuckDuckGo".
I think most people think of animals like African Elephants when they hear "elephant", though, in which case you're looking at a 13,000 pound animal versus a ~12 pound one if you use hits as your metric, or a ~95 lb one if you go by visits as your metric.
So an actually fair metaphor is if Google's an elephant, DuckDuckGo is somewhere between a goose and a hyena. Better watch out, Google.
Yeah, be really scared MSFT, two geeks from Stanford that wanted to sell for $800k are going to challenge you in the future.
A little (long-overdue) help from anti-trust authorities and Google might just have to watch out. After a while migration to another SE is logarithmic. https://duckduckgo.com/traffic.html Google pays for a lot of its traffic, see Traffic Acquisition Costs, directly and indirectly.
Exactly. Most people, when thinking about the relative size of animals, evaluate the 2-dimensional area of those animals. We certainly don't tend to consider mass, weight, or even volume. So the metaphor as originally given (mosquito vs elephant) works well.
In which case he would actually have been generous to DuckDuckGo while not being wrong by orders of magnitude.
Conclusion: OP meant linear size, not mass. He could have stated it explicitly though, because one would naturally think that he was referring to mass.
I did a quick search comparing the mass of a mallard to a Canadian goose (default options returned for 'weight of duck' and 'weight of goose'). A large mallard on avg is about 3.5 lbs, and a small Canadian goose on average is about 7.1 lbs. In this case, yes! Two ducks == 1 goose.
Depends on which species [0]. Elephants weight 3500-5900 kg[1], devide both numbers by 1100 to get 3.18-5.36 kg (7-11.8 pounds) and you have a simple interval-overlap search problem[2]. For example with dogs[3] the answer is a Japanese Chin[4]. So it should be "Google would be an elephant while DuckDuckGo is a ~~chihuahua~~ very small dog".
Not really, but "exact same end result" is far from the current situation. And I'm not saying they shouldn't do it, rather that they really need to catch up.
Google (not counting other Alphabet companies) does a lots of things. I think the original analogy is mostly right.
It might tell us how realistic scaling to the size of Google is, for example. I don't think it does, but it's an interest thought: could DDG manage Google's search traffic with ~1,000 employees (scaling linearly)? Could it do the same with fewer than that?
A very small part of Google employees actually work on search. If google were to sell everything but the search division, they'd be a small company. The ad business and infrastructure are used by search but not the product.
The author went out of his way to say that the scale of the metaphor was wrong. I don't find it that pedantic for someone to point out that the opposite is true, especially when it makes a huge difference to the reader understanding the scale of DuckDuckGo's accomplishments.
I agree with parent, it's not just a metaphor but a wildly inaccurate guess that makes the reader think duckduckgo is more irrelevant than it really is.
> That was one of the most HN comments I've ever read.
HN has a reputation among some[who?][weasel words] as a place where many commenters are some combination of pedantic, overly literal and out of touch with mainstream culture. Taking this metaphor to pieces would seem to check the first 2 boxes pretty neatly.
You actually make it sound bad. I would more likely to say they are precise and usually won't allow for inaccurate information to fly through the front page without pointing out the issues.
Let me use another metaphor to describe why some of us more "common folk" roll our eyes when we see objections like this prominent in the comments: while HN doesn't seem to miss the forest for the trees, it often seems as or more interested in focusing on a few individual trees than discussing the forest.
I don't think the correctness or incorrectness of the scale of the metaphor really changes the way the whole article reads. Yet it's the first discussion piece I see in the comments on HN, and that doesn't surprise me a bit.
That said, it's a known quirk of the site and not a problem in my mind. I don't hate the tendency even it does cause a regular eye roll from me.
There is definitely a culture of contrarianism on HN; there is a tendency for some users here to play devil's advocate or just argue against the logic of any article for the sake of feeling superior or clever. I'll probably get downvoted for this comment, but doesn't change the reality. And most of us are probably guilty of blending in with that culture here occasionally, while others seem to thrive on it.
I think they mean the tendency to take a statement and apply maths or logic to show that it is incorrect (or less frequently, correct).
I regularly do this (to the frustration of those around me) and I think there is a strong correlation between enjoying hacker news and having this personality trait.
No, the point is not simply using math to show something is wrong. It's using math to show something that doesn't need to be 100% correct to convey the desired meaning is wrong. It's pedantry to the point of wondering if the person responding has ever actually had a conversation with a human being that didn't involve a keyboard. If someone hears an elephant/mosquito metaphor in this context and starts doing math instead of simply taking it for what it is - a hyperbolic way of saying "this thing is really, really big/impressive/well-funded/whatever, and this other thing is... the opposite," that's a level of social awkwardness and borderline autism that is simply unnecessary in any discussion.
Okay, DDG is a hyena and not a mosquito. Great. Tell me how that changes the meaning of the article, or the strength of the facts used, or its conclusions, or the credibility of the author.
"I found a small detail in an irrelevant anecdote that is wrong (or more likely, that I misunderstood completely), therefore your entire argument is suspect and you're bad and you should feel bad."
HN comments have a tendency to obsess about almost irrelevant details. Exact, precise dictionary definitions of words clearly being used in context to mean something not exact matching the dictionary. Arguments over whether passing a pointer counts as pass by value or pass by reference. Taking metaphors and going off on numerical tangents that don't add anything to the meaning or discussion.
LOL - in a metaphorical / rhetorical contest, yes, the difference is irrelevant. What matters is the sense of overwhelming difference between the elephant and <insert most common animals or insects>.
In this context — absolutely.
Alas, the worrying trend I see more and more lately that people start to have trouble seeing context of anything. Sometimes it can be amusing, sometimes it looks dangerous.
Let's take a look here. The message being communicated was "google is much much larger than duckduckgo". Metaphor used carried that information. Communication achieved.
What's this about a goose and a mosquito? What's that adding to the communication? Seems pretty irrelevant.
I think it adds a lot. Comparing an elephant and a mosquito makes it seem like the mosquito is almost non-existent. A goose compared to an elephant is much more significant. A goose can stand up to an elephant. [1]
Google compared to DDK is like an elephant fighting a goose? That is not remotely the original meaning. The original meaning was nothing more than google being much bigger than DDG. You're vanishing into the very same HN comment mental masturbation that was mentioned.
>" That was one of the most HN comments I've ever read."
You know, that's just a gratuitous swipe. So was this in the blog post: "First, DuckDuckGo didn’t start as a nerd attempt to find the ultimate algorithm."
It really looks like you want to put down nerds to make yourself feel more sophisticated or popular. This isn't high school, though, and you're pissing in the same pool you're posting in. Turning "HN" into an adjective with a pejorative tone isn't going to win you any friends on HN, especially as a new poster. It's just mean spirited.
But doing that doesn't necessarily involves calculating a precise measurement of the ratios present in the metaphor. Human language conveys lots of meaning by evocative imaginery, too, so not all expressions should be taken literally.
Sometimes, taking the literal meaning might even distort the original message, like in this case changing the mosquito with a goose - if that had been the original expression, everyone would have wondered why the author chose such strange comparison.
The calculations were necessary to determine and illustrate how inaccurate and distorting the original version is. The size of the animals is being used to show sizes of companies, since everyone is generally familiar with the sizes of mosquitoes, geese and elephants. But the difference between a mosquito and a goose is drastic… handily, someone else already did the calculations to show how drastic. So, it is a bad original metaphor and it is misleading.
I was going to suggest that a good author would then to choose an animal about the size of the goose that has the same characteristics as a mosquito, but why? A mosquito is considered annoying, harmful and parasitic – if it was the size of the goose, things to be very different. It would be considered a terrifying predator of the jungle. So I don't see what qualities of a mosquito would make it an appropriate choice, and this is what makes the metaphor worth correcting. It is somewhat of a slur against DDG, more so since the numbers are so dramatically inaccurate.
But if there ever a place to be correct in what you say both factually and mathematically and then not be ridiculed for it, it's here. I support the parent argument.
Perhaps rather than thinking in terms of material mass, he was thinking in terms of subjective conceptual 'weight' of each in the minds of his audience. Mosquitos are annoying and can be quite dangerous due to carrying parasites and pathogens, so the 'weight' we give them in the messy associative matrix of our brains is far larger than their raw mass. In other words: maybe he didn't mean it precisely.
Or to put it in Edward Tuft's "Lie Factor" terms using a 5000 lbs elephant and 5mg mosquito I get a lie factor of 454046 (if someone can check out the math).
Great analysis. Still I see two problems. First, the company name Google comes from "Googol" and so it cannot be overestimated and second, why are you mixing lbs and kilogram in your analysis!? And using lbs is in general uglier than the wrong metaphor ;)
I find it funny that engineers ( on HN ? ) put on their I'm-a-rigid-logic-thinking-only-engineer cap when it suits them, and their I'm-an-imaginative-open-eyed-creative-rebel cap when it suits them as well. Traits of the first cap include: can't think outside the exact parameters given them, can't 'fill in the blanks' or inference laterally, take words and numbers to be exact limiting things. Traits of the second cap are basically the opposite of these, we can fill in the blanks, and so on.
I don't notice people wearing both hats at the same time much.
Of course, the OP could be satire. When you can't tell the difference between satire and your platform, then ...
Incidentally, if we look sideways a bit the metaphor fits like: Google isn't just linearly bigger, it has nonlinear increases in advantages because of its size, clout, network effects, and so on. So the order of magnitude discrepancy is perhaps justified by attempting to account for these affects. Conversely, it could be suggesting that smaller disruptive and startups ought to spread their influence virally, parasitically, or through insect-like nimbleness and hatching-of-the-1000-eggs reproduction, instead of relying on slower, more "mammalian" reproduction to propagate their influence. Who knows? Hard to say which interpretation is correct, when words are ambiguous and one leaves it to the imagination. You can't even rely on Occam's razor to decide, since the shortest path between two thoughts differs depending on the mind. But maybe that's not a bad thing. Speech & language is nothing more than successful miscommunication. You can't police what you don't understand.
Maybe he didn't mean the metaphor was the problem, but the sentence following it (about making things look better for ddg)? Depending on interpretation, that could be undue discredit to ddg...
That would make this a bit of an ironic comment, no? :p
If you'd like to go by searches rather than visits:
At DuckDuckGo we are averaging roughly 17 million searches a day right now (duckduckgo.com/traffic.html)
Google says they did 2 trillion searches last year[1], and while usually their search metrics include Youtube, Google Maps and Gmail queries let's assume it was all searches. That'd be roughly 5.5 billion searches a day.
So now we're talking ~1/322 or just shy of 40lbs. That's roughly the size of a Hamadryas baboon or about 22 Mallard ducks - not a bad little flock! :)
Disclaimer: DDG staff. Opinions are my own. No animals were harmed in the poor construction of this analogy.
Since when search metrics include YT, Maps and... Gmail?!? Unless you're talking about universal search results, I've never seen the numbers for all those products conflated.
If you look at the most recently public posted qSearch report on Feb 2016[2] you'll notice it talks about "Explicit Core Search" and "Powered By Reporting."
Powered By Reporting is listed as any search that is handled by Google or Bing. "Google’s “powered by” share is composed of searches conducted at "Google entities", as well as searches on AOL and Ask’s MyWebSearch "[3]
In Comscores qSearch product that you can subscribe to, when you hit "Google Sites" it provides a drop down detailing the breakdown of all the properties. Search is hard to define when companies get to that size, and searches on other Google properties are indeed 'searches' but they are also different than searches on DuckDuckGo.
Those are third-party numbers. The "trillions" statement came from Google, where nobody conflates searches on Gmail with those on Web search. Your original message implied that Google inflates search numbers; I'll only say that that's a major assumption to make. (I used to know the exact numbers.)
I know you kind of overdid it a bit, but that's exactly the kind of stuff that distracts me for an hour, because I have to do the same math you did. Nicely done and good to see that DuckDuckGo is already a hyena, rather than a mosquito.
DuckDuckGo is OK but they seem to be penalising single-page apps so my project websites rank low on DuckDuckGo but high on Google. So I can't support DuckDuckGo.
As I said, my websites are doing well on Google that's all the SEO I care about and I'm not willing to sacrifice user experience. It shouldn't be difficult to index web apps these days... There are so many open source headless browsing tools/libraries available.
That seems to be pretty one-sided view on decisions one makes out of trade-offs. Having iPhone 4 few weeks ago with js obviously off and now having SE with js:on, I'm not sure whether my web experience got any better or not at all (overall phone experience got better, no doubt).
Using Google to analyze its own competitor seems a bit questionable. Though with the difference in usage being what it is, I doubt Google would bother with any funny business.
Isn't DDG basically Bing + bangs? Doesn't seem fair to credit it at beating Google without highlighting the billions of dollars that MSFT has invested.
I mean, a probably more relevant summary of what DDG offers would be: Bing/Yahoo/Yandex mashup stripped out of personalized results + bangs + [until recently] community maintained instant answers.
Edit: Worth pointing out: as far as I am aware, they're using all the other search engines with an agreement with those companies, which makes it completely okay in my book to credit them as the one.
I credit my operating system to Canonical, even though the heavy lifting and behaving nicely on hardware is done by people contributing to Linux. Linux out of the box doesn't mean a thing to me. Bing out of the box doesn't mean a thing to me. DDG and Ubuntu do mean a lot to me.
> I credit my operating system to Canonical, even though the heavy lifting and behaving nicely on hardware is done by people contributing to Linux. Linux out of the box doesn't mean a thing to me. Bing out of the box doesn't mean a thing to me. DDG and Ubuntu do mean a lot to me.
That's a dangerous comparison to make because there isn't really any consensus* on which part of the stack to name the OS. For example, Linux is technically just the kernel. So most of the UX you deal with will be GNU and other user space tools. Depending on the DE you use, you might not even deal with much of Canonical's code. And Ubuntu does owe an awful lot to it's parent distribution, Debian.
In short, I get the point you're attempting to make but unfortunately your comparison is more contested than the original subject you were discussing.
Canonical did a good thing of mashing it all together into a single product. DDG does the same for search in my book.
They've both outsourced the heavy lifting to some third party.
In DDG, Bing/Yahoo/Yandex does all of the heavy lifting. In Ubuntu, it's the Linux kernel. They would be nothing without those products they depend on. But they bundle that with like hundreds of different, smaller in scale products to offer a nice experience out of the box.
For smaller in scale companies that are trying to compete with big players (like Microsoft and Google), this does seem like a winning combination in my book.
I'd say bangs are the whole product. When you fail to use bangs, it presents, as a plus, medium decent results. (Good for stuff like error messages on stack overflow and forums, not so good for vaguer queries).
DuckDuckGo uses multiple sources for our results, as well as our instant answers and bangs.
But the most important feature is that DuckDuckGo is the search engine that doesn't track you. We don't store your personal information and we don't sell you out to advertisers.
Our goal is to raise the standard of trust online, and so at DuckDuckGo you are not the product, search is. We think that's an important difference.
Still advertisers. It's a nuanced but important difference.
Case A - if you type in "Car" we show an ad about cars. Advertisers are paying to appear at the top of that page. They do not know who you are or what you previously searched.
Case B - if you type in "Car" you are shown an ad about cars. Advertisers are paying to target based on your interest in cars, and possibly other websites you've visited, online actions you've taken, re-marketing pixels from third-party websites, or your affinity to other demographic groups.
Most of those sources are search engines of some web pages, answer pages, cheat sheets and help documents. They even list tiny unit conversation services there. Give me a break.
> https://duck.co/help/results/sources
"We also of course have more traditional links in the search results, which we source from Bing, Yahoo, and Yandex."
I thought they where re-implementing search.. Are they "just" aggregating?
I recently spoke with Gabriel and his view was that they're going to continue to outsource / proxy / aggregate unless they need to implement their own search. Right now it's not worth the investment apparently.
It would make it hard to pretend that the DDG results were much better than those irrelevant Google results that more fanatical Googlephobes insist they get.
The thing with Duckduckgo is the choice has to be ideological or you will do a search again on Google again just to ensure you are not missing out.
This is Google marketing and brand perception at work because Google results of late, 3 years, have been unimpressive and you have to sift through pages of useless links and content to find any relevant information beyond the usual suspects one already knows, so their intensive spyware operations doesn't seem to help search quality.
It's surprising there are not more experimental search projects. One would have expected a steady stream of regular attempts but not a single credible effort exists. There was once an alternative search project called Cuil that just seemed to fizzle off.
I have to say, I go to the second page of Google search results maybe once every couple months. I'm not sure what you are searching for but that has not been my experience at all. It seems to always be getting better.
Sometimes I think there should be a search engine that has a button "these results are not satisfactory". When you click it, you can describe in detail what you were looking for... and hordes of cheap minimum wage or outsourced workers work 24/7 on these reports. Culling spammy search results, boosting good links, etc.. Solve what Google was always claiming to do purely algorithmically, with additional brute force labour.
(Actually, I suspect that a major ingredient of Google and Facebooks "algorithms" are really human reviewers... they want to keep this fact secret since human reviewers are responsible for judgement, whereas algorithms still count as impartial. This is not true of course, but it is the perception.)
Write a whole article about DuckDuckGo, and don't mention bang searches. !py searches the Python documentation, !w searches Wikipedia, !g searches Google, !a searches Amazon, and better yet, !gde, !wde, and !ade search the German Google, Wikipedia, and Amazon.
For natural language queries, I use !g, because Google is better at this sort of thing. For local queries I use !gde, because Google has better map integration. For most everything else, I prefer DDG, because it doesn't "did you mean DickDickGo?", and it doesn't "I don't know what you're looking for, but here are five ads instead".
DuckDuckGo has really, really improved on their results for me who do not live in USA. Before, it was pretty much unusable. But today I use DDG as my main search engine on all my devices.
People are conditioned to think Search has to work online.
This was true when Google was created.
No one had the processing or memory available on their desktop to search an entire index of the "useful" web.
Not anymore.
How large is a "useful" index of the web today? And can it fit on your laptop? The answer is yes.
Can the entire thing be queried fast? The answer is yes.
As an example take the entire stackexchange and wikipedia dumps in their entirety(including images). Compressed it comes to 50-60 GB range. Think about that number. That's an rough approximation of all known human knowledge.
It's not growing too fast. It has stabilized. To query the content you need an index.
So how large is an index to a 100 GB file? Generally around 1 GB. Let's say you use covering indexes with lot of meta data and up that to 5GB to support sophisticated queries.
With today's average hardware you can search the entire thing in milliseconds.
So why aren't we building better local search?
Because everyone is conditioned to believe, thanks to Google's success, we need to do it online. Which means baking in the problem of handling millions of queries a second into the Search problem. Guess what? This is not a problem that local search has.
Every time a chimp or a duck needs to build a protein in it's cell it doesn't query the DNA index stored in the cloud. Instead every cell has the index. Every cell has the processing power to query that index in the nanosecond time scale.
The cloud based search story is temporary.
If you want to index every reference to Taylor Swifts ass that every teenager in Norway, Ecuador and Cambodia are making, then yes you need a Google size index. But for useful human knowledge we are getting to the point where we don't need Google scale.
If you don't believe me look at what is possible TODAY in Dash/Zeal docset search for offline developer documentation search or Kiwix or with Mathematica.
Is there a way to easily sync search repositories, like Stackoverflow, Wikipedia etc., to your local computer with an automatically built search index?
Stabilised != final though, so you'll still need a layer in order to find all of the new knowledge and compare it against what everyone has locally (not going to download 50GB every time someone posts a new useful SO answer), and update the local store. Why not keep that index centrally for everyone instead of replicating it for 2bn people?
You vastly underestimate the amount of knowledge in random blogs and small websites. If you are only after the "mainstream" sites, sure, but that's quite the bubble. I want and need access to all the blogs, forums, wikis.
Why not download an index of the 100,000 or 1,000,000 sites which appear most often weighted by position? If they don't reach a certain threshold of quality (maybe the query is too obscure) then resend the query online?
I don't have a CS background so I'm not sure, is this a reasonable solution?
The obscure sites are what make the core of the internet, at least in my circles.
But having a way of making sure that a certain topic or genre of sites is well indexed locally would work. I.e. "my" top 100k. Oh, how awesome would that be.
Sometimes, it feels Google is increasingly useless at finding those niche websites because all the noise has time to do search engine """optimization""".
I’m glad I’m not the only one to think that. I wonder what a good way to measure that signal to noise ratio is?
This feels like an application of the 80/20 rule. I might not always find what I need in just that offline index, but the times I do would seriously disrupt google.
Oh, I wish there were a way to rate the results in a opposite way. Maybe it's just me, but sometimes I want even mark some site as 'a hidden gem'. Usually it's something rather niche though, like a bunch of great little articles about anti-aliasing filters design and sampling, full of engineering wisdom from the decades of experience. So I think it would be perfect to have a way to tag it by some topic.
Google is pretty terrible at finding that stuff (as well as DuckDuckGo.) Instead, the first two or three pages of results are the SEO kings, often with identical copy. https://millionshort.com is a search engine that is entirely a reaction to that.
So, practically speaking, how would one go about setting this up for local use? I would this would also work very well for locally stored books and docs.
The reason sites like Google and YouTube are good is not because you can search the larger sites well, but because they can search the long tail of the web well. Any search engine can index the top 5-10k sites and build something workable for that. The money maker is returning lesser known sites because, despite not being popular, they contain the best answer for the search query. YouTube got popular not because it has the most popular videos on the web (all video sharing websites have the popular videos) but because it contained and stored an insane amount of videos with like 100 views and would return them if you searched for something oddly specific.
I don't think search has much to do with YouTube's success. Just having all those long-tail videos is the main thing, and it has those videos because it's popular.
I've heard that a key early advantage of YouTube was that uploaded videos appeared immediately, not after a long processing delay. That helped it become popular, and once it was popular, it stayed popular and crushed the competition due to the network effect.
One aspect of this is trying to find content that has already been viewed. There are tools to archive web browsing locally for future re-use. Any improvements to local search would benefit this use case tremendously.
How would you keep the index up to date? How quickly would news stories appear in it and stale links disappear? Are you going to read the entire internet every day to keep it current?
maybe you could pull down changes on a daily basis? not a complete reindex but just updates that prune dead links and add new ones.
the problem is this would be completely useless for current events. but querying sites you use regularly, this could be an option. I often end a google search with modifiers like "wikipedia" or "reddit" or "stackoverflow". If I could store indexes to those sites offline that might be useful.
The creator of this offline search could potentially make money by indexing shopping sites like Amazon, eBay, etc with affiliate links.
Or I could let someone do it for me; say Google or DDG. I don’t have time for such nonsense. I’m not going to waste time hosting my own version of Apple Music, when I could just use Apple Music.
> So how large is an index to a 100 GB file? Generally around 1 GB. Let's say you use covering indexes with lot of meta data and up that to 5GB to support sophisticated queries.
You vastly underestimate index sizes. I'm not saying it wouldn't be realistic to download and index locally but they'll be much larger than 1-5%.
I don't think search is as simple as grepping a ton of files. Search Google for "movie where people can't have babies" and the #1 result is Children of Men. I think that's one area where Google pulls ahead of its competition is accurate results to vague queries like that and I think you need a lot of collected data from searches to provide that.
For the record anyone who hasn't seen Children of Men, do yourself favour and find it. One of the greatest and most underrated films of the last decade.
Edit: Oops turns out its 11 years old. Better make that the last two decades then. I'm gettin old
Problem is that knowledge gets outdated quickly as things change, I usually have to specify 'only show results from the last month', else the information is not relevant. Having to constantly download an index would be demanding, and to be honest why bother? It's not like google is slow.
Your idea doesn't go far enough. What good are a couple of stale indexes of Wikipedia and Stackoverflow going to be?
Instead, I am wondering why there isnt a federated open source search engine. It would be a cloud of nodes, each node spidering and indexing a small hash bucket of urls. With a million such nodes we could have a live updated, distributed search engine to replace Google. We could run millions of queries without paying someone - we'd pay back by serving as a node, just like in BitTorrent. With all the interest in privacy, I wanted to see more discussion of replacing Google with an open, non-censured and private protocol for search.
Who would host the index? You can't contact one million peers to run a search, so fanout-on-read doesn't work. If the index is also distributed (by search term?), then the search+indexing nodes will need to fanout-on-write, which is less latency-sensitive but still pretty onerous.
In the same vein, what good is an index of the web going to be? Googles value isn't their index, its how they translate your search terms into good results from that index. Anyone can index the web, not everyone can make sense of it from human questions.
Laptops generally come in 128GB or 256GB capacities. So we're talking roughly 20-50% of an average laptop's storage just to hold the search index for wikipedia alone.
Meaning it is not feasible to store a useful index of the web today on a laptop unless you want to dedicate that laptop to pretty much exclusively searching wikipedia's english content.
You're not searching that database in milliseconds on average laptop hardware, either, but you probably could get it to be "fast enough" for practical usages if you could somehow solve the harder parts like storage size, freshness, and indexing more than just wikipedia's english content.
That’s also assuming Wikipedia is worth having an offline index for. Wikipedia, for Bonn controversial, general knowledge has its value, but something gets lost with 1000 chefs in the kitchen. The beauty of the internet is that it isn’t Encarta.
So when there’s a new blog post on a topic of interest, how does that work? The internet isn’t an encyclopedia of a finite size.
Offline search is the equivalent of the Internet Yellow Pages from back in the day. New, relevant information is being added continuously. The search index 10 minutes ago is different than it is right now.
If you struggle to get any of your indexing working, let me know and I can Google it, whilst you grep through your stale Stack Overflow archive.
Your reply seems pointless, yes I could download an entire copy of Wikipedia, compress it and index it - why would I want to? Are you going to do this on every machine you own? What about the updates Wikipedia receives every minute?
You should start a company though to focus on this, maybe you could call it Encarta or something?
Yes, we can build a local encyclopedia. But from a 'Green' viewpoint, which is more environmental-friendly: more Terabyte hard-drives sold? or a few MB of data transferring over the internet?
Does anyone remember how back in the day, we had Lycos, Hotbot, Yahoo, Dogpile .. a whole host of search engines. You'd go to two or three to get an idea of what was out there. People had different indexes and there were a lot of players.
Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG.
The cost of entry to the search market is exceedingly high right now. This is a pretty good article of detailing how one person was able to come up with an idea and challenge the behemoth in very niche areas (privacy/the nsa leaks were probably the reason I started looking at/using it around 2013).
Yet I still miss the days of using multiple search engines; seeing a variety of results. I hate the de factor standard of Google. When a company controls that much of search, they get to define the narrative. They literally shape the way many people perceive the world.
I wonder if tech will get to the point where indexing will be easier and we'll see more solutions that are cheaper and that can crawl larger datasets with lower processing requirements. Maybe the next step will be distributed search with shared indexes?
In any case, Google can't remain on top forever (at least I hope not). It'd be nice to see more tech in this space, but it's an incredibly difficult problem. There is reason Google climbed to the top like it did.
I really like the localized index idea someone posted earlier. If I could get 60% of my searches answered locally for the cost of 100 gigs of space I would use it. Even more so if I could host it on a local server somehow.
While it would be nice if the barrier to entry was lower to enter the search space, I feel like if it's easier to get in Google will just be able to do more with what they have.
Storage, computation and bandwidth costs have never been cheaper. The problem is that Google has an extremely high market share which makes it really difficult to get users.
Google climbed to the top because they focused on what matters: the right results, quickly, no fuss. Their competitors gave you unrelated spam for most queries and were slow and focusing on useless things.
Googles results are getting worse, there's more fuss than ever and it's getting slower all the time. So there definitely is an opportunity (which DuckDuckGo is taking).
> Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG.
If DDG folks are reading this, here is an idea: Have a browser extension (so opt-in). If a user then does a !g search, forward the search terms and the search results to DDG and add them to your index. (I wouldn't go as far as Microsoft, who IIRC actually copied the results and the ranking for individual terms and added them to Bing.)
I see DuckDuckGo as an improved search bar more than as a search engine, though it's admirably fast at the latter.
DDG's bangs are effectively a CLI interface to web searches. !g does google, !yt does YouTube, !w does Wikipedia, !wa does Wolfram Alpha, !scholar does Google Scholar...
It's interesting, since browsers have had custom keyword searches for ages, and I've known that and just never taken the time to set them up.
Having DDG with its bangs I use them all the time, and if there's some specific new site that I want to search there's (so far) always been a fairly guessable bang already set up for it.
I really, really want to like DuckDuckGo. But every single time I’ve switched from Google, I’ve found them to be much worse at returning the best results for what I’m looking for.
For example, let’s say I want to look up “ear infection”. Google will spit out a bunch of info right on their results page, often saving me the trouble of even going to another site. DDG however will just give me 10 webmd links.
If they want to compete, the privacy angle isn’t enough. They need comparable functionality as well.
Try startpage.com. I moved away from google a year ago and never looked back. I've never once needed to fall back to google. That's probably because startpage.com is basically anonymized google.
I know we're supposed to root for the entrepreneurs here on HN, but DDG honestly seems like more hype / marketing than a decent product. You never hear anything about startpage.com (maybe because it has a boring name?) but it just works.
Thanks for the suggestion. I just gave them a try with my "ear infection" search, and found that while they provide similar links, they don't give the additional info (like the summary of what it is and the "People also ask" dropdown box) right on the results page that google search does. This is especially important to me on mobile, especially when I have a weak cell signal.
Do you know how they monetize? I know it's a bit of a cliche asking that (you're the product etc.) but I'm really curious - using the Google API for search isn't cheap at scale.
If they are using a video of Snowden, they may have some philanthropic money - or it may be something nefarious. I'd love to give them the benefit of doubt though.
Insofar as DDG is a mashup of many search engines, doesn't it technically have a massive team? If you add up the teams building the products that feed DDG, then you probably end up with a much larger team than the one building Google Search.
I might describe that as gaining on Google, but still losing quite badly.
I'm still impressed by DDG of course and welcome some alternative to Google, which is a dangerous monopoly with a search that I've come to despise in recent years due to its ignoring what I've actually typed.
But how good can a search engine be if it does not track its humans, and therefore cannot use browsing / search history as contextual clues for query ranking / semantics ?
It might be very good. But can such a system compete with one that does use search history as context?
Personally, I value my privacy more than the small amount of time tweaking my search query to get the results I actually wanted vs. the out-of-context results I got. I think anyone using DuckDuckGo is of the same opinion. When you think about it: total loss of internet privacy for some slightly better search results? Pretty massive trade off of with some serious consequences.
However, I do get your point. Without that extra context, the search results can only ever only be so good.
That solely depends on your definition of "good".
These comments have plenty of examples where DDG is "better" in terms of usability compared to Google, which is also an important factor besides search accuracy.
Imho using the search history as context can also lead to undesirable results, so can funneling people to overly popular results which still can be wrong, had both happen more than enough with Google.
It's a trade-off between spending time arguing with Google if I really meant to search for that word, regardless of " usage, and having to look at more results in DDG until I find the one that I was looking for.
There used to be a concept of google-fu, where writing search queries was a skill in itself. I got quite good at finding what's needed. Then, over time, search results became less and less useful. I noticed that search results seemed to be weighed more on my history than the query written. I'd get results with words synonymous to web page titles I'd recently visited, but otherwise no connection to my search. To get decent results now I have to use incognito or change the search settings to be "verbatim." I hoped things would get better, but years later things are the same.
DDG gives more predictable results. It reminds me of why I started using Google over other search engines decades ago.
I've been using DDG as my primary search for about 2 years now, it rarely ever falls short for me. When it does, it's usually related to some type of image search I'm doing, I think google still has nicer tools for that.
That's a strange assumption. I use my VPN provider's DNS servers and I imagine many of the relatively tech savvy users who hang out on HN and care about privacy enough to use alternative search engines know how DNS works.
557 comments
[ 2.4 ms ] story [ 149 ms ] threadSome sites popup dialogs and even if you delete all the elements they have set some of the elements in the original page to not allow scroll. With reading mode you just bypass all that crap.
Though it's possible to do a hell of a lot of site fixing in Stylish via CSS edits.
In Chrome, you can right click and select "Inspect" and then remove the relevant content from the display. :)
Other browsers have "reading mode" which tends to work well for this type of thing.
Several times I've come across an oddly formatted page and worried it'd lose styling required for the correct interpretation but not yet!
javascript:%20window.location%20=%20"http://outline.com/"%20+%20window.location
DuckDuckGo, on the other hand, is very very much worth your time, if you aren't familiar with it: https://duckduckgo.com
If anyone has doubts because they tried it years ago, I'd say go for it again.
And since I regularly browse via a Digital Ocean VM, the lack of CAPTCHAs on DuckDuckGo is refreshing as well.
If you don't mind my asking, why? And by which method do you usually accomplish this?
Edit: Thanks for explaining the CAPTCHA issue.
How?
ssh -D 8080 digital-ocean-vm-here.tld
Now port 8080 on your local machine is a socks proxy for your vm.
Google is really tor/proxy/anonymous user unfriendly. Requiring users to solve as many as four or five CAPTCHAs (seriously fuck you google).
They're like this for a reason. They didn't implement complex detection of proxies to annoy users. Just to keep everyone out who's not supposed to use their search.
But it's actually really nice to be able to pull a result or two from ddg by crawling, or to be able to use a VPN without having to solve a zillion captcha.
The only one hurt are the users.
Have you ever tried using Google's search over a VPN / Tor / public proxy? They're even worse than Cloudflare was with their CAPTCHAs).
> Why?
Probably to avoid fuckery by the local ISP and/or government.
> And how?
I'm assuming he meant OpenVPN or something, but I might be wrong.
Do you mean using a DO VM as a VPN, or via X11 forwarding to your desktop over SSH, or actually browsing on the DO VM's desktop using VNC? I've done all three in the past as experiments in private browsing, and the latter is too laggy to be comfortable.
So I figured if I had a choice of two search engines, where I get satisfying results in both of them, and one of them doesn't track me, why go for the one that tracks me?
I'd gladly do the same switch when it comes to Gmail, but I really really like Gmail's web interface, haven't gotten over that hurdle yet.
EDIT: I also switched from Google Chrome for pretty much the same reasons.
I really like the idea of Brave, but not having extension support is a major hurdle which I hope they get over.
I'm surprised there isn't a straightforward and user-friendly Google-less Chromium browser available.
Plus, Google doesn't get to see my mail any more. Ditto for Firefox vs Chrome.
Another issue I had was that my main email address was already in Google and Microsoft's systems for a couple reasons, and it wouldn't let me set it properly when changing my email address everywhere. So I have google@ and microsoft@ aliases just to work around their account management quirks.
I have multiple domains and aliases, and Fastmail is much better at those than Google Apps ever was (I also have the free plan but you can never change your initial domain), so I'm much happier.
A few for friends and family as well. --Friends I can tell to pay for themselves if I need to, but family, not so much.
Edit: As for checking them, most don't get many emails, and every good email app can handle multiple accounts easily, so I get notifications on my phone.
https://news.ycombinator.com/item?id=15029186
Really like Fastmail.
Do they have a spam filter? If yes, how do they filter emails without accessing it?
I also agree that Fastmail is great.
I could never get over the way using GMail felt like fighting with a sluggish toy version of email compared to desktop outlook (and I'm not really a fan of outlook, either). I just assumed that GMail's interface was the best a webapp could offer: it was the best I'd seen so far and had a no-longer-deserved "who could beat gmail at webmail?" bug in my mind.
I'm kinda surprised fastmail hasn't made larger inroads among the kind of techies that will occasionally bemoan giving their lives over to google.
Hard to avoid them in that case.
GOD YES! This gets incredibly annoying when searching for version-specific information, considering the differences there can be between version foo and version bar of $DISTRO. Nothing like searching for something specific to CentOS 7 and getting zillions of CentOS 6 and 5 hits.
Google will also helpfully ignore the "Verbatim" option, and God help you if you're trying to search for multiple specific phrases.
Google is looking more and more like Alta Vista in its waning days.
The problem is, whenever you can't find something in DDG, you assume it's because it's DDG, and search again with !g. Meaning I end up requerying 40% or so of my searches using !g (prefixing your query with !g redirects your query from DDG to encrypted google)
Here's some example queries I ended up !g yesterday that had much better results in google than DDG:
reduce kendo javascript file size - For me, the useful result was number 2 in google. In DDG it was number 11 ("Only What You Need | Kendo UI Getting Started").
ptr overwatch - PTR is the test patch of the game overwatch. It's regularly changed. The correct result "Overwatch PTR Now Available - August 29, 2017" is number 1 in google, in DDG the blog post August 12th is not available (well, correct from my perspective)
I tend to find anything speculative DDG is fairly terrible at. For example yesterday I was googling about trying to identify the source of some weird animation CPU cycles I was seeing in Chrome Performance Profiler, I didn't really get to the bottom of it, but I ended up ditching DDG and doing all the queries in google because I would otherwise have ended up searching everything twice.
I'm also getting quite frustrated with the "instant answer" functionality, it's generally terrible. One of the most annoying ones is the SO instant answer that just utterly sucks, you can't see the code, it usually cuts anything useful in half, and takes up a HUGE amount of space meaning on a laptop you've got to scroll to start seeing the results. I just want to be able to turn it off, but because they don't track you they don't offer that functionality.
Also the maps one is really bad. I almost always want directions, but clicking the map takes me to a really nonfunctional, bare-bones map that doesn't have directions and then I have to click another button to actually get the directions. There's a drop down where you can choose your map type, but it definitely doesn't work as expected, I just want it to always embed a google maps instead of whatever they're doing.
Basically UX ain't DDG's strong point.
Ironically, if you DDG: "turn off certain instant answers duckduckgo" it comes up with terrible search results and no answer. If you "!g turn off certain instant answers duckduckgo", the top result is at least relevant.
I also find the entire bang thing to be a gimmick. Who wants to use !r when !g with "reddit" at the end will always get you better search results, reddit's own search is abysmal (as is !so).
EDIT: I'm being overly negative, after all, I haven't actually switched back, like I did last time I tried to use DDG. So it's definitely worth a go, but I don't think it's ready for your Mum to use. Also, I have Bing on my phone's Chrome to avoid AMP, and it's actually quite good.
And I quite like using bangs, e.g. !w for Wikipedia or !i for images, rather than having to go to the initial search results on the search engine and then clicking again.
Finally, I like that on DDG, you can just arrow up/down through the search results, and then open one with enter (or cmd-enter to open it in a background tab), without reverting to the mouse.
You know you're onto something when the #1 documentation link is a key binding cheatsheet!
(And those on older versions can use VimFx until 57 is released.)
I don't ever find what I want a few times most days and need multiple sources/queries quite frequently.
That means more privacy because uses startpage.com
You can also use google on Firefox for Android. No AMP there.
Main reason: concerns about tracking and privacy.
Secondarily: politics.
DDG so far has been acceptable. As long as they keep their political opinions to themselves, the honeymoon will continue. My love affair with Google, on the other hand, is over. :(
No advertising, strong privacy policy, based in Australia (a country with a decent privacy track record that we know of).
It's $3/month which is a decent price for liberating a lot of data from advertisers.
The spam filter is playing catch-up to Google so that's a bit of a shock at first, but you can train it up well or use second layer measures like Sanebox.
Disclaimer: DDG staff - but personal opinion.
When DDG shows an ad, is there no tracking involved in that? For example, if I were to click on an ad (after turning my ad blocker off), does the destination site not get any information about me or specifically what I searched for?
For people afraid of getting off google, you can always search something like '!g my-search', it works the same for youtube(!yt), google image(!gi), or even hackernews(!hn)
And of course the best bang is the 'I am feeling lucky' one (!), i.e.: 'hackernews !'
You are linking to a ~5 year old thread that is discussing the state of things well before google switched everything over to ssl.
I can only see this as a good thing. If I want local search results, I'll add local qualifiers like "USA", "Texas", "Houston"
Usually if I need a certain engine I just guess the bang and it's usually supported and correct. !gm for google maps, !tineye for tineye, !wayback for the wayback machine...
Setting DuckDuckGo as the search engine for your browser's address bar means all these bangs work right there whenever you open a new tab.
!ddg google
Or if you are feeling lucky:
!!ddg google
:)
edit: typo
If you're on Google as your default search engine, it's not so convenient to see other engine's results (plus you're being tracked all the time).
Sounds like they have changed a lot since then, so I'll probably give it another shot.
Now that I've learned that !a works for amazon.com and duckduckgo.com gets affiliate revenue that way, then that's how I'll search amazon from now on. Need to boost up the underdog that cares about privacy, because google surely doesn't.
'<Song name> !' or '<song name> youtube !' takes you right to it.
I'd love a feature that opens the first result on a search engine like wikipedia does by default. Biggest use case for me would be imdb but there would be others.
DuckDuckGo has that by using a backslash, a space and a search term (\ bed intruder youtube) or an exclamation a space and the search term (although I believe the former is the official way now). I use it all the time, it's like a superpower.
Now if I type "h my-search" into the omnibar, it goes to google.com/search?q=site%3Anews.ycombinator.com+my-search which gives me only Google search results from HN.
This is using google's algorithm (not the search bar built into whichever website like DDG) which is still the best, especially if you constrain it to one domain name. I also don't have to type the "!"
The prefix messes up google's text prediction. I guess no one from the chromium team is using this feature, since it would be trivial to fix.
w https://www.google.com/search?&q=site%3Awikipedia.org+%s&btn... first result from wikipedia
h https://www.google.ca/search?q=site%3Anews.ycombinator.com+%...
r https://www.google.ca/search?q=site%3Areddit.com+%s reddit
y https://www.youtube.com/results?search_query=%s youtube
m https://www.google.ca/maps/search/%s google maps
b https://builtwith.com/?q=%s add "b " to a url to find out what tech is being used
wo https://www.wolframalpha.com/input/?i=%s wolfram alpha
i https://www.google.com/search?tbm=isch&q=%s google image search
g https://github.com/%s "g upspin/upspin" to jump to https://github.com/upspin/upspin.git
br https://github.com/Homebrew/homebrew-core/blob/master/Formul... read homebrew formula
c https://caniuse.com/#search=%s
(Title is google, but works for ddg amongst other things also)
We do have some of those tools, at DuckDuckGo they are called "Instant Answers."
For example the query: "2 lb to kg" will bring you to this results page https://duckduckgo.com/?q=2+lb+to+kg&ia=answer
You can see all our Instant Answers here: https://duck.co/ia
If something isn't triggering for you - let us know with the feedback tool on the site!
Disclaimer: DDG staff.
Get that 'in last year' filter already, DDG. I think I've complained about this through feedback before.
Also the autocorrect really throws my search terms off while Google does is right most of the time.
I especially like how programming questions usually provide a top result from Stack Overflow, a lot of times I don't even need to click the link to see the answer I'm looking for.
The privacy aspect was the driver for me, but it wasn't enough to make the switch all this time. The privacy aspect coupled with an easy way to get google results when I need them is. While I've known they had the bang queries for a long time, it was actually a youtube search result that finally made me shift. I forget what it was, but I was searching for something for my kids and it looked like my activity had polluted the results. That has been a problem for quite some time and this particular instance was enough for me to make the change. It was a harmless, but I really don't like the idea that my activity will potentially bleed into that of my family, and frankly I get irritated when their activity bleeds into mine. DuckDuckGo doesn't replace youtube search, but it was more of the general principal. I tried working with different accounts over the years, but it's not easy to switch on all platforms and if you've ever tried entering a 60 character password on a playstation, you know why that's a non-starter.
I also tried it a few years ago but was disappointed by the speed - at a time when the google search responses came back "instantly", the latency of DDG was noticeable and not good-enough.
Fast forward to 2017 and that appears to have been solved and the results are as fast as google are from what my brain/eyes can tell anyway - I would not be surprised to find out that google was faster if you timed it. In the past 6+ months I've been using DDG, I've only ever found the need to switch back to google once (and that was image search)
Is there anything more recent?
[1] http://highscalability.com/blog/2013/1/28/duckduckgo-archite...
I started using alternatives, for privacy, but also because they are often better. Google Maps can not even find near pharmacy in EU country.
edit: right corner
If Putin himself showed up to Yandex HQ and demanded a full data dump of every search you made, he is still incapable of touching you, whereas J. Random Agent at the FBI can throw you in a 6x8 steel box.
Some say the DDG bangs are a solution. What do I win by doing that? It only made me resort to !g all the time, because the results were so bad.
Now I use https://www.startpage.com/ with region set to Swedish. It's practically a proxy for Google search, so it gives me the right results but sans the filter bubble experience (yes, I want the regional bubble). If you're a non US user, I can recommend it.
Indeed, due to the bangs, I know use DDG as my proxy to a lot of other search engines, like Wikipedia (!w/!wda/!w..) and Wiktionary (!wikt), and even obscure ones like Memory Alpha (!memoryalpha). Very handy indeed.
I rarely use region specific DDG, but when I do, I find it ok. With DDG I can manually enable/disable and switch the region, while Google guesses based on my IP - very annoying, given that I frequently use VPNs. (Of course, if you're logged in to Google, you can specify region/language/etc., but if you don't want to be tracked and delete their cookies or use the browser in private/porn/incognito mode, then it'll just assume that you are looking for Dutch things in the Netherlands, because that's where the VPN exit is...)
As for bangs, I love skipping clicks. Most of the time I know what I want and can use bangs accordingly. I'm guessing my most common tags are !a, !yt, !gh, !wiki, !wolf and then some game(s) specific bangs.
BTW, for wikipedia, !w is enough. Wolfram Alpha is !wa.
StartPage doesn't seem to be helpful at all. It doesn't let me select language and region separately. I need to select English UK and then it sets my region as UK. Also, it's very slow (at least for me)
Why would this guy go out of his way to say this metaphor is actually literally informative and generous towards DuckDuckGo while apparently having exerted no mental effort and being incorrect by orders of magnitude on his own data?
If you're most generous to the writer, you get 12 million hits /day vs 13 billion hits /day according to his data from Wolfram, for a ratio of ~1/1100, which applied to a pygmy elephant of 5500 lbs yields a corresponding weight of ~5 lbs for the mosquito, or over 2,000,000 mg, versus the average mosquito weight of about 5 mg. Even if he meant a small 2000 kilogram pygmy elephant and a 20mg elephant mosquito, he's still 5 orders of magnitude too generous towards Google in mass comparison in his metaphor in which he is "actually making things better for DuckDuckGo".
I think most people think of animals like African Elephants when they hear "elephant", though, in which case you're looking at a 13,000 pound animal versus a ~12 pound one if you use hits as your metric, or a ~95 lb one if you go by visits as your metric.
So an actually fair metaphor is if Google's an elephant, DuckDuckGo is somewhere between a goose and a hyena. Better watch out, Google.
We also compare CPUs by die-area, not by their length or height.
Yeah, be really scared MSFT, two geeks from Stanford that wanted to sell for $800k are going to challenge you in the future.
A little (long-overdue) help from anti-trust authorities and Google might just have to watch out. After a while migration to another SE is logarithmic. https://duckduckgo.com/traffic.html Google pays for a lot of its traffic, see Traffic Acquisition Costs, directly and indirectly.
Though looking at size, an african elephant is 4m tall, while a small mosquito is 2mm. this is a ratio of 2000:1.
In which case he would actually have been generous to DuckDuckGo while not being wrong by orders of magnitude.
Conclusion: OP meant linear size, not mass. He could have stated it explicitly though, because one would naturally think that he was referring to mass.
[0] http://www.metzerfarms.com/DuckBreedComparison.cfm the most common duck (mallard) is 0.72–1.58 kg, the most common farm duck is 2-2.5 kg, but certain breeds are close to the right interval.
[1] https://en.wikipedia.org/wiki/List_of_heaviest_land_mammals
[2] https://stackoverflow.com/questions/4446112/search-for-inter...
[3] http://www.akc.org/content/dog-care/articles/breed-weight-ch...
[4] https://en.wikipedia.org/wiki/Japanese_Chin
Not really, but "exact same end result" is far from the current situation. And I'm not saying they shouldn't do it, rather that they really need to catch up.
Google (not counting other Alphabet companies) does a lots of things. I think the original analogy is mostly right.
My point can be rephrased as "given that the result is the same, do employee counts matter"? If not, then they don't matter in general.
It might tell us how realistic scaling to the size of Google is, for example. I don't think it does, but it's an interest thought: could DDG manage Google's search traffic with ~1,000 employees (scaling linearly)? Could it do the same with fewer than that?
the second one is doomed
Hey, it's just a metaphor.
Scales and ratios not needed.
edit: Wasn't meant to be negative ... as I could see myself writing something like that and my GF giggling at me for a week ...
> That was one of the most HN comments I've ever read.
What does this mean?
Well played.
I don't think the correctness or incorrectness of the scale of the metaphor really changes the way the whole article reads. Yet it's the first discussion piece I see in the comments on HN, and that doesn't surprise me a bit.
That said, it's a known quirk of the site and not a problem in my mind. I don't hate the tendency even it does cause a regular eye roll from me.
I regularly do this (to the frustration of those around me) and I think there is a strong correlation between enjoying hacker news and having this personality trait.
Okay, DDG is a hyena and not a mosquito. Great. Tell me how that changes the meaning of the article, or the strength of the facts used, or its conclusions, or the credibility of the author.
It means comments around here have a tendency to be overly pedantic and critical.
HN comments suffer from these and more.
What's this about a goose and a mosquito? What's that adding to the communication? Seems pretty irrelevant.
[1] https://m.youtube.com/watch?v=CURzgw60bFY
But the author said, specifically "I’m actually making things better", which isn't true.
You know, that's just a gratuitous swipe. So was this in the blog post: "First, DuckDuckGo didn’t start as a nerd attempt to find the ultimate algorithm."
It really looks like you want to put down nerds to make yourself feel more sophisticated or popular. This isn't high school, though, and you're pissing in the same pool you're posting in. Turning "HN" into an adjective with a pejorative tone isn't going to win you any friends on HN, especially as a new poster. It's just mean spirited.
Sometimes, taking the literal meaning might even distort the original message, like in this case changing the mosquito with a goose - if that had been the original expression, everyone would have wondered why the author chose such strange comparison.
I was going to suggest that a good author would then to choose an animal about the size of the goose that has the same characteristics as a mosquito, but why? A mosquito is considered annoying, harmful and parasitic – if it was the size of the goose, things to be very different. It would be considered a terrifying predator of the jungle. So I don't see what qualities of a mosquito would make it an appropriate choice, and this is what makes the metaphor worth correcting. It is somewhat of a slur against DDG, more so since the numbers are so dramatically inaccurate.
A good animal choose would be a… duck?
It's a metaphor. Not a fact.
English != Math :)
Video if you're unable to visualise - https://youtu.be/o0u4M6vppCI?t=2m42s
I don't notice people wearing both hats at the same time much.
Of course, the OP could be satire. When you can't tell the difference between satire and your platform, then ...
Incidentally, if we look sideways a bit the metaphor fits like: Google isn't just linearly bigger, it has nonlinear increases in advantages because of its size, clout, network effects, and so on. So the order of magnitude discrepancy is perhaps justified by attempting to account for these affects. Conversely, it could be suggesting that smaller disruptive and startups ought to spread their influence virally, parasitically, or through insect-like nimbleness and hatching-of-the-1000-eggs reproduction, instead of relying on slower, more "mammalian" reproduction to propagate their influence. Who knows? Hard to say which interpretation is correct, when words are ambiguous and one leaves it to the imagination. You can't even rely on Occam's razor to decide, since the shortest path between two thoughts differs depending on the mind. But maybe that's not a bad thing. Speech & language is nothing more than successful miscommunication. You can't police what you don't understand.
That would make this a bit of an ironic comment, no? :p
At DuckDuckGo we are averaging roughly 17 million searches a day right now (duckduckgo.com/traffic.html)
Google says they did 2 trillion searches last year[1], and while usually their search metrics include Youtube, Google Maps and Gmail queries let's assume it was all searches. That'd be roughly 5.5 billion searches a day.
So now we're talking ~1/322 or just shy of 40lbs. That's roughly the size of a Hamadryas baboon or about 22 Mallard ducks - not a bad little flock! :)
Disclaimer: DDG staff. Opinions are my own. No animals were harmed in the poor construction of this analogy.
[1]http://searchengineland.com/google-now-handles-2-999-trillio...
Powered By Reporting is listed as any search that is handled by Google or Bing. "Google’s “powered by” share is composed of searches conducted at "Google entities", as well as searches on AOL and Ask’s MyWebSearch "[3]
In Comscores qSearch product that you can subscribe to, when you hit "Google Sites" it provides a drop down detailing the breakdown of all the properties. Search is hard to define when companies get to that size, and searches on other Google properties are indeed 'searches' but they are also different than searches on DuckDuckGo.
[2]http://www.comscore.com/Insights/Rankings/comScore-Releases-... [3]http://www.comscore.com/Insights/Blog/comScore-September-201...
I wouldn't blame you, mine is too (since the majority of readers use Google), but I rank exceptionally higher in alternatives than I do on Google.
It's an alternative to datenkrake Google that does not track you and adds some neat features (bangs, keyboard navigation, etc.)
Edit: Worth pointing out: as far as I am aware, they're using all the other search engines with an agreement with those companies, which makes it completely okay in my book to credit them as the one.
I credit my operating system to Canonical, even though the heavy lifting and behaving nicely on hardware is done by people contributing to Linux. Linux out of the box doesn't mean a thing to me. Bing out of the box doesn't mean a thing to me. DDG and Ubuntu do mean a lot to me.
That's a dangerous comparison to make because there isn't really any consensus* on which part of the stack to name the OS. For example, Linux is technically just the kernel. So most of the UX you deal with will be GNU and other user space tools. Depending on the DE you use, you might not even deal with much of Canonical's code. And Ubuntu does owe an awful lot to it's parent distribution, Debian.
In short, I get the point you're attempting to make but unfortunately your comparison is more contested than the original subject you were discussing.
* though many do advocate GNU/Linux
Canonical did a good thing of mashing it all together into a single product. DDG does the same for search in my book.
They've both outsourced the heavy lifting to some third party.
In DDG, Bing/Yahoo/Yandex does all of the heavy lifting. In Ubuntu, it's the Linux kernel. They would be nothing without those products they depend on. But they bundle that with like hundreds of different, smaller in scale products to offer a nice experience out of the box.
For smaller in scale companies that are trying to compete with big players (like Microsoft and Google), this does seem like a winning combination in my book.
But the most important feature is that DuckDuckGo is the search engine that doesn't track you. We don't store your personal information and we don't sell you out to advertisers.
Our goal is to raise the standard of trust online, and so at DuckDuckGo you are not the product, search is. We think that's an important difference.
Disclaimer: DDG staff.
So... who's paying?
Case A - if you type in "Car" we show an ad about cars. Advertisers are paying to appear at the top of that page. They do not know who you are or what you previously searched.
Case B - if you type in "Car" you are shown an ad about cars. Advertisers are paying to target based on your interest in cars, and possibly other websites you've visited, online actions you've taken, re-marketing pixels from third-party websites, or your affinity to other demographic groups.
In "Case A" the search page is the product.
In "Case B" you are the product.
https://imgur.com/P1akOab
Do you have specific contracts that allows you to use their indexes anonymously or are you at their mercy?
Any evidence to suggest that most people care? Most people just want the best search results, quickly.
Or is this a conscious decision to keep DDG serving a 'niche'.
> and so at DuckDuckGo you are not the product, search is.
Is DuckDuckGo a paid for product then? Users pay to search do they? If it's ad (Or VC) supported, then the user very much IS the product.
But is that information collected by you or anyone affiliated with you or given to anyone?
How about information that doesn't directly identify you, but could be used to identify you? (things like browser fingerprints, IPs, cookies, etc)
https://duckduckgo.com/privacy
We do not collect or share any information that can be used to personally identify you.
What prevents Yahoo from (a) offering a competitive search to DDG or (b) Yahoo from shutting of DDG from using its search engine?
I thought they where re-implementing search.. Are they "just" aggregating?
I mean, even if they were literally proxying to anonymized Google results, wouldn't that cover the privacy benefit?
Also the system in place for aggregation isn't just Nginx, it actually does things, like bangs.
This is Google marketing and brand perception at work because Google results of late, 3 years, have been unimpressive and you have to sift through pages of useless links and content to find any relevant information beyond the usual suspects one already knows, so their intensive spyware operations doesn't seem to help search quality.
It's surprising there are not more experimental search projects. One would have expected a steady stream of regular attempts but not a single credible effort exists. There was once an alternative search project called Cuil that just seemed to fizzle off.
(Actually, I suspect that a major ingredient of Google and Facebooks "algorithms" are really human reviewers... they want to keep this fact secret since human reviewers are responsible for judgement, whereas algorithms still count as impartial. This is not true of course, but it is the perception.)
Proxied (therefore non-personalized) Google results: https://www.startpage.com/
Heavily privacy-focused, even proxying content: https://www.ixquick.eu/
Meta search engine with very feature-rich search, self-hostable: https://searx.me/
Search engine with very feature-rich search result pages: https://www.qwant.com/
Not yet released, peer-to-peer search: http://pearsearch.org/
If the second, as I suspect, it's not really long-term viable anyway, is it.
For natural language queries, I use !g, because Google is better at this sort of thing. For local queries I use !gde, because Google has better map integration. For most everything else, I prefer DDG, because it doesn't "did you mean DickDickGo?", and it doesn't "I don't know what you're looking for, but here are five ads instead".
Ummm... yes it does. It's literally the very first point the author discusses once he starts comparing the search results. Here's a link to the relevant subsection: https://fourweekmba.com/duckduckgo-vs-google/#Search_DuckDuc...
You should too.
Disclaimer: DDG staff.
Thanks for your hard work.
This was true when Google was created.
No one had the processing or memory available on their desktop to search an entire index of the "useful" web.
Not anymore.
How large is a "useful" index of the web today? And can it fit on your laptop? The answer is yes.
Can the entire thing be queried fast? The answer is yes.
As an example take the entire stackexchange and wikipedia dumps in their entirety(including images). Compressed it comes to 50-60 GB range. Think about that number. That's an rough approximation of all known human knowledge. It's not growing too fast. It has stabilized. To query the content you need an index.
So how large is an index to a 100 GB file? Generally around 1 GB. Let's say you use covering indexes with lot of meta data and up that to 5GB to support sophisticated queries.
With today's average hardware you can search the entire thing in milliseconds.
So why aren't we building better local search?
Because everyone is conditioned to believe, thanks to Google's success, we need to do it online. Which means baking in the problem of handling millions of queries a second into the Search problem. Guess what? This is not a problem that local search has.
Every time a chimp or a duck needs to build a protein in it's cell it doesn't query the DNA index stored in the cloud. Instead every cell has the index. Every cell has the processing power to query that index in the nanosecond time scale.
The cloud based search story is temporary.
If you want to index every reference to Taylor Swifts ass that every teenager in Norway, Ecuador and Cambodia are making, then yes you need a Google size index. But for useful human knowledge we are getting to the point where we don't need Google scale.
If you don't believe me look at what is possible TODAY in Dash/Zeal docset search for offline developer documentation search or Kiwix or with Mathematica.
Is there a way to easily sync search repositories, like Stackoverflow, Wikipedia etc., to your local computer with an automatically built search index?
I don't have a CS background so I'm not sure, is this a reasonable solution?
But having a way of making sure that a certain topic or genre of sites is well indexed locally would work. I.e. "my" top 100k. Oh, how awesome would that be.
I use https://chrome.google.com/webstore/detail/personal-blocklist... but it should come with pre-compiled lists, like uBlock Origin.
This feels like an application of the 80/20 rule. I might not always find what I need in just that offline index, but the times I do would seriously disrupt google.
Oh, I wish there were a way to rate the results in a opposite way. Maybe it's just me, but sometimes I want even mark some site as 'a hidden gem'. Usually it's something rather niche though, like a bunch of great little articles about anti-aliasing filters design and sampling, full of engineering wisdom from the decades of experience. So I think it would be perfect to have a way to tag it by some topic.
I've heard that a key early advantage of YouTube was that uploaded videos appeared immediately, not after a long processing delay. That helped it become popular, and once it was popular, it stayed popular and crushed the competition due to the network effect.
Relevant discussion: "Wallabag: a self-hostable application for saving web pages" | https://news.ycombinator.com/item?id=14686882 (July 2017)
>That's an rough approximation of all known human knowledge.
Are you taking the piss?
the problem is this would be completely useless for current events. but querying sites you use regularly, this could be an option. I often end a google search with modifiers like "wikipedia" or "reddit" or "stackoverflow". If I could store indexes to those sites offline that might be useful.
The creator of this offline search could potentially make money by indexing shopping sites like Amazon, eBay, etc with affiliate links.
You vastly underestimate index sizes. I'm not saying it wouldn't be realistic to download and index locally but they'll be much larger than 1-5%.
I get Baby Mama as #1 as well, but at least, from the movie synopsis, it doesn't appeat to be an irrelevant result.
Edit: Oops turns out its 11 years old. Better make that the last two decades then. I'm gettin old
Instead, I am wondering why there isnt a federated open source search engine. It would be a cloud of nodes, each node spidering and indexing a small hash bucket of urls. With a million such nodes we could have a live updated, distributed search engine to replace Google. We could run millions of queries without paying someone - we'd pay back by serving as a node, just like in BitTorrent. With all the interest in privacy, I wanted to see more discussion of replacing Google with an open, non-censured and private protocol for search.
Not sure if federated by they call themselves decentralised: https://yacy.net/en/index.html
Laptops generally come in 128GB or 256GB capacities. So we're talking roughly 20-50% of an average laptop's storage just to hold the search index for wikipedia alone.
Meaning it is not feasible to store a useful index of the web today on a laptop unless you want to dedicate that laptop to pretty much exclusively searching wikipedia's english content.
You're not searching that database in milliseconds on average laptop hardware, either, but you probably could get it to be "fast enough" for practical usages if you could somehow solve the harder parts like storage size, freshness, and indexing more than just wikipedia's english content.
Your indexing performance would also sharply decline when subjected to the random access seek times of those 2.5" spinning metal.
Offline search is the equivalent of the Internet Yellow Pages from back in the day. New, relevant information is being added continuously. The search index 10 minutes ago is different than it is right now.
Your reply seems pointless, yes I could download an entire copy of Wikipedia, compress it and index it - why would I want to? Are you going to do this on every machine you own? What about the updates Wikipedia receives every minute?
You should start a company though to focus on this, maybe you could call it Encarta or something?
Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG.
The cost of entry to the search market is exceedingly high right now. This is a pretty good article of detailing how one person was able to come up with an idea and challenge the behemoth in very niche areas (privacy/the nsa leaks were probably the reason I started looking at/using it around 2013).
Yet I still miss the days of using multiple search engines; seeing a variety of results. I hate the de factor standard of Google. When a company controls that much of search, they get to define the narrative. They literally shape the way many people perceive the world.
I wonder if tech will get to the point where indexing will be easier and we'll see more solutions that are cheaper and that can crawl larger datasets with lower processing requirements. Maybe the next step will be distributed search with shared indexes?
In any case, Google can't remain on top forever (at least I hope not). It'd be nice to see more tech in this space, but it's an incredibly difficult problem. There is reason Google climbed to the top like it did.
Google climbed to the top because they focused on what matters: the right results, quickly, no fuss. Their competitors gave you unrelated spam for most queries and were slow and focusing on useless things.
Googles results are getting worse, there's more fuss than ever and it's getting slower all the time. So there definitely is an opportunity (which DuckDuckGo is taking).
> Today, it's just Google. I've been using DDG for a few years, but about 1/3 of the time I add an !g because I don't find the results I need on DDG.
If DDG folks are reading this, here is an idea: Have a browser extension (so opt-in). If a user then does a !g search, forward the search terms and the search results to DDG and add them to your index. (I wouldn't go as far as Microsoft, who IIRC actually copied the results and the ranking for individual terms and added them to Bing.)
DDG's bangs are effectively a CLI interface to web searches. !g does google, !yt does YouTube, !w does Wikipedia, !wa does Wolfram Alpha, !scholar does Google Scholar...
Huge timesaver.
Having DDG with its bangs I use them all the time, and if there's some specific new site that I want to search there's (so far) always been a fairly guessable bang already set up for it.
I find it exceedingly convenient.
For example, let’s say I want to look up “ear infection”. Google will spit out a bunch of info right on their results page, often saving me the trouble of even going to another site. DDG however will just give me 10 webmd links.
If they want to compete, the privacy angle isn’t enough. They need comparable functionality as well.
I know we're supposed to root for the entrepreneurs here on HN, but DDG honestly seems like more hype / marketing than a decent product. You never hear anything about startpage.com (maybe because it has a boring name?) but it just works.
If they are using a video of Snowden, they may have some philanthropic money - or it may be something nefarious. I'd love to give them the benefit of doubt though.
I might describe that as gaining on Google, but still losing quite badly.
I'm still impressed by DDG of course and welcome some alternative to Google, which is a dangerous monopoly with a search that I've come to despise in recent years due to its ignoring what I've actually typed.
It might be very good. But can such a system compete with one that does use search history as context?
However, I do get your point. Without that extra context, the search results can only ever only be so good.
Imho using the search history as context can also lead to undesirable results, so can funneling people to overly popular results which still can be wrong, had both happen more than enough with Google.
It's a trade-off between spending time arguing with Google if I really meant to search for that word, regardless of " usage, and having to look at more results in DDG until I find the one that I was looking for.
DDG gives more predictable results. It reminds me of why I started using Google over other search engines decades ago.
Overall, super happy with DDG.
But I understand, I try to think of the internet as a non-privacy place.