HN Explorer

19 comments

[ 4.2 ms ] story [ 44.9 ms ] thread
zamalek
It would probably be more conspicuous if they didn't.
[deleted]
(comment deleted)
crazygringo
And?

Their robots.txt allows it [1] -- it only disallows URL's with a question mark in them (i.e. GET parameters) and none of the indexed pages have that:

  Disallow: /*?
And Google is indexing pages like:

  https://duckduckgo.com/Boeing
So not sure what the point of this submission is?

[1] https://duckduckgo.com/robots.txt

t0astbread
Someone found it mildly interesting. And it shows how you can mess up your robots.txt.
jrwr
Interesting, They block the Wayback Machine's Scraper
wmf
Google has a policy of not indexing search results so these will probably be removed although technically there's nothing wrong here.
raslah
It's weird that there some subtle defensiveness here. To heck with robots.txt and what not, when a site touts it's privacy, even this, however irrelevant, is slightly eyebrow-raising.
kchamplewski
How does this in any way reflect on either the privacy of DuckDuckGo (I don't think they ever claimed or even vaguely implied that if I search for, say, Banana, no one else will be able to see the results page for Banana, as that would be absurd), or Google, who are simply indexing publicly accessible pages, which is what they've always done since coming into existence.

What violation of privacy or hint thereof is happening here?

raslah
None. Thus my question of why the Curt responses and passive aggression. DDG has positioned itself as the anti Google and lo and behold Google still finds out what's being searched there, though not necessarily who searched it. If you're a bit cynical, it's vaguely interesting.
theboulevardier
If you look at the cached Google page, you will see that Google is only indexing the knowledge panel, not the results themselves. E.g https://webcache.googleusercontent.com/search?q=cache:t8Z5T9...
jtbayly
I'd love to see a feedback loop of Google and DDG both looking at each other for factual information to deliver alongside search results.
cocktailpeanuts
Have you ever thought, maybe that's how DDG intended it? small time search engines like DDG have no luxury to block any potential routes of people discovering them, especially through Google. I would even bet they are doing "SEO" to optimize their search results for Google.