I use `dnsmasq` and configured it to return `NXDOMAIN` for a giant list of ad domains I found. It doesn’t have feature-parity with this, but the software was readily available for my platform, which was nice.
dnsmasq can read from /etc/hosts so you can automate it just using `curl` (or other standard UNIX tools):
# the first command is a one time only job
cp -vip /etc/hosts /etc/hosts.default
# the following is the automation.
curl http://someonewhocares.org/hosts/hosts --silent > /etc/hosts.sowc
cat /etc/hosts.default /etc/hosts.sowc >> /etc/hosts
The other benefit of this is you can then just do a `dnsmasq reload` (since the hosts file is an external configuration file) rather than the full `dnsmasq restart` that dnsgate performs.
Oh that's handy. Would also make the update even easier since you no longer need to preserve any host-specific entries so a simple one liner in `curl` or `wget` would be enough.
I'm trying to think of a use case for addn-hosts in dnsgate. If the user is not using dnsmasq, then they are stuck with /etc/hosts, if they are using dnsmasq then they might as well use a dnsmasq syntax file so '--block-at-tld' can be used. dnsmasq.conf has an include syntax:
But having the entries in /etc/hosts should on Unix in most cases cause your computer to find it there right away without DNS lookup. Are you using dnsmasq locally on the host computer or to serve other computers as well? In the latter case, I see the point (though if I were to use dnsmasq I wouldn't intermix the hosts file).
There wouldn't be any point running dnsmasq only for a local machine :)
RE intermixing hosts file: dnsmasq supports reading from additional hosts files other than /etc/hosts (thank to creshal for that suggestion: https://news.ycombinator.com/item?id=10580082)
I run dnsmasq locally (and silently redirect all traffic from a certain uid with an iptables rule) to do adblocking and ... minimal parental control on my son's laptop. it is pretty great, I don't have to worry about him using the neighbor's wifi to get around my rules. ;)
Good point. Sadly that's pretty much how all adblockers keep up-to-date (albeit at least with TLS). Though pragmatically even if the update was manual, you still need to place some trust in the site when dealing with files of that size. Granted it would be better if those resources served TLS, but that would only prevent MITM and not the file being dodgy from source (eg site hacked).
Thinking a little more about the problem you raised, I guess you could grep it for IPs that aren't localhost (ie exit the shell script before writing the new hosts if `grep` returns a zero exit code). Aside that, I'd genuinely be interested in any ideas you have for keeping the file updated in a secure way (if at all possible).
You should still have a modicum of trust towards the host you're pulling from, as they can also insert something to potentially break the parsing code, which opens up a whole 'nother host of issues.
True. In dnsgate's case the domains are extracted and guaranteed (real hostname validation is on the todo) to point to 127.0.0.1 (or whatever '--dest-ip' is).
dnsmasq can read /etc/hosts, but /etc/hosts format can't do * blocking. So if you are using /etc/hosts (with dnsmasq) '--block-at-tld' wouldn't work.
Moving /etc/hosts to /etc/hosts.default is much better than the 'hosts example'. Updated it. Thanks. Still experimenting with 'dnsmasq reload' but if it works as advertised that's more general than restarting the service.
Correct. And it would be a problem if something was listening on 80/443/whatever. That's why '--dest-ip' can be set.
from man dnsmasq:
-B, --bogus-nxdomain=<ipaddr>
Transform replies which contain the IP address given into "No such domain" replies.
This is intended to counteract a devious move made by Verisign in September 2003
when they started returning the address of an advertising web page in response to
queries for unregistered names, instead of the correct NXDOMAIN response. This
option tells dnsmasq to fake the correct response when it sees this behaviour. As
at Sept 2003 the IP address being returned by Verisign is 64.94.110.11
Is there another way to get dnsmasq to return NXDOMAIN? If I am reading this correctly, one would need to do 'dnsgate --dest-ip 127.2.2.2' and then tell dnsmasq '--bogus-nxdomain=127.2.2.2'.
Take a look at PowerDNS Recursor or Unbound, both of which have scripting capabilities than enable this type of behavior and much more (Lua for PowerDNS, Python for Unbound).
I'd say a script for any of these would be a better choice.
I used just 'local-zone' entry. Unbound answers NXDOMAIN for those domains, and it prevents next HTTP request to 127.0.0.1. I'm not sure what approach is better.
I use unbound as my caching recursive server and I integrated blacklisting of those ad domains. Basically I generate config file for unbound using awk and reload unbound. Here's my script (for OpenBSD, but modification would be trivial): https://gist.github.com/vbezhenar/a0065fdb50f6aa601a20
probably because of TLS? You can pretty easily block IP ranges at the network layer but blocking domains is harder - blocking the DNS lookup is probably the easiest way to block a domain
In the case of this HN submission, they're the same thing. From what I can gather, Adsuck is essentially just a DNS forwarder that sends NX DOMAIN for blacklisted domains but forwards the DNS requests for all other DNS lookups. So "spoof" is a little misleading since it's actually doing the job of a normal DNS forwarder - albeit tuned with privacy in mind. So I think it's fair to say this method could more accurately be defined as "block[ing] domains" rather than "spoofing DNS". However I'd welcome a correction if I'm wrong.
Take a look at https://github.com/longsleep/adblockrouter - i hacked this together some months ago and use it since to provide DNS blacklisting to dnsmasq running on OpenWRT. If your OpenWRTC router has wget and ssl, it even runs directly on it.
This is a nice idea, but it's really easy to replicate using dnsmasq and a custom hosts file (I use "Someone Who Cares"[1], but others exist as well, [2][3])
dnsmasq can read from the hosts file and the whole thing is automatically updated once a week with the latest bad domains (a simple `curl` request is all that's needed)
This method also ports much easier to other *nixes and even Windows since dnsmasq already has wider platform support.
It's a lot less resource intensive not to use a hosts file. This might not be a concern for people with modern machines, but having a hosts file with 12,000 lines in it does take a certain amount of processing.
If I might mention my own site for a minute, I maintain a list of ad server (and tracking server) hostnames: http://pgl.yoyo.org/adservers/
You can view the list as a dnsmasq config file, a BIND config file, and a bunch of other formats.
Also, running your own DNS server means you can do this for every device, even those for which you can't easily access the HOSTS file or perhaps don't even have such a facility (locked-down mobile devices, embedded systems, etc) and not have to worry about keeping multiple copies of HOSTS files in sync and updated.
Although I don't have such devices in my network, I've heard that others do this to their "smart" TV/media box/creepy home surveillance gadgets.
Yes, absolutely! It's way easier to configure your wifi router (or cable modem, or whatever acts as the DHCP server on your network) to use a local DNS server that blocks ads. Then ad blocking just works, whatever you connect to your network.
is the best way to do this to run something like DD-WRT? Or is there a way to accomplish this with stock router firmware? I ask because I recently tried to install DD-WRT but ran into issue and had to revert to my stock firmware.
Another way to accomplish it if you've got a spare computer (even something like a Raspberry Pi) is to configure it as a DNS server and then use your stock router firmware, if it supports it, to point DNS servers to that computer's local IP. So one device in your home or network would then become the local resolver.
Personally I run dnsmasq on my file server (separate FreeBSD jail), but before then I was running it on my Asus router with pretty much the stock firmware. So I don't think there is a "best" approach specifically - just whatever works for you.
Very late in seeing this comment! It's a Netgear WNDR3700v4. DD-WRT is supported on it, and i did successfully get it installed, but I had no connection to the internet with it. There was a wiki page for my specific router model with instructions which I followed. I went back & forth with people on their forums and they were pretty much telling me "oh yeah, don't install that version that the instructions tell you to!" and "well yeah, this whole project is beta, so what do you expect?" All of that combined kind of turned me off to this project from now on :/
dnsmasq caches the hosts file so it's not that much an overhead. Anecdotally I've never had a problem with a 12k+ lines in hosts file (nor even several times that amount) on either Windows, FreeBSD nor Linux; and I certainly don't run powerful machines.
But that aside I wasn't suggesting that people run dnsmasq on their primary workstation. It's certainly an option, but I personally think it's better to have dnsmasq on a spare machine which is always on (doesn't need to be beefy, a Raspberry Pi would easily be more than capable). That way you can plum your dnsmasq IP into your DHCP server (or even use dnsmasq as your DHCP server as well) so that your phones, tablets and other devices all also benefit from ad blocking / tracking / etc.
Another option with dnsmasq is to run it directly on the router. Some consumer routers, such as Asus, allow you to access the device's command line via telnet / SSH. In the case of Asus, they literally just run dnsmasq for their DNS and DHCP daemons, so you could apply the hosts file directly on the router if you wished.
If you're Joe User, it's much easier to install a browser plugin or hosts file than it is to set up a DNS server of your own, route your machines to it (even internally), configure it properly...
Personally, it's also less annoying to shut off uBlock/etc per site than to go into hosts, commenting out a domain, then uncommenting when I'm done when visiting sites that break when you cut off google analytics or doubleclick
51 comments
[ 2.4 ms ] story [ 116 ms ] threadThanks for the recommendation.
echo "conf-file=/etc/dnsgate/generated_blacklist.conf" >> /etc/dnsmasq.conf
RE intermixing hosts file: dnsmasq supports reading from additional hosts files other than /etc/hosts (thank to creshal for that suggestion: https://news.ycombinator.com/item?id=10580082)
Thinking a little more about the problem you raised, I guess you could grep it for IPs that aren't localhost (ie exit the shell script before writing the new hosts if `grep` returns a zero exit code). Aside that, I'd genuinely be interested in any ideas you have for keeping the file updated in a secure way (if at all possible).
You should still have a modicum of trust towards the host you're pulling from, as they can also insert something to potentially break the parsing code, which opens up a whole 'nother host of issues.
Moving /etc/hosts to /etc/hosts.default is much better than the 'hosts example'. Updated it. Thanks. Still experimenting with 'dnsmasq reload' but if it works as advertised that's more general than restarting the service.
from man dnsmasq:
Is there another way to get dnsmasq to return NXDOMAIN? If I am reading this correctly, one would need to do 'dnsgate --dest-ip 127.2.2.2' and then tell dnsmasq '--bogus-nxdomain=127.2.2.2'.I'd say a script for any of these would be a better choice.
BIND has had RPZ support for years and other resolvers are working on the feature.
I know i could add it to other existing servers, but i'd really love a really small independent virtual machine which sole purpose would be DNS.
dnsmasq can read from the hosts file and the whole thing is automatically updated once a week with the latest bad domains (a simple `curl` request is all that's needed)
This method also ports much easier to other *nixes and even Windows since dnsmasq already has wider platform support.
[1] http://someonewhocares.org/hosts/
[2] http://winhelp2002.mvps.org/hosts.htm
[3] https://github.com/StevenBlack/hosts
If I might mention my own site for a minute, I maintain a list of ad server (and tracking server) hostnames: http://pgl.yoyo.org/adservers/
You can view the list as a dnsmasq config file, a BIND config file, and a bunch of other formats.
Although I don't have such devices in my network, I've heard that others do this to their "smart" TV/media box/creepy home surveillance gadgets.
What's your router model?
But that aside I wasn't suggesting that people run dnsmasq on their primary workstation. It's certainly an option, but I personally think it's better to have dnsmasq on a spare machine which is always on (doesn't need to be beefy, a Raspberry Pi would easily be more than capable). That way you can plum your dnsmasq IP into your DHCP server (or even use dnsmasq as your DHCP server as well) so that your phones, tablets and other devices all also benefit from ad blocking / tracking / etc.
Another option with dnsmasq is to run it directly on the router. Some consumer routers, such as Asus, allow you to access the device's command line via telnet / SSH. In the case of Asus, they literally just run dnsmasq for their DNS and DHCP daemons, so you could apply the hosts file directly on the router if you wished.
dnsmasq has the --addn-hosts parameter for that, so you can keep the list in a separate file.