I stumbled upon this when researching wether it was something to set up with my pi as DSN resolver but it was kinda a pita, does implementing DoT would be faster?
HTTP only adds a small size overhead due to header formatting but it won't add latency, and DNS requests are in general dominated by connection latency (outside of local network)
HTTP adds latency for the first request to a server, since you will have to do the 1-RTT certificate and key exchange before sending your HTTP request.
For many use-cases, that penalty will be paid on every request rather than just the first request, because many DNS resolution libraries aren't designed to have persistent state, so every time the library is used, it will have to set up a new connection to the server from scratch.
I think you're confusing http(s) and tls. Http(1) is a request/response protocol. Tls is the stream encryption and key negotiation protocol most heavily used.
Also, tls1.3 adds 0rtt for subsequent cqlls to the same server.
you have also to do this with plain QUIC or with plain TLS adding HTTP on top of both doesn't add RTT, the point was (DNS over QUIC) vs (DNS over HTTP over QUIC)
If "apt install bind9" (or whatever package manager your distro uses) and editing ~8 lines from 2 text files to make it a secure caching forwarder was too complicated I don't think this would be any easier since it's, at this point, still out of the norm.
All the major DoH providers appear to also support DoT, so DoT is a viable option. Also it is easier to implement, as it can just be a abstraction of existing implementations over DNS over TCP.
I'm kind of sad that we're using HTTPS as a transport though their use cases make sense. It seems this is intended to exist alongside the existing & new secure protocols? Is it foreseen that this will only be used by web apps?
QUIC (Quick UDP Internet Connections) is a protocol derived from Google's Speedy project. Another protocol would use QUIC to initiate a handshake between server and client. The cool thing is that it only takes one UDP packet to do what TCP does in the typical SYN/SYN-ACK/ACK.
Well, it's not quite exactly the same thing as the 3-way handshake over a single packet.
The 3-way handshake ensures that both ends are able to send traffic to their counterpart. Just because i receive a packet doesn't mean I can send one back.
HTTPS isn't only used for webapps. It's widely used for APIs from mobile phone apps (for example). There are also a ton of libraries etc. for handling HTTP and HTTPS making it easy to adopt.
This would certainly be huge overhead at the dns server right? I'm curious if anyone manages an authoritative DNS server? I currently manage four rather large ones and the collective whole serves up between 400,000 and 900,000 responses per second.
Possibly load would in fact go down. Current DNS requests over UDP can (and frequently are) spoofed because there is no way of validating the sender is who they say they are.
With DNSoverHTTPS, the sender must be able to receive replies at their IP address, so cannot spoof who they are easily, closing off a lot of avenues for DoS attecks on your and other peoples recursive resolvers.
Spoofing the sender IP is only useful to conduct DNS reflection attacks, where a small question is sent, and a much larger response is sent back to the (possibly spoofed source IP).
If you require the question to be at least as large as the response, amplification is not a concern any more, so UDP is perfectly fine. This is what the DNSCrypt protocol is doing.
1 Million responses per second on a 8 core 2 Ghz mahine means you have 16,000 clock cycles to formulate a response to each query.
Considering that [1] claims 1.43 cycles per byte for Chacha20, and assuming requests & responses are typically 200 bytes (they will compress well with HTTP header compression), the actual encryption part of https only consumes 1.43 * 200 = 286 cycles.
The remaining 15,716 clock cycles should be plenty to read the actual responses from the cache and parse the HTTP/2 request bytes.
Obviously all of the above assumes you design your infrastructure purely for performance. Writing everything in Python and NodeJS might suddenly burn up all those 15,716 clock cycles!
I checked the cpu stats - and what you describe is definitely accurate - these boxes spend most of their clock ticks on idle...which is amazing. At 3 Ghz and 8 cores - we're talking 24,000 clock cycles to come up with a response.
'[RFC2818] defines how HTTPS verifies the DoH server's identity.' that RFC is TLS over HTTP
How do you resolve the IP address of your DoH server? Is the answer to just use an IP address? In that case does it have a certificate for the IP addresses instead of for a FQDN?
You use a name by default, and if you want to bypass resolution with a system DNS, you hardcode it's IP alongside the name. For example, in firefox:
network.trr.bootstrapAddress - (default: none) by setting this field to the IP address of the host name used in "network.trr.uri", you can bypass using the system native resolver for it. This avoids that initial (native) name resolve for the host name mentioned in the network.trr.uri pref.
> In that case does it have a certificate for the IP addresses instead of for a FQDN?
It can be both: If you look at https://1.1.1.1/, the certificate is issued for *.cloudflare-dns.com, but it also contains the alternative names 1.1.1.1 and 1.0.0.1.
Please forgive me if this has been covered. AFAIK it has not.
Has anyone worked out how this will play nicely in corporate environments? i.e. avoiding leaking even more internal names, resolving internal systems first, etc? Or is it on IT departments to now customize the proxy settings for all browsers and API clients on all operating systems? I ask, because I know that most companies barely manage this on one browser, much less all of them. I am not in IT, but I am occasionally required to assist them and sometimes that requires capturing DNS requests for entire networks.
Correct me if I'm wrong, but I'm pretty sure that this is in fact designed to break that kind of arrangement. The problem is that the mechanics used for corporate monitoring are almost exactly the same mechanics used for your ISP spy on you or censor things. So yes, the end result is that we will need to move monitoring (and potentially blocking if you're doing that) onto the endpoints.
Not necessarily, it depends on the egress architecture. There are ways to do corporate monitoring that cannot be done by your ISP.
Web content filters (aka Proxies) in a transparent mode deployment that are not performing TLS interception would not be able to monitor these queries if wrapped in TLS.
However, an explicit mode deployment and all 80/443 traffic enforced by an egress firewall through the proxy that was performing TLS interception can still monitor this traffic. In fact, even if you’re not brokering TLS, the GET method described can still leak the query (but not response) unless your client uses tunneling (ie. CONNECT method).
ISPs can’t force you to use an explicit proxy, but corporations can.
That is a challenge for us as well. Many of our partners use MitM proxies, but our privacy, compliance and legal teams will not approve the usage of them. We promote a work-life balance which means we allow people to use work resources for personal use. Culturally, that is great. It does create this conundrum however. The company does not want to be liable for intercepting personal and financial data that does not belong to us.
> We promote a work-life balance which means we allow people to use work resources for personal use.
Personally I think that is foolish and naive. Not just from privacy perspective but also liability and tax reasons.
Postmen don't get to use the vans for personal errands, factory workers aren't permitted to run the machines to make t-shirts for themselves. Why are IT resources considered differently?
If you want to permit WLB then tell employees that they are fee to use their personal smartphones and 4G plans.
> The problem is that the mechanics used for corporate monitoring are almost exactly the same mechanics used for your ISP spy on you or censor things.
I disagree. With SSL getting more and more widely deployed, corporate monitoring has no choice but to add re-encrypting proxies.
In that case, all SSL sites you are visiting are signed by a company internal certificate which is more or less obvious depending on the browser (if you ask me, browsers don't make this nearly as obvious as they should)
However, this kind of monitoring absolutely continues to work fine even if the browser is using DNS over HTTPS.
What DoH does break though, is internal DNS resolution of internal intranet hosts. Those would obviously be corporate owned as they would obviously have a corporation specific domain anyways, so no MitM is needed anyways because the admin can just look at the traffic on the server side.
This means that as DoH becomes more widely used, a corporation has only two options to continue to be able to allow users to connect to internal sites, both of which are worse from a security standpoint than the status-quo:
* they could assign publicly resolvable names to all of their internal hosts. This is likely to cause issues with IDS systems which assume that to be a DNS reflection attack and it of course also means that they now have to leak internal names to the public which obviously isn't desirable.
* they could deploy browsers with a modified configuration. Now most corporation already do that anyways to deploy their internal monitoring cert, but some smaller companies might not be interested in monitoring their users and right now they can ship original browser installers from their original source with their original config whereas in the future they will need to use group policies or custom installers, both of which are less trustworthy for a user than the default installers with the default config (who knows what that group policy / custom installer is doing alongside of configuring DoH).
So while option 1 is worse for overall corporate security, option 2 is worse for security and trust for individual users when the corporation is not currently doing any monitoring.
This security downgrade hurts even more in light of the fact that real SSL interception is not at all stopped by this, so already malicious actors can continue to be malicious, but good faith actors have to accept a significant downgrade in either user-trustworthiness or security.
My first experience with BGP hijacking was accidental and not malicious - and over twenty years ago now: I dropped Sprintlink off the East Coast US for several hours. If LE had been active, I could've taken over any Sprintlink-hosted (or downstream) SSL cert simply by performing challenges to LE. If people aren't doing this out of China and Russia I'd be surprised.
LE might mitigate this with multiple checks (across multiple private links) but it currently doesn't do this: LE checks my challenge only once.
>What DoH does break though, is internal DNS resolution of internal intranet hosts. Those would obviously be corporate owned as they would obviously have a corporation specific domain anyways, so no MitM is needed anyways because the admin can just look at the traffic on the server side.
>This means that as DoH becomes more widely used, a corporation has only two options to continue to be able to allow users to connect to internal sites, both of which are worse from a security standpoint than the status-quo:
Why not just have an internal forwarder that does DNS-over-HTTPS? Browsers would send traditional DNS to the forwarder, and the forwarder would do DoH from there.
This would also assume all the users would know how to configure this specific endpoint, or that IT staff do this for all the users. Today there isn't a good story around automating this process in heterogeneous shops. (windows, mac, linux, misc api tools)
If a browser has a default option to use CloudFlare, Google, Amazon or Microsoft, then the user will click the button to "secure their DNS" and suddenly their internal applications may break or do odd things and companies start leaking even more internal records.
Some IT departments may try to fix this by deploying configurations for 80% of the use cases and some may just block the public DoH servers.
I can think of many ways this could be solved. The point I was trying to convey is that people have not thought this through and implemented this in browsers before corporate infrastructure was ready to accommodate the need. This creates contention and friction that could have been avoided, in my opinion. Adoption is likely to be embraced more widely if more use cases are accommodated.
I think gp was concerned more with not leaking internal dns names as DoH isn't using the system resolver and hence requires each application to be configured individually to point to an internal DoH server.
Exactly. If DoH included a change or extension to DHCP or could leverage an existing feature of DHCP, then this could be solved for corporate use. I would imagine something like the Proxy PAC "option wpad-url" using a different code to signify DoH. Even then, only browsers would be addressed by wpad. There would still need to be a mechanism to set DoH for non browsers.
We would also need all OS providers to get in front of this and have a package that supports internal DoH easily. i.e. AD snap-in for windows AD controllers. Perhaps an extension to sssd and/or Unbound and/or dhcp for Linux. That needs to happen before all the browsers start supporting this.
You are correct and that is my point as well. I have yet to see a company that manages all proxy settings for all browsers, operating systems, API clients, etc. At least, not very well. Shops that are strictly windows can enforce this from AD. Everyone else will have to shim something in. We are a mix of Windows, Mac and Linux. Mostly Mac.
Without a decent path forward, I could imagine that many corporations may go the heavy handed route of blocking all the known DoH resolvers. I envision IP and DNS lists like firehol and various RBL databases to eventually contain these if they do not already.
If you want to visit a webpage, you need to use the http (clear) proxy. Most web pages are blocked, and if you need one unblocked you open a ticket and wait a week (or escalate).
I've seen that too. One of our partners has such a stringent policy that the government of China allows them to entirely circumvent their "great firewall" with their own dedicated circuits. They track metrics on every single packet to and from anywhere. They essentially built their own world-wide ISP.
I worked for IBM a few years ago as a remote worker. Interestingly, they don't try to solve this problem (Intranet + managed settings for proxying out) at all.
Instead, they have the opposite arrangement: even sitting in an IBM 'work center' (that you needed to use your employee key-card to access), you're actually just on the public Internet, and you need to use a VPN to reach the IBM Intranet.
The VPN backend interacts with a backend that records the device-management (MaaS360) enrollment status of the device, so you can't get onto the Intranet unless your machine is conformant. And if you aren't connected over VPN to the Intranet, you may as well just be sitting in an Internet cafe with a regular laptop—there's no data available for you to exfiltrate.
DNS has always been available over TCP. The DNS packet header has a Truncated (TC) bit that tells you when you should query over TCP, as UDP responses are usually truncated at ~500 bytes.
DNS over TCP can often provide better throughput and lower latency, especially on networks with a lot of packet loss (which can be your network if you're, say, batch resolving IP address PTR records from a log where your DNS UDP requests overflow the kernel and NIC TX buffers).
DNS Queries over HTTPS.. which implies TCP. Weren't TCP DNS requests avoided like the plague? And why wrap DNS in HTTPS instead of just TLS if you are going down the TCP route?
Why not DNSCrypt which is already supported by a lot of servers? Or DNSCurve.. or DNS over QUIC.
You do both. Both Google and Cloudflare run DNSoTLS on port 853 and DNSoHTTPS on port 443. DNSoTLS and DNSoHTTPS in their short lives have already gained more traction than DNSCrypt ever got.
> DNSoTLS and DNSoHTTPS in their short lives have already gained more traction than DNSCrypt ever got
I'm not convinced at all that this is true. Cisco, Comodo, Infoblox, Yandex, support DNSCrypt only in their products. They have no reason to switch to protocols that would require 10x more traffic.
OpenNIC secure resolvers are all DNSCrypt. Cleanbrowsing resolvers probably get a lot of connections over DNSCrypt as well.
I run both public resolvers accessible over DoH and DNSCrypt, and the traffic over DoH is negligible compared to the DNSCrypt one.
67 comments
[ 2.6 ms ] story [ 82.7 ms ] threadhttps://datatracker.ietf.org/doc/draft-huitema-quic-dnsoquic...
For many use-cases, that penalty will be paid on every request rather than just the first request, because many DNS resolution libraries aren't designed to have persistent state, so every time the library is used, it will have to set up a new connection to the server from scratch.
Also, tls1.3 adds 0rtt for subsequent cqlls to the same server.
I'm in the process of working on a system resolver, but first have to refactor a bunch of stuff. We'll see how that goes.
The 3rd time this exact link (ietf.org) has been posted.
Except for emotions: do we realy expect a new and exciting discussion will take place?
https://news.ycombinator.com/item?id=18533056
See also DNS over QUIC.
https://tools.ietf.org/html/draft-ietf-quic-http-13
https://news.ycombinator.com/item?id=18427795
The 3-way handshake ensures that both ends are able to send traffic to their counterpart. Just because i receive a packet doesn't mean I can send one back.
https://news.ycombinator.com/item?id=16166202
https://news.ycombinator.com/item?id=17196415
https://news.ycombinator.com/item?id=17222861
https://news.ycombinator.com/item?id=17574516
I recently had a need to resolve DNS with http only and this came in handy.
[0] https://developers.google.com/speed/public-dns/docs/dns-over...
With DNSoverHTTPS, the sender must be able to receive replies at their IP address, so cannot spoof who they are easily, closing off a lot of avenues for DoS attecks on your and other peoples recursive resolvers.
Spoofing the sender IP is only useful to conduct DNS reflection attacks, where a small question is sent, and a much larger response is sent back to the (possibly spoofed source IP).
If you require the question to be at least as large as the response, amplification is not a concern any more, so UDP is perfectly fine. This is what the DNSCrypt protocol is doing.
Considering that [1] claims 1.43 cycles per byte for Chacha20, and assuming requests & responses are typically 200 bytes (they will compress well with HTTP header compression), the actual encryption part of https only consumes 1.43 * 200 = 286 cycles.
The remaining 15,716 clock cycles should be plenty to read the actual responses from the cache and parse the HTTP/2 request bytes.
Obviously all of the above assumes you design your infrastructure purely for performance. Writing everything in Python and NodeJS might suddenly burn up all those 15,716 clock cycles!
[1]: https://eprint.iacr.org/2013/759.pdf
https://twitter.com/PowerDNS_Bert/status/1064290946731384832
How do you resolve the IP address of your DoH server? Is the answer to just use an IP address? In that case does it have a certificate for the IP addresses instead of for a FQDN?
network.trr.bootstrapAddress - (default: none) by setting this field to the IP address of the host name used in "network.trr.uri", you can bypass using the system native resolver for it. This avoids that initial (native) name resolve for the host name mentioned in the network.trr.uri pref.
https://daniel.haxx.se/blog/2018/06/03/inside-firefoxs-doh-e...
It can be both: If you look at https://1.1.1.1/, the certificate is issued for *.cloudflare-dns.com, but it also contains the alternative names 1.1.1.1 and 1.0.0.1.
The name used to check the certificate and the URL are also distinct. For example, look at the stamp for Cloudflare:
https://github.com/DNSCrypt/dnscrypt-resolvers/blob/master/v...
It defines https://1.0.0.1/dns-query for the URL, but dns.cloudflare.com for the certificate name.
Has anyone worked out how this will play nicely in corporate environments? i.e. avoiding leaking even more internal names, resolving internal systems first, etc? Or is it on IT departments to now customize the proxy settings for all browsers and API clients on all operating systems? I ask, because I know that most companies barely manage this on one browser, much less all of them. I am not in IT, but I am occasionally required to assist them and sometimes that requires capturing DNS requests for entire networks.
Web content filters (aka Proxies) in a transparent mode deployment that are not performing TLS interception would not be able to monitor these queries if wrapped in TLS.
However, an explicit mode deployment and all 80/443 traffic enforced by an egress firewall through the proxy that was performing TLS interception can still monitor this traffic. In fact, even if you’re not brokering TLS, the GET method described can still leak the query (but not response) unless your client uses tunneling (ie. CONNECT method).
ISPs can’t force you to use an explicit proxy, but corporations can.
Security teams have competing directives.
Board & Investors: “Prevent customer data exfiltration”
Privacy / Legal: “Employee activity monitoring Violates our regulatory obligations”
HR: “BYOD is essential to talent retention, cultural comfort, and workforce optimization”
Employees: “I refuse to use VDI or Jump Hosts”
Personally I think that is foolish and naive. Not just from privacy perspective but also liability and tax reasons.
Postmen don't get to use the vans for personal errands, factory workers aren't permitted to run the machines to make t-shirts for themselves. Why are IT resources considered differently?
If you want to permit WLB then tell employees that they are fee to use their personal smartphones and 4G plans.
I disagree. With SSL getting more and more widely deployed, corporate monitoring has no choice but to add re-encrypting proxies.
In that case, all SSL sites you are visiting are signed by a company internal certificate which is more or less obvious depending on the browser (if you ask me, browsers don't make this nearly as obvious as they should)
However, this kind of monitoring absolutely continues to work fine even if the browser is using DNS over HTTPS.
What DoH does break though, is internal DNS resolution of internal intranet hosts. Those would obviously be corporate owned as they would obviously have a corporation specific domain anyways, so no MitM is needed anyways because the admin can just look at the traffic on the server side.
This means that as DoH becomes more widely used, a corporation has only two options to continue to be able to allow users to connect to internal sites, both of which are worse from a security standpoint than the status-quo:
* they could assign publicly resolvable names to all of their internal hosts. This is likely to cause issues with IDS systems which assume that to be a DNS reflection attack and it of course also means that they now have to leak internal names to the public which obviously isn't desirable.
* they could deploy browsers with a modified configuration. Now most corporation already do that anyways to deploy their internal monitoring cert, but some smaller companies might not be interested in monitoring their users and right now they can ship original browser installers from their original source with their original config whereas in the future they will need to use group policies or custom installers, both of which are less trustworthy for a user than the default installers with the default config (who knows what that group policy / custom installer is doing alongside of configuring DoH).
So while option 1 is worse for overall corporate security, option 2 is worse for security and trust for individual users when the corporation is not currently doing any monitoring.
This security downgrade hurts even more in light of the fact that real SSL interception is not at all stopped by this, so already malicious actors can continue to be malicious, but good faith actors have to accept a significant downgrade in either user-trustworthiness or security.
I do not think this is a fair trade-off.
I wasn't aware that SSL interception was possible without a CA trusted by the victim, which is trivial in Enterprise and nigh-impossible elsewhere.
https://www.internetsociety.org/blog/2018/04/amazons-route-5...
https://www.itnews.com.au/news/china-systematically-hijacks-...
https://www.theregister.co.uk/2018/11/13/google_russia_routi...
My first experience with BGP hijacking was accidental and not malicious - and over twenty years ago now: I dropped Sprintlink off the East Coast US for several hours. If LE had been active, I could've taken over any Sprintlink-hosted (or downstream) SSL cert simply by performing challenges to LE. If people aren't doing this out of China and Russia I'd be surprised.
LE might mitigate this with multiple checks (across multiple private links) but it currently doesn't do this: LE checks my challenge only once.
>This means that as DoH becomes more widely used, a corporation has only two options to continue to be able to allow users to connect to internal sites, both of which are worse from a security standpoint than the status-quo:
Why not just have an internal forwarder that does DNS-over-HTTPS? Browsers would send traditional DNS to the forwarder, and the forwarder would do DoH from there.
If a browser has a default option to use CloudFlare, Google, Amazon or Microsoft, then the user will click the button to "secure their DNS" and suddenly their internal applications may break or do odd things and companies start leaking even more internal records.
Some IT departments may try to fix this by deploying configurations for 80% of the use cases and some may just block the public DoH servers.
I can think of many ways this could be solved. The point I was trying to convey is that people have not thought this through and implemented this in browsers before corporate infrastructure was ready to accommodate the need. This creates contention and friction that could have been avoided, in my opinion. Adoption is likely to be embraced more widely if more use cases are accommodated.
We would also need all OS providers to get in front of this and have a package that supports internal DoH easily. i.e. AD snap-in for windows AD controllers. Perhaps an extension to sssd and/or Unbound and/or dhcp for Linux. That needs to happen before all the browsers start supporting this.
Without a decent path forward, I could imagine that many corporations may go the heavy handed route of blocking all the known DoH resolvers. I envision IP and DNS lists like firehol and various RBL databases to eventually contain these if they do not already.
If you want to visit a webpage, you need to use the http (clear) proxy. Most web pages are blocked, and if you need one unblocked you open a ticket and wait a week (or escalate).
Instead, they have the opposite arrangement: even sitting in an IBM 'work center' (that you needed to use your employee key-card to access), you're actually just on the public Internet, and you need to use a VPN to reach the IBM Intranet.
The VPN backend interacts with a backend that records the device-management (MaaS360) enrollment status of the device, so you can't get onto the Intranet unless your machine is conformant. And if you aren't connected over VPN to the Intranet, you may as well just be sitting in an Internet cafe with a regular laptop—there's no data available for you to exfiltrate.
much better to authenticate the policer.
At least we get a private connection to a known entity though.
DNS over TCP can often provide better throughput and lower latency, especially on networks with a lot of packet loss (which can be your network if you're, say, batch resolving IP address PTR records from a log where your DNS UDP requests overflow the kernel and NIC TX buffers).
DNSoQUIC is already in the pipeline.
I'm not convinced at all that this is true. Cisco, Comodo, Infoblox, Yandex, support DNSCrypt only in their products. They have no reason to switch to protocols that would require 10x more traffic.
OpenNIC secure resolvers are all DNSCrypt. Cleanbrowsing resolvers probably get a lot of connections over DNSCrypt as well.
I run both public resolvers accessible over DoH and DNSCrypt, and the traffic over DoH is negligible compared to the DNSCrypt one.