The email service luxsci.com scrubs a lot of information from your headers, if you wish [1].
Another well-known email provider allows you to scrub the IP address, but it isn't clear whether they mean to publicize it. So Google the name of your favorite email provider along with "hide ip address", and see what you can find.
I've just reproduced the described behavior via a test message from Gmail.
On the other hand, I've also just reproduced it via test messages from every other one of my email accounts, across multiple providers as well as the various instances I run for my own use. The only one that didn't repro was Office 365.
This is how SMTP is supposed to work. As a side note, messages sent with Outlook or webmail using O365 (or Exchange) don't go from your client to the O365 server over SMTP, but your IP address is still shown in the X-Originating-IP header.
This is "Working as Intended" (and Google told him exactly that).
Gmail and other providers have to split the balance between privacy and security. If you use the web interface and they authenticate your device directly, Google hides your IP and takes responsibility as the "source" of the email. If you're using SMTP it is possible you could be relaying mail for others (even possibly as an open relay) and they will disclose the IP of your client or mail server.
This is what is supposed to happen. I guess I agree though that we could get rid of it as normal users dont know this is how email normally works. It's useful though...
If host A is using gmail to spam me, then I can block host A, but if gmail would remove this I can only block all of gmail (or the gmail account, but a new account will be created 2 seconds later, whereas creating new IPs is much harder).
I'm all for privacy and prevention of leaking data such as IP addresses, but there are other concerns, as well, which I often see "privacy advocates" forget about. Maybe there is a better way to solve this problem than including the IP address – I would be all for it – but this post doesn't mention any of this.
Unfortunately, 100% privacy is often open to abuse from malicious actors, and a balance needs to be struck.
BTW: you can make a similar argument about the stamps the post office puts on your letters, which often includes the location and may be "private information".
Sending spam on the gmail web UI is considerably harder than a few lines of shell script needed to spam over SMTP. And arguably, the web UI should really add the same Received headers.
> Users looking to send email in a manner that keeps this information private from message recipients should use either Gmail’s web interface or an alternative mail provider.
Or, use a VPN rather than hamstring the ability to validate email or fight spam.
Can someone break down what this means for me? I'm assuming an example of say Thunderbird as the MUA connects to gmail's servers with SMTP over TLS or using STARTTLS and happens to include this header of the client's public OR (not xor) private IP address as per specification(s).
What's unclear for me it whether this is then stripped by gmail's MSA/MTA before being routed towards the destination. From how I'm reading the article in the OP, it seems to imply that the recipient of the message would know the sender's IP address. If this is the case then I'd say that's of some concern, but it seems people are saying this is NOT the case. Can someone please expand on the exact test setup?
Any random MTA including gmail's that receives your message over SMTP prepends Received header to your message that contains where the message was received from and that somewhat obviously includes the IP address where the message came from. It works as intended and if you at all care about privacy implications of that (whatever that might be) you simply should not use email/SMTP.
I'm not looking for an argument, I've never noticed this behavior before. Why is this done? People are saying this is some kind of way to combat spamming/spoofing/malicious email but I don't see how that's the case. If the source address of the packets matches a blacklist then you can correlate against that. While not impossible to forge an IP address, it's certainly more trivial to edit the SMTP header (I'm not saying that would avoid detection, mind you).
What benefit do we gain from including the origin's IP address throughout the entire header history?
This was done way before Google existed, way before they offered a GMail server or an SMTP interface to their servers. It is done to combat spam, if I get an email that was relayed through their servers I can see which IP connected to their servers to send it. Then every time the email is sent between servers the source IP is again added. It allows to track the real source of an email, remember that email inherently has no authentication, there's no way to really say the From address is authentic. But having a record of all the IP addresses used when transferring the email I can determine that pretty easily.
> remember that email inherently has no authentication, there's no way to really say the From address is authentic
SPF and DKIM have changed this quite radically. (Not always for the better, e.g. mailing lists traditionally had a legitimate need to set the From: address on the emails they relay. But one can argue that there's little need for those nowadays.)
Trouble being that SPF and DKIM only work when they've been implemented correctly and they rely on DNS. That requires DNS to be resilient against attacks (DNS isn't foolproof).
While yes, email has no implicit authentication and forging an email is trivial, pulling that attack off is super difficult due to modern SPF and DKIM mechanisms. I'm not super familiar with DKIM but I'll use SPF in my example below.
Let's say I'm running an email server for myself for example.com and I receive an email from 192.0.2.1 and it claims to be from bob@gmail.com. Okay, great. First I consult with my database of blacklists to make sure that 192.0.2.1 isn't blacklisted. If it is, into the trash it goes.
The blacklist reports the address is clear, I move to the next step - SPF. I do a DNS lookup for the txt records under gmail.com and find the one for SPF. But lo and behold, the SPF record does not include 192.0.2.1 as a valid address to submit mail on behalf of gmail.com - this email is likely forged. I drop the mail and submit a report to my favorite blacklist provider(s).
Now, could the DNS have been forged? Yes. With modern DNSSEC, is that likely? I don't think so. Could the packet's source address been forged? Yes, that's pretty easy if you're the ISP or state-sponsored. But if the source address has been forged, there's no way the IP address in the SMTP headers are going to do you any good - the attack is too sophisticated at that point.
If a receiver of mail fails to properly validate the source of the email, that onus is on them as far as I'm concerned, especially so if DNSSEC, SPF, DKIM, etc have all been implemented. While open relays aren't extinct, they sure are easy to detect and most best practices I've read on email say to reject them when detected.
I still don't see how having this trail of user IP addresses is useful. There's nothing implicitly in the message that can be trusted - that's what PKI is for.
I don't know what you mean by "modern DNSSEC", but virtually none of the most important names on the Internet are (or likely ever will be) signed, so whatever your obstacles to forging an email might be, DNSSEC won't be among them.
This shows the top 25 websites and that none of them have DNSSEC.
As for whether people will in the future, it’s impossible to say for sure. But chrome doesn’t support dnssec, which shows how seriously google takes it.
Thanks for the link, that's quite interesting. While web browsers can implement their own DNS resolver, that's completely irrelevant when we're talking about an MTA. The server(s) the MTA(s) are running on have their own underlying resolvers which can be DNSSEC-compatible, or better yet, if you're using a public resolver like quad9, they validate the DNSSEC for you.
Edit: Not that I'm saying you should trust quad9 full-stop, but it is a nice feature. Anyone could run their own private resolver but most choose not to because of the very same privacy concerns we're talking about in these headers, namely - making your traffic easier to profile.
It doesn't matter that your MTA could, in theory, do fully recursive signature-validating resolutions, because none of the domains you'll be looking at are signed.
Yes, I'm satisfied with the evidence presented that the "major" vendors aren't signing their zones, but I do think we're far off-topic at this point as to whether or not the client IP address has value in the message headers.
It's easy to work this out for yourself. Take any list of the most popular zones --- the Moz 500 is the simplest to download --- and then write a simple shell loop to "host -t ds" each of them. You'll see in a minute or so that it is as I say it is. With the exceptions of Cloud Flare, which sells DNSSEC services, and Paypal (but none of Paypal's subsidiaries like Venmo), nobody in the technology industry uses DNSSEC. For that matter, none of the major banks do, either. Look to any industry vertical where companies tend to have significant security teams: none of them use DNSSEC. DNSSEC is virtually absent among major domains on the Internet.
This despite the fact that DNSSEC has been under development for twenty five years, with repeated aggressive pushes for deployment.
Indeed, browsers have experimented with DNSSEC support... and then removed DNSSEC support from their builds when they discovered it was unworkable.
Maybe if kids these days weren't brought up with ONLY services like Gmail and Outlook for email, they wouldn't be so ridiculously surprised when they learn how actual email works.
This does illustrate the differences in expectations, though. We all know how it works. Most consumers of Gmail don't. It should be made clear to them, or it should be changed to protect them.
In other news iOS tells every web site you visit Your IP address! And no matter where you drive people have access to a unique vehicle ID assigned to your vehicle by the state!
43 comments
[ 3.7 ms ] story [ 105 ms ] threadBoth Protonmail and Gmail hide the IP if you're using the web interface.
https://protonmail.com/bridge/
tho not clear if dannyw tested with ProtonMail IMAP/SMTP Bridge or with ProtonMail web.
Another well-known email provider allows you to scrub the IP address, but it isn't clear whether they mean to publicize it. So Google the name of your favorite email provider along with "hide ip address", and see what you can find.
[1] https://luxsci.com/extranet/info/email-smtp-anon.html
On the other hand, I've also just reproduced it via test messages from every other one of my email accounts, across multiple providers as well as the various instances I run for my own use. The only one that didn't repro was Office 365.
Gmail and other providers have to split the balance between privacy and security. If you use the web interface and they authenticate your device directly, Google hides your IP and takes responsibility as the "source" of the email. If you're using SMTP it is possible you could be relaying mail for others (even possibly as an open relay) and they will disclose the IP of your client or mail server.
I'm all for privacy and prevention of leaking data such as IP addresses, but there are other concerns, as well, which I often see "privacy advocates" forget about. Maybe there is a better way to solve this problem than including the IP address – I would be all for it – but this post doesn't mention any of this.
Unfortunately, 100% privacy is often open to abuse from malicious actors, and a balance needs to be struck.
BTW: you can make a similar argument about the stamps the post office puts on your letters, which often includes the location and may be "private information".
Or, use a VPN rather than hamstring the ability to validate email or fight spam.
March 9, 2020 – Reported to Google security
March 10-19, 2020 – Assisted with steps to reproduce
March 20, 2020 – Resolved by Google as “Won’t Fix (Intended Behavior)”
March 20-31, 2020 – Worked to confirm that there were no future plans to fix
April 2, 2020 – Got confirmation that the team was aware of the issue and has no plans to fix
Sorry but this is just embarrassing. He wasted so much of his time and time of others while doing 15 sec of googling[1] would solve this.
[0] https://www.quora.com/Does-sending-email-from-Gmail-expose-y...
What's unclear for me it whether this is then stripped by gmail's MSA/MTA before being routed towards the destination. From how I'm reading the article in the OP, it seems to imply that the recipient of the message would know the sender's IP address. If this is the case then I'd say that's of some concern, but it seems people are saying this is NOT the case. Can someone please expand on the exact test setup?
What benefit do we gain from including the origin's IP address throughout the entire header history?
SPF and DKIM have changed this quite radically. (Not always for the better, e.g. mailing lists traditionally had a legitimate need to set the From: address on the emails they relay. But one can argue that there's little need for those nowadays.)
Let's say I'm running an email server for myself for example.com and I receive an email from 192.0.2.1 and it claims to be from bob@gmail.com. Okay, great. First I consult with my database of blacklists to make sure that 192.0.2.1 isn't blacklisted. If it is, into the trash it goes.
The blacklist reports the address is clear, I move to the next step - SPF. I do a DNS lookup for the txt records under gmail.com and find the one for SPF. But lo and behold, the SPF record does not include 192.0.2.1 as a valid address to submit mail on behalf of gmail.com - this email is likely forged. I drop the mail and submit a report to my favorite blacklist provider(s).
Now, could the DNS have been forged? Yes. With modern DNSSEC, is that likely? I don't think so. Could the packet's source address been forged? Yes, that's pretty easy if you're the ISP or state-sponsored. But if the source address has been forged, there's no way the IP address in the SMTP headers are going to do you any good - the attack is too sophisticated at that point.
If a receiver of mail fails to properly validate the source of the email, that onus is on them as far as I'm concerned, especially so if DNSSEC, SPF, DKIM, etc have all been implemented. While open relays aren't extinct, they sure are easy to detect and most best practices I've read on email say to reject them when detected.
I still don't see how having this trail of user IP addresses is useful. There's nothing implicitly in the message that can be trusted - that's what PKI is for.
Source?
>...Or likely ever will be
Source?
This shows the top 25 websites and that none of them have DNSSEC.
As for whether people will in the future, it’s impossible to say for sure. But chrome doesn’t support dnssec, which shows how seriously google takes it.
Edit: Not that I'm saying you should trust quad9 full-stop, but it is a nice feature. Anyone could run their own private resolver but most choose not to because of the very same privacy concerns we're talking about in these headers, namely - making your traffic easier to profile.
Thanks for the discussion!
This despite the fact that DNSSEC has been under development for twenty five years, with repeated aggressive pushes for deployment.
Indeed, browsers have experimented with DNSSEC support... and then removed DNSSEC support from their builds when they discovered it was unworkable.
This does illustrate the differences in expectations, though. We all know how it works. Most consumers of Gmail don't. It should be made clear to them, or it should be changed to protect them.