"In this paper, we study the stability of the public IP addresses a user device uses to communicate with our server. Over time, a same device communicates with our server using a set of distinct IP addresses, but we find that devices reuse some of their previous IP addresses for long periods of time. We call this IP address retention and, the duration for which an IP address is retained by a device, is named the IP address retention period.
We present an analysis of 34,488 unique public IP addresses collected from 2,230 users over a period of 111 days and we show that IP addresses remain a prime vector for online tracking. 87 % of participants retain at least one IP address for more than a month and 45 % of ISPs in our dataset allow keeping the same IP address for more than 30 days."
I suppose this only works if you combine IP with some other information, like username or browser fingerprint. Otherwise you could be tracking multiple 'users' at the same IP.
The whole multiple-users-on-one-ipv4 thing always feels like it distracts from what a tracker's goal really is. It is definitely beyond sufficient identification and more than needed for a tracker to start targeting you with ads and what not.
In some ways the fact that IP only tracking is lossy but still good enough is something to be afraid of. It is easy to 'taint' an IP and your households behind-the-scenes profile over a google search by your guest using your wifi.
Advertising that is relevant to my wife probably isn't relevant to me, using IP is not good enough. The title and focus here is on user tracking, which is the goal I'm commenting about. Its quite common to have multiple users behind the same IP, those users and their actions may or may not be related (I have no idea what my co-workers might be doing for example but we share a static IP).
I honestly didn't follow your Google search/wifi example.
Define "not good enough"(for who is it not good enough? for advertisers?).
I used to work on a project linking cookies together in anonymous profiles (for advertising), initially the plan was to separate household/individual profiles using different heuristics, but I think eventually it turned out that nobody cares that much - the advertisers just wanted a rough cross-device profile, not perfect accuracy. I mean sure, for marketing purposes (as well as engineering/ "performance" reasons) you needed figures to brag about accuracy and whatnot. But it's really really hard to get those figures right, and ultimately the thing that will convince advertisers is "using cross-device profile improved conversion rate by 10%"; everything else is a detail in comparison.
A user is usually defined as a human being. I'm a human being and I did not come with a network connection. The world, generally, does not assign IP addresses to human beings - thus an IP address cannot track me as a human, a human computer user.
The paper/study here used a combination of IP address and a UUID to track 'users' which are presumably but not necessarily individual humans. It was the use of this UUID that allowed the authors to draw conclusions about how often a given user is seen at the same IP address. The IP address by itself did not provide enough tracking to draw the conclusions in the paper.
Is IP address identification 'good enough' for advertising, maybe. But IP is no where near definitive for identifying individual humans.
The paper does not say how often they found multiple 'users' using the same IP address. The paper focuses on individual reuse of IPs. How often did they find different users using the same IP in their study?
In internet advertising no one expects every ad to trigger a sale. Since there is some amount of acceptable waste in the system this paper has decided that IP is 'good enough' for targeting - because its okay if we are throwing away 20% or more of the ads (we didn't expect any traction from them anyway seems to be the thinking).
And the authors admit that a larger or different dataset might show different results.
> Advertising that is relevant to my wife probably isn't relevant to me, using IP is not good enough.
Advertisers do not care about what you as a human do in the same way that e.g. some government agencies might. Both the paper and you talked about "user tracking" in the context of advertising. My claim is that "IP address identification is actually 'good enough' for advertising" (for some kinds of advertising, at least, but the discussion here is complex and I really don't want to get into it - on short, the problem is not so much with the 'tracking'/building the profile, but with being confident enough that you're not targeting the wrong person/household member during the delivery of the more sensitive ads; and with not spooking/startling the user with "how the hell do they know this about me" - sometimes the user needs to be e.g. logged in so that he understands why he received an ad).
In particular, there are large swaths of companies that don't care whether it's you, your wife or your child - Disney e.g. will happily serve you ads based on activity performed by your children or wife, and will make no effort to figure out if it was you or them because it doesn't matter.
Its only good enough when you expect some non-trivial number of the ads to be wasted. If you care about that waste then IP is not good enough.
For some brands, like Disney in you example, spraying ads everywhere is good enough. For other brands that is not good enough, feminine hygiene products would be an example (both age and gender specific where IP alone would be a poor indicator).
How do you even define "non-wasted ads"? One ad = one buy? One ad = one interested person?
Targeted ads, even imperfectly-targeted ones, are "less wasted" than other kinds of ads. They are not perfect, by far (thankfully, if I may add so!), but "not good enough?" Come on, what exactly is good enough, then?
>Targeted ads, even imperfectly-targeted ones, are "less wasted" than other kinds of ads.
How do you know that your 'targeting' is working? If you target females how do you measure how many females saw the ad? Targeted ads should go to the targeted audience or they likely went to waste - at least the advertiser didn't get what they paid for. The quality of your targeted audiences matters to your advertising customers.
An IP address wouldn't be good enough to perform identification, but it would likely be good enough to perform reidentification. Most people on a network are not clearing their browser caches at the same exact time. The real power here is in using IP addresses in combination with other fingerprinting techniques.
> The real power here is in using IP addresses in combination with other fingerprinting techniques.
I didn't think that was anything new.
It also sounds like a possible path for exploit, kind of like not requiring a password when you call voicemail from your own phone (one could spoof your number as the caller id and access your voicemail without a password).
> I honestly didn't follow your Google search/wifi example.
Sorry, wasn't clear there. I generally don't think the 'match' thresholds need to be that high for effective (in terms of dollars earned vs spent) targeted ads.
In the google example: if a friend comes over a searches for designer watches odds are the whole IP/home is a decent enough target for designer watches.
The big conclusion to be drawn from this paper is that IP address tracking can be combined with client-side tracking techniques to perform reidentification.
Most users will at the very least use two IP addresses - home broadband and mobile SIM broadband.
Then you have wifi hotspots, friends wifi. The average user uses many IP's and not limited to the range of one ISP.
SO whilst you can fingerprint devices and usage patterns, the IP address will by itself be useless to identify such users, it may well augment a little but is no solution.
But then IPv4 shares many IP addresses across mobile and broadband users in various ways. Most do not have a fixed IP and even those that do, do not have a fixed IP upon their mobile data activities - unless they VPN into their home broadband. Though if they use service VPN offerings, then another layer of IP ranges.
So the potential towards false assertions based upon an IP and user usage may well trip up and fail. Imagine using an ISP with dynamic IP and the next user of that IP uses it for crime, well with some bad logging and aggressive association, you can mislabel somebody for a crime they did not commit.
Roll on IPv6 and with that, mobile carriers would of been the obvious benefit of that, yet I'm not aware of any progressing that in any timely manner and chug along using a pool of IPv4 and various tricks to make those cater for many.
However in this paper the authors tried to show is it's stability over time. So overall their "findings" indicate ip addresses should not be overlooked in privacy protection.s they stated as follows:
"... Over time, a
same device communicates with our server using a set of distinct IP
addresses, but we find that devices reuse some of their previous IP
addresses for long periods of time. We call this IP address retention."
IPv6 improves this situation (now). At first, ipv6 was actually a lot worse, since the back 1/2 of your address was your MAC address, allowing your device to be tracked around the internet no matter where it went.
People quickly realized this flaw, and updated the standard so that basically your client gets to pick the second 1/2 of your address now. And the nice thing is, most major platforms will actually run multiple addresses in parallel, allowing new connections to use a new address while old connections keep using the old one.
So while ipv4 adds some protection by having multiple clients behind the firewall, ipv6 actually makes it better by looking like even more clients behind the firewall.
Combined with a browser that blocks fingerprinting you get slightly better privacy with ipv6.
Absolutely not.
Most ISPs will allocate you a fixed /64. You may well have privacy IPs in this /64, the prefix will always be the same..
A though day for privacy activists
Sure, just like with an IPv4 they allocate you a fixed /32.
But you get slightly more privacy by having the client able to randomize the other 1/2 of the address and use multiple addresses, which would confuse trackers. Or the trackers just look at the first /64 and ignore the rest and you're no worse off than you were with your ipV4 /32.
You are worse off as now you have some introspection into computers on the other side of that firewall; with NAT you could have thousands of computers and they would all get melded together as one, but unless you very carefully generate a new IP address for every single connection you make (which is how you can get back to where you were with NAT), you now have the ability to somewhat differentiate users who before would have been mixed. So it is at best the same but probably worse, at which point why not use the thing that is always at least as good if not much better?...
Sounds like we need shorter IPv6 leases and more rotation between the prefixes, but that somehow goes a bit against what IPv6 should provide us - freedom to hold on to an address.
Some systems already have a solution for this, since devices can have multiple IPv6 addresses at once. They have one permanent IPv6 address which is not used for outgoing connections but can be used for incoming connections, and then temporary IPv6 addresses used for outgoing connections which can be rotated arbitrarily often. The first address is permanent but can only be used if you already know it.
An improvement on this would for services to request the machine to hold a specific address, permitted if the address is valid on the current network and not already in use. Then services could each have their own address (generated on first use and then requested when they start) which is effectively permanent but not the same as the addresses used by other services on the same machine.
It's a wash, I think. Anyone who cares uses VPNs and/or Tor. You can do either IPv4 or IPv6 with VPNs, but the methodology is different. With IPv4, it's just NAT. With IPv6, VPN servers must assign a public address. And with Tor, sites only see the exit address.
>just like with an IPv4 they allocate you a fixed /32
Many (most?) ISPs that give you unique IPv4 addresses also use dynamic pools. Just reconnect your router and blend into the pool. Or, if you're behind NAT, you're already indistinguishable from others.
My router is connected for months on end, and I think this is the norm, not an exception. Even if I reconnect, most of the time my provider assigns me the same IP. The only solution here is a good VPN.
When I first installed my connection, provider forcefully disconnected my router every 23 (yes, 23) hours to prevent me from having static ip (and hosting servers on fast home connection). Now it seems my public ipv4 actually changes every several minutes, probably some kind of ipv6to4 gate. But ipv6 of one computer is unchanged since several years.
not really.
the case where someone in a household power-cycles the router and it gets a new ipv6-addy and then one has to power-cycle all networked devices too because they have no way of knowing that the old prefix is dead, is very common.
To add on this,many will allocate a much smaller prefix. If you are being tracked by IP, the whois of your v6 address will contain what prefix is delegated,but even if they don't use whois they can simply track "subnets" (prefixes) as a data point, if a cookie or browser fingerprint is seen on multiple addresses, the address part will help pin point a specific user.
With v4 you have NAT and CGNAT now that obscures your public address a bit, you lose that with v6. For privacy, you need your address to be shared by many other users. This is one reason VPNs are useful, your public IP is shared by many. It's the same problem with browser fingerprinting, the more you try to prevent it, the more trackable data points you create (e.g.: one more extension or one less default setting that makes you a few steps more unique than everyone else).
I didn’t get your description of IPv6 offering better privacy protection.
With IPv6, your ISP could give you one address for each of your devices for life without any grudges or pain, assuming it doesn’t mind losing out on the static IP add on pricing that some charge in the scarce IPv4 space. That would enable better long term tracking without additional tokens (like cookies) and fingerprinting. We’ll never run out of IPv6 addresses.
Your ISP only assigned you the first 1/2 of the address. You get to pick the second 1/2 yourself. Your client can (and in most cases will) constantly switch up the second 1/2 of the address.
Which means that trackers will still track you, but only based on the first 1/2 of your IPv6 address. Since that doesn't change (often) it's essentially as fixed as your old /32 IPv4 address was.
Except now you are potentially leaking information about the individual computers in the lower 64 bits; if you generate those randomly for every single connection then you can mitigate that, but is that how this is actually being implemented or are they just doing periodic cycling? With NAT you were just guaranteed that this would always be safe.
> With NAT you were just guaranteed that this would always be safe.
Another safety benefit of NAT is that if you use a private IP address block (ex 192.168/16) you can be reasonably certain that your devices aren't publicly routable without bothering to actually check the router firewall configuration. This is particularly nice for home networks - I don't usually want my home devices to be reachable from the internet at large.
To understand what I'm getting at here, imagine trying to walk an elderly relative through verifying their firewall configuration over the phone. A simple command to check the IP address was enough with NAT.
In mobile networks I've only ever seen one UE get one /64, meaning most IPv6 users in the world does not share their /64 with anyone else, if I'm not very mistaken. I think this is standardized in 3GPP but I won't dig for it.
Other sites should get a /56, including home users. That's in RFC 6177. This is a best practice RFC of "shoulds" rather than "musts" so technically it's still IPv6 if your ISP only give you a /64 for your site... but it's wrong.
Your LAN devices (if you only have one LAN in your site, typical for a home) could still all be in the same /64. But the difference between one home and one person is small in the context of tracking for purposes of advertisement and besides, the trackers get extra bits per device which were hidden when the same devices were behind one IPv4 NAT. Poorly hidden, as all devices probably had different user-agents or resolutions and a hundred other things to fingerprint anyway but still, slightly less accessible.
That seems backwards: with NAT you couldn't identify all of the individual computers on my Internet connection; but now, with IPv6, you either can at worst (as every device has its own IP address that it reuses) or, at best (generating a new address for every single connection), are just getting yourself back to where you were with NAT. I appreciate that for a while IPv6 was actively much worse as it allowed address correlation across multiple networks (due to the MAC address being the same in the lower 64 bits), but fixing that doesn't make it better than NAT: if you mostly care about privacy, it still seems to make the most sense to use NAT if at all possible.
There are also several other methods that often work, e.g. the X-Forwarded-For HTTP header if there is a local proxy, or a long list of non-HTTP protocols or tunnel-XYZ-inside-HTTP protocols that have the client IP as part of the protocol.
Temporary IPv6 addresses actually solve this, especially for P2P systems that do benefit from knowing the client IP in case there is a peer on the same LAN, because the address they get is the same one the remote server sees (no additional information) and then it's both a valid local address but hard to correlate with anything when every device can have hundreds of them at once.
Unless you have many thousands of devices on your network, I can't see it actually having any practical advantage. I have maybe a dozen devices on my network, that is going to require only a tiny amount of entropy to uniquely identify the devices, in conjunction with the first half of the address. You wouldn't even need an entire UA string in many cases. Resist fingerprinting != prevent fingerprinting.
The resist fingerprinting measures in browsers are intended to help you blend in with millions of other devices. You'll still likely stand out like a sore thumb if the sample size is your household.
> so that basically your client gets to pick the second 1/2 of your address now. And the nice thing is, most major platforms will actually run multiple addresses in parallel, allowing new connections to use a new address while old connections keep using the old one.
This is really interesting and I was not aware of this. Are there any links where I can read about this more? I tried searching around, but did not find anything.
I worked on a document a few years back on anonymizing IP addresses. If you find yourself in a situation where you need to balance anonymization of IP addresses with research needs, this paper may be useful.
IP is just one more data point. There are already so many ways a browser can be fingerprinted, it doesn't make things that much worse.
While you can limit your exposure a bit, I long ago reached the conclusion that strong privacy is impossible in the current client/server web model. There is too much surface area.
61 comments
[ 2.8 ms ] story [ 70.0 ms ] thread"In this paper, we study the stability of the public IP addresses a user device uses to communicate with our server. Over time, a same device communicates with our server using a set of distinct IP addresses, but we find that devices reuse some of their previous IP addresses for long periods of time. We call this IP address retention and, the duration for which an IP address is retained by a device, is named the IP address retention period. We present an analysis of 34,488 unique public IP addresses collected from 2,230 users over a period of 111 days and we show that IP addresses remain a prime vector for online tracking. 87 % of participants retain at least one IP address for more than a month and 45 % of ISPs in our dataset allow keeping the same IP address for more than 30 days."
In some ways the fact that IP only tracking is lossy but still good enough is something to be afraid of. It is easy to 'taint' an IP and your households behind-the-scenes profile over a google search by your guest using your wifi.
I honestly didn't follow your Google search/wifi example.
and your spouse's typed in search queries
and your playstation
and your smart coffee machine
and alexa actually is
thats all without getting me started on the analytics networks in other apps sharing with the same data brokers between apps
Define "not good enough"(for who is it not good enough? for advertisers?).
I used to work on a project linking cookies together in anonymous profiles (for advertising), initially the plan was to separate household/individual profiles using different heuristics, but I think eventually it turned out that nobody cares that much - the advertisers just wanted a rough cross-device profile, not perfect accuracy. I mean sure, for marketing purposes (as well as engineering/ "performance" reasons) you needed figures to brag about accuracy and whatnot. But it's really really hard to get those figures right, and ultimately the thing that will convince advertisers is "using cross-device profile improved conversion rate by 10%"; everything else is a detail in comparison.
The paper/study here used a combination of IP address and a UUID to track 'users' which are presumably but not necessarily individual humans. It was the use of this UUID that allowed the authors to draw conclusions about how often a given user is seen at the same IP address. The IP address by itself did not provide enough tracking to draw the conclusions in the paper.
Is IP address identification 'good enough' for advertising, maybe. But IP is no where near definitive for identifying individual humans.
The paper does not say how often they found multiple 'users' using the same IP address. The paper focuses on individual reuse of IPs. How often did they find different users using the same IP in their study?
In internet advertising no one expects every ad to trigger a sale. Since there is some amount of acceptable waste in the system this paper has decided that IP is 'good enough' for targeting - because its okay if we are throwing away 20% or more of the ads (we didn't expect any traction from them anyway seems to be the thinking).
And the authors admit that a larger or different dataset might show different results.
> Advertising that is relevant to my wife probably isn't relevant to me, using IP is not good enough.
Advertisers do not care about what you as a human do in the same way that e.g. some government agencies might. Both the paper and you talked about "user tracking" in the context of advertising. My claim is that "IP address identification is actually 'good enough' for advertising" (for some kinds of advertising, at least, but the discussion here is complex and I really don't want to get into it - on short, the problem is not so much with the 'tracking'/building the profile, but with being confident enough that you're not targeting the wrong person/household member during the delivery of the more sensitive ads; and with not spooking/startling the user with "how the hell do they know this about me" - sometimes the user needs to be e.g. logged in so that he understands why he received an ad).
In particular, there are large swaths of companies that don't care whether it's you, your wife or your child - Disney e.g. will happily serve you ads based on activity performed by your children or wife, and will make no effort to figure out if it was you or them because it doesn't matter.
For some brands, like Disney in you example, spraying ads everywhere is good enough. For other brands that is not good enough, feminine hygiene products would be an example (both age and gender specific where IP alone would be a poor indicator).
Targeted ads, even imperfectly-targeted ones, are "less wasted" than other kinds of ads. They are not perfect, by far (thankfully, if I may add so!), but "not good enough?" Come on, what exactly is good enough, then?
How do you know that your 'targeting' is working? If you target females how do you measure how many females saw the ad? Targeted ads should go to the targeted audience or they likely went to waste - at least the advertiser didn't get what they paid for. The quality of your targeted audiences matters to your advertising customers.
I didn't think that was anything new.
It also sounds like a possible path for exploit, kind of like not requiring a password when you call voicemail from your own phone (one could spoof your number as the caller id and access your voicemail without a password).
Sorry, wasn't clear there. I generally don't think the 'match' thresholds need to be that high for effective (in terms of dollars earned vs spent) targeted ads.
In the google example: if a friend comes over a searches for designer watches odds are the whole IP/home is a decent enough target for designer watches.
Then you have wifi hotspots, friends wifi. The average user uses many IP's and not limited to the range of one ISP.
SO whilst you can fingerprint devices and usage patterns, the IP address will by itself be useless to identify such users, it may well augment a little but is no solution.
But then IPv4 shares many IP addresses across mobile and broadband users in various ways. Most do not have a fixed IP and even those that do, do not have a fixed IP upon their mobile data activities - unless they VPN into their home broadband. Though if they use service VPN offerings, then another layer of IP ranges.
So the potential towards false assertions based upon an IP and user usage may well trip up and fail. Imagine using an ISP with dynamic IP and the next user of that IP uses it for crime, well with some bad logging and aggressive association, you can mislabel somebody for a crime they did not commit.
Roll on IPv6 and with that, mobile carriers would of been the obvious benefit of that, yet I'm not aware of any progressing that in any timely manner and chug along using a pool of IPv4 and various tricks to make those cater for many.
"... Over time, a same device communicates with our server using a set of distinct IP addresses, but we find that devices reuse some of their previous IP addresses for long periods of time. We call this IP address retention."
People quickly realized this flaw, and updated the standard so that basically your client gets to pick the second 1/2 of your address now. And the nice thing is, most major platforms will actually run multiple addresses in parallel, allowing new connections to use a new address while old connections keep using the old one.
So while ipv4 adds some protection by having multiple clients behind the firewall, ipv6 actually makes it better by looking like even more clients behind the firewall.
Combined with a browser that blocks fingerprinting you get slightly better privacy with ipv6.
its a larger identifying token serving less users, so no.
But you get slightly more privacy by having the client able to randomize the other 1/2 of the address and use multiple addresses, which would confuse trackers. Or the trackers just look at the first /64 and ignore the rest and you're no worse off than you were with your ipV4 /32.
An improvement on this would for services to request the machine to hold a specific address, permitted if the address is valid on the current network and not already in use. Then services could each have their own address (generated on first use and then requested when they start) which is effectively permanent but not the same as the addresses used by other services on the same machine.
Many (most?) ISPs that give you unique IPv4 addresses also use dynamic pools. Just reconnect your router and blend into the pool. Or, if you're behind NAT, you're already indistinguishable from others.
not really. the case where someone in a household power-cycles the router and it gets a new ipv6-addy and then one has to power-cycle all networked devices too because they have no way of knowing that the old prefix is dead, is very common.
With v4 you have NAT and CGNAT now that obscures your public address a bit, you lose that with v6. For privacy, you need your address to be shared by many other users. This is one reason VPNs are useful, your public IP is shared by many. It's the same problem with browser fingerprinting, the more you try to prevent it, the more trackable data points you create (e.g.: one more extension or one less default setting that makes you a few steps more unique than everyone else).
With IPv6, your ISP could give you one address for each of your devices for life without any grudges or pain, assuming it doesn’t mind losing out on the static IP add on pricing that some charge in the scarce IPv4 space. That would enable better long term tracking without additional tokens (like cookies) and fingerprinting. We’ll never run out of IPv6 addresses.
But then in reality one IPv4 /32 is often many users due to NAT. Doesn't that make privacy better with v4?
Another safety benefit of NAT is that if you use a private IP address block (ex 192.168/16) you can be reasonably certain that your devices aren't publicly routable without bothering to actually check the router firewall configuration. This is particularly nice for home networks - I don't usually want my home devices to be reachable from the internet at large.
To understand what I'm getting at here, imagine trying to walk an elderly relative through verifying their firewall configuration over the phone. A simple command to check the IP address was enough with NAT.
Other sites should get a /56, including home users. That's in RFC 6177. This is a best practice RFC of "shoulds" rather than "musts" so technically it's still IPv6 if your ISP only give you a /64 for your site... but it's wrong.
Your LAN devices (if you only have one LAN in your site, typical for a home) could still all be in the same /64. But the difference between one home and one person is small in the context of tracking for purposes of advertisement and besides, the trackers get extra bits per device which were hidden when the same devices were behind one IPv4 NAT. Poorly hidden, as all devices probably had different user-agents or resolutions and a hundred other things to fingerprint anyway but still, slightly less accessible.
Not if you're using WebRTC which would promptly leak the private IP address: https://news.ycombinator.com/item?id=12528184
Temporary IPv6 addresses actually solve this, especially for P2P systems that do benefit from knowing the client IP in case there is a peer on the same LAN, because the address they get is the same one the remote server sees (no additional information) and then it's both a valid local address but hard to correlate with anything when every device can have hundreds of them at once.
The resist fingerprinting measures in browsers are intended to help you blend in with millions of other devices. You'll still likely stand out like a sore thumb if the sample size is your household.
This is really interesting and I was not aware of this. Are there any links where I can read about this more? I tried searching around, but did not find anything.
As noted, Mac OS, Windows and Linux already implement this.
[1]https://en.wikipedia.org/wiki/IPv6_address#Temporary_address...
[2]https://tools.ietf.org/html/rfc4941
https://www.icann.org/en/system/files/files/rssac-040-07aug1...
While you can limit your exposure a bit, I long ago reached the conclusion that strong privacy is impossible in the current client/server web model. There is too much surface area.