88 comments

[ 2.4 ms ] story [ 123 ms ] thread
This article uses the terms "visit" and "pageview" in unusualy ways.

They give the example that when a user clicks through from yourwebsite.com/ to yourwebsite.com/page then the first is a unique visit while the second is a non-unique visit.

I would neither call a "visit" but rather a "pageview".

In my book, the visit is what started from the first pageview on yourwebsite.com site and continues to the last pageview.

I haven't touched analytics in 15 years, but back then what you describe for visit was called a session. That seems less ambiguous to me.
Author here. It's technically more correct to call those visits "unique page views" and "page views". I think visit is a very simple word to understand for most people. My tool is targeted at people who want a simple tool, that's why I want to keep the wording also simple.
How we used these words 20 years ago, when server logs were cool:

Uniques are visitors not pageviews. Hits, page views, and sessions are decreasingly granular content stats, not people stats. Visits and visitors are people stats. Sessions tend to be the matrix of page views (containing multiple hits) per visitor visit.

The same user will trigger many unique visits if coming from, for example, google search
This is true and that's a trade-off for not using fingerprinting or cookies.
I am wondering how they will try to compensate when the browser is actively blocking referrer information. In that case the data will be polluted like the website has received more unique visitors, but it is not the actual.

In earlier times, this kind of tracking was the norm. Big Analytic companies tried to distill this information and add value to the existing data to make sense. So the cookies and fingerprinting were added to distill that data.

Given you want discount internal site traffic from being unique, you could add a query param to URLs as the user navigates.
I think a more robust (and privacy-preserving) way to calculate uniques ( per time-frame as well) is to make use of the local storage / indexDB in the visitor's browser.

This is how it would work:

- Visitor X lands on your website.

- With some JavaScript you check local storage if `dailyUnique` for today's date is set.

- if yes, send `visit` signal

- if not set, send `unique visit` signal and set `dailyUnique` to local storage.

You can apply this to any analytics use case. Its is private and does not rely on referrers. We have been using this goal-attainment approach to do analytics at my company for quite some years.

Author here. Our tool is GDPR compliant out of the box. We can't use local storage (it's legal wise the same as a cookie). Although your suggestion will provide more accurate stats, it's a trade-off we are willing to accept.
How about referring from a https site? AFAIK, browsers don't send a referrer from s https site.
If the referring site is http and the site being referred to it won't send the referrer header.

http --> https = no referrer header

https --> https = referrer header

Actually:

https --> http = no referrer header

That is not really relevant because if a user comes from a different website and the referrer is not present, it will be counted as unique. Having a referrer of that other website or having no referrer does not change anything. We both treat them as unique.

The only referrer we actually use is the one from the same site to the same site. Those requests are always with the same protocol (and if you not you should change that). In that case we see both the hostname and the referrer are equal thus being non unique.

> it's legal wise the same as a cookie

I am not sure that is the case. GDPR makes provisions for personal data that can uniquely identify users. Blanket statements like: Local storage is not allowed I think are misleading. The state is persisted in the client's machine. Unlike cookies, which get attached to all requests in the specified path, local storage items are not transmitted with the request. Furthermore, in the approach I recommended earlier, no unique identifiers are being sent with the request at all. I am pretty sure that is GDPR compliant, but would love to be pointed to legal provisions that would suggest otherwise.

> it's a trade-off we are willing to accept.

Referrers, in my opinion, are not reliable enough to derive uniques, and I would assume (although I would not have any numbers to back it up), that the margin of error is very significant when you consider every condition under which referrers would not be sent (some very good cases when that happens are mentioned by other people in this very thread)

Functionally, local storage is the same as cookies with the only real difference being it requires an active request. Any information stored can immediately moved to and from the browser to the server with fetch or whatever. "Uniquely identifying users" would work with local storage the same as a cookie.
It could be moved, but until you actually do it it’s not tracking but merely logging.
The same is true for cookies.
But cookies are automatically sent with every request to your server. So you would need to take special care to ensure they aren’t sent until you have permission.
(comment deleted)
How about using quickly expiring cookies to tell if visit is unique (no cookie) or same (cookie exist).

It would get reset if external referer is recognized.

Setting a short expiring cookie is still setting a cookie.
Just like setting a cookie that I didn't agree for setting other cookies :)
How is this different from using a cookie, from a privacy perspective?
There's no difference between a cookie and local storage -- that's what a cookie is. But there is a difference between using an identifier of 2019-12-14 vs an identifier of 175c6328-0699-4577-8364-3db5071c2173.
Would this pass GDPR without a cookie notice?
> ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;

> ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction;

As far as the English text of the regulation goes, it's clear that the generation, detection, and use of this record count as "processing", as long as the record itself is "personal data".

It's not clear that this is the case. A record stating "this browser has visited XXXX website today" does nothing, in the absence of other records, to identify the person providing the record. But this is open to some interpretation. In particular, you might be getting this record from web requests that already identify the user by other means (perhaps they're logged in). In that case, someone could argue that the fact of the user having visited (or not) your site before on the same day is data that pertains to them specifically ("personal data"), and that your making note of it is prohibited by default under article 6 of the GDPR.

The counterargument would be that when the data is reified in your use and your records, it has already become impossible to relate to any individual person.

You'd have to rely on that counterargument, because article 6 won't help you at all:

> 1. Processing shall be lawful only if and to the extent that at least one of the following applies:

> (a) the data subject has given consent to the processing of his or her personal data for one or more specific purposes;

> (b) processing is necessary for the performance of a contract to which the data subject is party or in order to take steps at the request of the data subject prior to entering into a contract;

> (c) processing is necessary for compliance with a legal obligation to which the controller is subject;

> (d) processing is necessary in order to protect the vital interests of the data subject or of another natural person;

> (e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller;

> (f) processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child.

> Point (f) of the first subparagraph shall not apply to processing carried out by public authorities in the performance of their tasks.

None of these will apply unless you are an agent of the government.

-----

Thought experiment: as a hostile website operator, you decide to attack your users by filling their local storage. You generate random bytes and store them under random keys to the limit of what their browser will allow. You don't yourself know what the keys are.

Is this a GDPR violation? Those random bytes are highly entropic identifiers which you processed and associated with individual users.

Sorry, wouldn't work.

Directive 2009/136/EC (aka EU Cookie Law) is not limited to "cookies". The law itself doesn't even mention the word "cookie"!

It covers localStorage and any similar features that may come along in the future.

The GDPR law concerns data which can be used to identify / track a person. It does mention cookies once! Quote:

> Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.

This would not apply to the above method since it is not an identifier.

The problem is, there is another regulation called the ePrivacy directive (from 2009, the original 'cookie law') that states you need user consent before setting any kind of cookies other than the strictly necessary. In the privacy dialog, these are allowed to be pre-selected, unlike identifying cookies, but you still need to acquire consent explicitly.

Not a lawyer, this is not legal advice.

Very excited by the title, but very disappointed by the content...

This is not true unique visit counting. Which can be achieved using non unique values being set in browser storage, which is privacy friendly in my eyes.

I agree with you. It's privacy friendly to have a cookie stored on the browser with a boolean only. Unfortunately it's not allowed by GDPR without asking for consent first. Would be open to other ways without cookies and fingerprinting. I think there is no other way.
So even if it’s obvious that there is no unique ID in there, you still need to show the notice?

We really need to lobby for a smarter law.

I'm just glad someone is focusing on obeying the spirit of the law rather than tacking an ironic "we value your privacy" popup to the website and continue selling my data.
Yeah, I always read these popups as “we value your privacy, therefore we take it away from you and sell it to the highest bidder” and hit the back button.
There's nothing "unique" about those visits, just using the referrer isn't enough to make them unique.
Yeah, I had to keep re-reading it to make sure I didn't miss something clever.

The "trick" is that they use an unrelated metric instead? Heh.

It's always quasi-unique, because people might use some privacy protection measures that skew the results. This is less precise but more privacy friendly (for those who care).
The author's comment post discusses how the analytics are going to be flawed, but is emphasizing that the trends in the numbers over time can be very useful, even when we know the exact numbers are not precise.

I think it's a great trade-off to respect user privacy while removing annoying popup garbage from the user's screen about consent. Aren't we all sick and tired of these consent buttons everywhere online now???

The article says

> When a user lands on your website without visiting another website (direct visit) we will record it as a non unique visit

But then the image below says the contrary

> This will be tracked as a unique visit as the current website name is different from referer (not available).

Which is it?

Anyhow, this seems like this approach unfortunately makes the "unique visits" data completely wrong, since users might tend to visit your website multiple times a day.

Thank you. It was a typo, it's now fixed.

Unique visits data is never 100% accurate. What if a user switches from device or browser?

Well, your site could require authentication. 100% accurate unique visits data isn't really a good reason to do that, but it is a side effect if you already do.
Author here. Just to shine some light on the problem we are having and how this solves that problem. We get more and more customers that are required by law to have an active opt-in for cookies and fingerprinting. They basically can't use those techniques before a visitor actually ticks a checkbox (which has to unchecked by default).

This results in a lot of missed data. They can't run Google Analytics for the visitors that don't check this box. For them it's a big issue because they can't see how their visitors are behaving before checking that box.

Simple Analytics wants to fix this issue by not requiring consent. Customers can use our tool for every visitor that lands on their website.

One other thing that is a common misconception is that unique visits with cookies are accurate. They also have flaws:

- What if a user uses a different browser

- What if a user uses a different device

- What if a user blocks (third-party) cookies

The same goes for using IP based fingerprinting. All techniques have flaws. Although they are flawed they are used to get a number. It's not an accurate number but it's a number. That's also the case with using our technique. It's less accurate then using cookies, yes, but it's a number you can use. You can compare it to a previous period, you can see the unique views of a page, etc.

If you don't need to comply with privacy regulations you can use all the tools you want. We just focus on the companies who do care about those regulations and give them the big picture they are now missing.

> They can't run Google Analytics for the visitors that don't check this box

I don’t understand. They can. Just anonymize the ip. You can track all their activities. Am I missing something?

Google Analytics tracks your visitors with a cookie [1]. You need consent for placing that cookie under the GDPR and other privacy regulations.

[1] https://developers.google.com/analytics/devguides/collection...

I don’t think GDPR necessarily says that one must get consent for every cookie.
Google analytics does use first party cookies for the site stats, but it also s̶e̶t̶s̶ [edit] can set a third party DoubleClick cookie that tracks users across sites. So that anonymizing the IP doesn't anonymize the user.

There's also optional functionality that can track users via the first party cookie. Passing login id, for example, to GA.

    it also sets a third party DoubleClick cookie
I have never witnessed that. Can you link to a site that uses only Google Analytics where that happens?

Maybe you confused Analytics and Adsense?

"For customers that are using Google Analytics' Display Advertiser features, such as remarketing, a third-party DoubleClick cookie is used in addition to the other cookies"

So, I suppose I should have said "can set", though remarketing is very common.

https://developers.google.com/analytics/devguides/collection...

I would think that enabling the "Display advertisers features" only makes sense for webmasters who also use Google Adsense. And then their visitors get bombarded with all kinds of cookies anyhow. So I don't think it applies to the use case at hand. Where a webmaster wants to implement statistics without violating GDPR.
It's used for Adwords. That doesn't change anything on your site. You can remarket only on Google searches, for example...no AdSense involved.
It’s not a double click cookie, but if you have multiple sites under the same analytics account google will be able to track cross site sessions via a cookie they set to their own domain.
Not in a default configuration
There is much confusion as there is both the Directive 2009/136/EC (aka EU Cookie Law) AND GDPR.

EU mandated that all websites (residing in EU) to obtain informed consent before they can store or retrieve information on a visitor's computer or web-enabled device.

This is not limited to "cookies". The law doesn't even mention the word "cookie". It covers localStorage and any TBD technology in the future.

There are exceptions for "strictly necessary" like a store would not have to get consent to operate a shopping cart on the website as thats can be considered critical to the purchasing and checking out. However storing what you browsed for recommendation purposes is not, and therefore that would require consent.

On the Subject of GDPR,

Are there any website still blocking EU visitors or has that been solved?

I remember when GDPR was introduce lots of website simply decide to shut off EU IP access and redirect them to a page saying not available to EU.

I think a number of US news sites (e.g. LA Times) still block EU traffic.

Furthermore, GDPR has caused a massive decrease in the number of newly registered domains that are successfully recognized as spam:

> Prior to the implementation of GDPR, security researchers were able to identify and block 1.8 million newly registered malicious domains in October of 2017 alone. Fast forward to February of 2019 and that number drops to less than 160,000.[0]

And it has caused a major annoyance for the general public, who mindlessly consent to almost every "consent" prompt they are given.[1]

Companies are also at the mercy of their regional government when it comes to compliance. A company in Greece was fined 150,000 euros, not because the law made it illegal to process data in the way they did, but because the reason they provided for processing was the "wrong" one.[2]

> PWC asked its employees for permission to process their personal data when it should have used a different legal basis (combination of contract, legal obligations and legitimate interest). [(2)]

In effect, their effort to follow the law made them break the law, even though their actions were legal under Article 6, Section 1 of the GDPR.[3] This flies in the face of the EU's language in 2018, where they suggested that companies attempt to follow the "spirit of the law" rather than to worry about dotting i's and crossing t's.

Oddly enough, regulators seem to be pleased with this outcome of seemingly arbitrary enforcement.

[0]: http://www.circleid.com/posts/20191213_the_high_cost_of_priv...

[1]: https://www.cnbc.com/2019/05/04/gdpr-has-frustrated-users-an...

[2]: https://iapp.org/news/a/just-say-yes-gdpr-consent-is-not-as-...

[3]: https://gdpr-info.eu/art-6-gdpr/

GDPR is not a cookie law. You don’t need consent to run out-of-the-box GA with IP anonymization.
You most certainly do, even if it might not yet be prosecuted.

See article 6 1/a-f for lawfulness of processing.

Without consent there only remain select few conditions, none of which apply to the operation of GA for visitors without a legal contract with the site operator on a different level (ie. customer relationship).

It is only a matter of time..

I'd not label that as "certainly", quite a few lawyers disagree about 1f not being applicable for basic, ano-/pseudonymized analytics.
There is much confusion as there is both the Directive 2009/136/EC (aka EU Cookie Law) AND GDPR

EU mandated that all websites (residing in EU) to obtain informed consent before they can store or retrieve information on a visitor's computer or web-enabled device.

This is not limited to "cookies". The law doesn't even mention the word "cookie". It covers localStorage and any TBD technology in the future.

There are exceptions for "strictly necessary" like a store would not have to get consent to operate a shopping cart on the website as thats can be considered critical to the purchasing and checking out. However storing what you browsed for recommendation purposes is not, and therefore that would require consent.

> You need consent for placing that cookie under the GDPR

Do you have a reference for this? It not how I understand GDPR, and I am not clear which part of it would apply here when personally identifying information is not being tracked.

"Personal data which have undergone pseudonymisation, which could be attributed to a natural person by the use of additional information should be considered to be information on an identifiable natural person."

GA pseudoanonymizes visitor data, but the does not exempt site operators from adhering to the consent mechanisms of the GDPR.

But the cookie is not "Personal data which have undergone pseudonymisation", is it? Assuming we're talking about the case where no PII is being sent along with the tracking data (which I believe is the point of masking the IP address, among other protections) how could this be attributed to a natural person?

What I'm trying to understand is why the addition of an opaque cookie value necessarily changes the situation, such that consent is required.

It's very possible I'm missing something here; genuinely trying to learn what that is.

Have you considered using localStorage? It’s not sent across the wire (I’m sure you know), and as described you’re counting sessions not unique users.
“Simple Analytics wants to fix this issue by not requiring consent.”

Well that doesn’t sound incredibly sinister or anything.

A user can land your page from multiple referrers (or the same) more than once and some search engines including Google will hide the referrer information so the calculation will be off IMO.

The regulations are not strict if you're not identifying the individual people from your visitor/user data. If you're collecting user event data into your servers and can prove that you're using the data just for the analytics purposes (i.e. providing a better user experience to your users) there is nothing you can afraid of using cookies.

That may not be the case for third party analytics providers though because if you're using Google Analytics, you're responsible from how Google are using your customer data.

As I understand it, this is the method Webalizer has been using to calculate unique visitors for over two decades now. It is mindboggling that we have come full circle and this is now a business model again.
It's one of the methods we used when doing logfile analytics in the early 2000s. At the time it just appeared obvious to me. One problem with it (which is why we combined it with other techniques too) is that some browsers even then did not send Referer headers reliably, neither between pages on the same site nor between third party sites, so what they say in the article about "direct visit" is not reliable.
This explains that obnoxious URL google has for results. To be able to fully ensure they know who referred you. I use DDG at home but Google at work.
> some browsers even then did not send Referer headers reliably

Apart from https/http thing mentioned elsewhere in this thread, there's also Referrer-Policy [1]. I don't know how wide-spread its use is, but I find that most requests I get have no referrer header these days.

[1] https://developer.mozilla.org/en-US/docs/Web/Security/Refere...

Completely true, but I'm fine with this being posted because I do think people need reminding of how much we can do without Google tracking.
You could just add a unique value to every link between pages?
This way a user is still unique when he/she closes the tab and reopens, right?
What if they share the link on social? All the new traffic would seem to be non-unique.
PWA 1px image on top with cache-first show single visits. A cookie without a cookie
Creative - I love this solution
with cache expire time set to midnight you also can get daily users
> This referrer is very useful to figure out where traffic is coming from.

This is exactly why sending the referrer header is insane[1]. The header betrays the user's privacy by design.

Remove it ASAP. If your app breaks, that's unfortunate but necessary.

[1] https://news.ycombinator.com/item?id=20468115

Necessary for whom? Privacy purists seem to be surprised that website and app owners want/need analytics to meaningfully improve their app's usability. I praise SA's privacy oriented approach as a meaningful alternative to GA.

This sort of overly-critical hyperbole doesn't help the privacy conversation. If follow SA loses a feature it likely needs to remain competitive, and customers would have another reason to go back to Google Analytics. Hardly a necessary privacy win a my book.

> Necessary for whom?

Necessary for anyone interested in preserving democracy[1]. Unless we actively fight to preserve privacy, surveillance capitalism will continue to "disrupt" traditional forms of power.

> If follow SA loses a feature it likely needs to remain competitive, and customers would have another reason to go back to Google Analytics.

Continuing the betrayal of user privacy in the hope of appeasing the surveillance capitalists won't be "[privacy] for our time". Advertisers and other eavesdroppers have consistently demonstrated they will maximally exploit any tool available to them.

[1] http://nymag.com/intelligencer/2019/02/shoshana-zuboff-q-and...

> Privacy purists seem to be surprised that website and app owners want/need analytics to meaningfully improve their app's usability.

Both "privacy purists" and many other people see this as expediency at best, and a lie at worst (i.e. once marketing gets wind of the tracking built for development purposes; see e.g. recent Gitlab fiasco).

> Other analytics businesses use this technique (for example based on IP address). This seems privacy friendly but is considered fingerprinting. For which you need consent.

Do you need consent to count unique IPs that visit your site in the UK under GDPR? Aren't there laws that say unique IPs do not correspond directly to an individual (in the context of piracy), and yet for simple website analytics you count IP as uniquely identifying a user? Am I required by law to change the default config of apache or nginx to not log the IPs? Or is it enough if I promise not to analyze the logs?

(comment deleted)
It really doesn't solve any problems surrounding web analytics, user privacy or user experience. I think it shows the creators lack of industry experience in ecommerce, advertising and user tracking. It's like someone has sprinkled a flawed 20 year old method of tracking in glitter and saas'd it.

I think there is a very small subset of website owners that would be willing to pay for this. Then again he only needs to get a few people that aren't sales focused and are deluded enough to think this is the pinnacle of 'privacy'.

GDPR is just a mess. The bureaucrats who wrote it don't understand it, neither do the enforcers, or the folks under enforcement.

Brussels should just shut itself down.