Particularly in a topic specifically about a product competing with google. Like, GOOGLE! It's almost impossible not to be more privacy respecting / enforcing.
Its a US company, you have to trust them on their words no logs are being shared with NSA etc.
Also seems cloudflare is becoming one of the bigger nodes that if cloudflare does go down.
It might shut down big sections of the internet.
> The concept of a “visit” is key to this approach. Rather than count unique IP addresses, which would require storing state about what each visitor does, we can simply count the number of page views that come from a different site. This provides a perfectly usable metric that doesn’t compromise on privacy.
Say that again please? Is this tracking unique Referrer’s? What if a significant amount of traffic is direct?
Assuming here we mean individual "sessions", eg. session cookies which are removed after you navigate away from the site. Will have to read their JS to see what they're doing.
Yes, that explanation is a little bit baffling. Referrers are not very reliable (increasingly missing). Maybe visits will just not be very accurate with this.
Citation on "increasingly missing" please. I tried finding out more about that for security reasons (a customer of ours checks the referer for anti-CSRF purposes and it seems to work perfectly fine in any browser so long as you don't do something like https->http posting) but from what I saw it was actually quite reliable and add-ons that block it are rather rare even among privacy what privacy nuts generally recommend to install.
I would assume a direct access with no referrer is also considered a "visit". I think the main point here is subsequent clicks around the site with the referrer from the current site do not increment the "visit" counter.
> it's not like you give them much more information than they already have.
My thoughts exactly. I'm hoping that we'll see a move from Google Analytics towards more privacy friendly services as they improve.
Then again, I'm not sure marketing people will want to move away from Google Analytics. And I don't think marketing people will read news from cloudflare.. so we'll have to educate them.
A move away from google to a new private entity offering seemingly friendly service? That reminds me of why I installed Google Analytics in the first place.
What's the point?
You could go with good log parsing programs like awstats.
Options to purge analytics data and visitor logs would also be handy to give us more control over our users’ privacy, as well as the ability to transparently see how much of our user data you have collected and for how long it will be retained.
And while I have your ear, please consider supporting web servers hosted on non-default ports, and the setting of custom Host headers behind your reverse proxy. (in self-serve - hosting on privileged ports shouldn’t be a requirement of using your service for both security and IPv4 exhaustion reasons)
Love Cloudflare. It’s good to see your team self-aware about privacy concerns and working to mitigate some.
Can you elaborate on the "custom Host headers" ask?
Are you looking to rewrite the Host header as the request gets passed to the origin (without changing the resolution of the origin)? Or something more like domain fronting?
An example or two would be great so I can share with the team.
A custom Host header passed to the origin that differs from the actual Cloudflare-hosted subdomain, without changing its DNS resolution - correct. This would be helpful when the origin is hosted by an inflexible third party, or as a time saver to quickly switch between environments without needing to modify other parts of the infrastructure.
Example 1: A third-party service does not support custom domains, only responds to "Host: my-company.saas-helpdesk.com". For a clean user experience, I want "support.my-domain.com" to reverse proxy to "my-company.saas-helpdesk.com" while rewriting the Host header to their origin as "Host: my-company.saas-helpdesk.com". (I realize CSP headers and all sorts of pain might happen here - perhaps not the best example - but would be great flexibility to have around)
Example 2: My startup's origin only responds to "Host: beta.mydomain.com", and the developer with access to its code is on vacation. I want to release my product. I point "www.mydomain.com" to the IP hosting "beta.mydomain.com" but since that origin does not respond to "Host: www.mydomain.com" - it does not work. I need a Host header rewrite.
Unfortunately we (Cloudflare) can't allow Host header rewrites like this as they could be used to bypass other customers' Cloudflare security settings. However, both of your examples can be done with Workers. I wrote a more detailed explanation a while back here: https://news.ycombinator.com/item?id=25058579
So it's okay for users savvy-enough to write Workers to bypass other customers' security settings - but not for self-serve point-and-click customers to do the same. Interesting. I suppose users of Workers are more likely to have a verified payment method and thus are less likely to be engaged in suspicious behavior. Thanks for the link.
No no no, you misunderstood. The whole post I linked you to was about why Workers does not allow forged Host headers.
What Workers does allow is for you to send a request to a different hostname. In this case, the Host header matches the DNS lookup. So if the third party is on Cloudflare, the request will go through their Cloudflare security settings.
It also allows, via the `resolveOverride` option, for you to do DNS lookup that doesn't match the Host header, as long as the Host header specifies a hostname within your domain. You can't forge a Host header specifying someone else's domain.
Cloudflare is moving at an eye-watering pace and I can only imagine it's a testament to the environment you've helped build. I'm amazed at your product velocity and I have to say thanks as a customer, observer and shareholder :)
Can you have multiple trackers set up? I get the same tracking code for each domain I try every time I set it up, which isn't great for keeping relations between websites private.
The token makes it seem like it's per-user rather than a general one everyone gets.
Thank you for providing a great service. Hosting on the web was super stressful in the past, with having to worry about script kiddies disagreeing with you and causing your bandwidth bill to soar.
Small issue; when clicking the link you provided I just get redirected to my accounts dashboard.
Edit: There was a link to web analytics in the right pane, and that worked for me
It’s currently broken. I have a free account and wanted to have a look at it. First it didn’t show at all in my dashboard, after a while it showed up, and I was able to complete onboarding and look at the (empty) results page. When I then refreshed it, I got 404 and now the web analytics button is gone again.
I'm an existing Cloudflare (free) customer. If I click that link it just takes me to my list of sites. If I choose a site I see the normal analytics I've always seen with he prompt to upgrade to pro. Is there a way to sign up if you already have a site on CloudFlare?
As much as I applaud every alternative to Google services, I would think about these kind of analytics twice when running any kind of developer-centric service. Chance is that your target audience is running an adblocker / DNS blocker of some sort and that this will heavily skew the analytics results: The absolute numbers will be a lot lower and you will only capture folks not savvy enough for adblockers. I removed any external analytics about a year ago when I realized that more than 50% of visitors to my service were blocking Google Analytics and that GA numbers had become meaningless (cross-referenced against access.log).
One of the local news stations uses them on their website. They have a tiny "continue without supporting us" link that blends in nicely with the text around it. The weird thing is I always see that link while my wife never does. I don't mean she can't find it; it simply isn't there. This is using Chrome on Android. No root or anything special. Doesn't matter if her phone is on wifi or mobile. On the other hand, I see that link every time. It's been that way for about a year and I have never been able to figure it out -- although admittedly, I haven't tried that hard.
Getadmiral.com didn’t even get past DNS blacklists, so not sure what exactly you’re reporting: Blocked by 1Hosts (Lite), ABPVN List, Anudeep's Blacklist, Energized Blu, hostsVN, Lightswitch05 - Ads & Tracking and oisd.
Tech-related sites can have 40%+ desktop adblocking. Mobile blocking is usually <10% no matter the vertical. The less tech-heavy the site is, the less adblocking but it's usually at least 10% on desktop.
Also, on the pricing page there's a "Get Free Analytics" button that you should be able to click. If you have any issues you can let me know: james at getadmiral.
Admiral's paywall is so glitchy on mobile, when I don't have an ad blocker installed but it frequently thinks I do, that I've learned recoil in disgust from any brand that uses your service.
That's unfortunate! Usually when this happens the user has a network filter, enterprise firewall, or something else on the network that's blocking some traffic. Publisher's have an opportunity through our platform to offer subscriptions as an alternative to whitelisting so you can view the site without ads. They can also issue you a free subscription.
Products like Admiral are why the Internet sucks. Everybody treating the Internet like their own little capitalist fiefdom consuming everything you do so you can be repackaged and resold a million million times.
I'm old enough to remember an internet that respected the sacred open ungated plain of knowledge. Publishers lost the privilege to monetize when they insisted on tracking, whether the visitor paid or not.
This relates to an article I wrote and shared here recently about marketers being obsessed with bad data - ie looking at GA even though the data in GA is heavily skewed and potentially entirely misleading due to issues with adblocking, etc.
Keep in mind that the Internet actually has more bot traffic than human traffic these days. A 50% discrepancy between Google Analytics and access logs could be explained entirely by bot traffic with no ad-blockers involved.
It could be ad-blockers, though. And I want to emphasize a point you mentioned briefly: it very much depends on your target audience. An anecdote from several years ago, around with IE was still the plurality of traffic on most sites but no longer the majority. Two web properties run by the same company, with different audiences. On one of them, IE did not crack the Top 5 in browsers, and was actually outnumbered by Opera. On another, the entire Top 5 browsers were different versions of IE, and Chrome and Firefox combined were less popular than IE6, IE5.5, and AOL's browser (which apparently was a reskinned IE7).
The moral of the story is, what's true for everyone may not be true for you, and vice-versa.
The vast majority of bot traffic identifies itself though. Search engines from all over that you've never heard of, but they're all showing accurate user agents. I don't attribute every access by far, of course, but for a small website (with peaks to high HN ranks, so it's not unknown or that badly linked) it's manageable to tail -f the access log and see that the traffic you're getting matches what you expect after e.g. posting a link in a chat group. About 99% of traffic in such a case fall within the categories "human-looking user agent visiting this newly created URL that I just shared in the group" and "random bot traffic".
The vast majority of what I guess is "robotic, but javascript-running" traffic identifies itself as the (headless) browser it most likely uses, i.e. Chrome.
When such crawlers use a distributed network of machines to crawl, there's no easy way to identify them as such, other than _guess_ that they're robotic in nature due to them coming from "data centre" like IPs (i.e. AWS, Azure, GCP, waht-have-you).
While such requests may well be users using those "data centre" services for, say, a personal VPN... the overall pattern is what gives them away.
Not all users look at a page and then start requesting (maybe from a separate IP, mind you, but still from the same provider yet with the same UA) each URL in that page in succession.
Crawlers who want to stay under the radar are much, much harder to detect as such nowadays.
En masse, with a popular website, sure. But from my limited traffic website where I sometimes look at individual requests by hand and see that it's my friends from residential Dutch IP addresses loading the page, I'm quite sure these are no secret masqueraded internet robots. Hence I conclude that, since much of the traffic (in periods that I look at randomly, the vast majority I never look at) is identifiable as human or as clearly a crawler, such unidentifiable bot traffic is a very tiny percentage.
I'm sure it'll be there, lots of people will be interested in what differences a page shows from different vantage points or to robots, but it's probably less than a percent of all robot traffic.
Though checking the thread above mine, I'm not exactly sure what I was replying to. Didn't it say something about robot traffic that wasn't identifiable as such? I don't see that anymore.
Fundamentally I am much more concerned about blocking tracking that actual advertising. So I have my blocker set based on that. I'm pretty sure GA gets swept up as part of that.
Streaming structured JSON access logs from nginx into an ElasticSearch via logagent-js, and then query via kibana. But goaccess is fine if you just want quick aggregate information!
Privacy-first analytics is a contradiction in terms. Do not spy on people. Do not build software that spies on people. It's that easy. This project is despicable, and this company is despicable.
"An analytics product that doesn't track people" is nonsensical. It's like "water that isn't wet".
Importantly, real trust on the internet is built on math and logic, not on faith. And faith, in this case, in a corporate institution which has spent years expending its fullest effort to direct as much of the flow of internet traffic through its servers as possible. I am never going to take you at your word that you won't write any of this data down, I promise, for real, no subpoenas or subversion going on here, no sir!
And yes, I know who you are, and that it's no use convincing you that the company that you built is a net evil on this world.
Because it does track people. Cloudflare tracks people. Cloudflare excludes people. It is it's very purpose for existing as you well know. These feel good marketing campaigns and services you do are just to centralize more of the internet into cloudflare and that's a bad thing.
What happens to "visits" in Brave or other browsers that strip the Referer header in top-level navigations [1]? Also, what about direct traffic?
They shouldn't even try to report "visits" if it won't be accurate because now they're just going to have a bunch of confused publishers asking why the numbers are so different than GA.
I've just integrated it with my pet project at varten.com. It's a very smooth experience, and data points started to show up on the Cloudflare dashboard within a few minutes.
One minor downside: it doesn't work on clients with JavaScript turned off.
Thanks! If you do end up "orange clouding" and proxying your site through Cloudflare, we can show you lots of stats about your website, even from clients that don't support JS :)
I occasionally spend time going through the analytics section inside the dashboard. I suppose those are data points gathered by your edge servers (?). If true - how is that different from what is launched today?
what launched today allows you to use it even if you don't proxy your traffic, hell even if you don't have your nameserver pointed to cloudflare at all.
Again, not analytics. Traffic stats. Blurring meaning here again, packaging data lying around from CF ops. Same as a stats counter from 2001. Veiled comparision to Google, who's in a different category for the most part - they're more like 'business analytics' where ('privacy' aside) you can see how users surf, improve UX, make decisions to help goals.
Nice to see those numbers, but in addition to that, Why not show us the page load time. Come-on Cloudflare!! This is definitely being collected from our browsers!
This looks compelling - I've been wanting a simple way to track traffic volume on my static site. I don't have ads or anything and don't want to be creepy.
Basically just want to know roughly how many people are reading my blog posts and if any old posts are getting big upticks in traffic.
Still only lets you look at week long slices, not a competitor to GA.
Also browser/device/viewing environment info is a key piece of GA and not something anybody has replicated as well. (the ability to intuitively drill down and compare is second to none with GA. you have to create segments in omo witch is a like 10 click long process with 3 processing pauses vs clicking on `windows` to drill down by family to os version or clicking on a referrer to see the pages they refer traffic to by visit)
I want to know which pages are performing the best but for the life of me I can't find anything to do with pages. Even just a table of "page | views" would do it.
Am I just being completely incompetent? This feels like it should be a front and centre stats table somewhere
e: oh I was still on the old stats, same bug others reported.
Side note I don't believe for a second my personal blog has had 500 unique visits in the past 24h. I've never really trusted Cloudflare's ability to count visits properly, gotta be miscounting assets as unique visits or showing me Google bots or something there.
Cloudflare offers a ton of free services in hopes you get attached to it and like it enough to try out their paid stuff.
So far Cloudflare cdn in my experience is much faster than google or Amazon. Google cloud cdn really disappoints me. Cloudflare v8 isolate based edge workers are pretty phenomenal too.
120 comments
[ 3.1 ms ] story [ 170 ms ] threadNo need. Cookies are irrelevant, tracking is. Cookieless tracking already falls under the GDPR. This doesn’t seem to track users so it’s fine.
It's just generally very narrowly interpreted as 'if you use cookies you need to plaster your site in a ridiculous banner and options menu'.
EDIT: Wait, you're the Cloudflare CEO and you don't know about Cloudbleed?
Say that again please? Is this tracking unique Referrer’s? What if a significant amount of traffic is direct?
I've been using Simpleanalytics, which is pretty good overall, but they keep messing with their site and breaking things...
Also, if your site is already behind CF, it's not like you give them much more information than they already have.
My thoughts exactly. I'm hoping that we'll see a move from Google Analytics towards more privacy friendly services as they improve.
Then again, I'm not sure marketing people will want to move away from Google Analytics. And I don't think marketing people will read news from cloudflare.. so we'll have to educate them.
What's the point?
You could go with good log parsing programs like awstats.
I think it's a good solution for small side projects when you don't need very accurate analytics.
Marketers sadly don't seem to give a shit about that.
The cookie banner is required due to the ePrivacy law which predates GDPR.
GDPR consent would still be needed in what concerns personal managements and the reasons for collecting, storing and processing said information.
Options to purge analytics data and visitor logs would also be handy to give us more control over our users’ privacy, as well as the ability to transparently see how much of our user data you have collected and for how long it will be retained.
And while I have your ear, please consider supporting web servers hosted on non-default ports, and the setting of custom Host headers behind your reverse proxy. (in self-serve - hosting on privileged ports shouldn’t be a requirement of using your service for both security and IPv4 exhaustion reasons)
Love Cloudflare. It’s good to see your team self-aware about privacy concerns and working to mitigate some.
Are you looking to rewrite the Host header as the request gets passed to the origin (without changing the resolution of the origin)? Or something more like domain fronting?
An example or two would be great so I can share with the team.
Example 1: A third-party service does not support custom domains, only responds to "Host: my-company.saas-helpdesk.com". For a clean user experience, I want "support.my-domain.com" to reverse proxy to "my-company.saas-helpdesk.com" while rewriting the Host header to their origin as "Host: my-company.saas-helpdesk.com". (I realize CSP headers and all sorts of pain might happen here - perhaps not the best example - but would be great flexibility to have around)
Example 2: My startup's origin only responds to "Host: beta.mydomain.com", and the developer with access to its code is on vacation. I want to release my product. I point "www.mydomain.com" to the IP hosting "beta.mydomain.com" but since that origin does not respond to "Host: www.mydomain.com" - it does not work. I need a Host header rewrite.
Thank you for your time!
What Workers does allow is for you to send a request to a different hostname. In this case, the Host header matches the DNS lookup. So if the third party is on Cloudflare, the request will go through their Cloudflare security settings.
It also allows, via the `resolveOverride` option, for you to do DNS lookup that doesn't match the Host header, as long as the Host header specifies a hostname within your domain. You can't forge a Host header specifying someone else's domain.
The token makes it seem like it's per-user rather than a general one everyone gets.
At launch, we just support one token per account. We are already working on supporting multiple analytics tokens in one account though!
Small issue; when clicking the link you provided I just get redirected to my accounts dashboard.
Edit: There was a link to web analytics in the right pane, and that worked for me
(We actually did have a replication issue where some folks in the EU had trouble accessing this page)
Tired of this broken all green 'status pages'. :(
Sorry to hear that. We appear to have a client-side caching bug (ironic!)
Can you try refreshing on the 404 page and see if you have access now?
[1] https://getadmiral.com
(Full disclosure: I'm a co-founder at Admiral)
Any plans to make the "continue without disabling" button harder to click in the future?
But I’m curious - do you have any off the cuff stats on what is typical as far as % of users with Adblock across various types of audiences?
Also, on the pricing page there's a "Get Free Analytics" button that you should be able to click. If you have any issues you can let me know: james at getadmiral.
I'm old enough to remember an internet that respected the sacred open ungated plain of knowledge. Publishers lost the privilege to monetize when they insisted on tracking, whether the visitor paid or not.
How do you sleep at night?
https://news.ycombinator.com/item?id=25016532
It could be ad-blockers, though. And I want to emphasize a point you mentioned briefly: it very much depends on your target audience. An anecdote from several years ago, around with IE was still the plurality of traffic on most sites but no longer the majority. Two web properties run by the same company, with different audiences. On one of them, IE did not crack the Top 5 in browsers, and was actually outnumbered by Opera. On another, the entire Top 5 browsers were different versions of IE, and Chrome and Firefox combined were less popular than IE6, IE5.5, and AOL's browser (which apparently was a reskinned IE7).
The moral of the story is, what's true for everyone may not be true for you, and vice-versa.
When such crawlers use a distributed network of machines to crawl, there's no easy way to identify them as such, other than _guess_ that they're robotic in nature due to them coming from "data centre" like IPs (i.e. AWS, Azure, GCP, waht-have-you).
While such requests may well be users using those "data centre" services for, say, a personal VPN... the overall pattern is what gives them away.
Not all users look at a page and then start requesting (maybe from a separate IP, mind you, but still from the same provider yet with the same UA) each URL in that page in succession.
Crawlers who want to stay under the radar are much, much harder to detect as such nowadays.
I'm sure it'll be there, lots of people will be interested in what differences a page shows from different vantage points or to robots, but it's probably less than a percent of all robot traffic.
Though checking the thread above mine, I'm not exactly sure what I was replying to. Didn't it say something about robot traffic that wasn't identifiable as such? I don't see that anymore.
Importantly, real trust on the internet is built on math and logic, not on faith. And faith, in this case, in a corporate institution which has spent years expending its fullest effort to direct as much of the flow of internet traffic through its servers as possible. I am never going to take you at your word that you won't write any of this data down, I promise, for real, no subpoenas or subversion going on here, no sir!
And yes, I know who you are, and that it's no use convincing you that the company that you built is a net evil on this world.
https://news.ycombinator.com/newsguidelines.html
No, that's not true. And stop claiming you know what I think or believe.
They shouldn't even try to report "visits" if it won't be accurate because now they're just going to have a bunch of confused publishers asking why the numbers are so different than GA.
[1] https://www.cookiestatus.com/brave/#referrer
Thanks! If you do end up "orange clouding" and proxying your site through Cloudflare, we can show you lots of stats about your website, even from clients that don't support JS :)
I occasionally spend time going through the analytics section inside the dashboard. I suppose those are data points gathered by your edge servers (?). If true - how is that different from what is launched today?
That Google Analytics offers more in depth features is irrelevant for most people.
I’m disappointed.
I just need an analytics solution that somewhat guarantees I do or don’t need to use a cookie banner.
I need to know how many people visit my website without bothering them with annoying popups.
You always get what you pay for!
Basically just want to know roughly how many people are reading my blog posts and if any old posts are getting big upticks in traffic.
Hopefully this fills that need!
Particularly looking to add a CNAME record in my DNS to point to the script's domain and have it effectively served from a subdomain of my site.
Also browser/device/viewing environment info is a key piece of GA and not something anybody has replicated as well. (the ability to intuitively drill down and compare is second to none with GA. you have to create segments in omo witch is a like 10 click long process with 3 processing pauses vs clicking on `windows` to drill down by family to os version or clicking on a referrer to see the pages they refer traffic to by visit)
Am I just being completely incompetent? This feels like it should be a front and centre stats table somewhere
e: oh I was still on the old stats, same bug others reported.
Side note I don't believe for a second my personal blog has had 500 unique visits in the past 24h. I've never really trusted Cloudflare's ability to count visits properly, gotta be miscounting assets as unique visits or showing me Google bots or something there.
Is it only an entry-point to attract people on other paid services?
So far Cloudflare cdn in my experience is much faster than google or Amazon. Google cloud cdn really disappoints me. Cloudflare v8 isolate based edge workers are pretty phenomenal too.