There may be an argument that Chrome is marginally faster than Firefox on desktop, and while this may also technically be true on mobile, it's completely dwarfed by the fact that Chrome is being asked render 5x as much crap (with animations and sound and video and nagbars and ...) compared to Firefox mobile with uBlock Origin.
You will literally regain hours of battery life. Those Joules belong to you -- don't just shrug and hand them to Google.
I wish you could install umatrix, and it had a decent mobile UI. Umatrix has just transformed the way I browse the web and if anything it has educated me at the same time. I like seeing that this random site has about 2^5 different domain names contacted, ranging from ad-junk to CDNs. It tells me quite a lot about its developer. Sites with one hostname, minimally awful JS and few-to-none XHR requests get a thumbs up.
On the other hand, it probably means that I am a "false negative" in TFA's report. I'd love to know the correlation between the server logs and what Plausible shows for a connection. I'd also like to know how they infer OS -- e.g. for privacy reasons, my reported user agent is not accurate...
> I like seeing that this random site has about 2^5 different domain names contacted
You can see that in uBlock Origin already. The summary info has the number of domains connected and if you click on "more" you have the details, just like on desktop. You can also block JavaScript altogether for a particular site.
For Android, I'm using fennec with ublock origin and it's great. Also YouTube Vanced is very useful. And Infinity for a tolerable reddit experience on mobile.
Android offers Private DNS, just point that at nextdns.io and block things even within apps that aren't browsers.
The biggest benefit is definitely the sheer speed at which news websites load on my phone now, even when I'm on a crap connection.
Another benefit: This isn't a VPN, so you can still use a VPN whilst using Private DNS.
Edit: Also... lol, I can't access TFA as plausible.io is also blocked on my network and phone. I'm going to assume that the article is actually an analytics competitor whose pitch is "GA is blocked, use us instead!".
Edit 3: NextDNS stats from my home network: 11.45% of DNS requests blocked, but from my phone 23.7% of DNS requests blocked. You need this more on your mobile than in a web browser!
You don't have to do that... NextDNS ultimately uses Cloudflare DNS under the hood, so just point yourself at 1.1.1.1 and you're done but it will block nothing.
If you want a DNS server to block things, know that the definition of what to block is subjective and some people may disagree with what you want to block.
For that reason you get an account, which allows you to have a configure, and that needs to be resolve to you... so at some level an identifier of the configuration is needed. This is the id.
On Android I use personaldnsfilter (https://f-droid.org/en/packages/dnsfilter.android/), which creates a local VPN, there are also actual VPN apps that allow blocking ad hosts, and you can also use AdGuard DNS directly.
Out of curiosity, what does this bring you over hosts file blocking, and how much does it affect battery life? I use dead-simple modifications to /etc/hosts and it seems to be remarkably good. Do you MITM yourself for further adblocking, or just use it for the modified DNS?
My phone is not rooted, and I just switched to nextdns.io on suggestion of another commenter here. I can't say about the phone battery, but I don't think it makes a big difference.
NextDNS does comprehensive blackholing of tracking and ads, very limited battery impact.
Works on ios and android.
I think their free tier is very generous at 300k lookups/month; otherwise it's something cheap like 2€/month if you want unlimited lookups (i use this to block tracking on all my devices + home network)
I've had bad experiences, battery-wise, using on-device adblocking solutions (that inspect DOM). An alternative i tried was tunneling to my raspberry pi that ran pihole (dns blocker), and bouncing all my traffic off the home connection, but that was also battery intensive.
You're right, this dns solution is very unobtrusive when it comes to energy use.
If on iPhone, install NextDNS and enable a blocklist which blocks GA, flip the switch, now NextDNS handles your DNS and will answer requests to GA with a "sorry cannot find this".
Use Bromite, Vivaldi or Brave on Android. They have built-in ad tracker protection. The first two even support custom filtering lists.
On iOS the options are a bit worse but Brave has some kind of adblocking that works. Safari also supports blocking lists via AdGuard but it's more limited.
AdGuard (local app) works in Safari. I don't like safari, but the AdGuard blocking does not work in FireFox! I use DDG browser mostly and it works well. I do see the occasional ad.
The whole browser situation in iOS is really user unfriendly.
Firefox is another nice alternative if you have a good Android phone I guess.
Sadly, I have a quite old Android device (but its battery life it's still good) where Chromium-based browsers are already quite slow and Firefox feels even slower and drains the battery faster (the old Fennec was worse than the new one, it hanged for 30 seconds).
I surf in private mode. A super simple solution for all your tracking protection needs people. It boggles my mind how this comes up again and again yet we have private browsing since when? The 90s? Don’t be lazy - that’s the real reason, isn’t it? Having to login again..
No, we really did not have private browsing since the nineties. It's a new thing, for sure.
What we had in the nineties was that we could configure our browsers to ask for every single cookie that a site wanted to place on our computers. That got old really fast when internet advertising became a thing, and tracking cookies were all over the place.
> No, we really did not have private browsing since the nineties. It's a new thing, for sure.
Indeed! Seems the first browser to implement some sort of incognito/private/separated profile (specifically to hide your tracks, not general like Firefox Profiles) was Safari around the release of Mac OS X Tiger (April 29, 2005).
You do know that fingerprinting is a thing, right? Private browsing is designed to keep your local device clean of your tracks, not the wider web. At all.
Telling Facebook "hello, I am visiting baddragon.com from Mozilla Firefox 63.1 on Obscure Linux in Podunk, Saskatchewan, population 32" is barely an improvement over telling them "hello I am John Q. Smith and I am visiting baddragon.com".
And of course, private browsing is generally per-instance. Log in to your Google account in a private tab? All your other "private tabs" are also logged in to Google until you close the private browser.
There is a German movie about the system that is used to gather TV ratings. It's a special box that some users get which reports what they are watching. Small sample size goes into a big statistic (not sure how accurate the portrayal of the system in the movie is). These boxes are given to the people who pay the German public TV fee, which excludes i.e. students (they don't have to pay) and some other groups. This group of critical people figured that out and started to hack into these machines to fake ratings. They faked the ratings away from stupid trash TV towards some higher quality stuff, documentaries, culture, ...
Obviously in the movie then the country saw a renaissance, everyone got smarter, yadda yadda, you get it.
I feel like this is similar.
All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).
Maybe we should engage in large scale AdWords fraud. Send come fake traffic away from Facebook and over to Wikipedia.
What GP describes would be stronger than AdNauseum. Instead of sending clicks indiscriminately for every ad, you would send clicks for high quality content.
The idea is not so much of clicking on high quality content ads, but sending visits to high quality content pages and click on the ads of those pages, to get them more money. Would obviously stop working if the sales after the ad click are not made.
But, hey this isn't a gameplan, this was just a "what if" :D
Cannot agree more. Since the the advertisers have that much to spend, let's make them to spend more for nothing. Also by doing so, it will literally put enough noise into the data those trackers collect and renders the user profiling useless and effectively protecting us from being tracked.
If we only click ads on high quality contents, those content owner might benefit from it in short term but in long run it most likely going to back fire and make them penalized as for sure there is going to be counter measures. If we simply click every ad indiscriminately, there is no way to tell or they have to penalize everyone which almost equals to do nothing.
I think it wouldn't be about the ads, but the visit tracking. Like, block GA from seeing your visits on trash sites, but allow when it's a high quality post/content/source, so we skew the numbers for high quality content
This movie sounds great and I would love to watch it. It's a shame there's no legal way to watch many international movies like this if you are in the US.
I make a point of watching a non-US film on every international flight I take - I find it to be a unique opportunity to watch interesting non-US movies with English subs. I have discovered two excellent oddball comedies from in flight movies (Die Goldfische from Germany and Le Grand Partage from France), but when I tried to rewatch these films with others in the US, I discovered it was basically impossible. There must be so many other great movies from around the world we are missing out on.
> which excludes i.e. students (they don't have to pay)
That is incorrect, they do have to pay (once per household). But if you get BAFÖG (student loan/social benefit mix that require you and your parents to be below a certain income bracket), you don’t.
I know that very well, I had to pay it because there is no BAFÖG for foreign students. Just felt it might not be an important enough detail to explain BAFÖG to the international audience.
I can't quite understand this. By "exploiting" you simply mean targeting ads?
The real harm seems to be from the tech giants censoring speech and policing payments, but what's the harm that someone targets a pair of shorts that I might like or show an ad for a conference I might be interested in?
I.e. ads, facebooks feed, endless scrollers like 9gag -> they like to use dark patterns and exploit tricks against the human mind to keep and guide your attention.
The harm if targeted ads depends on your viewpoint.
A targeted ad might serve you something you were looking for anyway, or it might manipulate you into spending on something you don't actually need. I.e. look at Instagram influencers, showing off their fake perfect live, making the viewer feel small and then try to buy the same happiness by buying the same product.
At best, ads are information that you need, at worst, they use psychology to manipulate you.
Seriously, you're worried about someone spending more time on their phone or spending more money on goods and services? Spending their own money — not the public money? That's what freedom is to you? Not the freedom to talk to whoever you want, say whatever you want or pay whoever you want?
I have no clue where you derive any of these things from. I certainly never said any of it. I never even made a claim a out freedom.
I was talking about how ads, tv programming, trackers and such have a tendency to create a positive feedback loop, which leads people towards less quality and mindless consumption. And about a fun idea from a movie, to break this feedback loop and replace it with another one, that promotes higher quality content.
You then started talking about a different topic and are now accusing me of not being interested in free speech.
Some people don't want to be tracked or monitored by advertising companies and it should be enough to just say so without companies like Facebook always trying to sneak tracking back in via dark pattens, shadow profiles, etc, etc.
For example once you've seen a website offer you the same product for different prices based on arbitrary tracking it leaves a bad taste in your mouth.
From your answer and other people's answers I feel like everyone has been brainwashed to focus on minor things like this, and totally ignore the elephant in the room.
So Facebook and Google know your religion, politics and food preferences. So what? How can they harm you with this?
They harm you when they choose what you can say, who you can talk to and who you can do business with. But literally what's the harm of ads? Making your computer a bit slower? That's such a tiny issue it's not even worth talking about.
In England and Wales, all three types of fraud covered by the 2006 Act have as substantive elements of the offence a requirement that a defendant intends to either "make a gain for himself or another" or "cause loss to another or to expose another to a risk of loss", where gain and loss are defined as only referring to "money or other property".
Note also that the offence is framed in terms of intention to gain or cause loss. Even if no material gain or loss happens, the intention is what matters. (The equivalent mental element for theft in England is that one dishonestly intends to permanently deprive someone of the property being taken.)
This applies to online ad 'click fraud' in both ways—if a publisher fraudulently clicks on adverts to make money, that'd potentially be fraud to make a gain for themselves. If a competitor clicks on adverts to get their competitor to lose money on pay-per-click, that'd potentially be fraud to cause a loss to another.
I can't speak for jurisdictions other than England and Wales but I'd be surprised if a fair number of other jurisdictions didn't also define fraud in a way that covers both gain and loss scenarios.
It's not actual fraud. You're not bound by the terms of service of ad providers. Your device is actually sending requests to the ads, which they are counting themselves to determine payment rates. If it's fraud to send ad impressions to sites I like without visiting them, it could be fraud to deny ad impressions to sites that I do visit (with my ad blocker on.)
If the sites set it up themselves, it's fraud. If you conspire with the sites to set it up, it's fraud and conspiracy to commit fraud. But if I'm prosecuted for the crime of not actually looking at ads I request, that's just a judge with an agenda. Whether it went one way or another at every layer of appeal would probably be a coin flip, though.
Correct, fraud isn’t the correct term here. The op grabbed the first word that probably presented itself, and others nearly ad nauseam (lowercase) have discussed its merits as the term.
What’s needed is a better term.
I propose either AdTurfing (hat tip AstroTurfing) or AdLighting (hat tip GasLighting). My personal preference is the second.
Vandalism is, again a crime against property that is legitimately placed on the street or somewhere.
Ad biz is a business model, not property, and a predatory one as that. You are not obligated to enable someone else's business for free.
They are claiming their clicks and impressions mean something - its their problem to ensure they are accurate.
Imagine someone is doing a survey of sex habbits, and selling the results. Is it a fraud to lie? Ofcourse not, why should you be responsible for their profits.
The fact that people think they are legally obligated to enable this is really fucked up.
While legally we are probably far away from this being a fraud, nothing stops google from adding something to their TOS and banning your account on that basis... This is the only reason I'm not using those noise generators even though all ad-tech should burn in a trashcan fire in my opinion.
Is it the Simpsons or Seinfeld where in one episode one of the characters gets a Nielsen box that is used for measuring tv viewership and they can’t leave the house for fear of shows being canceled if they’re not home to watch it.
They did something similar in “Roseanne.” The Connors were selected to be a Nielsen family and Roseanne made the family watch nothing but PBS, documentaries, etc the whole time. She wanted to hack the ratings so that over time, regular folks like her family would get exposed to a better class of information.
Is this the fee you have to pay even if you are blind or deaf, or do not have a Radio or TV? Gives you the most boring TV News in the world, plus 600 movies per year nobody wants to watch, because they are horrible? But they collected the tax and they have to spend it, as the German actors guild is worst than a cartel? :-)
Not a big fan of the GIS either, but my stance is not as extreme. I see benefits, but there is problems as well. It could be a good system if there was a proper reform and if you get politicians out of the system.
> Is this the fee you have to pay even if you are blind or deaf [...]
That's not correct. If you are deaf, you get a reduction - if you are blind, you get a reduction. If you are deaf-blind, you don't pay at all. If you are already receiving financial help because of your blindness, you don't pay at all.
You are right about it, did that change recently? According to the reference below it depends on the degree of deafness or blindness...if you can still hear or see something still have to pay...
I personally prefer boring news backed by quality journalism and funded by people rather than sensational, outrageous and superficially controversial news! If I want the excitement I watch a drama or action movie. I think the news articles/pieces work like clickbait; the more head-turning they get, the more viewers they acquire which results in higher profits.
I am not talking about sensational or misleading like in FoxNews or CNN :-) I am talking about boring as in....very serious persons, on a serious background, reading very serious the Reuters or Ap news of the day. As the service was paid and it has to provided ;-) A better model would be instead having to provide good analysis or quality content otherwise your audience will go somewhere. As in for example, the FT or the Economist.
Actually thinking about it, the issue is wider than just the news and I think the financing mode of the mandatory tax is a big part of it. What is really a shame, as Germany has a rich culture of hundreds of years so great content in all forms should not be a problem.
You might comment the BBC financing model is similar and I would agree. I think the difference is that the BBC also embraced a highly commercial model of selling content like Top Gear and other stuff worldwide. In this case the English language content with its planetary audience, pushed for a more competitive/commercial model and the German TV stayed too insular in my view.
According to their numbers they netted $400,000 worth of Google, essentially for free. Mind, that hasn't been updated in close to a decade; nowadays their 819 shares would be worth about $2.4 million.
Don't modern internet-connected TV boxes snitch on what people are watching? Surely we don't need a sampling anymore?
Just thinking that the elderly population is probably the least likely to use those boxes (though I am not even sure of that), whereas they constitute the (dying) core of traditional TV viewership.
They could put an IR camera right next to the remote sensor. Since it's behind a glass that is opaque to visible-light you wouldn't even know the camera's there, and if you put tape on it your remote wouldn't work anymore.
>Since it's behind a glass that is opaque to visible-light you wouldn't even know the camera's there
If it's opaque to visible light how does the camera produce an image?
>if you put tape on it your remote wouldn't work anymore.
Not really? You just have to be more careful placing the tape. Besides, nowadays many smart TV have app remote controls, or RF-based remote controls so the IR sensor being blocked is a non-issue
> If it's opaque to visible light how does the camera produce an image?
It isn't opaque to infrared light though. The image is still high enough quality to make out people, it looks funny but it is good enough for their purposes.
Some sort of localised motion sensor would probably be sufficient to tell you how many people are watching without invading too much on people's privacy. You wouldn't know the age, but you already have ton of information by knowing the address and the subscriber already.
> Some sort of localised motion sensor would probably be sufficient to tell you how many people are watching
You'd also have pets captured, curtains flapping by an open window captured, and any toys kids are playing with (like balls) potentially captured as another viewer too.
Meanwhile pap who likes to sit motionlessly while the kids round around him, isn't detected.
> without invading too much on people's privacy
Motion detection feels like a pretty major privacy violation to me.
> You wouldn't know the age, but you already have ton of information by knowing the address and the subscriber already.
So I found out my neighbors were proud to be participating in some new Neilson study. Apparently they pay you to wear these boxes in your pocket that pick up sound. At the end of the day it syncs what it heard to their servers and you get points, which eventually translates to money.
Essentially the stated purpose is so that it can pick up what music you listen to throughout the day, but in reality it's picking up everything.
Modern TVs do snitch but there are two issues with relying on that:
1. There is a disparity between age groups and other demographic dividers who have newer TVs. This could significantly skew the results for some advertisers.
2. The data is going to the TV manufacturer, and they will not share that freely between themselves or with anyone else. This will complicate collating the data as there are several entities to negotiate with in order to get an overall picture.
Ironically, modern streaming apps have far more accurate numbers than anything Nielsen ever cooked up sampling people, but Netflix made the precedence that they should be "secret sauce" and not shared publicly and most of the "Streaming Wars" diaspora today are following that policy/precedent.
We're in something of a worst of both worlds situation where Nielsen has an increasingly small number of viewers where traditional TV boxes work to get decent samples, has to rely more than ever on surveys, and distrusts all streaming viewership numbers because they are cloak and dagger white lies between competitors, despite in theory being way more accurate than all the previous tools (the surveys and the TV boxes).
It's almost wild. The most forthcoming to shareholders/the general public over the years has been Hulu and Hulu's numbers at times have suggested Nielsen's data is very, very wrong right now, but Nielsen doesn't trust Hulu's data at all because it smells like lies because Netflix does nothing but lie or ghost them.
I have been exploring ways to avoid sending this data to them. To start: using an instance of Pi-Hole and running all your apps will remove some tracking from them.
Following this up with paying for your subscription but pirating all the shows might help remove you from the cycle completely.
I understand the second point is not realistic for the majority of people but I wonder if in the future we might have an easy to use version of Pi-Hole that most people can just flip on and strip a large chunk of tracking from the apps.
That just gets us back to the topic way above: if we aren't giving that tracking data to TV producers, it's tough to complain when our favorite nerdy/intelligent shows get cancelled.
That's the deep weird irony that we live in a world where we could have the best possible numbers (directly tracked statistics), and yet TV Producers are still relying (for the most part) on Phone and Snail Mail Survey Results because they don't think they can trust streaming provider numbers. Pi-Holing those numbers just gives those Producers even more reason to feel that they are lies or wrong. At what point do you Pi-Hole too many telemetrics to oblivion and aren't allowed to complain when your favorite TV shows get canceled because "no one" was watching it?
I guess this contributes to me watching less and less shows because I am tired of stuff being canceled midseason. It might be a self fulfilling prophecy. Less people watch these shows/appear to watch these shows, the networks stop producing it and these people don't come back to whatever the network does end up producing.
From what I am hearing, traditional forms of media are in decline because people are spending their time on Youtube/Twitch/Social media instead.
> They faked the ratings away from stupid trash TV towards some higher quality stuff, documentaries, culture, ... Obviously in the movie then the country saw a renaissance, everyone got smarter, yadda yadda, you get it.
> I feel like this is similar. All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).
I think this is right - ad-traffic is manipulative and actually I don't think it is a societal 'good' at all.
A few personal examples:
* On Youtube almost all my adverts are encouraging me to start Forex / Stock / Property investment and trading, and sign up for courses on these. These courses are scams (or at best, 'half-scams' and poor/generic advice repackaged and sold for thousands), and in general provide poor financial advice (either through extortionate courses, recommending you become too heavily leveraged or advising you to day-trade high-volatility stocks by just looking at charts). Presumably it does this because I am 32 and male, so I am considered 'prime' for this marketing.
* One of the friends I know is a girl, and she has never seen the above adverts. We were talking and she says every single advert is just about pregnancy and fertility. I wonder how many of these adverts are just reinforcing gender-stereotypes in a wider sense, i.e. while google claims to be progressive and care about 'equality' really is their business model at it's core really just targeting women and telling them that they should be getting pregnant, while telling guys that they should be the bread-winners and earn money via stocks/shares?
* While my adverts are for forex, and my apparently fertile friend is getting adverts for pregnancy tests, my older parents just get targeted adverts for pre-paid funerals. One or two are probably be fine, but they are just on constant repeat - and I can't help but think that I wouldn't the constant reminder of death before every youtube video.
* My laptop is convinced that I want to go camping. It's only my laptop, every advert is camping related. Sleeping bags, tents... and the strange thing is that when it started I didn't want to go camping, but it's been so consistent across the last few months now that I kinda wanna go camping. Like it's sold me this romantic vision which I know wasn't there before, so even though I would usually like to say I can't be manipulated through marketing, it's really made me realise I can be.
Is the above really making society better? And if it's not, why should we put up with it? IMO the biggest lie we have been told by Google is that 'personalised ads' are a good thing.
I agree on a personal level, although at a societal level I believe regulation is required.
As per the parent comment to my original one, I just fundamentally do not believe that most advertising contributes anything positive to society and mostly generates negative externalities.
Intuitively, I'd say it doesn't hold water: Entrenched incumbents would generally tend to have more money than new competitors, so the incumbent can afford more advertising to drown out that of the challenger.
The only net gain I can see is for the advertising industry, which extracts money from both challenger and incumbent.
> The advertiser has a tracker that it places on multiple sites and tracks me around. So it doesn't know what I bought, but it does know what I looked at, probably over a long period of time, across many sites. Using this information, its painstakingly trained AI makes conclusions about which other things I might want to look at, based on...
> ...well, based on what? …Probably what it does is infer my gender, age, income level, and marital status. After that, it sells me cars and gadgets if I'm a guy, and fashion if I'm a woman. Not because all guys like cars and gadgets, but because some very uncreative human got into the loop and said "please sell my car mostly to men" and "please sell my fashion items mostly to women."… You know this is how it works, right? It has to be. You can infer it from how bad the ads are. Anyone can, in a few seconds, think of some stuff they really want to buy which The Algorithm has failed to offer them, all while Outbrain makes zillions of dollars sending links about car insurance to non-car-owning Manhattanites. It might as well be a 1990s late-night TV infomercial, where all they knew for sure about my demographic profile is that I was still awake.
> You tracked me everywhere I go, logging it forever, begging for someone to steal your database, desperately fearing that some new EU privacy regulation might destroy your business... for this? [1]
As someone who has been in one or another meeting with German TV stations I can assure you this is not completely far fetched. The people deciding what is running at these stations are of the mindset (a translated quote):
> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them
and then they will serenade about how they would "love to have a bit more sophisticated things", but as they are the only ones who really understand their audience, they cannot allow this, although they support the values of the 68 generation etc. pp.
From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.
Well, you say that, but isn't that exactly why Netflix got rid of ratings?
All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.
It's the same for most content, I read a huge amount. I do read some intellectual books, but only occasionally. The rest of the time I read utter, thoroughly entertaining, trash.
I don't want to read about an existential crisis after programming all day, I want someone to hit something with a big sword and get the girl.
> It's the same for most content, I read a huge amount. I do read some intellectual books, but only occasionally. The rest of the time I read utter, thoroughly entertaining, trash.
I feel the same way. However I force myself to read things that will better me once in a while anyway. I too want to hit things with a big sword (without the pain of getting hit), and get the girl (without cheating on my wife), but the world including me is better if I do something else anyway. Which is why I do sometimes read a complex math book.
> Well, you say that, but isn't that exactly why Netflix got rid of ratings?
> All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.
This rather shows that ratings do work, but are used wrongly for giving recommendations:
If you want to create suggestions for a user, in many case the wrong answer is "suggest what has really high ratings", but rather "given the ratings that this user gave and the films that he watched, what will he also like."
The fact that these documentaries get high ratings (the same might hold for art house films) shows that there is some (niche?) audience which really loves this kind of films, but not that "John Doe" will love it, too.
I remember my professor talking about the ensemble approach the competitors were taking in a data mining class I took. I was saddened when Netflix ended up not using it because it was too slow and expensive or something.
I remember when that competition was released, I was working (as an undergrad) in GroupLens, which had MovieLens still around but it was getting a bit of bitrot.
It was always weird to me that GroupLens didn't spin up a team for it, but it seemed like everyone in the research group had moved on to other things and didn't want to context switch back. Someone mentioned something like "shame they didn't do this 10 years ago, a million dollars would have been nice". I think someone was doing tagging on movielens, but I don't remember the details.
I got the sense that neither Riedl nor Konstan (or any of the current grad students it seemed) wanted to pursue it (Terveen, I think, wasn't in to recommender systems at all in the first place).
I don't think the lab had any funding problems haha, so maybe that was what it came down to.
> This rather shows that ratings do work, but are used wrongly for giving recommendations
And I know I'm (possibly) a minority on hacker news, but I prefer the new system. I was giving everything I wanted to watch more 4 or 5, even when it was clearly not the case, but because I want recommendations of things I'm going to like AND actually watch
That is how it is with documentaries. Just because they are getting lower viewing numbers doesn't mean you shouldn't keep promoting them. Certainly there should be a mix of entertainment and public interest stuff, but following audience preferences for entertainment creates a feedback loop that damages society.
Things like documentaries are mostly watched by already well-off people (mostly middle class and up).
In Germany, public TV is paid for by (nearly?) every household. [0]
Forcing everyone, including poor people, to subsidize rich people's taste for documentaries seems a bit.. off?
Similar arguments apply to public libraries and opera houses, though at least there the financing is done mostly via progressive taxation.
Of course, you can argue that we sophisticated people know what's good for those unwashed masses, and if only they watched their documentaries like they are supposed to, they would soon see the light. Colour me skeptical.
[0] As far as I am concerned, private broadcasters can and should do what they feel like.
Well, when people complain about a growing wealth divide, and those who are doing okay financially say that childhood access to public libraries and documentaries made them who they are, shouldn't those things receive funding?
That's a bit like asking healthy people. You'll get answers ranging from things like exercise to homeopathy.
Answers from people can be used to suggest avenues for investigation, but shouldn't make you fund expensive stuff outright. (Assuming here that homeopathy is obvious nonsense, that people still swear by.)
And, of course, public libraries and documentaries don't have to just produce good effects. In order to justify public funding, you'd need to do a whole cost-benefit analysis and look at opportunity costs.
So look at what you get from public libraries vs libraries financed from private charity _plus_ whatever other good things the public money saved could do (including just outright giving it to poor people).
I deliberately picked public libraries here as a provocative examples. I suspect they might actually pass the cost-benefit test without too much contortion of metrics and data.
I am much less sanguine about the bang-for-buck of publicly financed opera houses, theatres and symphony orchestras, which Germany is quite fond of. And of course, publicly financed radio and TV broadcasters.
Without US public broadcasting, I and countless others would have grown up on GI Joe instead of Mr. Rogers.
Further, I generally strongly oppose charity as a component of future plans (leaving aside whether charity is good or necessary in the present). We should never be building society such that it depends on the funding whims of rich philanthropists.
Sure, there should be some kind of analysis of benefits, but some things simply have to exist for a society to be a society, because without them, the loss of their intangible and second-order benefits will cause a society to implode Idiocracy style, and nobody will know why.
On top of cost-benefit analysis, you will have to explain to everyone whose life trajectory was meaningfully improved by a resource what alternative path you are providing so that future people can also find their way to a better life.
Receiving education isn't about learning. At least not primary. It's about getting a piece of paper at the end.
Public libraries and free internet resources are one argument in this direction. Another: professors are usually more than happy for you to sit in on lectures, even if you don't pay any tuition.
People choose from what is presented to them. That's consumerism. It's not like people get to pick what gets produced. If more public interest material is available and advertised, it'll get watched more. The alternative is to watch less TV and engage with society directly more. Both of those outcomes would be preferable to the excessive production and consumption of entertainment.
Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces. Herman and Chomsky discuss this in Chapter 1 of Manufacturing Consent.
> The alternative is to watch less TV and engage with society directly more. Both of those outcomes would be preferable to the excessive production and consumption of entertainment.
I agree. And economically, if you want less of a good to be consumed, ceasing to subsidize its production with tax payer money is a good first step. If you want to go further, perhaps even tax its production.
> Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces.
How do market forces differ from public interest?
Or rather, what do you mean by 'public interest'? It's what the general public is interested in?
Market forces tend toward baseness. It takes active intent to elevate society above the lowest possible level. Just as one doesn't want to hire at the median skill level of their company lest the average continue to drop over time, a society has to aim higher than what the impulsive market optimizes for, or it will decay.
Revealed preference is just code for exploitation of vice.
When I was first living on my own barely making rent, PBS documentaries were the most interesting thing on broadcast TV. Infinitely more entertaining than drivel like the Bachelor. Other than available time and offered free content I doubt preference of documentaries is different among income classes.
Social class is correlated with income, but it's not the same.
Similarly, entertainment preferences are correlated with social class (and with income), but again, they are not perfect predictors of each other.
So a few anecdata wouldn't undermine anything here. Though in fact, your example actually strengthens the argument I am making: you are the kind of person that prefers watching documentaries over other drivel, and you are the kind of person who managed to get themselves out of poverty. That's likely because you have the preferences, habits and skills of someone who is at least middle class in a social sense, even if your income took a while to catch up.
(Keep in mind that we are talking about social class in a rather abstract fashion here. German middle class mores are different from American middle class mores.)
Wow, that is some overt classism right there. The world is a much better place is you just see people as people. Everybody is trying to achieve the same thing regardless of their "class". Security and safety for self and loved ones by whatever means are available to them with their skills and knowledge.
How many of these documentaries are actually public interest though? Versus propaganda by someone with an agenda to push under the guise of intellectualism?
Off course the so called "social issue" documentaries are nothing more than propaganda often created by think tanks or publicly funded "opinionators", these are very rarely not a waste of money and attention. It doesn't take away the fact that nature/science documentaries (if produced well, with sufficient funds) can capture imagination of a general knowledge of the layman public.
That is always the problem with data: it is reactive. Sure the average watcher is 65, and wants easy to understand stuff: that is what the data shows (I'll assume for discussion that is what the data shows, but I have no insight into if it is true or not). What the data doesn't show is if content would draw in day 25 year olds, they need several years of trying those other shows to see if it makes a difference - a very risky best that could run them out of business even if true (that is the older crowd stops watching faster than the younger crowd figured out it is worth watching meaning advertisers don't pay enough to keep producing content).
Of course TV in the US has figured out that the 65+ crowd is very valuable to customers (the advertisers, not viewers!), so even though they could get more viewers by not showing the nightly news, the nightly news is what they show.
This the same problem as interviewing people for a job nd collecting data about which “features” from interviews correlate with job performance… Without tracking the performance of any of the candidates who weren’t offered jobs—or turned down an offer.
Something, something, a diagram of a plane showing where the damage was on those that returned from missions.
> Something, something, a diagram of a plane showing where the damage was on those that returned from missions.
Ahahahah I love this dogwhistle. You always know you're in good company when someone waves their hands around and says "put the armor where there AREN'T bullet holes" and gives you a significant look. XD
Reminds me of a thread a while ago pointing out that a single show dominates the MTV programming lineup almost every day [1]. It's as if there is some algorithm running without human intervention that feeds on itself by reacting only to current viewing habits:
foreach (show s in lineup)
if (s.viewers > THRESHOLD)
lineup.replace_with_more (s)
Obviously resulting in this weird local maxima where no other shows get broadcast.
I don't have MTV so I had to look up what "Ridiculousness" is. Apparently they play Youtube/Tiktok/etc... clips? Sounds like it must be incredibly cheap to produce. This is what it looks like when you let a race to the bottom continue on indefinitely.
And the cable industry can't understand why people keep cutting the cord. Can you imagine shelling out $120/month to have some producers pick out Youtube clips for you?
I went to a big box bar/restaurant a couple years ago, and they had a big screen TV playing basically what you describe. I wonder if that is the audience, subscribers using is as essentially something in the background? It's hard to believe there are enough that want this to make it economically viable...
In the US there's actually an FCC requirement that television channels air news.
From https://www.fcc.gov/media/radio/public-and-broadcasting: "virtually every station has an obligation to provide news, public affairs, and other programming that specifically treats the important issues facing its community." The details are specific to the license, but almost every station is required to air at least an hour of news a day.
It's unclear, because the exact details are specific to the license and therefore may vary from station to station. My local television stations usually air the news at 11, which is after primetime.
This is precisely why the internet wins. It can both show the nightly news and a million other things at the same time, catering to a 60-year-old housewife, a 40-year-old car enthusiast, a 30-year old gardener, a 20-year-old fan of obscure Linux distributions and a Taiwanese 10-year-old kid living in Ireland, all at the same time. Youtube regularly recommends videos that I personally like, but that onlyhave a couple thousand views.
On the other-- we have a network that bills itself as educational and spends over a month marketing a full week of programming dedicated to propagandizing its audience to be maximally afraid of sharks.
It broadcast a wildly popular movie where a tornado full of sharks attacks a city.
German TV could be an order of magnitude worse than my parody and it still wouldn't even register on the American scale of stupidity.
It reminds me of the (possibly apocryphal) reasoning behind the cancellation of "Police Squad" back in the day: that people would have to pay attention to get the jokes.
The '68 generation' is pretty much the same generation that's now 65 years old and washing dishes..
> From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.
Well, that would be more bearable, if half the TV market wouldn't be allowed to essentially tax everyone to finance their drivel.
I stopped watching TV more than 10 years ago because I was worried to turn stupid from it. Not so far fetched after all.
There are some high quality shows and TV stations though. Namely Phoenix (similar to PBS in the U.S.) and some of the news magazines that run in the late evenings. Of course there are also all the other public stations with higher quality programs but I find the program most of the time quite random and sometimes even a bit elitarian.
> > Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them
I haven't watched german TV in ages, but I distinctly remember science shows degrading from science to thinly veiled ads - things like literally running a companies marketing video or making a "scientific comparison" where they hand out random style points at the end to make a specific product win. I think they even got into trouble over it since ads and science/education shows are taxed differently. Anyone pretending that they are doing that for their viewers is living in denial at best, but probably just outright lying.
> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them
> because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid
So basically they have a Hacker News mindset!
In every thread about a dumbed down GUI/website it is argued that granny wouldn't understand it otherwise. No power user allowed, because data shows user is monkey.
There is the Nielsen system in the US and I wonder if it is the other way around. That is, in reality, nobody has watched MTV since 1994 but Viacom bribed a Nielsen family to tune a TV to it and keep it there.
So you want more targeted ads for tech savvy people? I wouldn’t give ad tech that much credit. Many tech savvy people have cut the cord and watch TV through streaming. Anecdotally, I watch Hulu. Hulu knows so so so much about me yet zero of the ads are targeted. They very much have this capability (engineering resources) but due to a number of reasons I can only assume (network contracts, ad bids) it just probably isn’t going to happen. I would love if I could have an ad blocklist cause one more Progressive Ad will drive me bonkers.
I don't live in the ad tech world, so I only vague know what is interconnected. Do the tech giants just sell info back and forth to each other?
For example, When I watch youtube on my Roku, if im not signed in, does roku still aggregate what I watched on youtube, could that be sold to Hulu, for ads when I watch that on the same tv?
I work in the CTV ad-tech space. Viewership data is highly guarded by every company. It's how every company is trying to differentiate themselves from each other. So, no, YouTube and Hulu etc wouldn't share information.
Ad data is known by the DSP that serves the ad (if the company doesn't serve their own ads), but viewership data is secret as a competitive advantage.
I don't block ads. I remember what the internet was about to become before ads stepped in. Everything of value was going to be pull behind paywalls.
Let's say everyone get their wish and ads go away. Everything will require a purchase. Those purchases are logged to a real name/address. You end up with bigger privacy leaks.
People will still be tracking you the way they are now. And at the credit card level.
As an adult in the first world I can afford to pay for adfree solutions. Most people can't. Ads level the playing field.
While that speculation is plausible, another view would be that companies would offer quality content without tracking users and invading their privacy (how many paid services still flood you with ads?), and possibly free alternatives would start to come up.
From your point of view, free open source is something that wouldn't exist
The hobbiest websites will not go away. Open source existed long before the internet.
What you are left with is the hobbiest websites or the mega brands that want to funnel you into their ecosystem. To offer anything that cost resources that you are willing to spend you must be a megabrand using this as a loss leader opportunity.
I don't think we want that world. We may think we do but look at what happened in Mr. Robot.
> Let's say everyone get their wish and ads go away. Everything will require a purchase.
No, it won't. There was plenty of high quality stuff on the internet before ads or payment was even possible, and there's plenty of high quality stuff that don't track you or require payment right now. There's no reason to think that would all evaporate.
We will see plenty of high quality stuff still I agree. But much of the free stuff will be about converting you to the paid stuff.
With ads or not you are still the product. You will still be tracked because people want you to spend money on their service. People will sell that information. Companies will use it.
Sure, much of it will. But certainly not all of it. The amount of actually free content will certainly not decrease, and it would probably increase, even if by only a little.
> With ads or not you are still the product.
It depends on the site. There are lots of sites where the site operator has no interest in it generating an income, let alone a profit. You are not the product there.
Ads have bloated the useful internet to the point that it is more expensive and less functional. We have 8,000 websites trying to show me a recipe for chicken parm, most of them with pages of family history and backstory, because they are all trying to get me to see their ads. A lightweight Wikipedia for recipes and a few high value added websites charging a trivial amount for access to their recipe catalog would be highly value added for me, and run on a fraction of the infrastructure.
Ads obscure solutions, and add redundancy and complexity with zero value added, because solving a problem means you no longer are on the page seeing ads. Simplifying or automating a process means you are clicking less pages less often and not seeing ads. If you automate something to directly connect users with what they need, then they don’t need to come back and see your ads. So we have automations they bring us to some middle man that can show us some ads before we can get to what we need.
Ads mean that maximizing the time your attention is held is the core value. High quality content that leaves you informed/satisfied/fulfilled is worthless compared to low quality content that is just good enough to keep you from leaving, without being having enough substance to actual fulfill you, because then you might leave and not see the ads we have to show you.
Podcasts show us that a tiny minority of users able to pay for content subsidize an incredible amount of added value content for everyone, whether they can pay or not.
Ads don’t level the playing field. Ads are an ever growing tumor, sapping resources and weighing everything down in a mindless effort to replicate.
This is funny to me, because there's an American movie where Danny DeVito manipulates TV ratings to get his own (worse) shows on the air: https://www.imdb.com/title/tt0087979/
Most adblockers I know block Piwik and that's fully self hosted in most cases. I know because I gave it a spin and the entire interface was broken because of my adblocker.
Smuggling tracking through first party proxies is certainly an easier way to avoid privacy protection systems on your customers' devices, but it's definitely not the silver bullet people want it to be.
Adblockers are fully capable of blocking subdomains and individual scripts by name, as well as forbidding certain outbound connections.
You'll probably have to use inline scripts and send data back bundled with actual data necessary for your site to run if you want to sneak past the user's defenses. But if a site has that much of an adversarial with its users, maybe it's time to stop using that site.
I've noticed several plausible.js's blocked by uBlock origin. Not sure how, maybe pattern matching? I doubt it was manually in any blocklists as it was a small personal blog.
That's the main reason we promote the backend integration for Pirsch [0] so much. In the long run, JS will probably not be sufficient, depending on your target audience.
As a developer of an ad blocker[1] our stats would seem to back up what Plausible has found:
- Despite having equivalent desktop (macOS) and mobile (iOS) apps, most of our users (> 75%) use the app primarily on desktop
- Most users say the key reason for use is for privacy protection; even with Safari providing some inbuilt privacy tracking protection
- Our app is focused on providing a simple 'set and forget' ad blocking approach; so you would think the key audience would be less tech savvy users. However a large proportion of users are tech-inclined and knowledgeable users.
- A lot of tech heavy websites are some of the worst offenders in terms of tracker usage and advertising. For example, The Verge can load 3.5x faster simply by using an ad blocker that blocks the on-page trackers and ads[2]
Google has no incentive to support effective ad blocking on platforms they control (Android and Chrome). This has made us reluctant to develop for or invest time on these platforms – the platform owner would be working against what we would be trying to achieve.
What? Apple is even worse. There was a big cry when Google wanted to nuter adblockers API but Apple have done exactly the same thing. You fell into a pr stunt. Apple doesn't care for privacy. Just look at how they scan you pictures..
Are you referring to the "content blocker" API? Because that basically forces plug-ins to supply a list of blocked domains. Just like Google's upcoming changes. Neither allow you to run your own engine for blocking like what uBlock Origin does.
No, other cloud providers scan all the pictures you upload. Apple does a careful private set intersection (partially on the client, partially on the server) on all the pictures you upload to collect strictly less data than other cloud providers.
Blokada's UI is without peer and so it makes for a very good "just works" for the majority (in fact, from what I know, it is the most downloaded DNS-based content blocker on Android by far).
Forgive the questions, but you seem to be a good person to ask.. How does Blokada actually work? Its FAQ claims it:
> prevents apps and browsers installed on your device from sending your private data (known as tracking fingerprints) to the Internet.
Is it doing some kind of packet inspection?
As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.
> prevents apps and browsers installed on your device from sending your private data (known as tracking fingerprints) to the Internet.
For now, Blokada's utility is limited to DNS-based content blocking. It cannot and does not prevent most forms of fingerprinting.
> Is it doing some kind of packet inspection?
Yes, only DNS packet inspection, but even for the only thing it does, it is clumsy: It leaks DNS requests; that is, Blokada does not trap all DNS traffic on port 53, and it does not handle DNS queries sent over TCP. DNS66 has these same issues, too.
> As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.
Blokada uses the same trick (I mean, core parts of Blokada 4 code-base does bear similarities with DNS66 which preceded it... Blokada 5 however was re-written in Rust).
> Would Blokada work alongside that?
No, it cannot. But: Apps that support "DNS proxying" (like Nebulo [0]) can. It is quite an involved setup. I'd simply use Nebulo over DNS66, as it is not only more capable but also encrypts DNS traffic unlike Blokada 4 or DNS66.
> ...but you seem to be a good person to ask..
A disclosure, rather something to keep in mind: I have been accused of spreading fud by the Blokada lead developer and using it to "market" a "competitor" app I co-develop. In my defense, it wasn't / isn't fud what I spread, unless fud === uncomfortable truth.
Pretty sure Firefox did no such thing, instead IIRC the developer (gorhill) of both addons (uMatrix and uBO) realized that uMatrix was pretty much redundant given the options available in uBO and thus archived uMatrix.
On our website we have no analytics or trackers installed at all. The app usage statistics come via the Apple App Store. App Store users can opt out of these stats via an OS-level setting.
Why is it important to have "privacy" from someone selling me a new video streaming service, but not important to fight against censorship and policing of content by tech giants?
My experience running a web server also backs this up. The number of GET requests to the back end is around twice as much as I see in Google Analytics.
It appears that Magiclasso is not open source. That being the case, how can anyone trust your claim that "Magic Lasso Adblock doesn't see or have access to any of your web pages or browsing history?"
The Safari content blocking API makes sure of that. Content blockers can only provide a list of rules for content to block (based on URL regexes, CSS selectors, etc) but can't actually access the content itself.
Worth noting that if you're already willing to setup a first-party proxy like Plausible does in this comparison, you can do the same thing with Google Analytics using either the NYPL project [1] or send whatever you want to the Google Measurement Protocol API [2]. You can usually send whatever you want through a first-party proxy in basically any competent analytics product. Analytics is not ads.
I find it this comparison a little misleading because Plausible admits that their own script/endpoint are blocked by adblockers, just to a lesser extent than Google Analytics [3].
Seeing as Plausible is still a script, how much of the remaining percentage of Firefox/Linux users are blocking scripts entirely? My default uMatrix settings block everything but first party CSS and images.
Considering that most of the web is unusable for most people without javascript, I'm pretty sure no-script people are a very very small portion of the internet.
A lot works quite okay, reader view also works great a lot of the time. And if necessary I'll allow first party scripts and most tracking scripts are third-party.
This is good to hear. I didn't think the percentages would be that high.
"Data driven design" is kinda overrated anyway. It's better to actually listen to your users. Stats don't show what people think, and they're often tweaked to show what people want to see.
Collecting telemetry is essentially doing an impromptu social science study, and it suffers from the same problems - collecting good data and making correct inferences is a difficult task, and it requires specialized skills. Social scientists are trained to do this, and yet they still fail more often than not[0]. A random business trying to run product development off automated metrics? They don't stand a chance.
This is not to say the data is useless - it's just worth remembering that, unless you have strong statistical background, you're probably reading your data wrong, so your metrics should be treated as weak evidence at best.
--
[0] - It's not snark, it's a corollary of the replication crisis. This is a hard job.
"Listening to your users" is just a more manual form of data gathering though. Also, it's not like "hearing what you want to hear" has been invented by statisticians, that is super easy to do when talking to users as well.
Some of the problems in talking to users:
- people will often tell you what they think you want to hear rather than what they really think. (Very few users would ever tell an interviewing dev that they think the product is unsalvageable, the dev team wasted several months and that the user would never ever use it)
- Users often don't know what they want because they don't understand the full range of possibilities. Sometimes they dream of tech that would violate several laws of physics, at other times they fail to realize that for-loops exist and just ask for a faster way to manually click a button a thousand times.
- Fads affect users just as much as anyone else. If there has been a surge of news articles about how Google is doing fancy AI stuff, you can bet the bakery on the corner of the street will mention machine learning when asked what they want to improve their business.
It’s amazing how far the goalposts have moved on analytical data.
I worked retail in The Time Before The Web and we collected and analyzed data from our customers. We had a door clicker to give us a daily record of traffic. We did detailed analytics on what items customers bought when, to inform inventory and staffing decisions. We did regular people-counting studies to evaluate our merchandising in different parts of the store. We did coupons in local papers for which we could directly track redemption. This was all in the service of making the store work better for our customers.
Now, you might say that those are all anonymous aggregate piles of data. Well, so are most web analytics packages, including many client-side JavaScript.
The difference between the two is that there's no question that counting heads walking through a door and watching inventory patterns are not personal to the customer.
With telemetry, including web analytics, there is always a question because there is no visibility as to their actual practices. It all amounts to "trust us". And we've had more than enough experience with these things to know that such trust is frequently misplaced -- so none of it can be trusted.
And GA especially cannot be trusted, and also happens to be pretty much ubiquitous.
As a side note, it's become just as dangerous to be in a physical store as online now. People who prefer not to be spied on must pay in cash, be sure to put their phones into airplane mode before entering the store, and so forth. And with the increasing adoption of face recognition in stores, it becomes risky to even show your face. In other words, they have to be as "on guard" in the store as online.
Why is it good to prevent analytics? How can it benefit the user?
Why is it more important to hide the fact that you're interested in cars and Netflix, but not important to protect free speech, free enterprise and free association?
I'm absolutely shocked by why someone would want to protect against a vendor selling me a new mobile phone when the real threat of social media is promoting left-wing extremism and censorship.
I'll take your question at face value (even though you're very certainly a troll account given your post history).
It's less about "hiding" or "preventing" anything, and more about putting the user in control of sharing those things. If I tell you I like figure skating, here on an online forum, I'm making a conscious decision to share that fact about my life.
If a script completely silently figures out I like figure skating, because of websites I previously visited, that it was able to infer due to various data sharing setups (such as google analytics overprevalence etc), that is not a choice I made. It's a form of stalking.
The problem is accentuated because it's not just one hobby, it's everything. Your hobbies, your location, gender, religion, income, political affiliation, etc.
That it serves ads based on that is icky, but the problem is that it knows those things at all without me choosing to share that. And this is without even getting into "bad actor" territory.
Privacy is control over what you share with others.
Personally I hate analytics because it makes my equipment work against me, or at least for someone else. Consider that there are websites and apps that are so overloaded with js and ads, that they will run sluggishly on anything but the latest phones and laptops. I would rather not allow that.
I have no problem with static ads. I love watching creative ads on TV during a sports broadcast. Instagram ads are another example of advertising done well. Minimal distraction from the content, no slowdowns, easily dismissible.
As for censorship, I agree that is a more important issue. But we can focus on more than one issue at a time. I will take a win on either front.
> not important to protect free speech, free enterprise and free association?
Analytics is able to track your speech, enterprise and association to facilitate their encroachment. Part of the metadata collected from messaging apps, for example, is the network graph of your interactions with other users. Association. If you're not using something E2EE, that's speech as well. And so on.
> the real threat of social media is promoting left-wing extremism and censorship.
The percentage is misleading. it's not people intentionally blocking Google Analytics, it's anyone that runs any adblocker, which almost all block Google Analytics. I installed uBlock for my parents and they don't have the faintest clue what GA is. A significant portion of those 58% probably don't care either.
Just wireguard to your “safe haven” and you can open your browser anywhere.
Or, if you’re lazy like me, just use NextDNS when you’re outside. There’s even an option to only activate NextDNS when not connected to the Wi-Fi of your choice.
Including Reddit's audience as "tech savvy" is an outright lie. Reddit is one of the most popular websites in the world. (And even if it weren't, it doesn't take much browsing to see how non-tech-savvy its audience is. It's basically Facebook at this point.)
The audience isn't even sort of comparable to Hacker News though, so putting both in the same headline is weird. There was some correlation long ago, now not so much.
The Firefox numbers above are a much better tell of what a "largely tech savvy crowd" looks like. 88%.
Depends really on the subreddit in question. The traffic to the blog post that was analyzed came from /r/linux which should be more tech savvy than the average web user and also more tech savvy than the average Reddit user (i would put /r/linux audience close to the Hacker News audience)
I assumed it was more since posts I've had on the front page have more like 70-80% of traffic missing in GA. It could be because I only enable it when DoNotTrack is off though which I don't believe is the default.
One solution is to use its own Google Tag Manager server. In this case, the events are sent to the server container via a custom URL and the data is then transmitted to Google analytics.
That's why I prefer goaccess(1) over any other tool that uses client side Javascript.
It's basically a real time website log analyser which gives you enough information to know whats happening on your website but doesn't require any pesky Javascript etc to do it.
Also since it is works by analysing your log files it can never be blocked.
> Since you know they are bots, why couldn't you filter them?
A decent amount can be filtered by UA. This is inconvenient because UA is a very large piece to log and index on, so you need to do UA processing to do anything useful with it and … well by that point it just becomes a chore and I suspect there's good logging services that do this better than you'd spend your time doing yourself.
> On the other hand, couldn't there be bots that run JavaScript which would be tracked client-side?
They exist but they're more rare by nature, because running JS at bot scale is expensive.
> well by that point it just becomes a chore and I suspect there's good logging services that do this better than you'd spend your time doing yourself.
Which is why the OP suggested using tools like goaccess.
> One of my clients' website traffic is composed of over 75% bot traffic. Server-side logs are unusable for anything other than site performance.
I'm unclear how broad you intend that second sentence to be, but there's still a ton of info you can glean just from server-side logs:
- Referrer info, and, by extension, popular search terms being used to find your site;
- Paths on your site causing 5xx errors (so pages which might be triggering an error in a server-side script)
- Paths on your site causing 4xx errors, and associated referrers (might be broken links on your own site; might be stale search engine indexing)
- Mobile vs desktop access statistics
Finding this data among a bunch of bot-induced noise might be annoying, but if they're good bots and sending proper UA headers specifying their botness, it's easy enough to filter out. Even otherwise there might be typical bot-like behavior you can find and account for such as not sending a referrer header or trying to access known exploitable PHP scripts (in which case you should block that IP address for a few hours or days - there are programs which can do this sort of thing automatically but frustratingly I can't recall the name of one off the top of my head right now).
Granted, a lot of this can be spoofed, but I'm pretty sure the number of people sending spoofed referral or UA headers is dwarfed by the number of those (like me) who block Google Analytics and similar cruft entirely.
No, you're right, I shouldn't have written something so dismissive. (I do include error tracing as part of "performance" for what it's worth but those have their own system from within the app itself)
Frankly I would love to see some serious low-config solutions to analyzing server-side logs. Oh, especially Fastly. Client in question uses Fastly and it blew my mind to find out that there was nothing in place to answer simple questions such as "what are the slowest paths to respond", "which paths are a cache hit most often", "which paths are most hit overall", etc. And being able to look at various dimensions such as browser, bot traffic, country of origin, etc. If you have suggestions…
Any log analyzer will tell you which path is the most hit. For slowest paths, I think a server daemon could theoretically log how long it took to serve the page from request in to last byte out, but I don't know if any of them do that - you might have to set up a custom format for logging, and then from there you'd need to tell your analyzer how to interpret that field. For cache hits, I guess it'd depend on what sort of cache you have in mind, but that might be something you could only effectively log at the application level.
Are there relevant studies on the effectiveness of advertising?
I remember spotting such study.
I still wonder if advertising is always worth spending on, beacuse google and facebook make a big amount of money with it, but I'm still skeptical when I see the actual amount of people "engaging" in ads, when you see how much it actually costs.
Yes, pretty much every company first data person runs Cost-per-action (CPA) analysis to estimate how many sales can be attributable to ads vs. other ways of raising your brand. It’s an imperfect science (just use the word “attribution” to trigger shivers in any data person) but it’s very clear that targeted platforms offer dramatically cheaper leads for almost every product.
That’s why the hostility to ads feels misplaced: it works, it helps company to whom you want to give money to find you and people like you. It’s just that condescension from _some_ people in Ad platform and a press that insistently favours the worst people mean that people like me who argued for more user-friendly interfaces (ban certain content, prevent repeated ads) were routinely overruled.
> it helps company to whom you want to give money to find you
You mean, the companies who want me to give them money. I'm sure there are a few companies out there whose product I actually want and don't mind paying for, but their number is absolutely dwarfed by the number of companies that seek to induce FOMO/status anxiety/etc in order to get me to buy things I don't need and would not want if not for advertising.
every advertising platform wants to pretend they are delivering useful products that people would otherwise be unaware of, but almost exclusively deliver what you have identified.
Yup. I find it amazing the quoted myth lives on. It doesn't even make sense.
How many problems do most people have that can be solved with an existing product they are unaware of? The numbers can't be very high.
How do you explain well-known brands advertising the same product for decades on end? Surely Coke and Pepsi aren't suddenly enlightening many people to the existence of their drinks.
How about ads that get shown to the known-same individual time and time again after mere minutes? (See Hulu, at least back in the day, not sure what it's like now.)
The whole line about ads being mere consumer education is ridiculous and doesn't even stand up to a cursory thought.
Every business with a competent marketing department will continuously track and measure the effect of the ads they run, like how many users who click the ads turn into paying users.
My startup has tried a variety of marketing strategies from in-person campaigns on the street, video ads on YouTube, "free" PR through newspapers etc. In order to measure the effect of each approach we only did one at a time.
For us paid marketing on Facebook/Instagram was, unexpectedly, the most efficient form of marketing by far. But I would not assume that applies to all, or even most, businesses. So you should experiment with different strategies for your business.
It's the tracking part here that's hard. How do you know that the FB/IG ad was the first time the converting user heard of your product, or that it was the deciding factor? If you literally have no other way of discovering your product than this works, but it's easy for FB/IG to show your ad to users who were already going to convert and claim the conversion...
I see this so many times. Someone Googles for <product name> and then clicks on the ad for said product instead of their website which is the first organic result. Google claims it’s an ad conversion and gets the money, marketing monkey will happily take this as credit for their work and justification for further ad spend & their own salary, while the truth is that this user already made their decision to use this product (as they’ve searched for it) and didn’t need the ad.
I dislike ads in general. Specially Youtube ads. They are hysterical and for some god knows reason advertisers think its a good idea to repeat ad nauseum the same ad multiple times even on the same video. I end up hating the brand more than having some interest in the product.
(Paid) reviews on the other hand like unboxing, configuring and testing a product that I'm interested in are totally another thing. This applies to furnitures, house appliances, computers and so on. A good example is that I did not knew how much I wanted to build a fully silent computer before watching so many build videos of a certain fanless case that looks like a metal cube.
"All in all, the difference in stats would mostly come from people blocking the Google Analytics script. Google Analytics is listed on many blocklists while the Plausible proxy runs as a first-party connection and is not."
So what exactly does Plausible do to avoid being blocked in the same manner? What prevents GA from doing the same?
> So what exactly does Plausible do to avoid being blocked in the same manner?
It seems they serve plausible from the same server as the origin website (first party), probably under an unique name, which means you need to block the script manually on every domain instead of simply blacklisting `plausible.io` (for example).
> What prevents GA from doing the same?
That's a good question. Probably ease-of-use (no need to explain how to host it, easier updates) and the lack of need to do so. YouTube, for example, could also make it _far_ harder to block ads by simply sending a single video stream, but they don't for some reason.
Nice that Marko did a follow up by writing this blog post. A few months ago (in May), I also asked myself the question of "How plausible are our web analytics?", and I was able to see the same phenomenon [1]. Many people in my audience block client-side web trackers.
One option would now be to host a plausible proxy on-premise. But I didn't have time to try that out yet.
PS: We're paying plausible customers and our stats are publicly accessible [2].
One of the reasons for mobile apps is that adblocking is disabled. Explains why Reddit promotes it so much.
Similarly, wrapping websites like Discord or Slack in Electron also gives the website owners full telemetry and tracking that they can't get in a tech savvy browser.
Would an always on VPN, a remote pihole be the only way for privacy?
Honestly, if they'd abuse their power given through electron - surely, we the users would start boycotting one way or another. Ultimately, this purpusefully regressing UX for revenue - nobody likes it except the finance department.
Not sure if VPN would help you much against telemetry in a mobile app. A native foothold in your phone's system gives them access to much better data than they could infer on the server side.
E.g. if I wanted to know where you're hailing from, I'd browbeat you into granting me Location access privileges. If that's too difficult, I'd get you to grant me Files/Photos privileges (this one won't raise too many alarm bells with apps like Discord or Reddit), and then try to read EXIF geotags off your recent photos.
It must be very disheartening to work on reddit's mobile site. Not only is it deliberately made a miserable experience by forcing you through AMP via Google and insisting you "continue in browser" every time, but then you're greeted with a banner that outright says "this page is better in the app".
Say what you want about the implementation but the new design is definitely better for watching memes. And I wouldn’t be surprised if the design doc simply said “focus on memes”
I wonder how long they'll keep the API live with so few restrictions and fees. Twitter makes huge amounts of money from selling the kind of data that PushShift gives away for free from Reddit.
Agreed. I don't mind the new layout but it is still amazing that after years of this new layout that at least once a week I go to reddit and it can't load comments. I'm not a developer so maybe there is a valid reason but as a user it just seems ridiculous that the site can't do its main function reliably.
> One of the reasons for mobile apps is that adblocking is disabled.
That and revenue is much higher on apps than on the web. I made an android apps for a website. And just alone the android apps made more money than the web version. With less ads.
Blokada and others work by using the VPN functionality in Android, to implement DNS blacklists. Alternatively, AdGuard and NextDNS run DNS servers where you can customize the block list, a remote Pi-Hole as you said.
I'm using NextDNS as the system-wide private DNS on my Android phone, it works great and eats less battery than Blokada.
I found NextDNS to be relatively convenient and easy to set up even for a lay audience. Definitely easier than a PiHole or a custom `dnsmasq` setup, and it offers mobile configuration client apps.
What I do not know is if it will work also when apps begin using DNS over HTTPS… I suppose not?
I have been using NextDNS for couple of days, but since I don't have a static IP, it's obviously not so convenient, I have to reset my IP every time it changes
But otherwise, absolutely great, awesome statistics about blocked/requested domains, countries, etc...
Unless they provide a VPN it is only blocking the not-so-bad-actors. Everyone else use hardcoded DNS IPs. If you look at traffic from an Android phone you will get lots of DNS requests to Google DNS no matter if you use NextDNS or not. If you only provide one (primary) DNS IP in android 8.8.8 8 (Google DNS) will even be used by default together with your DNS provider. Same is going on in iOS. If they do provide a VPN then it isn't really for a lay audience IMO but it is the only thing that isn't like pissing in the wind.
NextDNS offers a DoH endpoint and is a selectable TRR in Firefox. Unfortunately that doesn't help with apps doing DoH to bypass DNS blocking. The current state of the Internet / computing is a bit problematic, but there are ways forward.
What I do and recommend everyone to do is:
1. Run an edge network device using network access controls and filter which devices on your network get outbound network access (in my case just the gateway device). Block all inbound traffic except what you choose to pinhole, block all outbound traffic except ports you choose to add to the allow list.
2. On every client device run a local application firewall (I like Vallum and Little Snitch on MacOS as examples) and filter applications by domain + port on outbound requests, block all inbound requests.
3. On every client device force it through a VPN to a gateway device internal to your network to get internet access, anything that falls off the VPN is then blocked from the internet. The gateway device can forcibly route traffic and perform additional filtering
4. On every client device, configure it to use an internal DNS on your network with a fallback to a trustworthy external provider, have the internal DNS use a trustworthy external provider over DoH. Block outbound DNS at the edge device
(blocks all non-encrypted lookups).
It's kind of a pain, and a mess, but it does greatly restrict the damage that rogue IoT / Smart devices can do.
I personally use doh_blacklist with around 170ips, where I block outgoing traffic for known (publicly and not so) internet reachable doh resolvers. There is no problem(+) - everything works perfectly.
ipset create blist_doh hash:ip hashsize 1024
for ip in `cat /etc/bin/blist_doh.txt`; do ipset add blist_doh "$ip"; done
iptables -A <insert some iptables placement specific to your outgoing/forwarded traffic> -m set --match-set blist_doh dst -j DROP -m comment --comment 'SPY:all ext DoH BLOCKED'
Still, as of 2021, doh rule is around 2% traffic logged compared to my other rule, where I simply block outgoing 53/udp (except my resolver). a LOT of your devices ignore your dhcp dns settings and try to circumvent it going directly to shady 8.8.8.8 etc.
(*) you shall every few months check and update it.
As it intentionally looks like HTTPS traffic, that's a different game; you kind of have to whitelist rather than blacklist.
Here's an idea that supposes you have some server available: Force all traffic over your network, even when roaming (VPN). Squid+Privoxy as HTTP proxy (your choice if you want to DPI and add own TLS CA as root on devices, in which case you can actually block it, or just do HTTP CONNECT). Optionally also set up a SOCKS5-proxy (microsocks). Configure all applications to go over proxy. Applications that don't support it can usually be "proxified" with tsocks or proxychains4.
Now you have more fine-grained control over your traffic. You can do fun stuff like setting up several incoming ports, several outgoing proxies (including Tor/I2P) and allow/block based on destination, domain, protocol, etc.
Finally disable outgoing traffic on your devices/route to a black hole. Or route it via your VPN at least, where you can do some rudimentary stuff with nftables/iptables. Oh, and something not everyone knows: you can route traffic based on uid/gid on Linux, so on your workstation you can leverage that too selectively control traffic on an application-basis.
On non-rooted Android and iOS, I suspect this is not perfectly achievable but if you have the time you can get pretty far otherwise. It's an arms race and I feel like we can keep up.
> Similarly, wrapping websites like Discord or Slack in Electron also gives the website owners full telemetry and tracking that they can't get in a tech savvy browser.
Discord, Slack and other similar webapps can (and maybe do) send telemetry in the same connections used for the app's features. You can't reliably block that.
I used to have Google Analytics blocked on my home networks (through Pihole/Adguard Home) with the exception of one network which my wife uses. She works with SoMe so not having access kinda interferes with her job.
I've since moved to NextDNS.io to have network agnostic ad/malware filtering. Still keep a "special" configuration for my wife :)
Does anyone have recommendation for static site hosting with simple server-side analytics? The only thing I can find is Netlify Analytics, but $9/mo is too much for hosting a few pages of OSS project documentations.
If there's no complex interactivity achieveable only through JS, I prefer to ship sites with plain HTML/CSS, so I don't really like to use any client-side analytics. And yes – I can live without all the fancy features to track and profile my visitors.
I don't want to host by myself either, since I enjoy using a simple git push to deploy my site.
* Use a CDN which provides access logs
(AWS Cloudfront, logs are stored in S3, Azure supports this too i think)
* Feed those logs into a self-hosted analytics solution, like goatcounter [1]
How about Pirsch [0]? It starts at $4/month and you can set up as many sites as you want for up to 10k page views/month. That should be sufficient for most smaller sites.
709 comments
[ 2.4 ms ] story [ 543 ms ] threadThere may be an argument that Chrome is marginally faster than Firefox on desktop, and while this may also technically be true on mobile, it's completely dwarfed by the fact that Chrome is being asked render 5x as much crap (with animations and sound and video and nagbars and ...) compared to Firefox mobile with uBlock Origin.
You will literally regain hours of battery life. Those Joules belong to you -- don't just shrug and hand them to Google.
My Joules, my rules.
On the other hand, it probably means that I am a "false negative" in TFA's report. I'd love to know the correlation between the server logs and what Plausible shows for a connection. I'd also like to know how they infer OS -- e.g. for privacy reasons, my reported user agent is not accurate...
You can see that in uBlock Origin already. The summary info has the number of domains connected and if you click on "more" you have the details, just like on desktop. You can also block JavaScript altogether for a particular site.
The biggest benefit is definitely the sheer speed at which news websites load on my phone now, even when I'm on a crap connection.
Another benefit: This isn't a VPN, so you can still use a VPN whilst using Private DNS.
Edit: Also... lol, I can't access TFA as plausible.io is also blocked on my network and phone. I'm going to assume that the article is actually an analytics competitor whose pitch is "GA is blocked, use us instead!".
Edit 2: https://news.ycombinator.com/from?site=plausible.io Yes, assumption was correct.
Edit 3: NextDNS stats from my home network: 11.45% of DNS requests blocked, but from my phone 23.7% of DNS requests blocked. You need this more on your mobile than in a web browser!
If you want a DNS server to block things, know that the definition of what to block is subjective and some people may disagree with what you want to block.
For that reason you get an account, which allows you to have a configure, and that needs to be resolve to you... so at some level an identifier of the configuration is needed. This is the id.
Also on F-Droid:
https://libreplanet.org/wiki/Group:IceCat/icecat-help-help
https://f-droid.org/en/packages/com.stoutner.privacybrowser....
Works on ios and android.
I think their free tier is very generous at 300k lookups/month; otherwise it's something cheap like 2€/month if you want unlimited lookups (i use this to block tracking on all my devices + home network)
Why would it use any extra battery? The phone has to make DNS lookups whether its with the one from nextDns or from the DHCP server.
You're right, this dns solution is very unobtrusive when it comes to energy use.
When I want to "try for free" I'm assigned a unique ID which is then used in all DNS resolution requests. How is that even remotely tracking-safe?
The unique id is to retain settings across different devices.
Of course you have to trust them not to sell you out.
On iOS the options are a bit worse but Brave has some kind of adblocking that works. Safari also supports blocking lists via AdGuard but it's more limited.
The whole browser situation in iOS is really user unfriendly.
Sadly, I have a quite old Android device (but its battery life it's still good) where Chromium-based browsers are already quite slow and Firefox feels even slower and drains the battery faster (the old Fennec was worse than the new one, it hanged for 30 seconds).
/sarcasm
What we had in the nineties was that we could configure our browsers to ask for every single cookie that a site wanted to place on our computers. That got old really fast when internet advertising became a thing, and tracking cookies were all over the place.
Indeed! Seems the first browser to implement some sort of incognito/private/separated profile (specifically to hide your tracks, not general like Firefox Profiles) was Safari around the release of Mac OS X Tiger (April 29, 2005).
Telling Facebook "hello, I am visiting baddragon.com from Mozilla Firefox 63.1 on Obscure Linux in Podunk, Saskatchewan, population 32" is barely an improvement over telling them "hello I am John Q. Smith and I am visiting baddragon.com".
And of course, private browsing is generally per-instance. Log in to your Google account in a private tab? All your other "private tabs" are also logged in to Google until you close the private browser.
And yeah probably need to use windows not tabs on desktop. On iPhone each tab wouldn’t see that another is logged in though
(Otherwise yes good to be aware of fingerprinting I agree with you)
I feel like this is similar. All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).
Maybe we should engage in large scale AdWords fraud. Send come fake traffic away from Facebook and over to Wikipedia.
https://adnauseam.io/
Edit: it's an adblocker that is supposed to click on EVERY ad that it blocks.
On the contrary, clicking only on trash, and having the high quality content have a much higher signal-to-noise ratio would have the desired effect.
Maybe.
But, hey this isn't a gameplan, this was just a "what if" :D
I'm fully on board with the AdNauseam model which fucks about indiscriminately. The ad industry can burn.
If we only click ads on high quality contents, those content owner might benefit from it in short term but in long run it most likely going to back fire and make them penalized as for sure there is going to be counter measures. If we simply click every ad indiscriminately, there is no way to tell or they have to penalize everyone which almost equals to do nothing.
Wouldn’t they just lose money? Ad clicks are not valuable to the businesses themselves. Only to the ad companies.
I always click on Taboola bitcoin scam links for that exact reason. It's like scambaiting an algorithm.
Might be interesting.
Is there any extension that blocks analytics selectively?
I make a point of watching a non-US film on every international flight I take - I find it to be a unique opportunity to watch interesting non-US movies with English subs. I have discovered two excellent oddball comedies from in flight movies (Die Goldfische from Germany and Le Grand Partage from France), but when I tried to rewatch these films with others in the US, I discovered it was basically impossible. There must be so many other great movies from around the world we are missing out on.
That is incorrect, they do have to pay (once per household). But if you get BAFÖG (student loan/social benefit mix that require you and your parents to be below a certain income bracket), you don’t.
The real harm seems to be from the tech giants censoring speech and policing payments, but what's the harm that someone targets a pair of shorts that I might like or show an ad for a conference I might be interested in?
The harm if targeted ads depends on your viewpoint.
A targeted ad might serve you something you were looking for anyway, or it might manipulate you into spending on something you don't actually need. I.e. look at Instagram influencers, showing off their fake perfect live, making the viewer feel small and then try to buy the same happiness by buying the same product.
At best, ads are information that you need, at worst, they use psychology to manipulate you.
I was talking about how ads, tv programming, trackers and such have a tendency to create a positive feedback loop, which leads people towards less quality and mindless consumption. And about a fun idea from a movie, to break this feedback loop and replace it with another one, that promotes higher quality content.
You then started talking about a different topic and are now accusing me of not being interested in free speech.
Thanks!
('E.g.' doesn't really mean 'example given' in English - it means 'exempli gratia' in Latin. But it's a useful mnemonic.)
Some people don't want to be tracked or monitored by advertising companies and it should be enough to just say so without companies like Facebook always trying to sneak tracking back in via dark pattens, shadow profiles, etc, etc.
For example once you've seen a website offer you the same product for different prices based on arbitrary tracking it leaves a bad taste in your mouth.
So Facebook and Google know your religion, politics and food preferences. So what? How can they harm you with this?
They harm you when they choose what you can say, who you can talk to and who you can do business with. But literally what's the harm of ads? Making your computer a bit slower? That's such a tiny issue it's not even worth talking about.
Be carefull - fraud is a crime. But I am under no obligation to provide AdWords any data, let alone true and reliable data.
Considering the tracking and spying, and my legitimate interest in privacy, there is no room for fraud argument here.
PS: this would be probably a Hoax
I don’t think AdNauseum users would be prosecuted, but you’re still wrong.
source - https://en.wikipedia.org/wiki/Fraud
Note also that the offence is framed in terms of intention to gain or cause loss. Even if no material gain or loss happens, the intention is what matters. (The equivalent mental element for theft in England is that one dishonestly intends to permanently deprive someone of the property being taken.)
https://www.legislation.gov.uk/ukpga/2006/35
This applies to online ad 'click fraud' in both ways—if a publisher fraudulently clicks on adverts to make money, that'd potentially be fraud to make a gain for themselves. If a competitor clicks on adverts to get their competitor to lose money on pay-per-click, that'd potentially be fraud to cause a loss to another.
I can't speak for jurisdictions other than England and Wales but I'd be surprised if a fair number of other jurisdictions didn't also define fraud in a way that covers both gain and loss scenarios.
Spoiler alert, it doesn’t involve your intentions or money.
https://en.m.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
The companies spying on me should be the first in line to be tried under CFAA
If the sites set it up themselves, it's fraud. If you conspire with the sites to set it up, it's fraud and conspiracy to commit fraud. But if I'm prosecuted for the crime of not actually looking at ads I request, that's just a judge with an agenda. Whether it went one way or another at every layer of appeal would probably be a coin flip, though.
What’s needed is a better term.
I propose either AdTurfing (hat tip AstroTurfing) or AdLighting (hat tip GasLighting). My personal preference is the second.
Ad biz is a business model, not property, and a predatory one as that. You are not obligated to enable someone else's business for free.
They are claiming their clicks and impressions mean something - its their problem to ensure they are accurate.
Imagine someone is doing a survey of sex habbits, and selling the results. Is it a fraud to lie? Ofcourse not, why should you be responsible for their profits.
The fact that people think they are legally obligated to enable this is really fucked up.
The idea that some die hard Alf fan just made they way past my post on hacker news really puts a smile on my face.
Edit: Note: Free Rainer also goes under the name "Reclaim your brain"
https://www.german-way.com/germanys-tv-tax-the-debate-over-t...
That's not correct. If you are deaf, you get a reduction - if you are blind, you get a reduction. If you are deaf-blind, you don't pay at all. If you are already receiving financial help because of your blindness, you don't pay at all.
https://www.rundfunkbeitrag.de/buergerinnen_und_buerger/info...
Actually thinking about it, the issue is wider than just the news and I think the financing mode of the mandatory tax is a big part of it. What is really a shame, as Germany has a rich culture of hundreds of years so great content in all forms should not be a problem.
"Why is TV in Germany so bad?" https://www.reddit.com/r/germany/comments/3d4vxz/why_is_tv_i...
"Why is German TV so crap?" https://www.exberliner.com/blogs/the-berlinale-blog/berlinal...
You might comment the BBC financing model is similar and I would agree. I think the difference is that the BBC also embraced a highly commercial model of selling content like Top Gear and other stuff worldwide. In this case the English language content with its planetary audience, pushed for a more competitive/commercial model and the German TV stayed too insular in my view.
You may like the "Google will eat itself" idea.
https://www.gwei.org/index.php
Nice cosmological time scale idea
Those 819 shares are worth $2,371,005 now
Just thinking that the elderly population is probably the least likely to use those boxes (though I am not even sure of that), whereas they constitute the (dying) core of traditional TV viewership.
If it's opaque to visible light how does the camera produce an image?
>if you put tape on it your remote wouldn't work anymore.
Not really? You just have to be more careful placing the tape. Besides, nowadays many smart TV have app remote controls, or RF-based remote controls so the IR sensor being blocked is a non-issue
It isn't opaque to infrared light though. The image is still high enough quality to make out people, it looks funny but it is good enough for their purposes.
https://www.washingtonpost.com/news/the-switch/wp/2017/03/07...
You'd also have pets captured, curtains flapping by an open window captured, and any toys kids are playing with (like balls) potentially captured as another viewer too.
Meanwhile pap who likes to sit motionlessly while the kids round around him, isn't detected.
> without invading too much on people's privacy
Motion detection feels like a pretty major privacy violation to me.
> You wouldn't know the age, but you already have ton of information by knowing the address and the subscriber already.
Except when you have friends or family over.
Essentially the stated purpose is so that it can pick up what music you listen to throughout the day, but in reality it's picking up everything.
Depends, are the TV networks buying that information? Or just advertisers?
1. There is a disparity between age groups and other demographic dividers who have newer TVs. This could significantly skew the results for some advertisers.
2. The data is going to the TV manufacturer, and they will not share that freely between themselves or with anyone else. This will complicate collating the data as there are several entities to negotiate with in order to get an overall picture.
Yes, which is why I will never own one.
We're in something of a worst of both worlds situation where Nielsen has an increasingly small number of viewers where traditional TV boxes work to get decent samples, has to rely more than ever on surveys, and distrusts all streaming viewership numbers because they are cloak and dagger white lies between competitors, despite in theory being way more accurate than all the previous tools (the surveys and the TV boxes).
It's almost wild. The most forthcoming to shareholders/the general public over the years has been Hulu and Hulu's numbers at times have suggested Nielsen's data is very, very wrong right now, but Nielsen doesn't trust Hulu's data at all because it smells like lies because Netflix does nothing but lie or ghost them.
Following this up with paying for your subscription but pirating all the shows might help remove you from the cycle completely.
I understand the second point is not realistic for the majority of people but I wonder if in the future we might have an easy to use version of Pi-Hole that most people can just flip on and strip a large chunk of tracking from the apps.
That's the deep weird irony that we live in a world where we could have the best possible numbers (directly tracked statistics), and yet TV Producers are still relying (for the most part) on Phone and Snail Mail Survey Results because they don't think they can trust streaming provider numbers. Pi-Holing those numbers just gives those Producers even more reason to feel that they are lies or wrong. At what point do you Pi-Hole too many telemetrics to oblivion and aren't allowed to complain when your favorite TV shows get canceled because "no one" was watching it?
From what I am hearing, traditional forms of media are in decline because people are spending their time on Youtube/Twitch/Social media instead.
> I feel like this is similar. All tech savvy people block ads and analytics and at least the known tricks they use against us. So the internet only tracks the defenseless people and is then built to serve them (and or exploit them).
I think this is right - ad-traffic is manipulative and actually I don't think it is a societal 'good' at all.
A few personal examples:
* On Youtube almost all my adverts are encouraging me to start Forex / Stock / Property investment and trading, and sign up for courses on these. These courses are scams (or at best, 'half-scams' and poor/generic advice repackaged and sold for thousands), and in general provide poor financial advice (either through extortionate courses, recommending you become too heavily leveraged or advising you to day-trade high-volatility stocks by just looking at charts). Presumably it does this because I am 32 and male, so I am considered 'prime' for this marketing.
* One of the friends I know is a girl, and she has never seen the above adverts. We were talking and she says every single advert is just about pregnancy and fertility. I wonder how many of these adverts are just reinforcing gender-stereotypes in a wider sense, i.e. while google claims to be progressive and care about 'equality' really is their business model at it's core really just targeting women and telling them that they should be getting pregnant, while telling guys that they should be the bread-winners and earn money via stocks/shares?
* While my adverts are for forex, and my apparently fertile friend is getting adverts for pregnancy tests, my older parents just get targeted adverts for pre-paid funerals. One or two are probably be fine, but they are just on constant repeat - and I can't help but think that I wouldn't the constant reminder of death before every youtube video.
* My laptop is convinced that I want to go camping. It's only my laptop, every advert is camping related. Sleeping bags, tents... and the strange thing is that when it started I didn't want to go camping, but it's been so consistent across the last few months now that I kinda wanna go camping. Like it's sold me this romantic vision which I know wasn't there before, so even though I would usually like to say I can't be manipulated through marketing, it's really made me realise I can be.
Is the above really making society better? And if it's not, why should we put up with it? IMO the biggest lie we have been told by Google is that 'personalised ads' are a good thing.
You, your friends, and your family personally don't have to put up with it. Ublock Origin can block ads on Youtube with ease.
As per the parent comment to my original one, I just fundamentally do not believe that most advertising contributes anything positive to society and mostly generates negative externalities.
I honestly don't know if that argument holds water.
The only net gain I can see is for the advertising industry, which extracts money from both challenger and incumbent.
> ...well, based on what? …Probably what it does is infer my gender, age, income level, and marital status. After that, it sells me cars and gadgets if I'm a guy, and fashion if I'm a woman. Not because all guys like cars and gadgets, but because some very uncreative human got into the loop and said "please sell my car mostly to men" and "please sell my fashion items mostly to women."… You know this is how it works, right? It has to be. You can infer it from how bad the ads are. Anyone can, in a few seconds, think of some stuff they really want to buy which The Algorithm has failed to offer them, all while Outbrain makes zillions of dollars sending links about car insurance to non-car-owning Manhattanites. It might as well be a 1990s late-night TV infomercial, where all they knew for sure about my demographic profile is that I was still awake.
> You tracked me everywhere I go, logging it forever, begging for someone to steal your database, desperately fearing that some new EU privacy regulation might destroy your business... for this? [1]
[1] https://apenwarr.ca/log/20190201
> Our average viewer is 65 years old and watches TV while doing the dishes, we must not show things that cannot be understood by them
and then they will serenade about how they would "love to have a bit more sophisticated things", but as they are the only ones who really understand their audience, they cannot allow this, although they support the values of the 68 generation etc. pp.
From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.
All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.
It's the same for most content, I read a huge amount. I do read some intellectual books, but only occasionally. The rest of the time I read utter, thoroughly entertaining, trash.
I don't want to read about an existential crisis after programming all day, I want someone to hit something with a big sword and get the girl.
I feel the same way. However I force myself to read things that will better me once in a while anyway. I too want to hit things with a big sword (without the pain of getting hit), and get the girl (without cheating on my wife), but the world including me is better if I do something else anyway. Which is why I do sometimes read a complex math book.
> All the documentaries were getting really high ratings, so would display highly in searches, but not many people actually watched them.
This rather shows that ratings do work, but are used wrongly for giving recommendations:
If you want to create suggestions for a user, in many case the wrong answer is "suggest what has really high ratings", but rather "given the ratings that this user gave and the films that he watched, what will he also like."
The fact that these documentaries get high ratings (the same might hold for art house films) shows that there is some (niche?) audience which really loves this kind of films, but not that "John Doe" will love it, too.
It was always weird to me that GroupLens didn't spin up a team for it, but it seemed like everyone in the research group had moved on to other things and didn't want to context switch back. Someone mentioned something like "shame they didn't do this 10 years ago, a million dollars would have been nice". I think someone was doing tagging on movielens, but I don't remember the details.
I got the sense that neither Riedl nor Konstan (or any of the current grad students it seemed) wanted to pursue it (Terveen, I think, wasn't in to recommender systems at all in the first place).
I don't think the lab had any funding problems haha, so maybe that was what it came down to.
And I know I'm (possibly) a minority on hacker news, but I prefer the new system. I was giving everything I wanted to watch more 4 or 5, even when it was clearly not the case, but because I want recommendations of things I'm going to like AND actually watch
In Germany, public TV is paid for by (nearly?) every household. [0]
Forcing everyone, including poor people, to subsidize rich people's taste for documentaries seems a bit.. off?
Similar arguments apply to public libraries and opera houses, though at least there the financing is done mostly via progressive taxation.
Of course, you can argue that we sophisticated people know what's good for those unwashed masses, and if only they watched their documentaries like they are supposed to, they would soon see the light. Colour me skeptical.
[0] As far as I am concerned, private broadcasters can and should do what they feel like.
Answers from people can be used to suggest avenues for investigation, but shouldn't make you fund expensive stuff outright. (Assuming here that homeopathy is obvious nonsense, that people still swear by.)
And, of course, public libraries and documentaries don't have to just produce good effects. In order to justify public funding, you'd need to do a whole cost-benefit analysis and look at opportunity costs.
So look at what you get from public libraries vs libraries financed from private charity _plus_ whatever other good things the public money saved could do (including just outright giving it to poor people).
I deliberately picked public libraries here as a provocative examples. I suspect they might actually pass the cost-benefit test without too much contortion of metrics and data.
I am much less sanguine about the bang-for-buck of publicly financed opera houses, theatres and symphony orchestras, which Germany is quite fond of. And of course, publicly financed radio and TV broadcasters.
Further, I generally strongly oppose charity as a component of future plans (leaving aside whether charity is good or necessary in the present). We should never be building society such that it depends on the funding whims of rich philanthropists.
Sure, there should be some kind of analysis of benefits, but some things simply have to exist for a society to be a society, because without them, the loss of their intangible and second-order benefits will cause a society to implode Idiocracy style, and nobody will know why.
- Good Will Hunting
edit: btw, speaking of libraries, copyrighteousness is destroying our society
Receiving education isn't about learning. At least not primary. It's about getting a piece of paper at the end.
Public libraries and free internet resources are one argument in this direction. Another: professors are usually more than happy for you to sit in on lectures, even if you don't pay any tuition.
assuming a normal distribution, wouldn't this then be the majority of people?
Arguing from definitions won't help us. (Keep in mind that 'middle class' doesn't mean 'median income' or something statistically simple like that.)
Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces. Herman and Chomsky discuss this in Chapter 1 of Manufacturing Consent.
https://ia802700.us.archive.org/31/items/pdfy-NekqfnoWIEuYgd...
I agree. And economically, if you want less of a good to be consumed, ceasing to subsidize its production with tax payer money is a good first step. If you want to go further, perhaps even tax its production.
> Private broadcasters do not pick material based on public interest or even their judgment of what is good. It is far more mechanical and influenced entirely by market forces.
How do market forces differ from public interest?
Or rather, what do you mean by 'public interest'? It's what the general public is interested in?
Market forces come from people.
Related: https://slatestarcodex.com/2015/09/11/book-review-manufactur...
Market forces tend toward baseness. It takes active intent to elevate society above the lowest possible level. Just as one doesn't want to hire at the median skill level of their company lest the average continue to drop over time, a society has to aim higher than what the impulsive market optimizes for, or it will decay.
Revealed preference is just code for exploitation of vice.
Of course, there are always enlightened and sophisticated people like yourself that know better what's good for the unwashed masses.
Are you implying that people who have better things to do than watch documentaries are stupid?
Similarly, entertainment preferences are correlated with social class (and with income), but again, they are not perfect predictors of each other.
So a few anecdata wouldn't undermine anything here. Though in fact, your example actually strengthens the argument I am making: you are the kind of person that prefers watching documentaries over other drivel, and you are the kind of person who managed to get themselves out of poverty. That's likely because you have the preferences, habits and skills of someone who is at least middle class in a social sense, even if your income took a while to catch up.
(Keep in mind that we are talking about social class in a rather abstract fashion here. German middle class mores are different from American middle class mores.)
Of course TV in the US has figured out that the 65+ crowd is very valuable to customers (the advertisers, not viewers!), so even though they could get more viewers by not showing the nightly news, the nightly news is what they show.
Something, something, a diagram of a plane showing where the damage was on those that returned from missions.
Ahahahah I love this dogwhistle. You always know you're in good company when someone waves their hands around and says "put the armor where there AREN'T bullet holes" and gives you a significant look. XD
1: https://twitter.com/MTVSchedule/status/1422934028253081603
And the cable industry can't understand why people keep cutting the cord. Can you imagine shelling out $120/month to have some producers pick out Youtube clips for you?
From https://www.fcc.gov/media/radio/public-and-broadcasting: "virtually every station has an obligation to provide news, public affairs, and other programming that specifically treats the important issues facing its community." The details are specific to the license, but almost every station is required to air at least an hour of news a day.
There's also requirements to air a certain number of hours of educational material for children (https://www.fcc.gov/consumers/guides/childrens-educational-t...).
Except, I thought, if they're explicitly registered as entertainment channels? (Like... Fox "News".)
So like, sticking only to the first and second derivative in soap opera plots about retired civil engineers?
No late-period Beethoven sonatas as background music?
That must be difficult for you as a German.
Meanwhile, here in America we're perhaps a few years away from something like the movie "Ass" from Idiocracy winning an Oscar.
On the other-- we have a network that bills itself as educational and spends over a month marketing a full week of programming dedicated to propagandizing its audience to be maximally afraid of sharks.
It broadcast a wildly popular movie where a tornado full of sharks attacks a city.
German TV could be an order of magnitude worse than my parody and it still wouldn't even register on the American scale of stupidity.
Very true, this is the first thing I thought of when I head of the popular show "Naked and Afraid".
https://boingboing.net/2014/07/04/police-squad-was-cancelled...
> From my standpoint the German television landscape is completely doomed, because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid and must not ever be confronted with content that shows them that there is still stuff to learn and understand in the world.
Well, that would be more bearable, if half the TV market wouldn't be allowed to essentially tax everyone to finance their drivel.
There are some high quality shows and TV stations though. Namely Phoenix (similar to PBS in the U.S.) and some of the news magazines that run in the late evenings. Of course there are also all the other public stations with higher quality programs but I find the program most of the time quite random and sometimes even a bit elitarian.
I haven't watched german TV in ages, but I distinctly remember science shows degrading from science to thinly veiled ads - things like literally running a companies marketing video or making a "scientific comparison" where they hand out random style points at the end to make a specific product win. I think they even got into trouble over it since ads and science/education shows are taxed differently. Anyone pretending that they are doing that for their viewers is living in denial at best, but probably just outright lying.
> because the people at the levers are in the illusion they do the good thing for "the small man" while in fact they just think the small man is incomprehensible stupid
So basically they have a Hacker News mindset!
In every thread about a dumbed down GUI/website it is argued that granny wouldn't understand it otherwise. No power user allowed, because data shows user is monkey.
For example, When I watch youtube on my Roku, if im not signed in, does roku still aggregate what I watched on youtube, could that be sold to Hulu, for ads when I watch that on the same tv?
Ad data is known by the DSP that serves the ad (if the company doesn't serve their own ads), but viewership data is secret as a competitive advantage.
Let's say everyone get their wish and ads go away. Everything will require a purchase. Those purchases are logged to a real name/address. You end up with bigger privacy leaks.
People will still be tracking you the way they are now. And at the credit card level.
As an adult in the first world I can afford to pay for adfree solutions. Most people can't. Ads level the playing field.
If you're willing to handle ads, they should at the least be untargetted.
From your point of view, free open source is something that wouldn't exist
What you are left with is the hobbiest websites or the mega brands that want to funnel you into their ecosystem. To offer anything that cost resources that you are willing to spend you must be a megabrand using this as a loss leader opportunity.
I don't think we want that world. We may think we do but look at what happened in Mr. Robot.
That's the world we have right now, though, just with the addition of the spying that advertising brings.
No, it won't. There was plenty of high quality stuff on the internet before ads or payment was even possible, and there's plenty of high quality stuff that don't track you or require payment right now. There's no reason to think that would all evaporate.
With ads or not you are still the product. You will still be tracked because people want you to spend money on their service. People will sell that information. Companies will use it.
> With ads or not you are still the product.
It depends on the site. There are lots of sites where the site operator has no interest in it generating an income, let alone a profit. You are not the product there.
Ads obscure solutions, and add redundancy and complexity with zero value added, because solving a problem means you no longer are on the page seeing ads. Simplifying or automating a process means you are clicking less pages less often and not seeing ads. If you automate something to directly connect users with what they need, then they don’t need to come back and see your ads. So we have automations they bring us to some middle man that can show us some ads before we can get to what we need.
Ads mean that maximizing the time your attention is held is the core value. High quality content that leaves you informed/satisfied/fulfilled is worthless compared to low quality content that is just good enough to keep you from leaving, without being having enough substance to actual fulfill you, because then you might leave and not see the ads we have to show you.
Podcasts show us that a tiny minority of users able to pay for content subsidize an incredible amount of added value content for everyone, whether they can pay or not.
Ads don’t level the playing field. Ads are an ever growing tumor, sapping resources and weighing everything down in a mindless effort to replicate.
Data poisoning for something the scale of Google I fear would be ineffective to the point of laughability, sadly.
Though, I would say this can actually hurt publishers in the long run
Believe it’s Scott Galloway who said advertising is a tax on the poor and technologically illiterate.
They are also JavaScript based, so how long before ad-blockers start blocking them too?
Isn't it time to come back to server side analytics?
By server side analytics I meant using a tool like AWStas or Webalizer which parse web server logs to tell you how much visitors you had on your site.
I can't see how it can be blocked.
Sorry for my misunderstanding.
Smuggling tracking through first party proxies is certainly an easier way to avoid privacy protection systems on your customers' devices, but it's definitely not the silver bullet people want it to be.
You'll probably have to use inline scripts and send data back bundled with actual data necessary for your site to run if you want to sneak past the user's defenses. But if a site has that much of an adversarial with its users, maybe it's time to stop using that site.
[0] https://pirsch.io/
Tools like AWStats or Webalizer are really outdated nowadays.
I welcome that; if I want to give feedback I'll do so myself. Analytics should not be opt-out.
- Despite having equivalent desktop (macOS) and mobile (iOS) apps, most of our users (> 75%) use the app primarily on desktop
- Most users say the key reason for use is for privacy protection; even with Safari providing some inbuilt privacy tracking protection
- Our app is focused on providing a simple 'set and forget' ad blocking approach; so you would think the key audience would be less tech savvy users. However a large proportion of users are tech-inclined and knowledgeable users.
- A lot of tech heavy websites are some of the worst offenders in terms of tracker usage and advertising. For example, The Verge can load 3.5x faster simply by using an ad blocker that blocks the on-page trackers and ads[2]
[1] https://www.magiclasso.co/
[2] https://www.magiclasso.co/insights/difference-adblocking/
But yes, in the end they are both crappy vendors.
Blokada's UI is without peer and so it makes for a very good "just works" for the majority (in fact, from what I know, it is the most downloaded DNS-based content blocker on Android by far).
However, it is disappointing that some of their decision-making is found wanting: https://www.reddit.com/r/privacytoolsIO/comments/papgeq/any_...
> prevents apps and browsers installed on your device from sending your private data (known as tracking fingerprints) to the Internet.
Is it doing some kind of packet inspection?
As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.
Would Blokada work alongside that?
For now, Blokada's utility is limited to DNS-based content blocking. It cannot and does not prevent most forms of fingerprinting.
> Is it doing some kind of packet inspection?
Yes, only DNS packet inspection, but even for the only thing it does, it is clumsy: It leaks DNS requests; that is, Blokada does not trap all DNS traffic on port 53, and it does not handle DNS queries sent over TCP. DNS66 has these same issues, too.
> As a secondary layer of blocking I use DNS66 which intercepts DNS requests and fails them for blacklisted domains, by installing itself as a virtual VPN - essentially a cooked /etc/hosts for Android.
Blokada uses the same trick (I mean, core parts of Blokada 4 code-base does bear similarities with DNS66 which preceded it... Blokada 5 however was re-written in Rust).
> Would Blokada work alongside that?
No, it cannot. But: Apps that support "DNS proxying" (like Nebulo [0]) can. It is quite an involved setup. I'd simply use Nebulo over DNS66, as it is not only more capable but also encrypts DNS traffic unlike Blokada 4 or DNS66.
> ...but you seem to be a good person to ask..
A disclosure, rather something to keep in mind: I have been accused of spreading fud by the Blokada lead developer and using it to "market" a "competitor" app I co-develop. In my defense, it wasn't / isn't fud what I spread, unless fud === uncomfortable truth.
[0] https://github.com/ch4t4r/Nebulo (fixed link, thanks u/NoGravitas)
https://github.com/Ch4t4r/Nebulo
IMO what really killed uM was major API changes by chrome and Firefox.
https://f-droid.org/en/packages/net.kollnig.missioncontrol.f...
I think it's pretty clear that they are discussing an App Store policy, not analytics collected by their app.
I find it this comparison a little misleading because Plausible admits that their own script/endpoint are blocked by adblockers, just to a lesser extent than Google Analytics [3].
[1] https://github.com/NYPL/google-analytics-proxy
[2] https://developers.google.com/analytics/devguides/collection...
[3] https://plausible.io/docs/proxy/introduction
"Data driven design" is kinda overrated anyway. It's better to actually listen to your users. Stats don't show what people think, and they're often tweaked to show what people want to see.
This is not to say the data is useless - it's just worth remembering that, unless you have strong statistical background, you're probably reading your data wrong, so your metrics should be treated as weak evidence at best.
--
[0] - It's not snark, it's a corollary of the replication crisis. This is a hard job.
Some of the problems in talking to users:
- people will often tell you what they think you want to hear rather than what they really think. (Very few users would ever tell an interviewing dev that they think the product is unsalvageable, the dev team wasted several months and that the user would never ever use it)
- Users often don't know what they want because they don't understand the full range of possibilities. Sometimes they dream of tech that would violate several laws of physics, at other times they fail to realize that for-loops exist and just ask for a faster way to manually click a button a thousand times.
- Fads affect users just as much as anyone else. If there has been a surge of news articles about how Google is doing fancy AI stuff, you can bet the bakery on the corner of the street will mention machine learning when asked what they want to improve their business.
I worked retail in The Time Before The Web and we collected and analyzed data from our customers. We had a door clicker to give us a daily record of traffic. We did detailed analytics on what items customers bought when, to inform inventory and staffing decisions. We did regular people-counting studies to evaluate our merchandising in different parts of the store. We did coupons in local papers for which we could directly track redemption. This was all in the service of making the store work better for our customers.
Now, you might say that those are all anonymous aggregate piles of data. Well, so are most web analytics packages, including many client-side JavaScript.
The difference between the two is that there's no question that counting heads walking through a door and watching inventory patterns are not personal to the customer.
With telemetry, including web analytics, there is always a question because there is no visibility as to their actual practices. It all amounts to "trust us". And we've had more than enough experience with these things to know that such trust is frequently misplaced -- so none of it can be trusted.
And GA especially cannot be trusted, and also happens to be pretty much ubiquitous.
As a side note, it's become just as dangerous to be in a physical store as online now. People who prefer not to be spied on must pay in cash, be sure to put their phones into airplane mode before entering the store, and so forth. And with the increasing adoption of face recognition in stores, it becomes risky to even show your face. In other words, they have to be as "on guard" in the store as online.
Why is it more important to hide the fact that you're interested in cars and Netflix, but not important to protect free speech, free enterprise and free association?
I'm absolutely shocked by why someone would want to protect against a vendor selling me a new mobile phone when the real threat of social media is promoting left-wing extremism and censorship.
It's less about "hiding" or "preventing" anything, and more about putting the user in control of sharing those things. If I tell you I like figure skating, here on an online forum, I'm making a conscious decision to share that fact about my life.
If a script completely silently figures out I like figure skating, because of websites I previously visited, that it was able to infer due to various data sharing setups (such as google analytics overprevalence etc), that is not a choice I made. It's a form of stalking.
The problem is accentuated because it's not just one hobby, it's everything. Your hobbies, your location, gender, religion, income, political affiliation, etc.
That it serves ads based on that is icky, but the problem is that it knows those things at all without me choosing to share that. And this is without even getting into "bad actor" territory.
Privacy is control over what you share with others.
Personally I hate analytics because it makes my equipment work against me, or at least for someone else. Consider that there are websites and apps that are so overloaded with js and ads, that they will run sluggishly on anything but the latest phones and laptops. I would rather not allow that.
I have no problem with static ads. I love watching creative ads on TV during a sports broadcast. Instagram ads are another example of advertising done well. Minimal distraction from the content, no slowdowns, easily dismissible.
As for censorship, I agree that is a more important issue. But we can focus on more than one issue at a time. I will take a win on either front.
Analytics is able to track your speech, enterprise and association to facilitate their encroachment. Part of the metadata collected from messaging apps, for example, is the network graph of your interactions with other users. Association. If you're not using something E2EE, that's speech as well. And so on.
> the real threat of social media is promoting left-wing extremism and censorship.
Oh, you're one of those.
Getting 100% blockage [0] on this adblock test [1].
[0] https://i.imgur.com/7hdaHmh.png
[1] https://d3ward.github.io/toolz/adblock.html
Haven't seen an ad in months. (kidding)
Or, if you’re lazy like me, just use NextDNS when you’re outside. There’s even an option to only activate NextDNS when not connected to the Wi-Fi of your choice.
The Firefox numbers above are a much better tell of what a "largely tech savvy crowd" looks like. 88%.
It's basically a real time website log analyser which gives you enough information to know whats happening on your website but doesn't require any pesky Javascript etc to do it.
Also since it is works by analysing your log files it can never be blocked.
(1) https://rt.goaccess.io/?20210826211303
One of my clients' website traffic is composed of over 75% bot traffic. Server-side logs are unusable for anything other than site performance.
I have two questions about this:
- Since you know they are bots, why couldn't you filter them?
- On the other hand, couldn't there be bots that run JavaScript which would be tracked client-side?
A decent amount can be filtered by UA. This is inconvenient because UA is a very large piece to log and index on, so you need to do UA processing to do anything useful with it and … well by that point it just becomes a chore and I suspect there's good logging services that do this better than you'd spend your time doing yourself.
> On the other hand, couldn't there be bots that run JavaScript which would be tracked client-side?
They exist but they're more rare by nature, because running JS at bot scale is expensive.
Which is why the OP suggested using tools like goaccess.
Using Lighthouse might be an interesting experiment though.
There is also an option to ignore IPs/Referrers too which can work very well for such cases.
(1) https://goaccess.io/features#:~:text=Spot%20aggressive%20hos...
I'm unclear how broad you intend that second sentence to be, but there's still a ton of info you can glean just from server-side logs:
- Referrer info, and, by extension, popular search terms being used to find your site;
- Paths on your site causing 5xx errors (so pages which might be triggering an error in a server-side script)
- Paths on your site causing 4xx errors, and associated referrers (might be broken links on your own site; might be stale search engine indexing)
- Mobile vs desktop access statistics
Finding this data among a bunch of bot-induced noise might be annoying, but if they're good bots and sending proper UA headers specifying their botness, it's easy enough to filter out. Even otherwise there might be typical bot-like behavior you can find and account for such as not sending a referrer header or trying to access known exploitable PHP scripts (in which case you should block that IP address for a few hours or days - there are programs which can do this sort of thing automatically but frustratingly I can't recall the name of one off the top of my head right now).
Granted, a lot of this can be spoofed, but I'm pretty sure the number of people sending spoofed referral or UA headers is dwarfed by the number of those (like me) who block Google Analytics and similar cruft entirely.
Frankly I would love to see some serious low-config solutions to analyzing server-side logs. Oh, especially Fastly. Client in question uses Fastly and it blew my mind to find out that there was nothing in place to answer simple questions such as "what are the slowest paths to respond", "which paths are a cache hit most often", "which paths are most hit overall", etc. And being able to look at various dimensions such as browser, bot traffic, country of origin, etc. If you have suggestions…
fail2ban
[0] https://pirsch.io/
I remember spotting such study.
I still wonder if advertising is always worth spending on, beacuse google and facebook make a big amount of money with it, but I'm still skeptical when I see the actual amount of people "engaging" in ads, when you see how much it actually costs.
That’s why the hostility to ads feels misplaced: it works, it helps company to whom you want to give money to find you and people like you. It’s just that condescension from _some_ people in Ad platform and a press that insistently favours the worst people mean that people like me who argued for more user-friendly interfaces (ban certain content, prevent repeated ads) were routinely overruled.
You mean, the companies who want me to give them money. I'm sure there are a few companies out there whose product I actually want and don't mind paying for, but their number is absolutely dwarfed by the number of companies that seek to induce FOMO/status anxiety/etc in order to get me to buy things I don't need and would not want if not for advertising.
How many problems do most people have that can be solved with an existing product they are unaware of? The numbers can't be very high.
How do you explain well-known brands advertising the same product for decades on end? Surely Coke and Pepsi aren't suddenly enlightening many people to the existence of their drinks.
How about ads that get shown to the known-same individual time and time again after mere minutes? (See Hulu, at least back in the day, not sure what it's like now.)
The whole line about ads being mere consumer education is ridiculous and doesn't even stand up to a cursory thought.
I have no hostility to ads themselves. I have a huge amount of hostility to the spying that comes with them.
My startup has tried a variety of marketing strategies from in-person campaigns on the street, video ads on YouTube, "free" PR through newspapers etc. In order to measure the effect of each approach we only did one at a time.
For us paid marketing on Facebook/Instagram was, unexpectedly, the most efficient form of marketing by far. But I would not assume that applies to all, or even most, businesses. So you should experiment with different strategies for your business.
No marketing dept is dumb enough to equate brand-keyword traffic with organic traffic.
(Paid) reviews on the other hand like unboxing, configuring and testing a product that I'm interested in are totally another thing. This applies to furnitures, house appliances, computers and so on. A good example is that I did not knew how much I wanted to build a fully silent computer before watching so many build videos of a certain fanless case that looks like a metal cube.
So what exactly does Plausible do to avoid being blocked in the same manner? What prevents GA from doing the same?
It seems they serve plausible from the same server as the origin website (first party), probably under an unique name, which means you need to block the script manually on every domain instead of simply blacklisting `plausible.io` (for example).
> What prevents GA from doing the same?
That's a good question. Probably ease-of-use (no need to explain how to host it, easier updates) and the lack of need to do so. YouTube, for example, could also make it _far_ harder to block ads by simply sending a single video stream, but they don't for some reason.
One option would now be to host a plausible proxy on-premise. But I didn't have time to try that out yet.
PS: We're paying plausible customers and our stats are publicly accessible [2].
-1: https://rugpullindex.com/blog#HowPlausibleareOurWebsiteAnaly...
-2: https://plausible.io/rugpullindex.com
Similarly, wrapping websites like Discord or Slack in Electron also gives the website owners full telemetry and tracking that they can't get in a tech savvy browser.
Would an always on VPN, a remote pihole be the only way for privacy?
E.g. if I wanted to know where you're hailing from, I'd browbeat you into granting me Location access privileges. If that's too difficult, I'd get you to grant me Files/Photos privileges (this one won't raise too many alarm bells with apps like Discord or Reddit), and then try to read EXIF geotags off your recent photos.
It must be very disheartening to work on reddit's mobile site. Not only is it deliberately made a miserable experience by forcing you through AMP via Google and insisting you "continue in browser" every time, but then you're greeted with a banner that outright says "this page is better in the app".
You can probably cross "mobile" out of this entirely.
Reddit have spent three years now building sites that are worse in every way than the decade old junker it's trying to replace.
So, maybe, they should have improved the old design instead of creating that insanity that is newreddit.
For now I still use it because it's significantly faster and easier to use, but I strongly suspect they want me on their new site.
The amount of times it crashes on mobile platforms is insane.
If they cared that much about whether or not what they were working on made the world better or worse they'd never have taken a job at Reddit.
It works very well, I highly recommend it.
That and revenue is much higher on apps than on the web. I made an android apps for a website. And just alone the android apps made more money than the web version. With less ads.
[1] https://www.zenz-solutions.de/personaldnsfilter-wp/
I'm using NextDNS as the system-wide private DNS on my Android phone, it works great and eats less battery than Blokada.
What I do not know is if it will work also when apps begin using DNS over HTTPS… I suppose not?
What I do and recommend everyone to do is:
1. Run an edge network device using network access controls and filter which devices on your network get outbound network access (in my case just the gateway device). Block all inbound traffic except what you choose to pinhole, block all outbound traffic except ports you choose to add to the allow list.
2. On every client device run a local application firewall (I like Vallum and Little Snitch on MacOS as examples) and filter applications by domain + port on outbound requests, block all inbound requests.
3. On every client device force it through a VPN to a gateway device internal to your network to get internet access, anything that falls off the VPN is then blocked from the internet. The gateway device can forcibly route traffic and perform additional filtering
4. On every client device, configure it to use an internal DNS on your network with a fallback to a trustworthy external provider, have the internal DNS use a trustworthy external provider over DoH. Block outbound DNS at the edge device (blocks all non-encrypted lookups).
It's kind of a pain, and a mess, but it does greatly restrict the damage that rogue IoT / Smart devices can do.
ipset create blist_doh hash:ip hashsize 1024
for ip in `cat /etc/bin/blist_doh.txt`; do ipset add blist_doh "$ip"; done
iptables -A <insert some iptables placement specific to your outgoing/forwarded traffic> -m set --match-set blist_doh dst -j DROP -m comment --comment 'SPY:all ext DoH BLOCKED'
Still, as of 2021, doh rule is around 2% traffic logged compared to my other rule, where I simply block outgoing 53/udp (except my resolver). a LOT of your devices ignore your dhcp dns settings and try to circumvent it going directly to shady 8.8.8.8 etc.
(*) you shall every few months check and update it.
Maybe for now, but it's just a matter of time until use of DoH to circumvent your attempts at redirecting DNS becomes more widespread as well.
The only approach that I could come up with to do so was to install a proxy to MITM all HTTPS connections to allow me to filter out DoH requests.
Here's an idea that supposes you have some server available: Force all traffic over your network, even when roaming (VPN). Squid+Privoxy as HTTP proxy (your choice if you want to DPI and add own TLS CA as root on devices, in which case you can actually block it, or just do HTTP CONNECT). Optionally also set up a SOCKS5-proxy (microsocks). Configure all applications to go over proxy. Applications that don't support it can usually be "proxified" with tsocks or proxychains4.
Now you have more fine-grained control over your traffic. You can do fun stuff like setting up several incoming ports, several outgoing proxies (including Tor/I2P) and allow/block based on destination, domain, protocol, etc.
Finally disable outgoing traffic on your devices/route to a black hole. Or route it via your VPN at least, where you can do some rudimentary stuff with nftables/iptables. Oh, and something not everyone knows: you can route traffic based on uid/gid on Linux, so on your workstation you can leverage that too selectively control traffic on an application-basis.
On non-rooted Android and iOS, I suspect this is not perfectly achievable but if you have the time you can get pretty far otherwise. It's an arms race and I feel like we can keep up.
Discord, Slack and other similar webapps can (and maybe do) send telemetry in the same connections used for the app's features. You can't reliably block that.
I've since moved to NextDNS.io to have network agnostic ad/malware filtering. Still keep a "special" configuration for my wife :)
I was as confused when i first saw job postings for "SoMe experience", and figured it was a MeToo thing :)
If there's no complex interactivity achieveable only through JS, I prefer to ship sites with plain HTML/CSS, so I don't really like to use any client-side analytics. And yes – I can live without all the fancy features to track and profile my visitors.
I don't want to host by myself either, since I enjoy using a simple git push to deploy my site.
Moving to a normal analytics, the number is closer to a dozen, maybe a few hundred.
I suspect Netlify doesnt remove bot traffic
* Use a CDN which provides access logs (AWS Cloudfront, logs are stored in S3, Azure supports this too i think) * Feed those logs into a self-hosted analytics solution, like goatcounter [1]
[1] https://github.com/zgoat/goatcounter
[0] https://pirsch.io/