How transparent is Scarf's product adoption metrics for OSS projects? https://about.scarf.sh/
I follow them on Twitter but haven't looked much into it other than reading their documentation, which makes me think that most of their telemetry is done at the point of the package distribution system: https://about.scarf.sh/package-sdks
Nope, nope, and more nope. You're not moving the Overton Window any more on me.
In fact it seems there's a clear correlation between the quality of software and how much spyware there is embedded in it. It's often merely another way to justify unpopular changes with "but the data says so".
IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.
> IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.
Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usable at all. Given that I’m unwilling to implement an opt-out system, I don’t really see a workable approach here at all.
What I hear you saying here is that people don't do what you want if you give them the choice, so you lean towards not giving them the choice rather than respecting their wishes.
> [Y]ou lean towards not giving them the choice rather than respecting their wishes.
> Is my interpretation correct?
I don’t think it is, no :) Rather, I’m not sure how to sell, to put it crassly, users on a choice when properly investigating or even being confronted with that choice would delay them seeing the dancing bunnies[1], but that would also, if I have any say about it, improve the bunnies in the future.
Does that mean there’s a shade of “I know better” in my problem statement? Of course it does, if I didn’t know better than the average user I’d have no business designing such choices. I don’t think there’s anything wrong about that, better than the average at an activity few practice is not a terribly high bar. Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though.
Basically, how do I make the user think, how do I give them the appropriate data to do so, and how do I deal with the obvious contradiction of that goal with principles of good design[2]? The potential benefits to the software and (thereby) the users are too much to give up without even asking those questions.
> I consider opt-out to be a manipulative approach.
So do I, which is why I wrote I’m unwilling to implement it :) The original (and, to be clear, purely theoretical) point was, opt-out is too manipulative while opt-in is likely useless.
Ah shoot. Did you take that to mean that I’m unwilling to implement an off switch at all? That wasn’t it, sorry for the confusion.
The struggle is real. As a developer, more data is obviously desirable and can make development much easier. I just can't think of a way to do telemetry that, if I were a user, I would accept. And I don't want to produce software that I wouldn't personally use.
I just don't know how to have my cake and eat it too.
As a developer your entire purpose is to make decisions for users. "Where should this service live, how should security work, how should I increment their billed service usage, when should I shut down their vm..."
I don't think the issue here is making decisions for users and not giving them a choice. 99.999% of software does not have a flag to change it. The issue seems to be more about the precise nature of this specific feature.
> The issue seems to be more about the precise nature of this specific feature.
Of course. Not really just this specific feature, but any and all features that can violate users privacy or security. In the end, I don't think these are decisions that developers should be making for users, because not all users have the same needs and getting this wrong can do harm.
That's why, for these sorts of things, meaningful user consent is critically important.
> people don't do what you want if you give them the choice, so you lean towards not giving them the choice rather than respecting their wishes
This nicely summarizes a very popular approach to telemetry – and to a variety of user-hostile behaviors. Web sites (for example) seem to have mastered the "fight against user preferences" approach, trying to play video when autoplay is blocked, using javascript modals since pop-up windows are blocked, fighting ad blockers, ignoring "do not track", etc..
If users are given any choice, usually it's a difficult opt-out process, which is more effective precisely because it makes it harder for users to make the choice that you don't want them to make, even if it isn't their actual preference. For an extreme example, see Facebook's (anti) privacy settings. Commonly used dark patterns further amplify user manipulation.
How many people are installing and using open source software but couldn't understand a pop-up explaining what data is collected and asking if they'd like to submit it? Is the non-technical nature of the user the problem or is it just that when you have an opt-in option most people make the choice to opt-out? That's the thing about respecting users by giving them choice, they get to say no. If they mostly say no, and you don't get enough data, that's the will of your users and therefore not really a problem.
> How many people are installing and using open source software but couldn't understand a pop-up explaining what data is collected and asking if they'd like to submit it?
I’ve taught probability theory using randomized response[1] as an exercise problem, and while people can understand it given time and motivation, it’s not immediately obvious. So I’m not exactly hopeful that a prospective Audacity, Blender, or even Free Pascal user (to take an arbitrary set of examples) would get what I mean if I say “I’m collecting no more than 10 bits of information about you using RAPPOR”[2], and I’m not willing to engage in comforting bullshit such as “all collected data is anonymous”, as I’ve been all too close to situations where the difference between the two might be one between freedom and prison.
> Is the non-technical nature of the user the problem or is it just that when you have an opt-in option most people make the choice to opt-out?
Both, because confirmation dialogs, especially privacy-related ones, have been thoroughly poisoned in users’ minds. But confirmation of obscure actions, however beneficial their consequences, is problematic in general—if I go on the street and ask people if they’d like caffeine in their tea or ascorbic acid in their apples, I expect (but have not checked) that the majority will say no, nevermind that both are normally there and intrinsic to the experience.
(The possibility of meaningful consent from a non-specialist is the subject of much discussion and few good answers in med school, or so I’ve heard.)
Whether the ultimate answer is to grant or deny permission, I’m not sure I can present the question in a way that will actually have it made on the basis of merit and not on “scary permission dialog, better say no” or “yes, yes, just let me through to my dancing bunnies[3]” or “yes, if I say no the installer will just tell me to GTFO”.
(In that respect the “Send crash report to vendor” button is unexpectedly good, because you’re not actually interposing yourself between the user and any prospective bunnies. But personally I don’t like to spend time and effort in order to send “feedback” into an unmarked hole where I’ve no idea if anybody will ever look at it. From that point of view, it is background data collection that’s unexpectedly good.)
And even if, for the purposes of this question, it would be best if people took the time to learn the necessary maths, computing, and operational security to make an informed choice, in reality I’m not sure that’s the best thing they can spend their life on.
So it may be the answer is that you simply can’t do telemetry well for the social reason that users won’t ever end up making an informed choice, or that the well has been poisoned so thoroughly that the rational choice is to reject everything. It’s just that I know that it’s basically possible in a technical sense, so I don’t want to give up that easily.
First, we are talking about Development tools, so not non-technical population.
Second, if Opt-in is considered difficult for the population, what does it say about the opt-out?
Opt-out is always, no exception, more difficult than opt-in.
Seems like you also[1] didn’t read the above the way I intended. I meant that I find explicit opt-out (as opposed to explicit opt-in) manipulative so I don’t want to implement it, not that I oppose having the ability to opt out at all.
The difficulty, though, lies not (entirely) with the default position of the toggle, the difficulty lies with making the user think about the question which is not relevant to their immediate task and which in any case they may not have the theoretical tools or time to evaluate properly. The default position of the toggle (if “off”, as I believe it should be) matters only because an opt-in process means you either confront the user with irrelevant questions on first launch or get essentially no data.
(I called this out as a a “general” question because I meant for it to apply not only to the Go toolchain, but to general-use software like Firefox or niche but non-programmer-oriented software like Audacity.)
The systemwide daemon proposed elsethread[2] would solve this nicely as well, but I have to admit that I’ve dismissed it from my thought process more than once before, because I didn’t think we were going to get one with any reasonable usage on any platform. Now that I’ve seen it put in writing, maybe it does deserve to be considered.
So how would you design a well-working system to make data-driven decisions rather than guesses? Most collection methods are notoriously bad, but partial collection is also bad since we now have to somehow put a weighing factor on presumed absent data, which turns choices into guesses again.
I think this is a really hard problem, and simply trying to guess in the dark as to what people want isn't the smartest way to go about finding the path forward / priorities / improvements / defects.
It also isn't something that we had in the past, because when we used to buy the IDE, buy the compiler, and then build software, sell that software, and let everyone know what cool tools we used, you'd have sales figures that would inform the creators how the tools were used. Now, the tools are available to everyone, anonymously, and everyone has an opinion on how well it works for them, but doesn't have the time to write a well-written report every time a release happens.
This is well done. It only exposes counters, and rather then pushing data up, the telemetry server must know the names of what it can ask for. No wildcards.
> Although the report would not include any identifiers, the TCP connection uploading the report would expose the system’s public IP address to the server if a proxy is not being used. This IP address would not be associated with the uploaded reports in any way.
Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software still phones home and the IP is still being collected and logged when it otherwise wouldn't be. Either when uploading the report or when downloading the “collection configuration”.
Honestly, assuming full transparency, I'm not opposed to the concept. I question how much telemetry is actually necessary, but I'm certain there will be times when it's nice to have. It'd also be interesting to see how it would go when for once people can see exactly what is collected, when, and from where.
I'm not sure that Google is the best place to showcase such a concept though. I'm sure there are a lot of people who have no problem with handing more data over to Google, but Google has abused the public's good will for the sake of data collection many times, and it's sure to put off some of the people who aren't already completely disgusted by the idea of their favorite open source projects collecting telemetry.
> Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software still phones home and the IP is still being collected and logged when it otherwise wouldn't be. Either when uploading the report or when downloading the “collection configuration”.
How do you verifiably not collect users’ IP addresses when receiving data from them? The verifiable part is the problem, of course you can (and should) just not log the addresses, but then the users can only trust you (and hope you or your uplink haven’t received any legal orders to the contrary). The only approach I can think of would be a Tor hidden service, but while it would technically work, as far as not exposing your users to scrutiny it actually sounds worse.
The only option is to have a proxy sit in the middle between the uploader and the server. You mentioned Tor but it doesn't have to be Tor, just some proxy most users would trust not to collude with the server and that doesn't itself derive benefit from seeing the IP addresses. If there were a different entity that could be relied upon to run servers doing this and were highly trusted by users, I'd be interested to use it. Failing that, the usual answer for an enterprise or company is to run their own HTTP proxy. The design explicitly supports that.
> their favorite open source projects collecting telemetry.
Their favorite Google open source project. This is specially important for project which can't realistically exist without main sponsor / benefactor. It also help people to pay whatever little/high cost in term of conscience when they take part or consume something willingly but do not approve of makers.
> the vast majority of projects, even large ones that would benefit, stay away from telemetry.
Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents?
Surveys about this are just asking humans to do something computers can do better.
Obviously privacy and consent are paramount concerns, but not only are they solvable, in open source they’re fully auditable (and a fork could fairly easily maintain a patch that removes it outright).
I think open source largely rejecting telemetry puts it at a huge disadvantage to proprietary and SaaS software where it is the norm. I’m very excited to see someone as thoughtful and well reasoned as Russ Cox to be trying to move the status quo forward.
The times "we" (previous companies) tried to implement telemetry in open source non-SaaS products (as distinct from "projects"), we either got huge blowback or users/customers simply blocked it at the firewall (and security teams at major enterprises were unwilling to open holes anyway).
The only workable solution I found was integrating this in a value-add way, so that something in the service/experience/etc was better for the user/customer as a result of enabling telemetry, without the dark pattern of making things intentionally awful/worse without it. We simply never got enough data to matter otherwise. But, again, that was products and not projects.
> The only workable solution I found was integrating this in a value-add way, so that something in the service/experience/etc was better for the user/customer as a result of enabling telemetry, without the dark pattern of making things intentionally awful/worse without it.
This sounds like a great concept, but I'm struggling to come up with concrete examples - how did you approach it?
On the contrary, I'd argue that the tracing visibility you're looking at isn't inherently a software trait at all. It's a deployment feature, which is something you address at-cost when building a product, but almost never when building FOSS software. It's not that people in FOSS don't see that upsides to it, it's that those upsides are insignificant relative to the cost of sustained market research. It's easier to just... make stuff, and have companies plaster over the gaps when their interests align.
Nothing wrong with what they've done here, but we already had most of these metrics. Nothing was really learned, and it took Red Hat and a few thousand users to get here. For smaller-scale projects, imagine how much smaller the returns would be.
We have to ask for permission on our SaaS products to collect this data as it's not necessary to collect it for the product to function. The EU GDPR mandates this.
Russ Cox is suggesting that there is no permission step and that the data is collected by default.
From my reading focused on this specific issue of the GDPR and the national laws of member states, this is not the case. Opt-in is specifically required for personal information. The telemetry data outlined in the proposal would not fall under this requirement. You can even retain time-limited IP logs with some special caveats. The GDPR is actually quite reasonable and fair.
Russ Cox is a very intelligent and effective engineer. He has a history of projects where he first analyses the problem space, then arrives at great solutions. He puts a lot of effort into discussing the problems and proposal with the community, especially after the widely criticized go mod decision by the go team (which is now mostly accepted as unfortunate, but in the end, the correct decision, I would think).
My point is: We all suspect Google and telemetry to be bad. But can we be charitable enough to separate the Go project, that is run by individual humans, and telemetry from our superficial cliches to actually read the proposal?
Google or Russ Cox's reputation is irrelevant. The idea stands alone. I'm merely crediting him with the idea.
I read the proposal. There is no discussion of the legality of this at all. I'd expect anyone with any level of supposed technical competence to consider this in relation to global data protection. I suspect there has been no legal review as mentioned in the thread because I know how slow the lawyers in this space work and the timeline between publishing this and now is too short to have had a conclusive answer.
As for your point about GDPR, I think if you apply your right to withdraw from opt out data collection and what that entails and then ask how this glaring defect is missing from RSC's paper, then you'll see exactly how much privacy consideration really went into this.
I dunno. It sure makes sense to me to collect telemetry from free software installations, but I feel that having every platform or even piece of software to do it on its own with opt-out will inevitably lead to people being overwhelmed and angry.
I would, personally, prefer a single non-profit service that would list publicly what is being collected and publish the results as open data for anyone to use. Applications (at least on Linux) would not submit their reports directly, but would use a local relay service that could be turned off completely or that could filter what reports to send to the server and what to /dev/null.
Distributions and other software stores would then make it mandatory for software to use this relay and either patch out any other telemetry from their packages or straight out forbid those that would not comply.
I think the issue of telemetry is fundamentally a human issue of incentives and trust. The system you describe is wise because it recognizes this and attempts to address it.
The difficulty with telemetry is that even if we design the perfect, privacy-preserving system to begin with, once the pattern of having a network port open is established, there's nothing to prevent us (humans) from changing our policies about what we're allowed to push/pull over that port.
In real-world analogues for these kinds of thorny policy problems, we have centralized arbiters to solve these problems. That might be a fruitful course of research for people interested in this problem to explore.
Unfortunately, even though this problem has software as its medium, it is a problem that cannot be solved by clever software alone, despite any appearances to the contrary.
> The difficulty with telemetry is that even if we design the perfect, privacy-preserving system
The other difficult is what you mentioned: trust. Even if a piece of software really does telemetry in a perfect, privacy-preserving way -- as a user, I have to take the developer's word for that in the end. That's a hard hurdle to pass, because that trust has been violated so much in the past that nobody gets the benefit of the doubt anymore.
> Unfortunately, even though this problem has software as its medium, it is a problem that cannot be solved by clever software alone
I agree entirely. At the heart of it, this is not a technological problem. It's a human one.
Honestly, this may be unpopular with hacker news, but just add your own telemetry. If people don't like it they can turn it off, and telemetry is essential for a good product.
That is fine, but in this case telemetry trades you (and other more hardline users) as a user for all the extra users you gain from instant crash reports, quick feedback, and generally better productivity.
I would never personally make that trade-off, and would always put (disableable) telemetry in my projects.
Well according to our telemetry 0% of users turn it off so it seems pretty popular.
But more realistically what you gain in privacy you give up in having your voice heard by the devs. The decisions about the future of the product/project will be driven by the data, specifically the data from the kind of people who leave telemetry on.
If 0% of your users disable it, that kind of screams there's something wrong with your opt-out mechanism. Is it broken? Hidden? Difficult to do?
I mean, with any group of people, there will always be a percentage that will disable it. If the telemetry is popular, that percentage might be very small, but it would be non-zero.
See, I _am_ a dev. I run telemetry on my infrastructure, I analyse it and fix what's broken, and if necessary, try and get upstream fixed. Also, I'm not opposed to telemetry in general, but if a switch like this is turned on by default, trust is broken for good.
Software which does any type of computing without it's user's informed consent is classifiable as malware, mind you.
Those design decisions were never made in a vacuum. They relied on telemetry (or, less scarily, user testing, user feedback, user research, etc) to figure out what works best. As a reminder, our intuition doesn't come from nowhere, rather centuries of survival and expectations. If you do not know what these expectations are, and if you do not know how your users interact with your product based on these expectations, you cannot make a good product. Certainly, you can make a product that appeals only to you, but how many of yous exist?
The design of a teapot is a great one. It didn't magically appear with handles and a spout and a place to hold leaves, but after years of refining based on usage. As shocking as that is, tea wasn't even discovered on purpose, let alone having a specific vessel for it right out the gate.
So yes, telemetry is essential. Taste is personal.
I think maybe even making telemetry mandatory with an open license, and customizable with a support license might be a sustainable way to run an open source company.
For many open source project (anything with an attached business model), either telemetry or tight communication around usage patterns will be necessary to inform development. The latter of those two options consumes business resources.
Mandatory with an open license might shoot yourself in the foot when people fork your code.
For me, the harshest you can go is to have telemetry and not prompt for the setting on start with the user opted in by default. Ideally, you ask on first load, and that's what I'd probably aim for.
You can't ever try to force people to do anything in open source. They'll run right by you and make it do what they want it to do with or without you.
I'd even imagine paying customers are broadly more happy with telemetry than open source ones. And their needs more important anyways.
Just because Linux is open source doesn't mean you can't have both Fedora and Red Hat (an enterprise version built on the same codebase)
I don't think any closed source goes into Red Hat, it's just the patch delivery pipelines, package repositories, etc that require a license. And support of course.
Same with any distributed system whose core contributors could gain insight from telemetry. All the components are open source, but they can package it all up and make it available under different terms.
If there's a community version and an enterprise version, you can then make telemetry required in the community version. If people don't like it, they can pull the package apart and put it back together however they want, or they can pay for an enterprise license.
You can make submitting telemetrics a condition of some other agreement, such as copyright license on the Red Hat name, or a B2B support contract. That, however, is pretty far removed from what's discussed here; if the software license itself makes telemetrics "mandatory", then it's no longer an open license.
The company ships one product which is a community edition of a bundle of open source software. That version has telemetry enabled and can't be disabled. Users who want to patch the code manually can of course disable it, just like they can disable the Ad lens in Ubuntu if they want to build it themselves, and those users will be off the beaten path and likely to run into issues that aren't easy to find paved solutions for.
They also offer an enterprise edition with a support license, on which telemetry is enabled by default, but can also be disabled.
> bundle of open source software. That version has telemetry enabled and can't be disabled.
I'd recommend saying "and cannot be configured off without code changes", or something. Of course it can be disabled, if it's open source.
Go compiler without telemetrics opt-out would fragment the community even harder than Go compiler with telemetrics default on. While your scenario is, of course, possible, it's not very relevant to the Go compiler.
As for your original "might be a sustainable way to run an open source company", if the primary feature one gets buy paying is "easy to turn off telemetrics", that just doesn't sound like enough value.
Well I mean. In a lot of the popular Linux distributions you kind of get what the distro comes with.
Can you configure openSUSE to use apt instead of zypper? I mean, sure, probably, it's open source.
Is it going to be straightforward? I don't know. I suspect it's going to be harder than it's worth, and you might need to rebuild the operating system image to change the directory layout to work with apt's assumptions or something.
So in practice, even though these things are all open source, people who bundle complex software lay down the paths that 99.9% of people will take.
I wasn't proposing "disabling telemetry" as the primary business proposition.
Instead I was suggesting that open-source companies spend a lot of time dealing with issues from people who use the open-source software in unanticipated ways. If "the beaten path" for using the software without support includes telemetry, they get value from that.
If people want to use the software without telemetry, then they're paying for a support license, so the issues they run into which the supporting company has no telemetric insight into are at least better aligned with their support resource allocation.
Init system used to be a big point of deleniation, but I think systemd is the standard now (for better or worse)
The filesystem and networking stack still have some variability.
There's still default applications, kernel modules, a gui app installer, the desktop, included drivers, and many more things that go into a distro. If you go with a distro that uses KDE and you switch to GNOME for example, you might lose a lot of GUI support for customization, might have to build addon packages yourself, etc.
It's all open source at the end of the day, but that optionality leads to a less streamlined user experience and lack of guardrails as soon as you step off the beaten path, than you would get with something like OSX
The company ships one product which is a community edition of a bundle of open source software. That version has telemetry enabled and can't be disabled. Users who want to patch the code manually can of course disable it, just like they can disable the Ad lens in Ubuntu if they want to build it themselves, and those users will be off the beaten path and likely to run into issues that aren't easy to find paved solutions for.S
They also offer an enterprise edition with a support license, on which telemetry is enabled by default, but can also be disabled.
Telemetry in open source exists for a long time. Debian has the popcon package that can be installed and reports weekly usage of the software packages. The telemetry data are published in the open. The Debian popcon FAQ could be used as guideline for other telemetry needs.
https://popcon.debian.org/
It does sound quite similar. But in addition to the crucial difference of opt-in vs. opt-out there's also an interesting contrast in how it's framed.
Debian talks about what you, the user, can do: help out, participate and vote. If you choose to do so.
The Go team talks about what the developers and their software will do to the user's machine, but the user is completely passive in their description. This is also reflected in the term "telemetry" itself: the software is not a tool in the user's hands but rather a remote-controlled probe in the user's habitat that pokes at the user to elicit interesting responses.
popcon should not be used as an example of how to do telemetry, as it is far worse for privacy than the Go proposal:
1. Sends names of private packages to the server, and publishes them.
2. Sends a unique identifier (a UUID stored in /etc/popularity-contest.conf) to the server, which is stored.
3. Doesn't use sampling, so if you use popcon you will be submitting a report once a week (Go's telemetry would average just one report a year).
4. Submits over plaintext protocols by default.
popcon may be opt-in (in the sense that the prompt during installation has "No" selected by default) but the prompt doesn't disclose the large privacy risks.
People are not appreciating the thought that has gone into the Go proposal to minimize the collection of private data, either intentionally or by accident, such as the client-enforced requirement that the the names of counters be published in a tamper-proof log so anyone can verify that, for example, no private package names are being disclosed. Everyone is focusing on opt-in vs out-out, but to me these other details are far more important.
I've been a pretty strong advocate of the idea that analytics should always be minimal, 100% anonymous, aggregated, and open to the public - otherwise it’s spying. This is how we do analytics on our websites today[0][1], and how we plan to do it in games we release in the future. Maybe one day I will start a dedicated FOSS service that people can use for exactly this with some trusted reputation/transparency/auditability to it.
I think what Russ has described here is decent and well-reasoned. I also think that Go being a product (it is, whether you like that word or not) makes it more fair to desire analytics of this form. I think it being opt-out is reasonable (after all, if it is not, they will make decisions using data that does not come from the vast majority of users, may as well not have analytics at all then.)
But I am afraid of this becoming pervasive not just in products (like CLI tools), but also in libraries, imagine every Go/npm package you use wants to ping the network because the authors want to know 'is this popular? can we deprecate XYZ method?' etc. If transparent telemetry in the form Russ and I have been viewing it becomes a more common thing, it won't be a surprise if more library authors begin to try to adopt something like this and it becomes a pervasive problem IMHO.
I am concerned about run-time telemetry in libraries as well. It might make sense for language ecosystems to offer more data about library usage gathered at build time eventually, as a different system than the one I'm posting about today. I think when you get to that level of detail you probably need to start thinking hard about differential privacy and probably cryptographic solutions like ESA or Prio. I don't think we know enough to design the library solution yet.
Telemetry embedded in libraries is simply abusive, in my opinion. At the very least, the decision about whether or not to include telemetry should be made by the application developers, not the toolmakers.
Right. My hope would be that language tooling offering library developers visibility into compile-time information about library usage would reduce their desire to insert run-time collection instead.
> the authors want to know 'is this popular? can we deprecate XYZ method?'
This is something that was common for internal libraries at some of the places I've worked. I'm honestly a little surprised it isn't a thing we see externally. I for sure do not want to see it, but I'm surprised we don't. Its probably enough to look at the public usage on GitHub, and make inferences and post notice on future-major-versions of libraries. Github honestly should make a tool to do this, they'd have a huge opportunity to inspect the data.
Yes. I fundamentally don't care how "good" Go telemetrics would be, because I don't want the FOSS ecosystem as a whole to take any more steps down that slippery slope. There will not be a way back from this.
Checking out Mach and seeing it's written in Zig. Bit surprised to see it being "used in anger" given that Zig is currently at 0.10.1. Can you share how your experience with the language has been so far? Thanks.
Opt-in should be the default where the tool asks for consent politely. Information on how the application is being interacted with belongs to the user not the developer thus it should favour their choice over sneakily enabling it for those that didn't pay attention or don't understand it.
It's on the project to convince the user to turn on telemetry rather than the user having to remember to turn it off. Excuses such as "nobody would turn it on" don't apply.
> 100% anonymous, aggregated, and open to the public
I don't believe the "100% anonymous" is a thing with AI anymore. When AI can identify/fingerprint you by your walk pattern[1], you can't really tell what data can and can't be used to fingerprint you.
This is a good plan, very simple and clear, and I like the list of system properties at the end. The solution is pretty tailored for the Go toolchain, which is a good strategy that has worked for them in the past.
A more general purpose metrics tool I'm watching closely is Divvi Up https://divviup.org/, a research project by ISRG, the same org that runs LetsEncrypt. The basic idea is to divide up each metric into two parts and publish each part to separate collection servers (one run by you and the other by divviup). Then the servers separately aggregate their half and combine the results, the idea being that each half is useless on it's own but when combined it's still useful.
I wouldn't suggest it for this application, but for the majority of typical apps it would be a vast improvement to privacy compared to the status quo.
I am all for transparency and limited intrusiveness of telemetry.
But in practical terms, the problem with this approach -- if I'm understanding it correctly -- is that it has no way to detect and reject outliers, and therefore the data can't be validated in any way. It only makes sense if all your clients are 100% trustworthy.
Let's say you want to know whether to keep supporting ARMv5, and your data says 10% of users are using it. There's no way to tell whether that's accurate, or if you have 0.01% of die-hard users who modified their telemetry code to report 1000x as frequently as they're supposed to. Even if you suspect this is happening (and you might not), there's no way to identify the culprit and filter out their data without tracking personal identifiers such as IP addresses.
So even if most of the time the telemetry data is valid, over time it will trend toward uselessness, because it can be endlessly second-guessed unless it confirms a decision you wanted to make anyway.
I don’t really see why a classic community-driven open source project would care about what non-contributing users are doing with the software. In that case, helpful users come with built-in telemetry (pull requests).
But I guess this could be helpful corporatized read-only repo projects, or other groups that aren’t sure if they are building a community or a customer base.
Because pull requests aren't the only reason you might do a community-driven open source project. Perhaps you're just altruistic, or want to populatize some technology etc.
> When you hear the word telemetry, if you’re like me, you may have a visceral negative reaction to a mental image of intrusive, detailed traces of your every keystroke and mouse click headed back to the developers of the software you’re using.
But that's not my only objection to telemetry. Equally important to me is that so many bad decisions are justified based on telemetry. It's very easy to misunderstand the data, because telemetry leaves out so much, but developers often treat it as if it's giving a complete picture.
As an example, I have seen developers drop really important functionality on the basis that it is rarely used. While that was true, it was also true that when those rare times happen, that functionality was absolutely critical to have.
Or they use the data as an accelerant: move rarely used features to places where they're even less discoverable, making them even less used, and then remove them altogether. The justification then becomes a self-fulfilling prophecy.
I haven't worked with golang in some time. How do golang devs generally obtain the compiler?
If you're getting it from distro repos, it should be straightforward to convince the distro package maintainer to disable the telemetry / patch it out.
Or is it a nvm/pyenv/rustup situation where you prefer to use bespoke toolchain managers to download upstream's compilers?
It depends on which compiler you want to use, but precompiled binaries include the gc compiler by default. If you want to compile from source yourself, you can use gc or gccgo assuming you already have the go toolchain, otherwise you would need to bootstrap from an existing binary.
There are tons of cases where the person installing go won't know that telemetry is enabled otherwise. For example, let's say you're at a bootcamp and the go installation instructions from your teacher don't mention telemetry -- how would the person know to disable telemetry? My concerns are around nation state actors, domestic abuse, journalist privacy, lawyer confidentiality and fully believe that this sort of telemetry can and will be abused in someway somehow, eventually, and probably in some obscure fashion.
Would be nice if this system threw something to stderr at runtime every. single. time. unless the message was explicitly disabled. Something like:
Go telemetrics are enabled! We are collecting {json:object} from your machine for X purposes. If you would like to opt out, run "echo GOTELEMETRY=off >> $(go env GOENV)". To disable this message, "echo GOTELEMETRYWARNING=off >> $(go env GOENV)"
If you run your build inside of a docker container, wouldn't it be enabled by default as well? Docker containers do not inherit the environment from the host that I know of.
From the description of the implementation, it would only send telemetry data after seven days of being on. So, yes, as long as the docker container is up for that long. I didn't see anything that mentions whether it would communicate to any telemetry server on initialization.
Not only is it going to be opt-out (because of course it would be coming from Google), I really like the whole "wait a week before sending telemetry" part that just coincidentally has the benefit of sneaking right past people that actively look for suspicious network activity when they've freshly installed something.
Totally agree. Telemetry has been around and matured and benefits users. I’m not sure the benefits for Go would be as significant as other software but, really, why not?
I fail to see what’s that got to “too much” restore activity doesn’t tell you anything actionable about your software, if anything it’s creepy as hell.
"Is it safe to remove support for X?" is one use case. Right now the strategy more or less amounts to "remove and see if anyone complains, possibly too late to change".
Microsoft deprecated the disk-image backup in Windows 7 because it was infrequently used... buy random grandparents.
It was basically a "free" wrapper on top of the Volume Shadow Service (VSS) built into the operating system, but only IT professionals ever used it, so... it had to go.
I think they should all be opt-in as well. However, as a developer and pretend sysadmin, I am generally a nice guy about not turning off telemetry on software products with a user facing UI that I use frequently.
What the hell? "People act like they discover telemetry in 2023"? Are we just going to ignore the fact that the issue of companies spying on their users has been a hotly discussed controversial topic since the practice began? Do you think objection to telemetry first appeared in 2023?
Since you asked, yes you are being uncharitable. It's rather hard to imagine that the people who are details-oriented enough to look for suspicious network activity after installing something wouldn't notice the disclosure on the download page (edit: or the release notes). On the other hand, the explanation given by Russ for delaying a week (so people have ample time to opt-out) makes sense.
Do you actually think Russ' explanation is just a pretext so they can evade detection by people who monitor for suspicious network activity (yet don't notice the disclosure on the download page)?
I am jaded and probably being a little uncharitable. However, I don't know Russ personally so I have no reason to place a high level of confidence that a Google employee isn't going to make decisions that align more with Google's interests vs privacy interests.
Regardless, there are plenty of ways to upgrade the Go tool chain (snaps, distro packages, fetching latest via curl, etc) that won't result in the changes being immediately visible. Given that, I think you are painting an overly optimistic picture of a world though where everyone that cares about this is going to be immediately aware that opt-out telemetry has been added vs a lot of installs being silently swept up into this by sheer ignorance.
Also, this is going to require me to go and set environment variables in about a dozen environments to disable the collection and while I can pretty easily manage that task via ansible I'm not happy about having to jump through hoops to turn off telemetry for a freaking compiler tool chain.
> I am jaded and probably being a little uncharitable. However, I don't know Russ personally so I have no reason to place a high level of confidence that a Google employee isn't going to make decisions that align more with Google's interests vs privacy interests.
If the nature of this data were different, I would be suspicious too. But it's really hard for me to see how a set of counters (whose names have various protections to ensure they can't contain private information) being sent approximately once a year is going to help with Google's advertising interests (which is what I assume you meant by "Google's interests"; I think they also have an interest in making Go better and the telemetry proposal aligns with that). This is literally the first time I've been OK with telemetry.
> Regardless, there are plenty of ways to upgrade the Go tool chain (snaps, distro packages, fetching latest via curl, etc) that won't result in the changes being immediately visible. Given that, I think you are painting an overly optimistic picture of a world though where everyone that cares about this is going to be immediately aware that opt-out telemetry has been added vs a lot of installs being silently swept up into this by sheer ignorance.
I agree there will be people who won't notice the disclosure (which will also be in the release notes), but again I tend to think that the people sniffing network traffic after installing a program would also scrutinize release notes instead of just blindly installing upgrades, which is why I find it pretty improbable that Russ' explanation was a pretext.
> Also, this is going to require me to go and set environment variables in about a dozen environments to disable the collection and while I can pretty easily manage that task via ansible I'm not happy about having to jump through hoops to turn off telemetry for a freaking compiler tool chain.
I think the best suggestion I've seen is that there should be a single environment variable (e.g. $TELEMETRY) that all programs should respect, to avoid the need to do work for every application.
> there should be a single environment variable (e.g. $TELEMETRY) that all programs should respect, to avoid the need to do work for every application.
There was a proposal for that some years ago, but that didn't really go anywhere, partially because of the author's rather unpleasant attitude towards projects he wanted to implement it and their overly broad definition of "tracking" (which includes e.g. update checks).
> I think the best suggestion I've seen is that there should be a single environment variable (e.g. $TELEMETRY) that all programs should respect, to avoid the need to do work for every application.
This is a nonstarter, as DNT demonstrated in spades.
Google is institutionally incapable of producing software that doesn't track its users over the internet.
One example is the stock calculator app on Android, which according to their privacy statement may track your app interactions, device id, and email. Like what if users actually subtract more than they add or something.
On-by-default makes me question whether rsc's judgement has been compromised, which leads me to question continuing to use the language. A strange miss for him.
Off-by-default in a scale likely means that there is no telemetry at all. I would not cancel a guy or programming language based on just suggesting that. He has given a lot of though for that if you read the blog posts.
I would expect nothing less of him than to give a topic a great deal of thought and devise a principled and rational solution.
I am, however, reminded of a quote from Peter Drucker: “There is nothing quite so useless as doing with great efficiency something that should not be done at all.”
I’m not picking nits regarding the overall proposal. I’m questioning the judgement that concluded/rationalized “on by default” is the right thing to do.
Also not “cancelling” at the moment, either, but definitely reassessing my future language choices and taking a more critical appraisal of Go’s direction/choices. This isn’t really an isolated incident, and trust has accumulated some dents.
313 comments
[ 4.9 ms ] story [ 275 ms ] threadI follow them on Twitter but haven't looked much into it other than reading their documentation, which makes me think that most of their telemetry is done at the point of the package distribution system: https://about.scarf.sh/package-sdks
In fact it seems there's a clear correlation between the quality of software and how much spyware there is embedded in it. It's often merely another way to justify unpopular changes with "but the data says so".
IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.
Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usable at all. Given that I’m unwilling to implement an opt-out system, I don’t really see a workable approach here at all.
Is my interpretation correct?
> [Y]ou lean towards not giving them the choice rather than respecting their wishes.
> Is my interpretation correct?
I don’t think it is, no :) Rather, I’m not sure how to sell, to put it crassly, users on a choice when properly investigating or even being confronted with that choice would delay them seeing the dancing bunnies[1], but that would also, if I have any say about it, improve the bunnies in the future.
Does that mean there’s a shade of “I know better” in my problem statement? Of course it does, if I didn’t know better than the average user I’d have no business designing such choices. I don’t think there’s anything wrong about that, better than the average at an activity few practice is not a terribly high bar. Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though.
Basically, how do I make the user think, how do I give them the appropriate data to do so, and how do I deal with the obvious contradiction of that goal with principles of good design[2]? The potential benefits to the software and (thereby) the users are too much to give up without even asking those questions.
(See nearby comment for extended discussion.)
[1] https://blog.codinghorror.com/the-dancing-bunnies-problem/
[2] https://sensible.com/dont-make-me-think/
> Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though.
I'll pull out just this point, though, to perhaps illustrate how different our worldviews are. I consider opt-out to be a manipulative approach.
So do I, which is why I wrote I’m unwilling to implement it :) The original (and, to be clear, purely theoretical) point was, opt-out is too manipulative while opt-in is likely useless.
Ah shoot. Did you take that to mean that I’m unwilling to implement an off switch at all? That wasn’t it, sorry for the confusion.
The struggle is real. As a developer, more data is obviously desirable and can make development much easier. I just can't think of a way to do telemetry that, if I were a user, I would accept. And I don't want to produce software that I wouldn't personally use.
I just don't know how to have my cake and eat it too.
I don't think the issue here is making decisions for users and not giving them a choice. 99.999% of software does not have a flag to change it. The issue seems to be more about the precise nature of this specific feature.
Of course. Not really just this specific feature, but any and all features that can violate users privacy or security. In the end, I don't think these are decisions that developers should be making for users, because not all users have the same needs and getting this wrong can do harm.
That's why, for these sorts of things, meaningful user consent is critically important.
This nicely summarizes a very popular approach to telemetry – and to a variety of user-hostile behaviors. Web sites (for example) seem to have mastered the "fight against user preferences" approach, trying to play video when autoplay is blocked, using javascript modals since pop-up windows are blocked, fighting ad blockers, ignoring "do not track", etc..
If users are given any choice, usually it's a difficult opt-out process, which is more effective precisely because it makes it harder for users to make the choice that you don't want them to make, even if it isn't their actual preference. For an extreme example, see Facebook's (anti) privacy settings. Commonly used dark patterns further amplify user manipulation.
I’ve taught probability theory using randomized response[1] as an exercise problem, and while people can understand it given time and motivation, it’s not immediately obvious. So I’m not exactly hopeful that a prospective Audacity, Blender, or even Free Pascal user (to take an arbitrary set of examples) would get what I mean if I say “I’m collecting no more than 10 bits of information about you using RAPPOR”[2], and I’m not willing to engage in comforting bullshit such as “all collected data is anonymous”, as I’ve been all too close to situations where the difference between the two might be one between freedom and prison.
> Is the non-technical nature of the user the problem or is it just that when you have an opt-in option most people make the choice to opt-out?
Both, because confirmation dialogs, especially privacy-related ones, have been thoroughly poisoned in users’ minds. But confirmation of obscure actions, however beneficial their consequences, is problematic in general—if I go on the street and ask people if they’d like caffeine in their tea or ascorbic acid in their apples, I expect (but have not checked) that the majority will say no, nevermind that both are normally there and intrinsic to the experience.
(The possibility of meaningful consent from a non-specialist is the subject of much discussion and few good answers in med school, or so I’ve heard.)
Whether the ultimate answer is to grant or deny permission, I’m not sure I can present the question in a way that will actually have it made on the basis of merit and not on “scary permission dialog, better say no” or “yes, yes, just let me through to my dancing bunnies[3]” or “yes, if I say no the installer will just tell me to GTFO”.
(In that respect the “Send crash report to vendor” button is unexpectedly good, because you’re not actually interposing yourself between the user and any prospective bunnies. But personally I don’t like to spend time and effort in order to send “feedback” into an unmarked hole where I’ve no idea if anybody will ever look at it. From that point of view, it is background data collection that’s unexpectedly good.)
And even if, for the purposes of this question, it would be best if people took the time to learn the necessary maths, computing, and operational security to make an informed choice, in reality I’m not sure that’s the best thing they can spend their life on.
So it may be the answer is that you simply can’t do telemetry well for the social reason that users won’t ever end up making an informed choice, or that the well has been poisoned so thoroughly that the rational choice is to reject everything. It’s just that I know that it’s basically possible in a technical sense, so I don’t want to give up that easily.
[1] https://en.wikipedia.org/wiki/Randomized_response
[2] https://blog.cryptographyengineering.com/2016/06/15/what-is-...
[3] https://blog.codinghorror.com/the-dancing-bunnies-problem/
The difficulty, though, lies not (entirely) with the default position of the toggle, the difficulty lies with making the user think about the question which is not relevant to their immediate task and which in any case they may not have the theoretical tools or time to evaluate properly. The default position of the toggle (if “off”, as I believe it should be) matters only because an opt-in process means you either confront the user with irrelevant questions on first launch or get essentially no data.
(I called this out as a a “general” question because I meant for it to apply not only to the Go toolchain, but to general-use software like Firefox or niche but non-programmer-oriented software like Audacity.)
The systemwide daemon proposed elsethread[2] would solve this nicely as well, but I have to admit that I’ve dismissed it from my thought process more than once before, because I didn’t think we were going to get one with any reasonable usage on any platform. Now that I’ve seen it put in writing, maybe it does deserve to be considered.
[1] https://news.ycombinator.com/item?id=34716342
[2] https://news.ycombinator.com/item?id=34709836
I think this is a really hard problem, and simply trying to guess in the dark as to what people want isn't the smartest way to go about finding the path forward / priorities / improvements / defects.
It also isn't something that we had in the past, because when we used to buy the IDE, buy the compiler, and then build software, sell that software, and let everyone know what cool tools we used, you'd have sales figures that would inform the creators how the tools were used. Now, the tools are available to everyone, anonymously, and everyone has an opinion on how well it works for them, but doesn't have the time to write a well-written report every time a release happens.
It's an excuse to not respect user's choices and absolve oneself of the blame, or regress to a lowest-common-denominator, "because the data says so".
"Not everything that counts can be counted, and not everything that can be counted, counts."
Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software still phones home and the IP is still being collected and logged when it otherwise wouldn't be. Either when uploading the report or when downloading the “collection configuration”.
Honestly, assuming full transparency, I'm not opposed to the concept. I question how much telemetry is actually necessary, but I'm certain there will be times when it's nice to have. It'd also be interesting to see how it would go when for once people can see exactly what is collected, when, and from where.
I'm not sure that Google is the best place to showcase such a concept though. I'm sure there are a lot of people who have no problem with handing more data over to Google, but Google has abused the public's good will for the sake of data collection many times, and it's sure to put off some of the people who aren't already completely disgusted by the idea of their favorite open source projects collecting telemetry.
How do you verifiably not collect users’ IP addresses when receiving data from them? The verifiable part is the problem, of course you can (and should) just not log the addresses, but then the users can only trust you (and hope you or your uplink haven’t received any legal orders to the contrary). The only approach I can think of would be a Tor hidden service, but while it would technically work, as far as not exposing your users to scrutiny it actually sounds worse.
Their favorite Google open source project. This is specially important for project which can't realistically exist without main sponsor / benefactor. It also help people to pay whatever little/high cost in term of conscience when they take part or consume something willingly but do not approve of makers.
Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents?
Surveys about this are just asking humans to do something computers can do better.
Obviously privacy and consent are paramount concerns, but not only are they solvable, in open source they’re fully auditable (and a fork could fairly easily maintain a patch that removes it outright).
I think open source largely rejecting telemetry puts it at a huge disadvantage to proprietary and SaaS software where it is the norm. I’m very excited to see someone as thoughtful and well reasoned as Russ Cox to be trying to move the status quo forward.
The times "we" (previous companies) tried to implement telemetry in open source non-SaaS products (as distinct from "projects"), we either got huge blowback or users/customers simply blocked it at the firewall (and security teams at major enterprises were unwilling to open holes anyway).
The only workable solution I found was integrating this in a value-add way, so that something in the service/experience/etc was better for the user/customer as a result of enabling telemetry, without the dark pattern of making things intentionally awful/worse without it. We simply never got enough data to matter otherwise. But, again, that was products and not projects.
This sounds like a great concept, but I'm struggling to come up with concrete examples - how did you approach it?
Look at GNOME, which recently pushed for it's users to contribute telemetry: https://linuxiac.com/gnome-survey-results/
Nothing wrong with what they've done here, but we already had most of these metrics. Nothing was really learned, and it took Red Hat and a few thousand users to get here. For smaller-scale projects, imagine how much smaller the returns would be.
We have to ask for permission on our SaaS products to collect this data as it's not necessary to collect it for the product to function. The EU GDPR mandates this.
Russ Cox is suggesting that there is no permission step and that the data is collected by default.
That is the issue.
Russ Cox is a very intelligent and effective engineer. He has a history of projects where he first analyses the problem space, then arrives at great solutions. He puts a lot of effort into discussing the problems and proposal with the community, especially after the widely criticized go mod decision by the go team (which is now mostly accepted as unfortunate, but in the end, the correct decision, I would think).
My point is: We all suspect Google and telemetry to be bad. But can we be charitable enough to separate the Go project, that is run by individual humans, and telemetry from our superficial cliches to actually read the proposal?
I read the proposal. There is no discussion of the legality of this at all. I'd expect anyone with any level of supposed technical competence to consider this in relation to global data protection. I suspect there has been no legal review as mentioned in the thread because I know how slow the lawyers in this space work and the timeline between publishing this and now is too short to have had a conclusive answer.
As for your point about GDPR, I think if you apply your right to withdraw from opt out data collection and what that entails and then ask how this glaring defect is missing from RSC's paper, then you'll see exactly how much privacy consideration really went into this.
https://gdpr.eu/eu-gdpr-personal-data/
I would, personally, prefer a single non-profit service that would list publicly what is being collected and publish the results as open data for anyone to use. Applications (at least on Linux) would not submit their reports directly, but would use a local relay service that could be turned off completely or that could filter what reports to send to the server and what to /dev/null.
Distributions and other software stores would then make it mandatory for software to use this relay and either patch out any other telemetry from their packages or straight out forbid those that would not comply.
The difficulty with telemetry is that even if we design the perfect, privacy-preserving system to begin with, once the pattern of having a network port open is established, there's nothing to prevent us (humans) from changing our policies about what we're allowed to push/pull over that port.
In real-world analogues for these kinds of thorny policy problems, we have centralized arbiters to solve these problems. That might be a fruitful course of research for people interested in this problem to explore.
Unfortunately, even though this problem has software as its medium, it is a problem that cannot be solved by clever software alone, despite any appearances to the contrary.
The other difficult is what you mentioned: trust. Even if a piece of software really does telemetry in a perfect, privacy-preserving way -- as a user, I have to take the developer's word for that in the end. That's a hard hurdle to pass, because that trust has been violated so much in the past that nobody gets the benefit of the doubt anymore.
> Unfortunately, even though this problem has software as its medium, it is a problem that cannot be solved by clever software alone
I agree entirely. At the heart of it, this is not a technological problem. It's a human one.
Do let people turn it off though please.
If I, perchance, encounter software I use phoning home without my explicit permission it's done on my systems. Period.
I would never personally make that trade-off, and would always put (disableable) telemetry in my projects.
But more realistically what you gain in privacy you give up in having your voice heard by the devs. The decisions about the future of the product/project will be driven by the data, specifically the data from the kind of people who leave telemetry on.
I mean, with any group of people, there will always be a percentage that will disable it. If the telemetry is popular, that percentage might be very small, but it would be non-zero.
Software which does any type of computing without it's user's informed consent is classifiable as malware, mind you.
Up until now, you've had to make these design decisions on your own, relying only on perplexing intangibilities like 'taste' and 'intuition'.
Especially if you haven't had the chance to develop an intuition yet and are new to the field. Without data to correct you, how do you get better?
The design of a teapot is a great one. It didn't magically appear with handles and a spout and a place to hold leaves, but after years of refining based on usage. As shocking as that is, tea wasn't even discovered on purpose, let alone having a specific vessel for it right out the gate.
So yes, telemetry is essential. Taste is personal.
For many open source project (anything with an attached business model), either telemetry or tight communication around usage patterns will be necessary to inform development. The latter of those two options consumes business resources.
For me, the harshest you can go is to have telemetry and not prompt for the setting on start with the user opted in by default. Ideally, you ask on first load, and that's what I'd probably aim for.
You can't ever try to force people to do anything in open source. They'll run right by you and make it do what they want it to do with or without you.
I'd even imagine paying customers are broadly more happy with telemetry than open source ones. And their needs more important anyways.
If it's mandatory to run the code that does telemetrics, it's not a very open license.
I don't think any closed source goes into Red Hat, it's just the patch delivery pipelines, package repositories, etc that require a license. And support of course.
Same with any distributed system whose core contributors could gain insight from telemetry. All the components are open source, but they can package it all up and make it available under different terms.
If there's a community version and an enterprise version, you can then make telemetry required in the community version. If people don't like it, they can pull the package apart and put it back together however they want, or they can pay for an enterprise license.
The company ships one product which is a community edition of a bundle of open source software. That version has telemetry enabled and can't be disabled. Users who want to patch the code manually can of course disable it, just like they can disable the Ad lens in Ubuntu if they want to build it themselves, and those users will be off the beaten path and likely to run into issues that aren't easy to find paved solutions for.
They also offer an enterprise edition with a support license, on which telemetry is enabled by default, but can also be disabled.
I'd recommend saying "and cannot be configured off without code changes", or something. Of course it can be disabled, if it's open source.
Go compiler without telemetrics opt-out would fragment the community even harder than Go compiler with telemetrics default on. While your scenario is, of course, possible, it's not very relevant to the Go compiler.
As for your original "might be a sustainable way to run an open source company", if the primary feature one gets buy paying is "easy to turn off telemetrics", that just doesn't sound like enough value.
Can you configure openSUSE to use apt instead of zypper? I mean, sure, probably, it's open source.
Is it going to be straightforward? I don't know. I suspect it's going to be harder than it's worth, and you might need to rebuild the operating system image to change the directory layout to work with apt's assumptions or something.
So in practice, even though these things are all open source, people who bundle complex software lay down the paths that 99.9% of people will take.
I wasn't proposing "disabling telemetry" as the primary business proposition.
Instead I was suggesting that open-source companies spend a lot of time dealing with issues from people who use the open-source software in unanticipated ways. If "the beaten path" for using the software without support includes telemetry, they get value from that.
If people want to use the software without telemetry, then they're paying for a support license, so the issues they run into which the supporting company has no telemetric insight into are at least better aligned with their support resource allocation.
The package manager is part of it, sure.
The filesystmem is another big part of it (though it's possible most are following XDG now https://specifications.freedesktop.org/basedir-spec/basedir-... )
Init system used to be a big point of deleniation, but I think systemd is the standard now (for better or worse)
The filesystem and networking stack still have some variability.
There's still default applications, kernel modules, a gui app installer, the desktop, included drivers, and many more things that go into a distro. If you go with a distro that uses KDE and you switch to GNOME for example, you might lose a lot of GUI support for customization, might have to build addon packages yourself, etc.
It's all open source at the end of the day, but that optionality leads to a less streamlined user experience and lack of guardrails as soon as you step off the beaten path, than you would get with something like OSX
I don't think I made myself clear.
The company ships one product which is a community edition of a bundle of open source software. That version has telemetry enabled and can't be disabled. Users who want to patch the code manually can of course disable it, just like they can disable the Ad lens in Ubuntu if they want to build it themselves, and those users will be off the beaten path and likely to run into issues that aren't easy to find paved solutions for.S
They also offer an enterprise edition with a support license, on which telemetry is enabled by default, but can also be disabled.
No, it isn't, and the idea that is is toxic.
But I'm confident I'll make a better product with telemetry vs without.
While I leave it turned on for personal projects, several projects at work require disabling it.
I have spent hours auditing through transitive dependencies to turn it off. It should not be this painful.
Debian talks about what you, the user, can do: help out, participate and vote. If you choose to do so.
The Go team talks about what the developers and their software will do to the user's machine, but the user is completely passive in their description. This is also reflected in the term "telemetry" itself: the software is not a tool in the user's hands but rather a remote-controlled probe in the user's habitat that pokes at the user to elicit interesting responses.
1. Sends names of private packages to the server, and publishes them.
2. Sends a unique identifier (a UUID stored in /etc/popularity-contest.conf) to the server, which is stored.
3. Doesn't use sampling, so if you use popcon you will be submitting a report once a week (Go's telemetry would average just one report a year).
4. Submits over plaintext protocols by default.
popcon may be opt-in (in the sense that the prompt during installation has "No" selected by default) but the prompt doesn't disclose the large privacy risks.
People are not appreciating the thought that has gone into the Go proposal to minimize the collection of private data, either intentionally or by accident, such as the client-enforced requirement that the the names of counters be published in a tamper-proof log so anyone can verify that, for example, no private package names are being disclosed. Everyone is focusing on opt-in vs out-out, but to me these other details are far more important.
Right. That's fully opt-in, to the point the package isn't even installed by default, which is the only moral way to do this.
I think what Russ has described here is decent and well-reasoned. I also think that Go being a product (it is, whether you like that word or not) makes it more fair to desire analytics of this form. I think it being opt-out is reasonable (after all, if it is not, they will make decisions using data that does not come from the vast majority of users, may as well not have analytics at all then.)
But I am afraid of this becoming pervasive not just in products (like CLI tools), but also in libraries, imagine every Go/npm package you use wants to ping the network because the authors want to know 'is this popular? can we deprecate XYZ method?' etc. If transparent telemetry in the form Russ and I have been viewing it becomes a more common thing, it won't be a surprise if more library authors begin to try to adopt something like this and it becomes a pervasive problem IMHO.
[0] https://hexops.com/privacy
[1] https://machengine.org
This is something that was common for internal libraries at some of the places I've worked. I'm honestly a little surprised it isn't a thing we see externally. I for sure do not want to see it, but I'm surprised we don't. Its probably enough to look at the public usage on GitHub, and make inferences and post notice on future-major-versions of libraries. Github honestly should make a tool to do this, they'd have a huge opportunity to inspect the data.
And opt-in.
> I also think that Go being a product (it is, whether you like that word or not) makes it more fair to desire analytics of this form.
Not by stealing it.
It's on the project to convince the user to turn on telemetry rather than the user having to remember to turn it off. Excuses such as "nobody would turn it on" don't apply.
I don't believe the "100% anonymous" is a thing with AI anymore. When AI can identify/fingerprint you by your walk pattern[1], you can't really tell what data can and can't be used to fingerprint you.
[1] https://ieeexplore.ieee.org/document/8275035
A more general purpose metrics tool I'm watching closely is Divvi Up https://divviup.org/, a research project by ISRG, the same org that runs LetsEncrypt. The basic idea is to divide up each metric into two parts and publish each part to separate collection servers (one run by you and the other by divviup). Then the servers separately aggregate their half and combine the results, the idea being that each half is useless on it's own but when combined it's still useful.
I wouldn't suggest it for this application, but for the majority of typical apps it would be a vast improvement to privacy compared to the status quo.
But in practical terms, the problem with this approach -- if I'm understanding it correctly -- is that it has no way to detect and reject outliers, and therefore the data can't be validated in any way. It only makes sense if all your clients are 100% trustworthy.
Let's say you want to know whether to keep supporting ARMv5, and your data says 10% of users are using it. There's no way to tell whether that's accurate, or if you have 0.01% of die-hard users who modified their telemetry code to report 1000x as frequently as they're supposed to. Even if you suspect this is happening (and you might not), there's no way to identify the culprit and filter out their data without tracking personal identifiers such as IP addresses.
So even if most of the time the telemetry data is valid, over time it will trend toward uselessness, because it can be endlessly second-guessed unless it confirms a decision you wanted to make anyway.
But I guess this could be helpful corporatized read-only repo projects, or other groups that aren’t sure if they are building a community or a customer base.
But that's not my only objection to telemetry. Equally important to me is that so many bad decisions are justified based on telemetry. It's very easy to misunderstand the data, because telemetry leaves out so much, but developers often treat it as if it's giving a complete picture.
As an example, I have seen developers drop really important functionality on the basis that it is rarely used. While that was true, it was also true that when those rare times happen, that functionality was absolutely critical to have.
If you're getting it from distro repos, it should be straightforward to convince the distro package maintainer to disable the telemetry / patch it out.
Or is it a nvm/pyenv/rustup situation where you prefer to use bespoke toolchain managers to download upstream's compilers?
Would be nice if this system threw something to stderr at runtime every. single. time. unless the message was explicitly disabled. Something like:
(it did say "transparent", like a service people opt into that could relate installations to github URLs)
Not only is it going to be opt-out (because of course it would be coming from Google), I really like the whole "wait a week before sending telemetry" part that just coincidentally has the benefit of sneaking right past people that actively look for suspicious network activity when they've freshly installed something.
Am I being uncharitable?
People act like they discover telemetry in 2023.
I don't think it's a big deal, ultimately it's to improve Go and the proposal makes it very easy to disable it ( single env variable ).
Does it? Telemetry mostly seems used to justify removing features I need on grounds that they’re little used.
As an other user noted, if telemetry is your yardstick, the average backup software removed the “restore” feature because that’s barely ever used.
"Is it safe to remove support for X?" is one use case. Right now the strategy more or less amounts to "remove and see if anyone complains, possibly too late to change".
What the hell, it's a freaking compiler. What do you mean, "too late to change"? Fail the compilation is suddenly a showstopper bug?
If go wants to deprecate features, just follow the same procedure done by literally all other non-spyware compilers.
It was basically a "free" wrapper on top of the Volume Shadow Service (VSS) built into the operating system, but only IT professionals ever used it, so... it had to go.
These are both Microsoft products. Microsoft's position is well known.
> Java etc ...
Really? Which Java distribution?
It's definitely opt-in in Jetbrains IDEs.
Do you actually think Russ' explanation is just a pretext so they can evade detection by people who monitor for suspicious network activity (yet don't notice the disclosure on the download page)?
Regardless, there are plenty of ways to upgrade the Go tool chain (snaps, distro packages, fetching latest via curl, etc) that won't result in the changes being immediately visible. Given that, I think you are painting an overly optimistic picture of a world though where everyone that cares about this is going to be immediately aware that opt-out telemetry has been added vs a lot of installs being silently swept up into this by sheer ignorance.
Also, this is going to require me to go and set environment variables in about a dozen environments to disable the collection and while I can pretty easily manage that task via ansible I'm not happy about having to jump through hoops to turn off telemetry for a freaking compiler tool chain.
If the nature of this data were different, I would be suspicious too. But it's really hard for me to see how a set of counters (whose names have various protections to ensure they can't contain private information) being sent approximately once a year is going to help with Google's advertising interests (which is what I assume you meant by "Google's interests"; I think they also have an interest in making Go better and the telemetry proposal aligns with that). This is literally the first time I've been OK with telemetry.
> Regardless, there are plenty of ways to upgrade the Go tool chain (snaps, distro packages, fetching latest via curl, etc) that won't result in the changes being immediately visible. Given that, I think you are painting an overly optimistic picture of a world though where everyone that cares about this is going to be immediately aware that opt-out telemetry has been added vs a lot of installs being silently swept up into this by sheer ignorance.
I agree there will be people who won't notice the disclosure (which will also be in the release notes), but again I tend to think that the people sniffing network traffic after installing a program would also scrutinize release notes instead of just blindly installing upgrades, which is why I find it pretty improbable that Russ' explanation was a pretext.
> Also, this is going to require me to go and set environment variables in about a dozen environments to disable the collection and while I can pretty easily manage that task via ansible I'm not happy about having to jump through hoops to turn off telemetry for a freaking compiler tool chain.
I think the best suggestion I've seen is that there should be a single environment variable (e.g. $TELEMETRY) that all programs should respect, to avoid the need to do work for every application.
There was a proposal for that some years ago, but that didn't really go anywhere, partially because of the author's rather unpleasant attitude towards projects he wanted to implement it and their overly broad definition of "tracking" (which includes e.g. update checks).
Some discussions:
https://news.ycombinator.com/item?id=27746587
https://lobste.rs/s/htbkqd/console_do_not_track
This is a nonstarter, as DNT demonstrated in spades.
One example is the stock calculator app on Android, which according to their privacy statement may track your app interactions, device id, and email. Like what if users actually subtract more than they add or something.
https://play.google.com/store/apps/datasafety?id=com.google....
Or the wallpapers they include on your phone - you guessed it!
https://play.google.com/store/apps/datasafety?id=com.google....
If that's the kind of environment you work in I'm not surprised this proposals seems modest by comparison.
If I make it opt in, nobody is going to give me the dollar, but that doesn't make opt out morally justifiable.
I am, however, reminded of a quote from Peter Drucker: “There is nothing quite so useless as doing with great efficiency something that should not be done at all.”
I’m not picking nits regarding the overall proposal. I’m questioning the judgement that concluded/rationalized “on by default” is the right thing to do.
Also not “cancelling” at the moment, either, but definitely reassessing my future language choices and taking a more critical appraisal of Go’s direction/choices. This isn’t really an isolated incident, and trust has accumulated some dents.