Submission title is a bit confusing: “opt-in” means you have to opt-in, that is, the default is “off”. The sentence “opt-in by default” is the opposite of what I assume the poster meant.
Storybook’s telemetry is opt-out as stated at the top of the page.
This also looks like something that should be posted as a text submission with commentary - simply editorializing a link + title is discouraged in HN.
That's correct, but it's a relatively new thing. 150 years ago two or more negatives meant an emphatic no. The switch was caused by Frege and contemporaries.
Yeah, this is more complex than a double positive isn’t it. Hmm.
“Opt in , by default” would seem to mean you have to opt in, unless some unnamed external condition was met, so normally you would have to choose to participate… which is the opposite of this case , where to NOT participate you must, by default, opt OUT. And if you do not you are automatically committed to participate.
"Opt" means to "To choose; select" [0]. If you opt in (i.e. make the choice to go in), that implies that you were not already in. In other words, it implies that the default (when you don't make a choice) is "out", i.e. disabled.
"Opt-in" and "opt-out" describe what the default is, so adding "by default" doesn't make any sense. If you can choose to go in ("opt-in"), then the default is "out". If you can choose to go out ("opt-out"), then the default is "in".
Some companies appear to be trying to twist this into "The choice is yours, and we have taken the liberty of making that choice on your behalf.".
Opt in by participation is implicit all the time. If I play soccer I am not "opting in" to get hit by a ball, but rest assured, it is going to happen.
There are topics dear to every subgroups heart, that the members will claim, surely must be exempt from this. As far as I can tell, the exemptions are mostly arbitrary.
This is essentially about checkboxes. “Enabled by default” would mean tracking is always on, but doesn’t tell you if it can be disabled by the user.
In the realm of privacy choices in software, we then started referring to these as “opt-in” (default off) or “opt-out” (default on) to communicate the default setting + the ability to change it.
Advertising trackers, for example, are mandated by law to be “opt-in” (they pre-select the checkbox for you of course, without submitting, but that’s a whole other story).
In the soccer scenario, as you say, the only choice to make is whether to participate. Whether to get hit by the ball can't be chosen independently. So the only opting you can do is playing (and getting hit) or not playing (and not getting hit).
In that scenario, the default is to not play soccer. If you want to play, you have to choose it, i.e. opt to play soccer, i.e. "opt in". So the default is that you're "out". Thus, playing soccer is "opt-in".
If people were born into soccer matches and had to actively try to escape, that would be "opt-out". You make a choice (opt) to escape out of the soccer match.
Notice that "opt-in" and "opt-out" describe the inverse of the default value: "opt-in" is the same as "out by default" and "opt-out" the same as "in by default". Mixing these phrases with opposite meanings (e.g. "opt-in by default") creates confusion, perhaps intentionally, about what the default value is.
I could not disagree more. You can't "opt-in" someone else because they're not opting in at all. I think this is an important and meaningful distinction that we should call out, particularly since we're seeing more companies being deceptive on this point and trying to trick people through this confusing language.
Isn't there a telemetry manifesto? I understand that developers want to peek into what their users are doing. That is the fruit of their labor after all. However telemetry became increasingly easy lately, and it is impossible (in theory and in practice) to do it right.
Collecting data of your users is
- stupid
- illegal
- harming your users
- makes your software a spyware
so in summary, don't do that. Instead, do the exact opposite, and excessively delete any data that you may store. Also make sure that your users are exposed against 3rd party (isp, cnds and such) to the minimum when they check and get new updates. (Randomized update checks over TOR, for example.) They trust you, and allow your program to handle their network, so use it wisely.
Also there are sane methods to know what your users are doing, for example you can buy usage blogposts from them, which you can showcase, and link in the manuals and support.
This is pretty basic stuff, but developers and the HN audience have a hard time to accept that.
Ask yourself what value you got from your telemetry lately. Also how much effort you put into it. Do you even look at it regularly? Does it still make sense?
Maybe consider the legality? Hint - if it isn't opt-in you are most likely doing something illegal. Congratulations! Does it still make sense?
Maybe consider not being a total jackass and insult your users? Does it still make sense?
I don’t mean to dismiss the importance of users owning and controlling their data, but isn’t telemetry a valid tool for software projects to gain valuable insights into how their product is being used and thus how to improve it? Yes, it can be (and, probably frequently is) abused. Yes there should be a way to opt-out of telemetry if a user desires for whatever reason. But I can’t help but feel this is posted in the spirit of “telemetry by default is bad” and I don’t really agree.
Full disclosure: this is the first I’ve ever heard of Storybook, so if there’s something particularly gross about telemetry for this use case I’m not aware of it.
If Telemetry is valuable, it's largely valuable to the degree it's representative. Opt-in sacrifices this and even a well optimized opt-in process won't prevent that.
This is not to say that Telemetry should be opt-out, just that you place other justifications ahead of the usefulness of Telemetry (which is fine).
These days I reckon running an LLM over the storybook discord and reddits would yield more valuable data. How big peoples projects are on average doesn’t matter. 100 people asking the same question does.
I completely agree that qualitative feedback is the most valuable kind of product input.
However, quantitative metrics like telemetry are used to answer different kinds of questions.
> How big peoples projects are on average doesn’t matter.
To give concrete examples, at my current company we do not collect telemetry and have regularly encountered problems where some assumption about customer usage (the number of items in the largest project, the size of the largest document, the smallest browser window, the oldest browser version, the usage of a deprecated feature) turns out to be incorrect, sometimes wildly so, and we only find out after a customer finally complains that they've been struggling with something for months.
If we had telemetry we could have known about these issues well ahead of time and in many cases avoided the problem in the first place. We still choose not to have telemetry for many of the reasons presented in other comments, but we should at least be honest that it can provide immense value to the product team.
You're right. Opt-in results in a survey population that is self-selected, and therefore largely meaningless. But compulsory participation in surveys is deeply objectionable; just because it's easy doesn't make it OK.
So you can't have representative telemetry that is also optional (for the user). So firms should stop trying to use telemetry, unless they're OK with telemetry data from a narrow pool of users, e.g. users of Beta versions of software.
The problem is most people really don’t care about telemetry, but don’t even care enough to opt themselves in. It’s simply not in their focus.
I think a very pronounced opt-out is fine, because it means anyone who is remotely against telemetry can leave, but you still get data from those who genuinely don’t care. Opt-in means you only take data from those who are explicitly for telemetry, which is a very small and not representative number.
Opt-out does have to be visible and pronounced though, and not have any drawbacks. “opt-out” through 30 secret menus which are only known to those who read internet forums doesn’t count, because you’re still secretly collecting telemetry from those who aren’t aware; and wording like “opt-in by default” is flat-out misleading (it’s like saying “made by metal artisans” when the “metal artisans” are literally robots). I’m pretty sure it’s also not necessary: even if there’s a big checkbox “enable telemetry” which is on-by-default, or in this case a flag `telemetry: false` which is very well-documented, the majorly of people choose the path of least resistance and will keep it enabled.
"Better products" don't spy on me. As a user, I don't want any information about my usage leaving my machine. In addition to privacy issues, this is now code that needs to be maintained, discussed, debugged... that has nothing to do with why I'm using the software. It's an entire feature set that costs money and time and technical complexity that provides literally no value to people using the tool for its intended purpose.
Software is best when it's designed and built by an expert with an opinion. You can look at things like Linux or vim as an example.
All software - including Linux and vim - makes some choices that benefit the developer over the user. Any choice of technology or software architecture with consideration of developer ergonomics is doing this. Every app that ever shipped with a runtime abstraction (classes, dependency injection, testable code patterns, debug logging, etc) chose to make the software fractionally worse for the user, to help the developers to actually write the software.
There is a meaningful and debatable tradeoff in all such decisions, and it's not clear cut. It is often (but not always) sensible for software to include extra code that, yes, must be maintained etc, which exists purely to make the development process of the software more effective. All extra code like this is strictly speaking a bad thing for users when considered in isolation, but overall improvements in development processes (better software, fewer bugs, quicker fixes) mean it's a net benefit for the user in the end, hopefully.
In this case, these developers genuinely do think the telemetry will help them build better software. You may not agree! That's an interesting point to explore, but their opinion is that the maintenance cost is worthwhile for the benefits that this information will provide. They think that the downsides of telemetry will be outweighed by the resulting significant improvements for people using the tool for its intended purpose.
They may well be totally wrong - there's an interesting balance there that is worth discussing, in terms of concrete benefits and real-world costs. Reducing all DX code to "literally no value" though is unhelpful and misses the real discussion.
I have the same opinion on telemetry but I have never opted in for any tool no matter how politely they asked and also never seen any other technical person doing it either. So maybe for software developers it doesn't work if it's opt-in and opt-out is a good strategy?
People don't want their privacy violated. It doesn't matter what data you're collecting. If you peer into my windows every day in order to document the colour of the paint on my walls, you don't get to use a defence of "I was only interested in unidentifiable information and besides, they wouldn't let me do it when I asked nicely".
Software installed on my machine is in a privileged position. It has more access to my personal files than my close friends and relatives. I'd rather that software respected some simple boundaries.
I think its value is considerably overstated from the use I've seen made of it.
Executives absolutely love it though. It makes them feel scientific in their decision making and gives them ass-cover when justifying their decisions.
I really think that the ability to collect opt in opinions at strategic points in the app is vastly more effective, not to mention less creepy and invasive.
Absolutely agreed with the second point. I’m not sure I agree with the first though. At least not that that’s the main reason telemetry is desired from product teams. (Though obviously telemetry does provide fodder for that type of stupidity for sure.)
To give a concrete example from my past experience, I used to work on a product that essentially knitted a bunch of services from a major cloud provider together to provide a user experience comparable to traditional HPC clusters. We never included telemetry, but we always wanted to in order to gain insight into how many compute queues people were using, how many VMs were in each of them, how many jobs were run over a certain amount of time, how many VMs those jobs ran across, etc. The sole reason we wanted this information was because the configuration exposed for this product was extremely complex, and we wanted to put layers on top of it to more easily enable the most common use cases.
Is this not a legitimate reason to collect telemetry? Is the concern that, even though the dev team wanted the data for legitimate purposes, eventually bad actors will use the capability to start collecting data for more nefarious purposes?
Edit: I see you amended the original to include a statement about opt-in opinions. What do you mean by that?
My edited statement wasnt directed at you (Ive only just read your reply) but it does indirectly answer your question.
What I mean is that you survey your users via your app on, say, the config page and ask them if they think its too complex and if it would benefit from layers or whatever and always let then enter an optional opinion when you do.
Telemetry is fine, it's just that under GDPR and co, it has to be opt-in; the user needs to make an informed decision on whether or not the app can collect and share telemetry.
The data is not likely to be truly anonymous in practice. Making sure it's truly anonymous takes more work and makes it less useful than not-quite-anonymous, so there's no incentive to do a good job of it.
Instead you end up with e.g. Microsoft's constant stream of "oops, we forgot to inform you of the data collection", "oops, we collected more data than intended", "oops, we sent it to our servers accidentally", "oops, it wasn't really anonymous" (e.g. because they simply hashed an ID to anonymize it), and so on.
If telemetry is so valuable these companies should give you discounts or payments for enabling it instead of sneaking in like a thief in the night to take it.
A lot of modern software seems overloaded with dark patterns to trick users into unintended actions that make the company more money. I suspect for many companies telemetry's real value is for marketing research and A/B testing the effectiveness of the latest dark pattern implementations.
> isn’t telemetry a valid tool for software projects to gain valuable insights into how their product is being used and thus how to improve it?
"Valid" is doing a lot of work here.
Trivially, telemetry can provide insights. That's obvious. But how are users supposed to divine which insights the telemetry is supposed to provide? What if the data-collector doesn't know what they're looking for, and is just scraping everything?
> the spirit of “telemetry by default is bad” and I don’t really agree.
You're entitled to your view. My view is that telemetry (i.e. remotely observing what I do on my computer) is definitely "bad", if I don't consent to it explicitly.
I took some interest in the concept of hashing IPs to avoid PII issues [0], and it turns out that since there's only so few IPv4 addresses, it's trivial to simply take the salt the project uses and calculate all the hashes. On my laptop it would take around two hours with the most naive implementation, so I don't really think this solves anything. This problem would be sidestepped by IPv6, but we all know how that is going so far.
The classic better approach would be to drop part of the IP address entirely, so it's impossible to recreate the full value.
For example, dropping the last octet (turning 123.4.5.6 into 123.4.5.0) means you can usually still do useful IP things (where in the world do our users come from, very roughly) while making it impossible to track unique users, or to definitively recognize real individuals.
I've seen one octet masking used as standard, probably just because it's easy, but you could do any other similar change you like, optimising for more or less anonymity/granularity as required.
Storybook is one of those projects, like Postman, that has completely lost the plot. It's unclear what objectives they are pursuing but in terms of developer experience they seem increasingly developer hostile. The CSF format is an unreadable joke and doesn't play that well with TypeScript, the number of plugins you need to do anything sane is ridiculous, and this telemetry story is a farce - open source should not skim data by default.
I remember writing a super simple wrapper around Jasmine to do the same kind of thing, and was glad when I could ditch it for Storybook many moons ago. But if I was starting a project now I would probably rawdog it again, Storybook just causes too much pain.
Is anyone aware of any similar projects that are more developer friendly?
What should people use instead of Postman these days? It seems to still be the only decent client for testing particular protocols, like OAuth 2.0. There's no way I'm using curl for that.
I wonder how do you know the things you build is useful if you don’t know it is being, well, used?
On the other hand, back in the server side everything days, I can have more or less all metrics I want from nginx log. I wonder was it equally frowned upon or is the frowned upon an SPA phenomenon?
68 comments
[ 3.0 ms ] story [ 70.9 ms ] threadStorybook’s telemetry is opt-out as stated at the top of the page.
This also looks like something that should be posted as a text submission with commentary - simply editorializing a link + title is discouraged in HN.
(“Opt-in by default” is a double positive, a less common sibling of the dreaded double negative)
Opt-in by default: you are in You can’t not select opt-in: you are in
“Opt in , by default” would seem to mean you have to opt in, unless some unnamed external condition was met, so normally you would have to choose to participate… which is the opposite of this case , where to NOT participate you must, by default, opt OUT. And if you do not you are automatically committed to participate.
"Opt-in" and "opt-out" describe what the default is, so adding "by default" doesn't make any sense. If you can choose to go in ("opt-in"), then the default is "out". If you can choose to go out ("opt-out"), then the default is "in".
Some companies appear to be trying to twist this into "The choice is yours, and we have taken the liberty of making that choice on your behalf.".
[0] https://en.wiktionary.org/wiki/opt#English
There are topics dear to every subgroups heart, that the members will claim, surely must be exempt from this. As far as I can tell, the exemptions are mostly arbitrary.
This is essentially about checkboxes. “Enabled by default” would mean tracking is always on, but doesn’t tell you if it can be disabled by the user.
In the realm of privacy choices in software, we then started referring to these as “opt-in” (default off) or “opt-out” (default on) to communicate the default setting + the ability to change it.
Advertising trackers, for example, are mandated by law to be “opt-in” (they pre-select the checkbox for you of course, without submitting, but that’s a whole other story).
In that scenario, the default is to not play soccer. If you want to play, you have to choose it, i.e. opt to play soccer, i.e. "opt in". So the default is that you're "out". Thus, playing soccer is "opt-in".
If people were born into soccer matches and had to actively try to escape, that would be "opt-out". You make a choice (opt) to escape out of the soccer match.
Notice that "opt-in" and "opt-out" describe the inverse of the default value: "opt-in" is the same as "out by default" and "opt-out" the same as "in by default". Mixing these phrases with opposite meanings (e.g. "opt-in by default") creates confusion, perhaps intentionally, about what the default value is.
The policy is "opt out" but you are "opted in" by default.
The title is now "Telemetry" and 90% of the discussions makes no sense anymore.
https://www.justus.pw/garden/telemetry.html
It’s opt out, and it doesn’t even completely disable it (unless given an env var as well)
The devs know about this:
https://github.com/storybookjs/storybook/discussions/19910
Collecting data of your users is
so in summary, don't do that. Instead, do the exact opposite, and excessively delete any data that you may store. Also make sure that your users are exposed against 3rd party (isp, cnds and such) to the minimum when they check and get new updates. (Randomized update checks over TOR, for example.) They trust you, and allow your program to handle their network, so use it wisely.Also there are sane methods to know what your users are doing, for example you can buy usage blogposts from them, which you can showcase, and link in the manuals and support.
Ask yourself what value you got from your telemetry lately. Also how much effort you put into it. Do you even look at it regularly? Does it still make sense?
Maybe consider the legality? Hint - if it isn't opt-in you are most likely doing something illegal. Congratulations! Does it still make sense?
Maybe consider not being a total jackass and insult your users? Does it still make sense?
Full disclosure: this is the first I’ve ever heard of Storybook, so if there’s something particularly gross about telemetry for this use case I’m not aware of it.
There should be a way to opt-in. Anything else is egregious.
This is not to say that Telemetry should be opt-out, just that you place other justifications ahead of the usefulness of Telemetry (which is fine).
However, quantitative metrics like telemetry are used to answer different kinds of questions.
> How big peoples projects are on average doesn’t matter.
To give concrete examples, at my current company we do not collect telemetry and have regularly encountered problems where some assumption about customer usage (the number of items in the largest project, the size of the largest document, the smallest browser window, the oldest browser version, the usage of a deprecated feature) turns out to be incorrect, sometimes wildly so, and we only find out after a customer finally complains that they've been struggling with something for months.
If we had telemetry we could have known about these issues well ahead of time and in many cases avoided the problem in the first place. We still choose not to have telemetry for many of the reasons presented in other comments, but we should at least be honest that it can provide immense value to the product team.
You're right. Opt-in results in a survey population that is self-selected, and therefore largely meaningless. But compulsory participation in surveys is deeply objectionable; just because it's easy doesn't make it OK.
So you can't have representative telemetry that is also optional (for the user). So firms should stop trying to use telemetry, unless they're OK with telemetry data from a narrow pool of users, e.g. users of Beta versions of software.
I think a very pronounced opt-out is fine, because it means anyone who is remotely against telemetry can leave, but you still get data from those who genuinely don’t care. Opt-in means you only take data from those who are explicitly for telemetry, which is a very small and not representative number.
Opt-out does have to be visible and pronounced though, and not have any drawbacks. “opt-out” through 30 secret menus which are only known to those who read internet forums doesn’t count, because you’re still secretly collecting telemetry from those who aren’t aware; and wording like “opt-in by default” is flat-out misleading (it’s like saying “made by metal artisans” when the “metal artisans” are literally robots). I’m pretty sure it’s also not necessary: even if there’s a big checkbox “enable telemetry” which is on-by-default, or in this case a flag `telemetry: false` which is very well-documented, the majorly of people choose the path of least resistance and will keep it enabled.
Software is best when it's designed and built by an expert with an opinion. You can look at things like Linux or vim as an example.
All software - including Linux and vim - makes some choices that benefit the developer over the user. Any choice of technology or software architecture with consideration of developer ergonomics is doing this. Every app that ever shipped with a runtime abstraction (classes, dependency injection, testable code patterns, debug logging, etc) chose to make the software fractionally worse for the user, to help the developers to actually write the software.
There is a meaningful and debatable tradeoff in all such decisions, and it's not clear cut. It is often (but not always) sensible for software to include extra code that, yes, must be maintained etc, which exists purely to make the development process of the software more effective. All extra code like this is strictly speaking a bad thing for users when considered in isolation, but overall improvements in development processes (better software, fewer bugs, quicker fixes) mean it's a net benefit for the user in the end, hopefully.
In this case, these developers genuinely do think the telemetry will help them build better software. You may not agree! That's an interesting point to explore, but their opinion is that the maintenance cost is worthwhile for the benefits that this information will provide. They think that the downsides of telemetry will be outweighed by the resulting significant improvements for people using the tool for its intended purpose.
They may well be totally wrong - there's an interesting balance there that is worth discussing, in terms of concrete benefits and real-world costs. Reducing all DX code to "literally no value" though is unhelpful and misses the real discussion.
No, as a user I really don't want my usage data leaving my machine. There is no justification worth that to me.
Software installed on my machine is in a privileged position. It has more access to my personal files than my close friends and relatives. I'd rather that software respected some simple boundaries.
Executives absolutely love it though. It makes them feel scientific in their decision making and gives them ass-cover when justifying their decisions.
I really think that the ability to collect opt in opinions at strategic points in the app is vastly more effective, not to mention less creepy and invasive.
To give a concrete example from my past experience, I used to work on a product that essentially knitted a bunch of services from a major cloud provider together to provide a user experience comparable to traditional HPC clusters. We never included telemetry, but we always wanted to in order to gain insight into how many compute queues people were using, how many VMs were in each of them, how many jobs were run over a certain amount of time, how many VMs those jobs ran across, etc. The sole reason we wanted this information was because the configuration exposed for this product was extremely complex, and we wanted to put layers on top of it to more easily enable the most common use cases.
Is this not a legitimate reason to collect telemetry? Is the concern that, even though the dev team wanted the data for legitimate purposes, eventually bad actors will use the capability to start collecting data for more nefarious purposes?
Edit: I see you amended the original to include a statement about opt-in opinions. What do you mean by that?
What I mean is that you survey your users via your app on, say, the config page and ask them if they think its too complex and if it would benefit from layers or whatever and always let then enter an optional opinion when you do.
Instead you end up with e.g. Microsoft's constant stream of "oops, we forgot to inform you of the data collection", "oops, we collected more data than intended", "oops, we sent it to our servers accidentally", "oops, it wasn't really anonymous" (e.g. because they simply hashed an ID to anonymize it), and so on.
A lot of modern software seems overloaded with dark patterns to trick users into unintended actions that make the company more money. I suspect for many companies telemetry's real value is for marketing research and A/B testing the effectiveness of the latest dark pattern implementations.
"Valid" is doing a lot of work here.
Trivially, telemetry can provide insights. That's obvious. But how are users supposed to divine which insights the telemetry is supposed to provide? What if the data-collector doesn't know what they're looking for, and is just scraping everything?
> the spirit of “telemetry by default is bad” and I don’t really agree.
You're entitled to your view. My view is that telemetry (i.e. remotely observing what I do on my computer) is definitely "bad", if I don't consent to it explicitly.
- Storybook unique identifier: One-way hash generated during Storybook
- One way hash of the IP address
I really doubt that that's "completely anonymous data"
[0] https://github.com/storybookjs/storybook/discussions/19910#d...
For example, dropping the last octet (turning 123.4.5.6 into 123.4.5.0) means you can usually still do useful IP things (where in the world do our users come from, very roughly) while making it impossible to track unique users, or to definitively recognize real individuals.
I've seen one octet masking used as standard, probably just because it's easy, but you could do any other similar change you like, optimising for more or less anonymity/granularity as required.
I remember writing a super simple wrapper around Jasmine to do the same kind of thing, and was glad when I could ditch it for Storybook many moons ago. But if I was starting a project now I would probably rawdog it again, Storybook just causes too much pain.
Is anyone aware of any similar projects that are more developer friendly?
https://github.com/tajo/ladle looks promising if you use React
[1]: https://histoire.dev/
I wonder how do you know the things you build is useful if you don’t know it is being, well, used?
On the other hand, back in the server side everything days, I can have more or less all metrics I want from nginx log. I wonder was it equally frowned upon or is the frowned upon an SPA phenomenon?