Yep. Some features are complex and cannot be completed in one sprint. On the other hand, we have constantly changing front-end code and leaving code in a branch too long might make it drift and might get broken in a lower-layer change because constantly syncing gets annoying. The other case is when the front-end is ready but the back-end is not or there is a public announcement that is scheduled for a later date.
Drifting branches seems to be the main argument pro feature flags. I wonder how teams avoid breaking parts sleeping behind feature flags, by the ongoing development of its surroundings.
If the flag is obtained from a configuration file, you have a testing configuration where the flag is toggled and you run the tests in your CI/CD pipeline. E.g. we have a development plan where everything is enabled and tested.
>I wonder how teams avoid breaking parts sleeping behind feature flags, by the ongoing development of its surroundings.
Just duplicate the testing jobs with the feature flag flipped. Configure new job to turn expected-failure test cases into expected-pass (or whatever analogous way you keep track of it).
The team uses them. Outside of looking for bugs I still have not been convinced that it is useful to check if blue versus green button has useful conversion rates.
I guess you could do A/B testing with feature flags but honestly, it won't be any good. If you want a good answer if to go with A or B, you want to have many data points, not just from one specific customer.
Gradual rollout and specific customer opt-in to certain features can certainly be done with feature flags, I wouldn't call that A/B testing unless you're running experiments and reaching a conclusion that A is better than B (or vice-versa).
You can totally implement A/B tests on top of a feature-flag framework. Your “pass” function should be hashed on the User ID/Cookie/whatever and then you can distribute users into pass/fail. You should do this anyway for reproducibility. If you log whether they passed/failed and then have metrics, you compute experimental results.
Put simply it disconnects merge and deploy from launch. This is very useful when your changes rely on other teams or third parties having gone live. Having a feature flag (we call them toggles) lets you get it into production turned off, without having to coordinate deploys with other systems.
The downsides are it is extra development overhead and adds tech debt (you should go back in and remove the toggles after a successful launch). Generally we try to devise a solution that can be deployed safely without a feature flag but they are often required.
Our team uses them extensively. It's also tied to our A/B and QA testing infrastructure as it performs a similar function of "turn this/that on/off for these particular users". This enables us to do continuous deployment (dead/unfinished code goes to live all the time) and running QA for features on live infrastructure that has actual production loads.
It's also a life saver when issues arise, though the correct term for this is "operational toggles". Flip a switch when functionality is causing issues and it's gone.
HN seems like 90% web developers, and I don't really see the point outside of the web. I mean, we have other ways of gradually rolling out and configuring new software builds.
I've used them going back decades, long before they were a buzzword, only in back-end. The most common use case is where you have a bug fix or new feature that is highly data-specific. Often you'll have a user or set of users with the problematic data pattern. You code up a fix and redeploy, but with the fix disabled. Then enable the fixed code path for the user that reported the problem and ask them to re-test. From their feedback and/or log data you can confirm the fix and then enable for all users. This is helpful for example when there would be privacy issues debugging/testing a fix on real user data.
We use them not as an A/B testing tool as I saw it suggested in a comment but more like a way to have controlled rollouts of features. As our team uses trunk based development we don't have a "development" branch so everything goes to master (we have a staging env and manual judgement prior to production deployment tho)
Not all projects are of the same duration or complexity. Most of the features get developed in a 2 week sprint, but from time to time, there are projects that take a month or so. When there are multiple teams with different working schedules, feature flags add a lot of value during deployments and reduce the dependencies on other teams. The business/product benefits that arise from feature flags are equally valuable.
I know you said frontend, but we use them for both FE and BE. They work fine, but our problem comes with DB migrations. We just haven't found a good way to deal with DB changes and flags.
This isn't a perfect solution but we "solved" this by allowing services to know if migrations have been run. We store the migration data in a shared database and every service that depends on a specific migration, directly or indirectly, is configured to verify that migration has been run before health checks will pass.
The system is designed to support automatic migrations and deployments, but I don't trust it enough yet. (It's easy to write a migration that works on a local database but consumes too many resources in production, so it's just a matter of having a proper preprod, that I haven't created yet.)
We don't really use feature flags, this migration code is just our way to get closer. But it would be possible to write a migration that handles the feature flag swaps, assuming it's a value in a database or something that works somewhat like a database (redis, storage bucket, k8s configmap, etc.) That's how I would do it anyway.
The migration content is all committed to a central repository (we're using a monorepo) and they can basically run "migrate all" to get them. There's also a script that creates a basic environment from scratch, but it comes with only minimal test data, and not nearly enough for load testing.
The dev would need to watch for updates to the main branch to know if something there requires their attention, which may not be totally scalable. We're miniscule and pre-launch so it's not a big deal for us yet.
Build a service/loop/CI/cron job/whatever to pull feature flag state from prod and check into your relevant branch(es). Make the dev deployment apply this state by default.
You’ll get a lot of extra value out of this if you have an automated test pass that exercises the bulk of the product. You get bonus points if you can integrate individual feature flags back to your branch(es) independently, because it will let you easily identify which flag has broken automation.
My team has basically implemented this twice, for two different products.
One (product A) does individual integrations from the production feature flag system to the repo when the flag is at 100% in production. It’s quite successful.
The other (product B) does bulk integration every few hours from production to the repo, bringing all flags at once. It enables flags that have been “approved” (as opposed to fully enabled in production). It’s less successful because of the bulk behavior, which makes it more difficult to isolate breaks.
Both of these integrations go through the same validation gates as pull requests (they are actually implemented as pull requests). This ensures that flag changes through this system cannot break pull requests.
The ideal system for protecting pull requests (or whatever other validation) is to require validation to pass before the flag can be enabled in production. I have not implemented this as a hard gate, but I do have an “fyi” validation run as post of the flag enabling for product A so engineers see if they’ve broken something before they start turning on the flag.
Getting philosophical, I strongly believe that feature flags should not be enabled by default in the validation environment until they are fully enabled (or close to fully enabled) in production. This ensures that the base state (flag off) is validated. You always want it to be safe to turn off the flag, which means you want validation running with the flag off. Product B does not have this behavior for legacy reasons but I think it’s a poor technical decision.
What kind of problems do you run into? And how is your DB deployed/updated/migrated/whatever?
I’ve found it necessary to either always update the DB before the code/binaries or to expose DB version to the code so that it can automatically turn off features if the DB schema isn’t updated yet. Which of these options makes sense depends on how you deploy. If you can rigidly enforce that the DB always updates before binaries, that’s a simpler model (but your DB queries need to be backwards/forwards compatible, depending on whether they are in code or in stored procedures).
The other thing that’s been hugely successful is to have a comprehensive test suite that can be exercised in intermediate states. e.g. New DB with old binaries. Absent a strong test suite, it’s always human judgement when it’s safe to roll out.
I’ve also found success with taking certain actions out of the deployment entirely. e.g. Adding indexes to large tables gets done independent of deployments, because if things go south we don’t want to be trying to resolve the indexing issue in the middle of a deployment. It’s easier to do this in isolation from other changes.
We use them a lot, but in my case mostly on the back end. We have a 24/7 service that runs and our customers depend on our service being up to make money. So if a new feature or change goes out that has any risk, we feature flag it so that if there are any problems we can flip the new feature off immediately rather than have to wait for a fix and deploy, or wait for a rollback.
Our feature flags are nice to work with in that you can just add them in code. If the flag doesn't exist in the DB, it is created with a default value. This makes them pretty painless to work with for us.
You can activate feature flags one server at a time as well to roll things out gradually if you want.
We have a simple web ui in our admin site where you can see them, what they are set to, when last updated etc. A good idea which we haven't done yet, is to log who changed it each time, and why as well.
Being able to find flags that haven't changed in a long time is useful to identify ones you can clean up.
Seems like a lot of folks here and at spotify do, as do we at my gig. Where do you keep your feature flags? Do you have to perform a reset to update them or do you cache them for X minutes?
Not just feature flags, but "knobs" - we use a percentage (of requests or customers) instead of a boolean to enable features so we can dial them up/down.
This let us slowly roll out new features to a subset of users in case there are any issues.
Using feature flags also requires testing the default (not enabled state), ensuring you have a robust realtime configuration manager to control the knobs, and metrics for everything - not just how many requests/customers are opted-in, but also the progress & state of the configuration change.
It does no good to first enable a feature at 1% if only 1% of your servers have received the updated configuration - that's only .01% impact. It also tells you when your rollback is complete - you want to be sure when you disable something that there's not some stuck server with the feature still enabled...
Biased [0], but we use feature flags a ton. It means that as an engineer we can throw new ideas at the wall, turn them on for ourselves and then a small group of trusted customers and super quickly iterate until we have a feature that works well and that we can safely release to the rest of the world.
We mostly use frontend feature flags for this, so we'd only show the link in the menu or the specific component if the feature flag is turned on.
Extensively. Nearly every single change to the codebase is flagged. Even switching copy is behind a flag sometimes if the copy is across multiple places. I find it excessive and irritating; it feels like cargo cult programming to me.
It's risk mitigation. Feature flagging everything is a pointless waste of time until suddenly you have to roll something back in production and it's not feature flagged. The value of feature flags increases with the greater the cost of a mistake making it to production.
For iOS builds particularly, that might have a 1-2 day review delay before you can get a new build approved, they're invaluable.
There are other uses if your flagging system is not "all or nothing". They can be very useful if they support directing a subset of users, customers, or traffic to new code/infra. The bigger a team gets, the more value you get from this.
In fact, FF like this can be used for A/B testing too.
Depends on how long the time is between commit and live-in-production, and on how severe the impact is. And also on the number of developers committing changes to the same codebase/app.
If it takes you >20 minutes from when you commit to when it's live in production, and an incident arises due to the change where data loss/corruption increases in "blast radius" more by having the breakage in production longer, then a feature flag might be a great way to give you an immediate "kill it" switch.
One possible alternative that could give an immediate "kill it" switch is to deploy the last-known-good build, but that only works IF other non-revertable changes haven't shipped since your change (like non-reversible schema migrations), and also IF your time-to-deploy-existing-build is sufficiently fast (it's pretty rare that you actually have instant deploys in live production apps).
If you're in the situation where you have multiple developers committing unrelated changes to the same codebase, and non-reversible changes are a possibility, and your CI/deployment time is sufficiently-long (>5minutes, maybe?), then yeah, feature flags are probably a better fit than "just revert the commit".
For one-man projects with a slow rate of change and a fast CI/CD pipeline, sure, feature flags are overkill.
Sure, though if your back-end has multiple services this becomes harder. If your back-end uses more than one server, then you can run into issues.
Feature flags also let you do partial rollouts, where you release to 0.1% of users and see if stuff breaks. Or A/B test to see if your feature improves whatever its meant to improve. It also lets you roll out the feature selectively to certain users if it's a breaking changes and each user needs time to migrate.
In a CD setup without feature flash, commit equals deploy equals launch.
With feature flags, commit equals deploy. Launch is controlled by feature flags.
For a more mundane analogy, would you rather have a stove that allows cold or burning only, or one with every possible nuance of warm in between?
No, we do monthly releases instead and make sure they're stable or fix them with a follow up bug fix if they're not. If a feature isn't ready yet, it doesn't get into the release (or develop branch).
I personally think feature flags are useful if you're deploying very frequently, but they just add confusion to software that's meant to be released/stable, especially for those developing it (what's with all the half done code and TODOs everywhere?)
Feature flags aren't a gateway to half-done code and TODOs unless they're misused: all code should be production ready, whether it's behind a feature flag or not. A feature flag severs the relationship between the availability of a feature and its appearance in code, which is very useful for lots of reasons -- pushing half-baked code isn't one!
Feature flags can be used as a gate for in progress code. This is common in companies that enforce a linear commit history and encourage many frequent, small commits. It helps avoid merge conflicts that could happen with long living feature branches.
We do. We have one React app running on multiple instances that service different customers and not all customers need the same set of features. Some of features are only supposed to work on our staging instance until they're ready for production.
Also biased! [0], we use them a lot, mainly on the front end. Most of the new features we are working on are gate-able via front end UI elements, and so we push out pretty much all our new features using our own platform.
We are also doing interesting stuff like controlling what features are in our Open Source Docker container via flags in the platform that are baked into our Docker images when they are built.
Once you start using and relying on flags, it's hard to go back, and helps with a bunch of 'good' engineering processes and patterns.
Another interesting aspect is that once you have gated a feature based on a flag, you can then AB test that feature, almost for free, as you have a way of bucketing users and showing or hiding the feature using most feature flag platforms.
We do, not sure if we do it "right", but we do have flags (both in the backend and frontend) to enable features only on "dev" while they are being worked-in-progress, this allows us to not delay pushing to prod. We also have feature flags in our infrastructure-as-code actually (we use CDK, a typescript based infra-as-code framework)
Worked for a bigtech well known name, large and extremely important project, literally the core of a service serving an enormous number of users, and feature flags were mandatory, no exceptions.
I can't imagine working without feature flags. Being able to enable new features in particular deployment rings (canary, dogfood, various production rings or regions), or per users / user groups, enabling gradually (percentage) and so on, is invaluable. I really can't overstate this.
Heck, we went as far as using feature flags for risky bugfixes even.
We had also internal tools to easily work with and track feature flags. A downside is that although normally you'd want to remove old feature flags that become obsolete, this hasn't been done very often.
What I suggested and we started doing was to tag the feature flags with the name of the author and the date at which they were added, and the same for the config updates, and usually ticket number and title for both case. This did help with tracking obsolescence, but obviously there was still a need to plan and do the actual work. Automating this process further was out of the question, due to the high risks involved.
That is the one problem we’ve found with feature flags. It’s very easy to forget to gut the “old” parts when they are turned off. In many cases I’ve seen two or three year-old feature flags whose stale end still remains because the developer and / or team that did the branch never cleaned up. It isn’t usually malicious or lazy… it’s just how things would pan out.
Then we’d be nervous removing the old stuff cause who knows why it was left there and who knows if they’d want it back on again…
That's not a problem with feature flags per se, it's a problem with lazy implementations of feature flags. Flags should be associated with an expiry date, and company comms tooling should be consistently yelling in some public channel when expired flags still exist in the codebase.
> That's not a problem with feature flags per se, it's a problem with lazy implementations of feature flags.
Oh absolutely. Feature flags are great, but you definitely need discipline to make sure you clean things up. The longer the unused code rots, the harder it is to remove it.
I have found it can be good to 'remove' the flag at the same time you create it, but just don't merge the removal until later. I wrote up this idea in a blog post a while ago, if anyone finds it interesting: https://launchdarkly.com/blog/how-to-use-feature-flags-witho...
I've seen this used, but as PRs get added, these 'cleanup' PRs move to the bottom and are usually ignored by other team members.
To me, it's about having enough time to do this in a sprint, and that means it really needs to be a post-launch Jira ticket. Which I've seen done maybe once.
yeah, this is definitely a risk. I agree that the PR needs to be tracked as a post-launch task.
The advantage to this approach is that you do the hard part of removing the flag (ie, thinking through all the parts of the code that need to be cleaned up) while everything is fresh in your mind. Otherwise, you end up spending more time regaining all of the context, and are more likely to leave some vestigial dead code because you aren't sure it isn't needed any more (this is probably less of a risk with languages that lend themselves well to static analysis that can identify dead code, but these tools are never perfect).
Starting new at a startup-with-traction several years ago, I had reason to check out the feature flag configs for my first feature. It was hilarious.
- There were hundreds of them, some of them going back to the garage days, multiple years old. Some would turn on/off major functionality core to the product. For example, this was an ecommerce product - one toggle was "show/hide the buy button". (I think that one is staying in.)
- This was all hand-rolled, pre-LaunchDarkly stuff, but at least they were all in one place. I diffed Production, Staging, and a one-off UAT environment - the toggles were MASSIVELY different across each.
Same. Feature flags need a yearly audit, but who has bandwidth for that? I'd file it under tech debt / tech health, or a great project for entry level or new hire.
> A downside is that although normally you'd want to remove old feature flags that become obsolete, this hasn't been done very often.
I figure this should be somewhat automatable since the relevant bits of code have references to the exact flag's identifier. Was it not done b/c nobody was incentivized in any way to spend any kind of time on cleanup?
edit: OK that might be a bit naive on the organizational side. Force everyone to give every flag an expiry date for review or something maybe?
A typical pattern I saw was a team adds a new flag, then gradually rolls it out. Once done the removal is added to the backlog. “High priority” bug fixes and feature work gradually moves the ticket down in the system. Two months later a reorg happens, the team no longer exists, and the work is lost.
In the case of my team, although we recognize we should clean it up, we’re usually prioritizing pushing out more features. That tendency leads to a lot of old feature flags.
In a startup, being able to get features out is more important than cleaning up old flags. If we don’t achieve product fit before we run out of runway, then the whole thing can get shut down. If we don’t achieve customer validation, same thing.
Usually, at some point, someone will suggest a rewrite. That pretty much never goes well.
> A downside is that although normally you'd want to remove old feature flags that become obsolete, this hasn't been done very often.
We use feature flags a ton and this is something that burns us a little bit too.
The upsides are worth it though. Having that surgical precision in production is essential when you have a complex product that touches many lines of business.
Yes, we do, not in the way a lot of people are describing here (push without disruption, a/b testing). We deploy on prem applications that can be managed from the cloud, so when we add a new feature that involves said applications we have to observe which features the deployed application can use (til upgraded) and only display the appropriate ones from the cloud UI.
Not anymore. We do still have the ability. Some teams have found that it introduced an overhead (moving part, point of failure) and long lived staleness into certain parts of the codebase, which is actually reflective of our teams' priorities. Feature flags bring an overhead that teams should be aware of an acknowledge the management required, before introducing it into a tech stack.
I worked at the biggest retailer in the world and we used them a ton. They got even more useful once we built the concept of a versioned flag that would allow us to be true or false, depending on the app version we were on. That kind of flexibility is a must.
142 comments
[ 4.0 ms ] story [ 199 ms ] threadJust duplicate the testing jobs with the feature flag flipped. Configure new job to turn expected-failure test cases into expected-pass (or whatever analogous way you keep track of it).
Gradual rollout and specific customer opt-in to certain features can certainly be done with feature flags, I wouldn't call that A/B testing unless you're running experiments and reaching a conclusion that A is better than B (or vice-versa).
Put simply it disconnects merge and deploy from launch. This is very useful when your changes rely on other teams or third parties having gone live. Having a feature flag (we call them toggles) lets you get it into production turned off, without having to coordinate deploys with other systems.
The downsides are it is extra development overhead and adds tech debt (you should go back in and remove the toggles after a successful launch). Generally we try to devise a solution that can be deployed safely without a feature flag but they are often required.
It's also a life saver when issues arise, though the correct term for this is "operational toggles". Flip a switch when functionality is causing issues and it's gone.
Never modify a column Never drop a column or table
The system is designed to support automatic migrations and deployments, but I don't trust it enough yet. (It's easy to write a migration that works on a local database but consumes too many resources in production, so it's just a matter of having a proper preprod, that I haven't created yet.)
For example, a new full-stack SWE is onboarding and has an env representing production, populated with test data.
That engineer wants to run the product as it currently exists on production including the state of feature flags.
How does this developer update this state, or remain notified if a FF state changes that affects something they are working on?
The migration content is all committed to a central repository (we're using a monorepo) and they can basically run "migrate all" to get them. There's also a script that creates a basic environment from scratch, but it comes with only minimal test data, and not nearly enough for load testing.
The dev would need to watch for updates to the main branch to know if something there requires their attention, which may not be totally scalable. We're miniscule and pre-launch so it's not a big deal for us yet.
You’ll get a lot of extra value out of this if you have an automated test pass that exercises the bulk of the product. You get bonus points if you can integrate individual feature flags back to your branch(es) independently, because it will let you easily identify which flag has broken automation.
Have you implemented this, or seen write-ups of this pattern described at length? Or perhaps commercial products providing for this scenario?
Curious about prior art on this subject.
One (product A) does individual integrations from the production feature flag system to the repo when the flag is at 100% in production. It’s quite successful.
The other (product B) does bulk integration every few hours from production to the repo, bringing all flags at once. It enables flags that have been “approved” (as opposed to fully enabled in production). It’s less successful because of the bulk behavior, which makes it more difficult to isolate breaks.
Both of these integrations go through the same validation gates as pull requests (they are actually implemented as pull requests). This ensures that flag changes through this system cannot break pull requests.
The ideal system for protecting pull requests (or whatever other validation) is to require validation to pass before the flag can be enabled in production. I have not implemented this as a hard gate, but I do have an “fyi” validation run as post of the flag enabling for product A so engineers see if they’ve broken something before they start turning on the flag.
Getting philosophical, I strongly believe that feature flags should not be enabled by default in the validation environment until they are fully enabled (or close to fully enabled) in production. This ensures that the base state (flag off) is validated. You always want it to be safe to turn off the flag, which means you want validation running with the flag off. Product B does not have this behavior for legacy reasons but I think it’s a poor technical decision.
What kind of problems do you run into? And how is your DB deployed/updated/migrated/whatever?
I’ve found it necessary to either always update the DB before the code/binaries or to expose DB version to the code so that it can automatically turn off features if the DB schema isn’t updated yet. Which of these options makes sense depends on how you deploy. If you can rigidly enforce that the DB always updates before binaries, that’s a simpler model (but your DB queries need to be backwards/forwards compatible, depending on whether they are in code or in stored procedures).
The other thing that’s been hugely successful is to have a comprehensive test suite that can be exercised in intermediate states. e.g. New DB with old binaries. Absent a strong test suite, it’s always human judgement when it’s safe to roll out.
I’ve also found success with taking certain actions out of the deployment entirely. e.g. Adding indexes to large tables gets done independent of deployments, because if things go south we don’t want to be trying to resolve the indexing issue in the middle of a deployment. It’s easier to do this in isolation from other changes.
Our feature flags are nice to work with in that you can just add them in code. If the flag doesn't exist in the DB, it is created with a default value. This makes them pretty painless to work with for us.
You can activate feature flags one server at a time as well to roll things out gradually if you want.
We have a simple web ui in our admin site where you can see them, what they are set to, when last updated etc. A good idea which we haven't done yet, is to log who changed it each time, and why as well.
Being able to find flags that haven't changed in a long time is useful to identify ones you can clean up.
This let us slowly roll out new features to a subset of users in case there are any issues.
Using feature flags also requires testing the default (not enabled state), ensuring you have a robust realtime configuration manager to control the knobs, and metrics for everything - not just how many requests/customers are opted-in, but also the progress & state of the configuration change.
It does no good to first enable a feature at 1% if only 1% of your servers have received the updated configuration - that's only .01% impact. It also tells you when your rollback is complete - you want to be sure when you disable something that there's not some stuck server with the feature still enabled...
We mostly use frontend feature flags for this, so we'd only show the link in the menu or the specific component if the feature flag is turned on.
[0] https://posthog.com/docs/user-guides/feature-flags
For iOS builds particularly, that might have a 1-2 day review delay before you can get a new build approved, they're invaluable.
In fact, FF like this can be used for A/B testing too.
At $PRIOR_JOB, a revert/deploy or rollback takes tens of minutes of the main application.
If it takes you >20 minutes from when you commit to when it's live in production, and an incident arises due to the change where data loss/corruption increases in "blast radius" more by having the breakage in production longer, then a feature flag might be a great way to give you an immediate "kill it" switch.
One possible alternative that could give an immediate "kill it" switch is to deploy the last-known-good build, but that only works IF other non-revertable changes haven't shipped since your change (like non-reversible schema migrations), and also IF your time-to-deploy-existing-build is sufficiently fast (it's pretty rare that you actually have instant deploys in live production apps).
If you're in the situation where you have multiple developers committing unrelated changes to the same codebase, and non-reversible changes are a possibility, and your CI/deployment time is sufficiently-long (>5minutes, maybe?), then yeah, feature flags are probably a better fit than "just revert the commit".
For one-man projects with a slow rate of change and a fast CI/CD pipeline, sure, feature flags are overkill.
Feature flags also let you do partial rollouts, where you release to 0.1% of users and see if stuff breaks. Or A/B test to see if your feature improves whatever its meant to improve. It also lets you roll out the feature selectively to certain users if it's a breaking changes and each user needs time to migrate.
In a CD setup without feature flash, commit equals deploy equals launch. With feature flags, commit equals deploy. Launch is controlled by feature flags.
For a more mundane analogy, would you rather have a stove that allows cold or burning only, or one with every possible nuance of warm in between?
I personally think feature flags are useful if you're deploying very frequently, but they just add confusion to software that's meant to be released/stable, especially for those developing it (what's with all the half done code and TODOs everywhere?)
We are also doing interesting stuff like controlling what features are in our Open Source Docker container via flags in the platform that are baked into our Docker images when they are built.
Once you start using and relying on flags, it's hard to go back, and helps with a bunch of 'good' engineering processes and patterns.
Another interesting aspect is that once you have gated a feature based on a flag, you can then AB test that feature, almost for free, as you have a way of bucketing users and showing or hiding the feature using most feature flag platforms.
[0] https://flagsmith.com/ & https://github.com/Flagsmith/flagsmith
I can't imagine working without feature flags. Being able to enable new features in particular deployment rings (canary, dogfood, various production rings or regions), or per users / user groups, enabling gradually (percentage) and so on, is invaluable. I really can't overstate this.
Heck, we went as far as using feature flags for risky bugfixes even.
We had also internal tools to easily work with and track feature flags. A downside is that although normally you'd want to remove old feature flags that become obsolete, this hasn't been done very often.
What I suggested and we started doing was to tag the feature flags with the name of the author and the date at which they were added, and the same for the config updates, and usually ticket number and title for both case. This did help with tracking obsolescence, but obviously there was still a need to plan and do the actual work. Automating this process further was out of the question, due to the high risks involved.
Edit: added the last paragraph.
Then we’d be nervous removing the old stuff cause who knows why it was left there and who knows if they’d want it back on again…
Oh absolutely. Feature flags are great, but you definitely need discipline to make sure you clean things up. The longer the unused code rots, the harder it is to remove it.
To me, it's about having enough time to do this in a sprint, and that means it really needs to be a post-launch Jira ticket. Which I've seen done maybe once.
The advantage to this approach is that you do the hard part of removing the flag (ie, thinking through all the parts of the code that need to be cleaned up) while everything is fresh in your mind. Otherwise, you end up spending more time regaining all of the context, and are more likely to leave some vestigial dead code because you aren't sure it isn't needed any more (this is probably less of a risk with languages that lend themselves well to static analysis that can identify dead code, but these tools are never perfect).
- There were hundreds of them, some of them going back to the garage days, multiple years old. Some would turn on/off major functionality core to the product. For example, this was an ecommerce product - one toggle was "show/hide the buy button". (I think that one is staying in.) - This was all hand-rolled, pre-LaunchDarkly stuff, but at least they were all in one place. I diffed Production, Staging, and a one-off UAT environment - the toggles were MASSIVELY different across each.
I figure this should be somewhat automatable since the relevant bits of code have references to the exact flag's identifier. Was it not done b/c nobody was incentivized in any way to spend any kind of time on cleanup?
edit: OK that might be a bit naive on the organizational side. Force everyone to give every flag an expiry date for review or something maybe?
In a startup, being able to get features out is more important than cleaning up old flags. If we don’t achieve product fit before we run out of runway, then the whole thing can get shut down. If we don’t achieve customer validation, same thing.
Usually, at some point, someone will suggest a rewrite. That pretty much never goes well.
I suggest tagging the PO/manager who requested the feature.
We use feature flags a ton and this is something that burns us a little bit too.
The upsides are worth it though. Having that surgical precision in production is essential when you have a complex product that touches many lines of business.
Having them everywhere was great, we could flip things off feature by feature just by changing the flags.
Having the old ones still around can be handy.
They have their place, but they certainly have a cost as well.