> Make feature flags short-lived. Do not confuse flags with application configuration.

This is my current battle.

I introduced feature flags to the team as a means to separate deployment from launch of new features. For the sake of getting it working and used, I made the mis-step of backing the flags with config files with the intent to get Launch Darkly or Unleash working ASAP instead to replace them.

Then another dev decided that these Feature Flags look like a great way to implement permanent application configs for different subsets of entities in our system. In fact, he evangelized it in his design for a major new project (I was not invited to the review).

Now I have to stand back and watch as the feature flags are being used for long-term configurations. I objected when I saw the misuse- in a code review I said "hey that's not what these are for"- and was overruled by management. This is the design, there's no time to update it, I'm sure we can fix it later, someday.

Lesson learned: make it very hard to misuse meta-features like feature flags, or someone will use them to get their stuff done faster.

For those that dont know about the project, check out Open Feature https://openfeature.dev/ which is sort of like Open Telemetry but for feature flags. Helps avoid vendor lock in. We're a young project and looking for help and to build the community!

This feels a bit like the dicta on 12 Factor: rules handed down from a presumed authority without any discussion of the tradeoffs. Engineering is tradeoff evaluation. Give me some discussion about the alternatives, when and why they're inferior and don't pretend like the proposed solution doesn't have shortcomings or pitfalls.

Item #1 depends on the reason you're using feature flags.

For a more nuanced and careful discussion of the topic I like to reference: https://martinfowler.com/articles/feature-toggles.html

I dedicated a day to evaluating feature flag software based on specific criteria:

    - Must support multiple SDKs, including Java and Ruby.
    - Should be self-hosted with PostgreSQL database support.
    - Needs to enable remote configuration for arbitrary values (not just feature flags). I don't run two separate services for this.
    - Should offer some UI functionality.
    - it should cache flag values locally and, ideally, provide live data updates (though pooling is acceptable).

    - Unleash: Impressive and powerful, but its UI is more complex than needed, and it lacks remote configuration.
    - Flagsmith: Offers remote configuration but appears less polished with some features not working smoothly; Java SDK error reporting needs improvement.
    - Flipt: Simple and elegant, but lacks remote configuration and local caching for Java SDK.
    - FeatureHub: Offers fewer features than Unleash and Flagsmith; its Java API seems somewhat enterprisly but supports remote configuration and live data updates.

As an engineer, I am generally against feature flags.

They fracture your code base, are sometimes never removed, and add complexity and logic that at best is a boolean check and at worse is something more involved.

I'd love a world where engineers are given time to complete their feature in its entirety, and the feature is released when it is ready.

Sadly, we do not live in that world and hence: feature flags.

I just item 1 (“Enable run-time control. Control flags dynamically, not using config files”) and it’s almost exclusively focused on what to do but not on why to do it.

It seems to be skipping past the use-cases and assumptions, in particular, describing what a system with feature flags looks and acts like, what the benefits and drawbacks are.

Background: I work at Block/Square, on the team that owns (but didn't build) our internal Feature Flag system, and also have a lot of experience with using LaunchDarkly.

I like the idea of caching locally, although k8s makes that a bit more difficult since containers are typically ephemeral. People will use feature flags for things that they shouldn't, so eventually "falling back go default values" will cause production problems. One thing you can do to help with this is run proxies closer to your services. For example, LaunchDarkly has an open source "Relay".

Local evaluation seems to be pretty standard at this point, although I'd argue that delivering flag definitions is (relatively) easy. One of the real value-add of a product like LaunchDarkly is all the things they can do when your applications send evaluation data upstream: unused flags, only-ever-evaluated-to-the-default flags, only-ever-evaluated-to-one-outcome flags, etc.

One best practice that I'd love to see spread (in our codebases too) is always naming the full feature flag directly in code, as a string (not a constant). I'd argue the same practice should be taken with metrics names.

One of the most useful things to know (but seldom communicated clearly near landing pages) is a basic sketch of the architecture. It's necessary to know how things will behave if there is trouble. For instance: our internal system uses ZK to store (protobuf) flag definitions, and applications set watches to be notified of changes. LaunchDarkly clients download all flags[1] in the project on connection, then stream changes.

If I were going to build a feature flag system, I would ensure that there is a global, incrementing counter that is updated every time any change is made, and make it a fundamental aspect of the design. That way, clients can cache what they've seen, and easily fetch only necessary updates. You could also imagine annotating that generation ID into W3C Baggage, and passing it through the microservices call graph to ensure evaluation at a consistent point in time (clients would need to cache history for a minute or two, of course).

One other dimension in which feature flag services vary is by the complexity of the rules they allow you to evaluate. Our internal system has a mini expression language (probably overkill). LaunchDarkly's arguably better system gives you an ordered set of rules within which conditions are ANDed together. Both allow you to pass in arbitrary contexts of key/value pairs. Many open source solutions (Unleash, last I checked, some time ago) are more limited: some of them don't let you vary on inputs, some only a small set of prescribed attributes.

I think the time is ripe for an open standard client API for feature flags. I think standardizing the communication mechanisms would be constricting, but there's no reason we couldn't create something analogous to (or even part of) the Open Telemetry client SDK for feature flags. If you are seriously interested in collaborating on that, please get in touch. (I'm "zellyn" just about everywhere)

[1] Yes, this causes problems if you have too many flags in one project. They have a pretty nice filtering solution that's almost fully ready.

[Update: edited to make 70% of it not italics ]

The system we're building now meets most of these but not necessarily in the way described.

First, we're building a runtime configuration system on top of AWS AppConfig. YAML/proto validation that pushes to AppConfig via gitops and bazel. Configurations are namespaced so the unique names is solved. It's all open in git.

Feature flags are special cases of runtime configuration.

We are distinguishing backend feature flags from experimentation/variants for users. We don't have (or want) cohorting by user IDs or roles. We have a separate system for that and it does it well.

The last two points - distinguishing between experimentation/feature variants and feature flags as runtime configuration are somewhat axiomatic differences. Folks might disagree but ultimately we have that separate system that solves that case. They're complimentary and share a lot of properties but ultimately it solves a lot of angst if you don't force both to be the same tool.

>Organizations who adopt feature flags see improvements in all key operational metrics for DevOps: Lead time to changes, mean-time-to-recovery, deployment frequency, and change failure rate.

Is this true? unfortunately there's no sources indicated, and a quick check on scholar doesn't show me anything of the sort.

We use Unleash. There are many things you can do with feature flags and Unleash helps with a lot of them. However, my feeling is 80% of the value comes from 20% of the features. Even a much simpler system provides a ton of benefit. For me, it is top of the list after having automated tests and automated deployments.

With regard to web-based services, once you’ve got the ability to do canary testing, IMO flags/toggles are less compelling — busier code and logic you’ll have to pull out later.

I know features flags like the flags of FF, but they are in fact config files. How does one realize dynamic control of features flags?

Are they using a kind of logic to determine to turn on/off a feature or do they query a central database to know that?

Can someone explain its basic mechanism? Thanks

More principles

- Require in code defaults for fault tolerance

- Start annoying the flag author to delete if the flag is over a month old

- Partial rollout should be by hash on user id

- Contextual flag features should always be supplied by client (e.g. only show in LA, the location should be provided by client)

I couldn't find an easy link from these docs to the product page on mobile. Seems like a wasted opportunity. I had to edit the URL to get to the company website.

Offtopic but relevant:

TL;DR if you break long posts into pages, at least have an option to see the whole thing in a single page.

I use a browser extension to send websites to my Kindle. It's great for long-ish format blog posts that I want to read, but I don't have the time at the moment. However, whenever I see long blog posts that are broken into sections, each one in it's own page, it becomes a mess. It forces me to navigate each individual page and send it to my Kindle. Then in the Kindle I have a long list of unsorted files that I need to jump around to read in order.

I understand breaking long pieces of text into pages makes it neater and more organized, but at least have an option to see the whole thing in a single page, as a way to export it somewhere else for easy reading.

Open Source knowledge sharing. I like it:

"Unleash is open-source, and so are these principles. Have something to contribute? Open a PR or discussion on our Github."

Is it just me or does this articles text structure and wording strongly indicate that is was written by gpt?