Block deploys to keep dead feature flags out of our code

TL:DR — have a test fail which stops deploys if feature flags aren’t removed within an expected timeframe.

I previously wrote about how we use feature flags a lot. We still do.

A problem we encountered was while we were very fast to add feature flags, we were much slower to remove them from the code.

To combat this I instituted some tests that would fail if feature flags weren’t removed within their expected timeframe.

The feature flags we use are Flipper which I highly recommend.

The code is in ruby but I’m hoping its quite a friendly language so its still readable for non-rubyists.

the code of the tests. This is available at the end of the blog

How it works

Every time an engineer adds a Flipper feature flag they also add the name of the flipper and the expected removal date to an array in our test file.

The tests then check that

a) we have an expected removal date for every feature flag in the system

b) each feature flag has not passed its expected removal date

How its working

Its working well. At our worst point we had about 25 feature flags with 20 of them being turned on permanently.

Now we’ve kept our feature flags under 10 for quite a while and we rarely have dead ones in for very long.

Can’t you just game it and keep postponing?

Yes of course. If a test fails likely the first action will be to update the date to the following day or week. But you will also contact the person who added the flag to let them know. Nobody wants to keep dead code in the codebase. Nobody wants to delay deploys by having their dead code cause tests to fail. So it nudges you towards doing something about it. And these nudges have shown to be very effective.

Inspiration

This approach was heavily inspired by Justin Searls snappily named todo_or_die gem.

The code

describe "Flipper" do
flippers = [
  ["hot_dogs_for_fingers", "2022-11-08"],
  ["please_be_kind", "2023-01-01"],
  ["the_everything_bagel", "2022-12-21"]
]
# when adding any new flipper we add the flipper name and expected removal date to the flippers array
it "tests all the flippers we have" do
  tested_flipper_names = flippers.map { |flipper| flipper[0] }.sort
  flipper_names = Flipper.features.map { |flipper| flipper.name.to_s }.sort
  expect(tested_flipper_names).to eq flipper_names
end
  flippers.each do |flipper|
    it "#{flipper[0]} has not passed its expected removal date" do
      expect(date_has_passed(flipper[1]) && flipper_exists(flipper[0])).to eq false
    end
  end
  def date_has_passed(date)
    Time.zone.today > Date.parse(date)
  end
  def flipper_exists(flipper_name)
    Flipper[flipper_name.to_sym].exist?
  end
end