Check a Flagsmith experiment's split before launch

Flagsmith's experiment setup is in beta and stores the hypothesis, variants, and primary metric, which the check reads from the live flag.

How it works

  1. Connect the Flagsmith MCP server at https://mcp.flagsmith.com to your AI assistant or coding agent and sign in with OAuth, or use an API key header where your client runs without a browser, noting that experimentation in Flagsmith is currently in Beta.
  2. Install the ExpLint MCP at https://explint.com/mcp to enable your agent to audit tests.
  3. Run the prompt below to audit these plans before the work starts.

How the check runs

  1. It reads the experiment's name and hypothesis, the multivariate flag behind it with the current value as control, the rollout share entering the experiment beside the variation split, and the primary metric with its expected direction.
  2. It sends that text to ExpLint with explint__lint, which returns errors, warnings, and passes, each finding naming the rule behind it.
  3. You edit the experiment in Flagsmith and ask for a fresh read, keeping the findings in chat because the MCP tools change setup fields without storing a verdict on the experiment.

Copy and paste this prompt

Change the words in square brackets to name your own project, board, or test.

Prompt for your assistant
Use the Flagsmith MCP tools to read the experiment [experiment name]. Build one text from its name and hypothesis, variants with the control marked, rollout share and variation split, and primary metric with its expected direction. Lint it with explint.com against project [your ExpLint project name]. Show each error and warning with its Flagsmith field. Do not launch or edit the experiment yet. Suggest fixes here and re-check after I update Flagsmith.

What ExpLint looks for

ExpLint ships over 24 rules, and a project switches each one on or off or rewrites it in plain English. These are the rules the plans this guide reads break most often. An error means the test cannot answer its question as written, and a warning means the result will be hard to judge when it comes in.

RuleLevelIn plain words
hypothesis/is-falsifiableerrorAn experiment that only claims the change will help gives the setup nothing a finished result could contradict.
variants/control-is-describederrorThe current flag value is the control, and the plan should say what that value shows people today.
randomization/states-the-spliterrorRollout share and variation split are separate numbers in Flagsmith, and the plan should show both so the reader knows how many people enter and how they divide.
decision/has-criteria-before-launcherrorThe setup names no decision field, so the plan should record what result would roll the flag out and what result would turn it off.

Read more

Portrait of Kameron Tanseli

Kameron Tanseli

Founder, ExpLint

Kameron Tanseli is the founder of ExpLint and a growth engineer who has run more than 1,500 experiments across a nine-year career. As the founding growth hire at Fyxer he launched 540 experiments in a year while the company grew from $1M to $35M in annual revenue, and he now runs growth engineering at FLORA, where a one-person team ships six to eight experiments a week.

Don’t let a bad test make it to production

Install the MCP and start flagging ticket slop before it reaches your users