Lint PostHog experiments with explint.com before you launch
Read an experiment from PostHog with its MCP server, check the plan with explint.com in the same chat, and use PostHog's own sample size help beside the check so the test is worth running and written clearly.
What you need before you start
- A PostHog project with an experiment drafted, or an idea ready to become one, with a feature flag behind it if the experiment needs one.
- Access to PostHog's MCP server for your assistant, which PostHog offers as a hosted server you add with its wizard or by hand.
- An A/BLint account with a project for your team, in the same assistant as PostHog.
- Rough numbers for your current rate and traffic, because PostHog can help estimate sample size and run time when you give it real figures to work from.
Connect PostHog and A/BLint
- Add PostHog's MCP server in your assistant at https://mcp.posthog.com/mcp using PostHog's MCP install help for your tool, and sign in to the PostHog project that holds your experiments. You can narrow the tools to experiments when you want a shorter list, using the feature choice PostHog documents.
- Add A/BLint in the same assistant at https://explint.com/mcp and finish the browser sign-in, or use an API key from the MCP page in your A/BLint workspace if your assistant asks for a key instead.
- Ask the assistant to list your draft experiments in PostHog, because seeing the right project come back saves confusion when you have more than one PostHog project.
- Name the experiment and your A/BLint project, so the assistant reads one draft and checks it against one rule set.
How the check runs
- The assistant reads the experiment from PostHog, including its hypothesis, flag, variants, metrics, and any sample size or run time notes.
- Where PostHog offers a sample size estimate, the assistant adds that figure to the text in plain words, so the checker can judge whether the plan shows its working.
- It sends the full text to A/BLint with ablint__lint and brings back errors, warnings, and passes.
- It explains which findings are about the writing in PostHog and which point at a setting, such as a second metric that needs demoting to a guardrail.
- You update the experiment in PostHog, then ask for a fresh read and a fresh lint before you launch.
Copy and paste this prompt
Change the words in square brackets to name your own project, board, or test.
Prompt for your assistant
Use the PostHog MCP tools to read the experiment called [experiment name], and include any sample size or run time estimate PostHog gives for it. Build one text from its hypothesis, variants and what each one changes, primary metric, secondary and guardrail metrics, traffic split, and planned run time. Lint that text with explint.com against the project [your A/BLint project name]. Show me each error and warning with the PostHog field it came from. Do not launch or edit the experiment. Suggest fixes in this chat, and re-check after I update PostHog.
What A/BLint looks for
A/BLint scores the text against the rules switched on in your project. The default project uses the rules below, and your team can turn rules off, change how strict they are, or add its own rules in plain English. An error means the test cannot answer its question as written, and a warning means the result will be hard to judge when it comes in.
| Rule | Level | In plain words |
|---|---|---|
| hypothesis/is-falsifiable | error | The idea makes a clear guess that a result could prove wrong, instead of saying the change will improve things in general. |
| hypothesis/names-metric-and-direction | error | The guess names the number it expects to move and which way it should move, such as a rise in checkout completion. |
| metrics/has-one-primary-metric | error | One main number decides the test, so the team agrees before the test starts which result matters most. |
| guardrails/are-listed | error | The write-up names the numbers that must not get worse, such as speed, errors, refunds, or people leaving. |
| variants/each-differs-from-control-stated | error | Each version says exactly what changes, so the team can tell afterwards which change caused the result. |
| power/has-a-sample-size-calculation | error | The plan shows how many people the test needs, worked out from the current rate and the smallest change worth caring about. |
| power/states-the-minimum-detectable-effect | error | The plan names the smallest change that would matter to the business, which is the number the sample size rests on. |
| duration/covers-full-business-cycles | error | The test runs long enough to cover full weeks, so a busy Monday and a quiet Sunday both count in the result. |
An example you can compare against
Before
Onboarding checklist v2
- Hypothesis
- The new checklist will activate more users.
- Metrics
- Activation, retention, clicks
- Flag split
- 50/50
After, checked again
Shorter onboarding checklist
- Hypothesis
- If we cut the onboarding checklist from six steps to three and move the optional steps after the first report, then the share of new teams that reach their first report in the first session will rise, because our funnel shows most new teams stop at the data connection step.
- Main number
- Share of new teams that reach a first report in their first session.
- Guardrails
- Support chats in the first week must not rise, and data connection completions within seven days must not fall by more than 5 points.
- Versions
- Control keeps six steps. Test shows three steps and moves the rest after the first report, with no change to the steps themselves.
- Split and size
- New teams, split by team, 50/50, about 4,000 teams over three full weeks, enough to spot a 3 point rise at our current rate.
- Decision rule
- Ship if the main number rises and the guardrails hold.
Still flagged on the rewrite
- errorguardrails/have-launch-thresholdsThis guardrail has no number, unlike the one that follows it.
- warningdecision/has-criteria-before-launchThe plan ships on a rise and says nothing agreed about the other results.
The stronger experiment uses PostHog language your team already knows, such as teams as the split unit, and it shows the sample size working in the same text as the guess. That habit helps twice, because the person who reads the results later can see what the team hoped for and what the numbers needed to prove it.
Fix what it finds and check again
Start with the errors, and have the assistant suggest each rewrite in the chat before anything is written back to PostHog, so wording your team already agreed stays yours to approve. Rewrite the experiment plan so a teammate reading it cold could say what will change, what number will judge it, and what would stop a launch, then lint the same experiment plan again and ask which rules now pass that failed before.
Limits and things to check yourself
- PostHog's MCP server can create and edit experiments and flags, so say plainly that you want suggestions in the chat first, because a loose prompt can turn a check into a change you did not mean to make.
- Insights and funnels that define a metric may sit outside the experiment read, so paste the metric definition in plain words when a finding seems to miss it.
- PostHog's sample size help works from the traffic and rates you give it. Wrong inputs give a confident looking estimate that still rests on sand, so check the base rate against a recent week.
- A passing score says the write-up covers what the rules ask for. It cannot say the idea is worth testing, that the traffic exists to reach a clear result, or that the change is safe to build, and those calls stay with your team.