Split traffic and record what each variant actually did
Two versions of a checkout, two confident opinions, and whichever one ships becomes the thing that was always obviously right. An A/B test replaces the argument with an assignment and a number.
Experiments fail for boring reasons far more often than statistical ones
None of these are analysis problems. They are all assignment problems — and assignment is what a feature flag system is already for.
Assignment done badly
- A user gets variant A on the web and variant B in the app
- Someone reassigns buckets mid-test to “even out” the split
- The same cohort lands in the treatment arm of four consecutive experiments
- Two systems with two opinions about which variant a user is in
Assignment that holds still
- The same person gets the same arm on every request, all test
- Widening or narrowing never reshuffles who is already assigned
- The feature key is in the hash, so experiments stay independent
- One system decides, and it is the one already controlling the rollout
Bucketing
Three properties, three ways experiments stop going wrong
Stable per user
No flip-flopping between versions on refresh.
Identical across services
Your API, web app and mobile app agree — as long as they pass the same bucket key.
Independent between experiments
The users in arm A of this test are not the users in arm A of the next one. Your experiments do not quietly become one big correlated experiment.
Variants
Arms that carry their own parameters
type Paywall = { headline: string; trialDays: number };
const evaluation = featureflow.evaluate('paywall-copy');
const paywall = evaluation.jsonValue<Paywall>();
renderPaywall({
headline: paywall?.headline ?? 'Start your free trial',
trialDays: paywall?.trialDays ?? 14,
});
// record the outcome against whichever arm this user is in
featureflow.track('trial-started');
featureflow.track('subscription', { value: 49.95, plan: 'pro' });Check the split before you trust the result. If you configured an even three-way split and one arm is answering a fraction of the traffic, no amount of analysis downstream will fix it.
Results
Evaluation counts per variant, every time
Where Featureflow stops, honestly
Featureflow is an assignment and delivery layer with goal tracking. It is not a statistics engine — it will not run sequential tests, apply variance reduction, or tell you a result is significant.
If you need statistics
The right shape is Featureflow deciding who is in which arm and emitting goal events, with a dedicated platform doing the analysis.
If you need an answer
“Did the new onboarding make things worse?” is answered by a clean split with a tracked goal.
Knowing which you have
Being clear about that saves buying a platform for the wrong problem.
One flag does the whole lifecycle: it ramps the feature safely, holds a stable split while you learn which variant wins, and turns the whole thing off if the experiment itself is the problem — without a deploy at any point.
Frequently asked questions
Do I need a separate A/B testing tool?
For the delivery half, no — a multivariate flag is an experiment. Featureflow assigns users deterministically, records evaluation counts per variant, and gives you a track() call for goals. For sequential testing, variance reduction and automated significance calls you will want a dedicated platform, with Featureflow as the assignment layer feeding it.
How are users assigned to variants?
By hashing the salt, feature key and the user's bucket key together and taking the result modulo 100, giving each user a stable position from 0 to 99 for that feature. The same user gets the same variant on every request and in every service passing the same bucket key.
How do I record a conversion?
Call track() with a goal key and the user. The optional third argument is either a number or an object with an optional numeric value plus your own fields — the same shape as the OpenFeature tracking API. Goal events are batched and flushed alongside evaluation events.
Can a variant carry more than a name?
Yes. Each variant carries a JSON payload of up to 32KB alongside its key, so an experiment arm ships its own parameters — copy, thresholds, which model to call — rather than requiring a code branch per arm.
How is this different from a progressive rollout?
The mechanism is nearly identical; the intent is not. A rollout asks 'is this safe yet?' and ramps toward 100%. An A/B test asks 'which of these is better?' and holds a stable split until you have an answer. Same flag object, different question.
What should I check before trusting a result?
The evaluation counts per variant. If you configured an even three-way split and one arm is answering a fraction of the traffic, the experiment is broken — usually a targeting rule above the split rule is matching first. It takes ten seconds and it is the single most useful habit.
Turn your next flag into an experiment
Multivariate flags, traffic splits and goal tracking are on the free tier — no credit card.
What else Featureflow solves
Kill switch
Turn a bad feature off without shipping a hotfix
Progressive rollout
Reach 1% of users before you reach all of them
Release pipelines
The rollout process runs itself instead of being remembered
Approvals & audit
Answer 'who changed production?' without an enterprise contract
Remote config
Change settings without a deploy or an app store review
Mobile releases
Update the app your users already installed