1. Home
  2. Meta Hub
  3. Tracking & Operations
  4. Meta A/B Testing in Ads Manager and Experiments: Setup, Variables, Confidence and When to Use Lift Studies
Tracking & Operations · 8 min read

Meta A/B Testing in Ads Manager and Experiments: Setup, Variables, Confidence and When to Use Lift Studies

By the Power Ads teamUpdated 2026-11-20Checked against 20 sources1,543 words

Turning two ad sets on and comparing their numbers in Ads Manager feels like a test, but Meta says it isn't true experimentation. Without its A/B testing tool, the system doesn't split the audience evenly but treats the ad sets in combination, which skews delivery and budget distribution. This guide covers Meta's own testing tools: where to create an A/B test, what you can test, how the audience is split, duration and budget guidance, how winners and confidence are reported, creative tests inside an existing campaign, and when a lift study is the better tool. Each rule comes from Meta's Help Center, measurement pages or developer documentation. Practitioner advice is labelled as such.

How Meta's A/B test works, and where to create one

An A/B test compares versions of an ad strategy that differ in a variable such as the image, text, audience or placement. Meta divides your audience into random, non-overlapping groups so nobody sees more than one version, then measures each version on a cost per result or cost per conversion lift basis. Meta recommends against testing by switching ad sets or campaigns on and off by hand, which can lead to overlapping audiences, inefficient delivery and unreliable results.

Meta says all the ways to create a test use the same underlying technology, and finished results are available in the Experiments tool.

  • Ads Manager toolbar (Meta's recommended method): tick the campaigns or ad sets to use, click A/B test and pick a variable. Choosing Custom duplicates the selection so you can edit any variable in the new version.
  • Duplicate an ad set or ad: click Duplicate and choose New split test campaign. Version A stays identical and you change Version B. The item must be published and not already in a test, and Meta says this option is rolling out gradually.
  • Experiments tool: select Create test, then A/B Testing. You can duplicate a campaign or compare up to 5 existing campaigns, but not campaigns already in another test or using the reservation buying type.
  • During campaign creation: listed by Meta, but it notes the option may not be available to you because of A/B test updates.

What you can test

Meta's Help Center lists four kinds of variable. Its split testing developer documentation also names delivery optimization and budgets. In Ads Manager, the variables offered depend on what you duplicate: an ad set shows audience and placement variables, an ad shows image and text variables.

  • Creative: different images, text or creative types, such as a video against a carousel. Managed at the ad level.
  • Audience: for example, adding or removing regions, or a custom audience against an interest-based one. Managed at the ad set level.
  • Placements: for example, automatic placements against placements you choose.
  • Custom: several variables changed at once. Meta warns you then won't know which one produced the winner, and recommends starting with a single variable.
Power AdsUnlimited Meta agency ad accounts on our corporate cards, with 24/7 support on Telegram and WhatsApp, for teams spending roughly $100,000+/mo. 4% per top-up, no setup fee. Apply for access →

Duration, budget and estimated power

In Ads Manager you can schedule an A/B test for 1 to 30 days. Meta recommends at least 7 days and says shorter tests may be inconclusive. If customers usually take more than 7 days to convert after seeing an ad, it suggests running longer, giving 10 days as an example.

Meta doesn't publish a minimum test budget. It says the budget should produce enough results to confidently determine a winner, and that both versions should get the same budget. Duplicating a campaign duplicates its budget too. Creating a test costs nothing extra, but Meta's A/B testing FAQ says an adequate budget is what ensures an even split and a confident result. When you create a test by duplicating, Ads Manager shows an estimated power, the likelihood of a statistically significant result given your schedule and budget. Meta recommends at least 80%, and notes this is an estimate, not a guarantee.

Meta also says the audience should be large enough for the test and not used by any other campaign running at the same time, since overlap can contaminate results. Because the audience is divided, test ad sets are more exposed to under-delivery, so you may need to broaden audiences more than usual or raise the budget.

Once a test is Running, you can't change the variable being tested. Changing the schedule or cancelling is still possible, but Meta says either affects the reliability of results so far. Cancelling doesn't stop delivery; pause the campaigns in Ads Manager to stop spend.

How winners and confidence are reported

Without a holdout, the winner is the version with the lowest cost per result for the key metric you chose at setup, such as cost per purchase. Meta simulates possible outcomes tens of thousands of times from the test data and reports a confidence percentage: the chance of similar results if the test were repeated. Meta's confidence article says that for A/B tests, 65% or higher represents a winning result. For lift tests, the bar is 90% or higher.

Results appear in Experiments, and in Ads Manager a beaker icon marks campaigns in a test. They include a cost per result chart, an explanation of the best performer marked with a green trophy, and age and gender breakdowns. With no clear winner, Meta shows top performers on other metrics; with none at all, it may suggest next steps such as extending the test. Meta notes a declared winner can still have higher costs if the test was too short or had too few results.

Creative tests inside an existing campaign

Ads Manager also has a creative test feature, set up at the ad level of an existing campaign. You compare 2 to 7 copies of an ad, and Meta provides delivery to the new test ads. Because the test runs in place, winners can keep running without being moved to another campaign where, Meta notes, learnings would reset. Meta's Marketing API documents creative tests as split tests across 2 to 7 treatment cells, with one ad per cell.

  • Supported only in campaigns using the Highest volume bid strategy.
  • Open the ad, go to the Creative test section and click Set up test. You can change each copy's creative after confirming.
  • You choose how much of the campaign or ad set budget goes to test ads. Meta suggests no more than 20%, and aims to spend that amount per day on them.
  • Test ads start on their ad set schedule when published; the test schedule only sets when results are measured.
  • Results identify top-performing test ads, but no confidence level is included. Nothing changes automatically: test ads keep running until you turn them off.

When to use a lift study instead

An A/B test tells you which strategy has the lower cost per result, not how many results would have happened without the ads. Meta positions A/B testing for tactical optimizations and Conversion Lift for measuring the conversions your ads directly caused, which supports decisions such as where to increase budget. Lift tests compare a randomized test group that can see your ads with a control group that can't, and Meta notes the control groups can raise the overall cost per impression of the campaigns in the test. For geo holdouts and MMM, see our incrementality testing guide.

  • Conversion Lift: as a guide, self-serve tests need a campaign that started in the past year with at least $5,000 USD spend and 500 conversions under a 1-day click, 7-day click or 1-day view setting, prorated beyond 90 days. The account also needs Conversions API sending an event with an Event Match Quality score above 5, or a supported data source sending a low-funnel or app install event (not available to advertisers mainly targeting countries affected by the ePrivacy directive).
  • Brand Lift: measures awareness, recall, perception or purchase intent through surveys across Meta apps. Meta's Experiments overview lists $120,000 USD ad spend in the last 90 days, and campaigns must also meet a per-country minimum budget, $15,000 by default for unlisted countries (as of September 15, 2026). Meta-managed brand lift tests go through a Meta account representative, and eligibility isn't guaranteed.

Practitioner advice and common mistakes

Practitioner advice, from our experience rather than Meta's documentation: write down the hypothesis, the key metric and the decision you'll make for each outcome before launch, and prefer a metric tied to revenue, such as cost per purchase, over cost per click. If estimated power sits well under 80%, lengthen the schedule, raise the budget or test a bigger difference before launching. Use creative tests for ongoing creative iteration and full A/B tests for structural questions. Our creative testing framework and offer testing framework cover what to test, and our guide to budgeting for testing vs. scaling covers how much to set aside.

Power Ads supplies Meta agency ad accounts at 4% per top-up; we don't run campaigns or tests for clients. The mistakes below are ones Meta's guidance warns against.

  • Testing by switching ad sets on and off instead of using the tool.
  • Versions that are too similar. Meta's example: past shoppers aged 18 to 20 against those aged 20 to 22 may not produce a confident winner.
  • Changing several variables, then crediting one of them.
  • Using the test audience in other campaigns at the same time.
  • Ending early or editing mid-test: tests under 7 days may be inconclusive, and schedule changes affect reliability.
  • Treating a result under 65% confidence as a winner, or expecting a confidence figure from a creative test.
  • Reading an A/B winner as proof of incremental sales. That needs a lift test.

Key takeaways

  • Use Meta's A/B test tool, not manual on/off tests: it splits the audience into random, non-overlapping groups.
  • Create tests from the Ads Manager toolbar, by duplicating an ad set or ad, or in Experiments with up to 5 campaigns.
  • Schedules run 1 to 30 days; Meta recommends at least 7 days, equal budgets and at least 80% estimated power.
  • The winner has the lowest cost per result; Meta treats 65% confidence or higher as a winning A/B result.
  • Creative tests compare 2 to 7 ad copies inside a Highest volume campaign and report no confidence level.
  • For incremental impact, use Conversion Lift or Brand Lift, which have their own requirements.

FAQ

How long should a Meta A/B test run?

Ads Manager lets you schedule an A/B test for 1 to 30 days. Meta recommends at least 7 days and says shorter tests may be inconclusive. If customers usually take more than 7 days to convert, run it longer; Meta gives 10 days as an example.

What budget does a Meta A/B test need?

Meta doesn't publish a minimum. It says the budget should produce enough results to confidently pick a winner and that each version should get the same budget. When you create a test by duplicating, Ads Manager shows an estimated power, and Meta recommends at least 80%.

What confidence level does Meta use to declare a winner?

Meta says that for A/B tests, a confidence percentage of 65% or higher represents a winning result, and for lift tests 90% or higher represents a statistically reliable one. Confidence is the chance of similar results if the test were repeated.

Can I edit an A/B test while it's running?

While it's In review or Scheduled you can change anything. Once it's Running, you can't change the tested variable, though you can change the schedule or cancel, which Meta says affects reliability. Cancelling doesn't stop delivery; pause the campaigns to stop spend.

What's the difference between an A/B test and a creative test?

An A/B test splits your audience into separate groups and reports a winner with a confidence percentage. A creative test compares 2 to 7 copies of an ad inside an existing Highest volume campaign, using a share of its budget, and identifies top performers without a confidence level.

When should I use Conversion Lift instead of an A/B test?

Use an A/B test to choose between strategies on cost per result. Use Conversion Lift to measure how many conversions your ads actually caused. Self-serve Conversion Lift has spend, conversion volume and signal quality requirements, such as a Conversions API event with an Event Match Quality score above 5.

Need agency ad accounts?Unlimited Meta agency accounts on our corporate cards, shared to your Business Manager through partner access. 4% per top-up, no setup fee, unused balance refunded within 24 hours (business days), free replacements, 24/7 on Telegram and WhatsApp.
@power_ads_hqGet the next one in Telegram.Meta policy shifts, spend-limit fixes and scaling notes from the Power Ads team. Join 3,000+ subscribers. Free, read-only, one tap to leave.Join channel

Sources

Power Ads background
By application only

We don't talk. We solve.

Unlimited Meta agency ad accounts on our corporate cards for teams spending roughly $100,000+ a month. 4% per top-up, no setup fee, 24/7 on Telegram and WhatsApp.

Telegram support · 24/7
TelegramWhatsAppApply