Creative: are retests worth the cost?

Business result

Additional profitable winners per unit of retest spend

Next step

Design a controlled test

Working output

SQL / notebook

Unit of analysis

Creative ID × launch attempt × branch × setup

Decision owner

To agree

Controlled test design

• Hypothesis: State the change and the expected business outcome.
• Comparison: Define treatment, control and the unit of assignment.
• Specify assignment, group overlap, exposure and contamination checks.
• Choose a primary metric, detectable effect and completed outcome window before calculating sample size.
• Set risk budget, guardrails, checkpoint and stop or restore conditions.

Acceptance checks

• Each promotion is linked to an identified attempt.
• Cohorts have the same completed horizon; later value is not duplicated across ad sets.
• Decision owner, risk budget, guardrails, checkpoint and restore condition are specified.
• Assignment, detectable effect and outcome horizon are specified before the test starts.

Deliverables

• A pilot design with success and stop criteria
• Read-only queries or notebook, parameters, results and a reproducible analysis dataset.
• A dated finding with source period, cutoff, methods, limitations and next action.