Process documentation for the recurring "which price wins" decision on Meta Ads. Companion to the
meta-price-test-analysiscode repo — this repo explains how and why, the code repo implements the current best version of it.
The methodology is locked: every Meta price-test analysis runs the same three tests in one image.
| # | Test | What it measures | Method |
|---|---|---|---|
| 1 | Profitability ROAS | Return per $1 spent | Point = Σrevenue / Σspend. CI = day-level resample (n=3,500 or 10,000 per request), recompute ratio each iter, percentile(2.5, 97.5). |
| 2 | SARIMAX 60-day forecast | Trajectory of daily purchases | Auto-select order via pmdarima.auto_arima. Forecast 60 days forward, plot with 95% CI. |
| 3 | Purchase Rate (CVR) | Purchases / clicks | Point = Σpurchases / Σclicks. CI = day-level resample, recompute ratio, percentile(2.5, 97.5). |
The synthesis rule (when to call a winner): 2 of 3 must hold:
- Bayesian
P(treat > ctrl) > 0.95 - MC
P(treat > ctrl) > 0.70 |Cohen's d| > 0.2
See meta-price-test-analysis repo for the full Bayesian / MC pipeline
(this repo documents the methodology; the code repo ships the working
implementation).
meta-price-campaign-analysis/
├── README.md # this file
├── SKILL.md # trigger conditions, three-test rule, pitfalls
├── LICENSE # MIT
├── .gitignore
├── requirements.txt
├── references/
│ ├── bootstrap_on_ratios.md # why you can't naive-bootstrap ROAS or CVR
│ ├── meta_export_layout.md # the column-index trap that bites every time
│ └── sarimax_model_selection.md # the decision tree for short price-campaign series
└── templates/
└── analyze_template.py # the reusable 3-test template
-
meta_export_layout.md— the single most important document in this repo. Meta's export xlsx has two header rows and the integer column index is offset from the column name position. Get this wrong and every number in your report is wrong, in a way that passes your own eyeball checks. -
bootstrap_on_ratios.md— why you cannot naive-bootstrap a ratio metric (ROAS, CVR). The short version: bootstrap-of-the-mean-of-the-ratio is biased; you must resample days and recompute the ratio each iter. -
sarimax_model_selection.md— training series lengths vary from 20 days (Denton) to 121 days (Waukesha). Defaultauto_arimawill happily fit SARIMA on 20 days and give garbage. The decision tree in this document is the one that actually works.
templates/analyze_template.py is a
drop-in 3-test script you can adapt to any new Meta price-test
data. Set FILES at the top to point at your xlsx exports, set
N_BOOT per the user's request, run, get a 3-panel horizontal viz per
city.
The user almost always asks for the same three tests in one image;
multi-city runs (3-4 cities in one session) are now standard. Trigger
conditions are spelled out in SKILL.md.
- Code implementation:
Pouyasharp/meta-price-test-analysis— the full pipeline (3 stages, 5 cities, 10 viz) - Time-series theory:
Pouyasharp/arimax-sarimax-tutorial— PhD-level deep-dive on the SARIMAX half of the pipeline - Weather-sensitivity companion:
Pouyasharp/poseidon-weather
MIT. See LICENSE.