# pymc-marketing-sweep

Fast prior screening for fitted PyMC-Marketing marketing mix models.

The package takes **one fitted PyMC-Marketing MMM**, normally saved as `model.nc`, and evaluates many alternative prior configurations without refitting the model for every candidate. It uses the existing posterior draws, exact prior-density ratios for supported priors, and Pareto-smoothed importance sampling (PSIS) to approximate the posterior under each proposed prior.

It is designed for questions such as:

- Is `saturation_beta = HalfNormal(sigma=0.50)` too loose?
- What happens if I tighten it to 0.40, 0.30 or 0.20?
- Can I change the adstock `Beta(alpha, beta)` prior without materially moving channel contributions?
- Which candidate is the smallest safe prior change?
- What would posterior contributions or future predictions approximately look like under that prior?
- What exact PyMC-Marketing `Prior(...)` should I use for the final real validation fit?

## Important distinction

This package **does not refit PyMC-Marketing for every candidate prior**.

It approximates the candidate posterior using the reference posterior:

```text
reference fitted posterior
        +
new prior / old prior
        +
PSIS smoothing
        =
approximate posterior under new prior
```

That is what makes the sweep fast.

A candidate that passes the sweep should still receive **one real PyMC-Marketing MCMC validation fit** before it is promoted to production.

## PyMC-Marketing 1.x target

This repository targets the stable **PyMC-Marketing 1.x** API and requires **PyMC-Marketing >=1.0.0,<2** on **Python >=3.12**. PyMC-Marketing 1.0.0 was released on 7 August 2026 and moved the supported stack to PyMC 6 and ArviZ 1.2+.

The implementation intentionally uses the 1.x public interfaces directly:

```python
from pymc_marketing.mmm import MMM
```

Saved models are loaded with `MMM.load(...)`. PyMC-Marketing 1.x stores fitted model artefacts as an `xarray.DataTree`, and the sweep reads the native `MMM.posterior` dataset accessor rather than relying on the older InferenceData compatibility surface.

The project does **not** maintain PyMC-Marketing 0.x compatibility branches. If a model was produced by an older release, migrate or validate it with PyMC-Marketing 1.x before using it as a production sweep reference.

PyMC-Marketing 1.0 also rejects legacy dict-style prior definitions in `model_config`. Production model configuration should use native `pymc_extras.prior.Prior` objects or supported PyMC-Marketing special priors such as `LogNormalPrior` and `LaplacePrior`. Serialised dictionaries may still appear internally when the sweep inspects `Prior.to_dict()` output, but they are not emitted as the final model configuration.

See `SOURCES.md` for the exact 1.0.0 API and dependency references used for this migration.

---

# 1. What input data is required?

## Required input 1: a fitted PyMC-Marketing model

Normally:

```text
inputs/model.nc
```

Create it from a fitted MMM using PyMC-Marketing itself:

```python
mmm.fit(X, y)
mmm.save("inputs/model.nc", engine="h5netcdf")
```

The package also accepts a PyMC-Marketing 1.x `.zarr` model path.

A normal PyMC-Marketing 1.x save writes an `xarray.DataTree` containing the information the sweep needs:

```text
posterior draws
model configuration
sampler configuration
model metadata
fit data
coordinates / dimensions
```

The most important groups for the sweep are:

```text
posterior
fit_data       # needed for full model reconstruction / future prediction
```

### What is inside `posterior`?

Typical PyMC-Marketing posterior variables include:

```text
intercept
adstock_alpha
saturation_lam
saturation_beta
gamma_control
gamma_fourier
channel_contribution
channel_contribution_original_scale
y_original_scale
...
```

The exact variables depend on the MMM configuration, saturation transformation, adstock transformation, panel dimensions and custom effects.

Do not guess them. Inspect the saved model first:

```bash
pmm-sweep inspect inputs/model.nc
```

## Required input 2: the sweep YAML

For example:

```text
configs/sweep.yaml
```

This tells the package which priors you want to test.

You **do not** need a different `.nc` file for every proposed prior.

One reference `.nc` is used to screen all candidates.

## Optional input: future predictor data

For example:

```text
inputs/future.csv
```

Use this only if you want approximate future posterior predictions from the recommended candidate.

It must contain the same predictor structure expected by the fitted PyMC-Marketing model:

- date column
- channel columns
- control columns
- any custom dimensions such as `geo`, `brand`, `customer` or `DMA`

Example:

```csv
date,search,social,price_index
2026-09-07,120000,45000,1.02
2026-09-14,125000,47000,1.01
```

## What is *not* required?

For a normal full PyMC-Marketing save, you do not need to separately provide:

```text
training.csv
sales.csv
media.csv
model_config.py
one NetCDF per candidate
```

The full `model.nc` should already contain the fit data and serialised model configuration.

---

# 2. Installation

Use Python 3.12 or newer. Create the repository and install it in an isolated environment:

```bash
python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -e ".[dev]"
```

On Windows PowerShell:

```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install --upgrade pip
pip install -e ".[dev]"
```

Run the tests:

```bash
pytest
```

---

# 3. First step for every model: inspect it

Before defining a sweep:

```bash
pmm-sweep inspect inputs/model.nc
```

The command prints:

1. installed PyMC-Marketing version
2. chain and draw counts
3. serialisable top-level model priors
4. nested prior paths such as `likelihood.sigma`
5. the likely posterior variable name for each prior path and whether it exists
6. every posterior variable in the fitted model

This matters because PyMC-Marketing model structures differ.

For example, one model might contain:

```text
saturation_beta
saturation_lam
adstock_alpha
```

while another saturation transformation can use different parameter names.

---

# 4. How to specify new priors

Candidate priors belong in YAML, not in the sweep engine.

Example:

```yaml
model:
  path: ../inputs/model.nc

sweep:
  mode: product
  max_candidates: 10000
  priors:

    saturation_beta:
      prior_path: saturation_beta
      posterior_variable: saturation_beta
      dist: HalfNormal
      parameters:
        sigma:
          values: [0.50, 0.40, 0.35, 0.30, 0.25]

constraints:
  max_pareto_k: auto
  min_reweight_ess: 200
  min_reweight_ess_fraction: 0.10
  max_relative_rmse: 0.03

outputs:
  directory: ../out
  comparison_variables:
    - channel_contribution_original_scale
  summary_variables:
    - channel_contribution_original_scale
    - y_original_scale
```

This means:

> Take the fitted model's `saturation_beta` prior as the baseline and test HalfNormal sigma values 0.50, 0.40, 0.35, 0.30 and 0.25.

## Preserve the fitted distribution

If you only want to alter prior parameters, omit `dist`:

```yaml
saturation_beta:
  parameters:
    sigma:
      values: [0.50, 0.40, 0.30]
```

The package keeps the original distribution and dimensions from the fitted MMM.

This is usually the safest workflow.

## Sweep multiple priors

Example:

```yaml
sweep:
  mode: product
  priors:

    saturation_beta:
      parameters:
        sigma:
          values: [0.50, 0.40, 0.30]

    adstock_alpha:
      parameters:
        alpha:
          values: [1.0, 2.0, 3.0]
        beta:
          values: [3.0, 4.0, 5.0]
```

The package evaluates the product of the grids. By default it refuses to expand beyond **10,000 candidates** so an accidental agent-generated Cartesian grid cannot consume unbounded time or memory. Set `sweep.max_candidates` explicitly if a larger grid is intentional.

In this example:

```text
3 saturation candidates
x
3 alpha candidates
x
3 beta candidates
=
27 candidate prior designs
```

## Channel-specific prior arrays

A candidate value can itself be a list.

For example, if the fitted prior has `dims="channel"` and there are three channels:

```yaml
saturation_beta:
  parameters:
    sigma:
      values:
        - [0.50, 0.40, 0.30]
        - [0.40, 0.30, 0.25]
        - [0.30, 0.25, 0.20]
```

Each inner list is one candidate vector.

The array length must match the corresponding posterior dimension.

---

# 5. Nested PyMC-Marketing priors

Some PyMC-Marketing priors are nested.

A typical likelihood can conceptually look like:

```python
Prior(
    "Normal",
    sigma=Prior("HalfNormal", sigma=1),
    dims="date",
)
```

The actual sampled child parameter is usually a different posterior variable from the observed likelihood itself.

For a nested prior, specify both the prior path and posterior variable explicitly:

```yaml
y_sigma:
  prior_path: likelihood.sigma
  posterior_variable: likelihood_sigma
  parameters:
    sigma:
      values: [1.0, 0.75, 0.50]
```

The sweep only changes the child prior being tested.

Use `pmm-sweep inspect` to confirm the posterior variable name first.

---

# 6. How the PSIS sweep works

Suppose the reference model was fitted with prior:

```text
p0(theta)
```

and produced posterior draws:

```text
theta_1, theta_2, ..., theta_S ~ p0(theta | y)
```

For a proposed prior:

```text
p1(theta)
```

the raw importance weight for each joint posterior draw is proportional to:

```text
p1(theta_s) / p0(theta_s)
```

The likelihood cancels because it is unchanged.

The package then applies **Pareto-smoothed importance sampling** to stabilise these ratios.

This gives an approximate posterior under the new prior without running NUTS again.

## Why joint weights matter

If you sweep both:

```text
saturation_beta
adstock_alpha
```

the package calculates one weight for the **joint posterior draw**, not separate independently normalised weights for each variable.

That preserves posterior dependence between parameters.

---

# 7. What does Pareto k mean?

`pareto_k` measures whether the fitted reference posterior has enough overlap with the proposed candidate posterior.

Lower is better.

A rough interpretation is:

```text
< 0.5       strong overlap
0.5 to 0.7  increasingly fragile
> 0.7       normally reject
```

The default `auto` threshold uses a sample-size-aware threshold capped at 0.7.

If a proposed prior is very different from the original fitted prior, the reference posterior may simply contain too few relevant draws.

That candidate should not be trusted just because its weighted posterior mean looks attractive.

The correct response is usually:

1. run a real PyMC-Marketing fit with an intermediate prior
2. use that model as the new reference
3. sweep again around the new region

---

# 8. What does reweighting ESS mean?

The package reports:

```text
reweight_ess
reweight_ess_fraction
```

These describe the effective number of reference posterior draws remaining after importance weighting.

For example:

```text
reference draws:       4,000
reweight_ess:          2,600
reweight_ess_fraction: 65%
```

means the candidate is well represented by the reference posterior.

A result like:

```text
reweight_ess:          75
reweight_ess_fraction: 1.9%
```

means the candidate is relying on very few effective draws and should normally be rejected.

### This is not NUTS ESS

`reweight_ess` is **importance-sampling ESS**.

It is not a prediction of the future NUTS bulk ESS that the candidate would obtain when refitted.

The final validation fit should still check:

```text
bulk ESS
tail ESS
R-hat
divergences
tree depth
BFMI
```

The package deliberately does not label importance ESS as sampler ESS. It therefore does not claim that a candidate will improve NUTS ESS before a real validation fit. If future sampler-performance prediction is required, calibrate it from actual historical refits rather than treating PSIS overlap as a substitute.

---

# 9. Posterior distortion guards

A candidate can have acceptable PSIS overlap but still move the MMM more than you want.

The package can compare candidate-weighted posterior means against the original fitted posterior for native PyMC-Marketing deterministics.

For example:

```yaml
outputs:
  comparison_variables:
    - channel_contribution_original_scale
```

For each candidate it calculates:

```text
RMSE
MAE
maximum absolute movement
relative RMSE
```

The default recommendation can enforce:

```yaml
constraints:
  max_relative_rmse: 0.03
```

meaning the change in the configured comparison variables must remain small relative to their baseline scale.

You can inspect other posterior variables if present, for example:

```yaml
comparison_variables:
  - channel_contribution_original_scale
  - saturation_beta
  - adstock_alpha
```

---

# 10. How a recommendation is chosen

A candidate is feasible only if it passes all configured checks:

```text
Pareto k
reweighting ESS
reweighting ESS fraction
posterior distortion
```

Among feasible candidates, the package recommends the **smallest changed prior** first.

If the fitted prior itself is included in the grid, it is retained as a zero-change benchmark but is never selected as the recommendation. This prevents the reference configuration from winning automatically just because its importance weights are uniform.

That means the package is not trying to find the most aggressive prior possible.

The decision rule is closer to:

> Among the prior changes the user explicitly asked to test, find the least invasive modification that is still reliably represented by the fitted posterior and stays within the permitted model-movement limits.

The candidate table is written to:

```text
out/sweep_results.csv
```

---

# 11. Run the sweep

```bash
pmm-sweep run configs/sweep.yaml
```

Typical outputs:

```text
out/
├── sweep_results.csv
├── recommended_scenario.yaml
├── recommended_model_config.py
├── recommended_importance_weights.npz
└── recommended_weighted_posterior_summary.nc
```

## `sweep_results.csv`

One row per candidate with values such as:

```text
scenario_id
pareto_k
pareto_k_threshold
reweight_ess
reweight_ess_fraction
mcmc_reff
prior_distance
max_relative_rmse
changed
feasible
recommended
```

plus distortion fields for each configured comparison variable.

## `recommended_scenario.yaml`

Machine-readable record of the recommended priors and metrics.

## `recommended_model_config.py`

Native PyMC-Extras `Prior(...)` or supported PyMC-Marketing special-prior objects that can be copied into the production PyMC-Marketing 1.x model configuration.

For example:

```python
from pymc_extras.prior import Prior

recommended_model_config_patch = {
    "saturation_beta": Prior(
        "HalfNormal",
        sigma=0.30,
        dims="channel",
    ),
}
```

## `recommended_weighted_posterior_summary.nc`

Weighted posterior means and configured quantiles for the variables listed under:

```yaml
outputs:
  summary_variables:
```

This is the approximate posterior under the recommended candidate prior.

---

# 12. Approximate future predictions under the new prior

Add a prediction section:

```yaml
prediction:
  data: ../inputs/future.csv
  draws: 1000
```

Then run the sweep normally:

```bash
pmm-sweep run configs/sweep.yaml
```

For the recommended candidate, the package:

1. takes the fitted joint posterior
2. applies the PSIS candidate weights
3. importance-resamples complete posterior draws
4. temporarily places those resampled draws into the loaded PyMC-Marketing model
5. calls PyMC-Marketing's own `MMM.predict_posterior`

Outputs:

```text
out/recommended_future_predictions.nc
out/recommended_future_prediction_summary.csv
```

This preserves the model's own PyMC-Marketing prediction machinery, including media transformations and model structure.

### Critical limitation

These are **approximate predictions under a reweighted posterior**.

They are useful for comparing candidate priors before spending compute on a refit.

They are not a replacement for fitting the recommended configuration and generating final production predictions from that validated fit.

---

# 13. How to apply the recommended prior to PyMC-Marketing

Suppose the sweep recommends:

```python
Prior("HalfNormal", sigma=0.30, dims="channel")
```

Apply it when creating the validation model:

```python
from pymc_extras.prior import Prior

model_config = {
    "saturation_beta": Prior(
        "HalfNormal",
        sigma=0.30,
        dims="channel",
    ),
}
```

Then build the same PyMC-Marketing model structure and perform one real fit:

```python
mmm = MMM(
    ...,
    model_config=model_config,
)

mmm.fit(X, y)
```

Finally validate sampling and model outputs.

Do not promote the recommendation solely because the sweep marked it feasible.

---

# 14. Using Codex, Claude or Cursor

The repository is intentionally agent-neutral and targets PyMC-Marketing 1.x only.

`AGENTS.md`, `CLAUDE.md` and `.agents/skills/pymc-marketing-sweep/SKILL.md` all point to the same workflow.

The coding agent should edit the YAML candidate grid, not the statistical engine.

## Example Codex instruction

```text
Read AGENTS.md and README.md first.

Inspect inputs/model.nc and confirm the current saturation_beta prior and its dimensions.

Create configs/saturation_sweep.yaml.
Keep the fitted saturation_beta distribution and dimensions unchanged.
Test sigma values 0.50, 0.40, 0.35, 0.30, 0.25 and 0.20.
Do not change adstock or controls.

Require the automatic Pareto-k threshold, at least 10% reweighting ESS,
and no more than 3% relative RMSE in channel_contribution_original_scale.

Run the sweep.
Tell me which candidate is recommended, why it passed, and the exact Prior(...) I should use for the final PyMC-Marketing validation fit.
Do not describe reweighting ESS as NUTS ESS.
```

## Follow-up instruction

```text
Now add adstock_alpha candidates with alpha [1, 2, 3] and beta [3, 4, 5].
Keep the saturation candidates unchanged.
Run the product sweep and compare the new recommendation with the previous one.
```

## Prediction instruction

```text
Use inputs/future.csv for an approximate posterior prediction under the recommended candidate.
Generate the prediction outputs, but clearly label them as importance-reweighted approximations.
Then give me the exact model_config prior patch required for the real validation refit.
```

More examples are in `examples/AGENT_PROMPTS.md`.

---

# 15. Supported prior distributions

The fast prior-ratio engine currently supports common scalar or element-wise independent PyMC priors:

```text
Normal
HalfNormal
LogNormal
Gamma
Beta
Laplace
Exponential
Uniform
StudentT
HalfStudentT
Cauchy
HalfCauchy
TruncatedNormal
```

It also understands the current PyMC-Marketing `LogNormalPrior` and `LaplacePrior` special-prior serialisations. A non-hierarchical `LogNormalPrior(mean=..., std=...)` is evaluated using the exact positive-scale mean/std parameterisation used by PyMC-Marketing.

`MaskedPrior` is intentionally not reweighted by the fast engine because the active-mask variable structure needs model-specific treatment.

It supports scalar and dimension-aligned array parameters.

## Hierarchical priors

Nested priors can be swept by explicitly targeting the child prior path and matching posterior variable, for example:

```text
prior_path: likelihood.sigma
posterior_variable: likelihood_sigma
```

Directly reweighting a top-level hierarchical prior whose parameters are themselves changing random variables is intentionally rejected unless the child prior is targeted explicitly. This applies to ordinary nested `Prior` objects and hierarchical `LogNormalPrior` / `LaplacePrior` parameters.

## Transformed priors

Custom `Prior(..., transform=...)` objects are rejected by the fast density engine unless support is explicitly added for that transform.

This avoids silently applying the wrong Jacobian.

---

# 16. Reference-model choice

Importance reweighting works best when the reference posterior has broad enough support to cover the candidate region.

A sensible workflow is:

```text
reasonably broad reference prior
            ↓
       one good MCMC fit
            ↓
       pmm-sweep grid
            ↓
   reject poor-overlap priors
            ↓
 choose smallest feasible change
            ↓
 one real validation MCMC fit
```

If the reference prior is already extremely tight, you cannot reliably use those posterior draws to predict a dramatically broader candidate posterior.

The PSIS guardrail is there to detect this problem.

---

# 17. Demo model

After installing the project in a Python 3.12+ environment with PyMC-Marketing 1.x, you can create a small example model:

```bash
python examples/build_demo_model.py
```

This creates:

```text
inputs/model.nc
inputs/future.csv
```

Then run:

```bash
pmm-sweep inspect inputs/model.nc
pmm-sweep run configs/sweep.example.yaml
```

To enable approximate future predictions, uncomment the `prediction:` section in `configs/sweep.example.yaml`.

---

# 18. PyMC-Marketing 1.x integration QA

The repository contains `tests/test_integration_pmm1.py`, a real integration test intended for a normal connected Python 3.12+ environment. It performs a small but genuine:

```text
MMM.fit
  -> MMM.save
  -> MMM.load through pmm-sweep
  -> baseline and changed-prior sweep
  -> recommendation generation
  -> importance-resampled MMM.predict_posterior
```

Run it explicitly with:

```bash
pytest -q -m integration
```

The ordinary unit and adversarial suite does not require PyMC-Marketing to be imported at collection time, so the statistical engine can be tested independently. CI should run both layers.

---

# 19. Production checklist

Before trusting a sweep:

- [ ] `model.nc` is the intended fitted PyMC-Marketing model.
- [ ] `pmm-sweep inspect` confirms the prior paths and posterior variable names.
- [ ] Candidate distributions and dimensions are appropriate for the existing MMM structure.
- [ ] Pareto-k thresholds are not relaxed merely to force a candidate through.
- [ ] Pathological candidates are recorded as infeasible rather than being allowed to abort the full sweep.
- [ ] Reweighting ESS is large enough for the intended comparison.
- [ ] Contribution or other configured posterior movement is within tolerance.
- [ ] The recommendation is the smallest feasible changed prior, not simply the tightest prior.
- [ ] Any unchanged fitted-prior row is treated only as a benchmark.
- [ ] Any future prediction is labelled as an importance-reweighted approximation.
- [ ] The recommended prior is run through one real PyMC-Marketing validation fit.
- [ ] Final NUTS diagnostics and business outputs are checked on that real fit.

---

# 20. Repository structure

```text
pymc-marketing-sweep/
├── README.md
├── SOURCES.md
├── AGENTS.md
├── CLAUDE.md
├── pyproject.toml
├── requirements.txt
│
├── src/
│   └── pymc_marketing_sweep/
│       ├── runtime.py
│       ├── config.py
│       ├── loader.py
│       ├── prior.py
│       ├── reweight.py
│       ├── summaries.py
│       ├── sweep.py
│       ├── predict.py
│       ├── recommend.py
│       └── cli.py
│
├── configs/
│   └── sweep.example.yaml
│
├── examples/
│   ├── build_demo_model.py
│   └── AGENT_PROMPTS.md
│
├── inputs/
│   └── README.md
│
└── tests/
    └── test_extreme.py
```

# 21. Adversarial and extreme-case testing

The test suite includes deliberately pathological cases, not only normal candidate grids. The current local suite contains **34 passing unit/stress tests plus one real PyMC-Marketing 1.x integration test**. The integration test is skipped only when PyMC-Marketing is unavailable in the execution environment. Current stress tests cover:

- an unchanged prior, which must produce exactly uniform importance weights;
- a **10x wider HalfNormal prior**, which must show poor overlap / low reweighting ESS;
- an **extremely tight HalfNormal prior** (`sigma=0.003` versus a `0.5` reference), which must collapse effective sample size and produce a severe PSIS diagnostic;
- a near-degenerate prior (`sigma=1e-6`), where a non-finite Pareto tail fit is converted to `inf` and rejected rather than leaking `NaN` into rankings;
- a candidate whose support contains **none** of the reference posterior draws, which must fail with a clear zero-density error;
- an invalid scale (`sigma=0`) inside a larger sweep, which must mark only that candidate invalid while the other candidates continue;
- nearly degenerate importance weights concentrated on one joint posterior draw, checking that joint-variable relationships remain intact during resampling;
- non-finite posterior values, which must be rejected;
- oversized Cartesian grids, which are stopped by `sweep.max_candidates`;
- missing comparison variables when a distortion guardrail is requested, which must fail rather than silently disabling the constraint.

Run the complete suite with:

```bash
pytest
```

A passing stress suite does not remove the final requirement to validate the recommended prior with a real PyMC-Marketing MCMC fit. Its purpose is to make the fast screening layer fail safely under extreme proposals.

The engine contains no client-specific files, fitted traces, user-local filesystem paths, credentials or production data.
