> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dema.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Terminology

> Key terms and concepts used in Dema's incrementality testing documentation.

<Info>
  This guide explains the key terminology used throughout Dema's incrementality
  testing documentation to help ensure clarity and consistency.
</Info>

## Test Structure Terms

### Treatment and Control Groups

**Treatment Group**
Regions where marketing spend is modified for the experiment. There are three types of treatment:

* **Holdout treatment**: Regions where ads are paused or reduced
* **New channel treatment**: Regions where new channel ads are added
* **Scaling treatment**: Regions where marketing spend is increased or decreased

**Control Group**
Regions where marketing spend continues as usual (business as usual). Used as the baseline for comparing treatment effects.

### Geographic Terms

**Commute Zones**
Clusters of zip codes that represent real-world consumer movement patterns, accounting for how people shop across city boundaries.

**Regions**
Geographic areas (typically groups of zip codes) used to segment test and control groups.

## Statistical Terms

### Confidence and Precision

**Confidence Interval**
The range of values within which the true incremental impact is likely to fall, indicating the precision of results.

**MDE (Minimum Detectable Effect)**
The smallest change in performance that can be reliably detected by the test given its design.

### Experiment Metrics

**Treatment Group Proportion**
The percentage of regions assigned to the treatment group (e.g., 13%). A lower proportion means fewer regions are affected by the test intervention.

**Estimated Lift**
The predicted percentage change in performance (e.g., -9.96% indicates an expected decrease in performance when ads are paused in a lift test).

**Minimum Detectable Effect (MDE)**
The smallest percentage change the test can reliably detect (e.g., -10% means the test can identify true effects of this size or larger). It describes the design's sensitivity — not the lift the model measured.

**Average Detected Lift** (shown in the app as **Minimum potentially detectable lift**)
The average lift the synthetic-control model actually recovered during back-testing at the smallest effect size that reaches the power threshold — a measure of the design's sensitivity (the smallest lift it could potentially detect), not what the test inferred. Distinct from the MDE: the MDE is the smallest effect the design *can* detect, while average detected lift is what the model *measured* when a known effect of that size was injected.

**Power**
The probability that the test detects a real effect of a given size when one is present. Higher power means the design is more likely to catch a genuine lift.

**Treatment Group Correlation**
How well the treatment and control groups match historically (e.g., 94.61%). Higher correlation (>90%) indicates well-matched groups for reliable comparison.

**P-value**
The probability of observing the measured difference if there were truly no effect (the null hypothesis). A low p-value means a gap this large would rarely occur by chance.

### Model Quality Diagnostics

These metrics distinguish a genuinely reliable market split from one that fit the past too closely (overfitting). See [Analyze a suggested experiment](/guides/incrementality-testing/analyze-suggestion#model-quality-diagnostics) for how to read them together.

**False Positive Rate**
How often the model detected a lift when none was applied, across historical A/A back-tests. Good: at or below the experiment's significance level (alpha).

**Lift Estimation Error**
How far the model's detected lift was from a known lift injected during back-testing. Good: close to 0%.

**Out-of-sample Prediction Error**
On periods the model didn't train on, how far its prediction was from actual sales, as a share of average sales. Cannot be inflated by overfitting. Good: low (e.g. under \~10%).

**Generalization Score**
In-sample error divided by out-of-sample error, scored from 0 to 1 (higher is better). Good: close to 1 (the held-out fit is about as good as the pre-period fit — it generalizes); near 0 signals overfitting.

**Out-of-sample Stability**
How consistent the out-of-sample error stays as the back-testing window slides through history, scored from 0 to 1. Higher is better: above 0.5 stable, 0.3–0.5 moderate, under 0.3 fragile.

**Effective Control Markets**
The effective number of control regions the synthetic control actually leans on (`~N of M` available). A low number means the comparison rests on few markets and is more fragile.

**Back-testing / Lookback Windows**
How many times the proposed match was re-tested on earlier slices of history; each window shifts back by one period. Typical: 3.

### Analysis Methods

**Synthetic Control**
A weighted blend of control regions created to best match the treatment region's historical performance patterns.

**Incremental Factor**
The ratio of incremental value to spend, representing either incremental ROAS or epROAS depending on the target variable.

## Test Types

### Platform-Specific Terms

**User-level Treatment**
Used in platform studies (like Meta's Conversion Lift), where individual users are randomly selected to not see ads.

**Regional Treatment**
Used in Dema's geo-testing, where entire geographic regions have their marketing spend modified.

### Time Periods

**Pre-treatment Period**
The baseline period before the test begins, used to verify group matching.

**Post-treatment Window**
The observation period after the test ends to capture delayed effects of marketing.

<Note>
  For any terms not covered here or for more detailed explanations, please reach
  out to your Dema specialist.
</Note>