---
title: What is Holdout group?
url: https://markin.ai/glossary/holdout-group
category: Experimentation and causality
aliases: Control group
---

# Holdout group

> A holdout is a randomly selected set of customers deliberately excluded from an action, kept as the counterfactual. A global holdout, excluded from all activity, measures the total incremental contribution of a growth programme rather than of any single campaign.

## Why it matters for ARPU

Without a holdout, a programme reports the revenue it touched, not the revenue it created. Every Markin experiment carries one for exactly this reason.

## Related terms

- [Incrementality](https://markin.ai/glossary/incrementality), Incrementality is the portion of an outcome that would not have happened without the action.
- [A/B test](https://markin.ai/glossary/ab-test), An A/B test randomly assigns customers to two or more variants and compares a pre-declared metric between them.
- [Experiment](https://markin.ai/glossary/experiment), An experiment is a controlled release of a candidate action against a randomised holdout, sized in advance so the result can distinguish a real effect from noise.
- [Uplift model](https://markin.ai/glossary/uplift-model), An uplift model estimates the change in outcome caused by treating a customer, rather than the outcome itself.

## Go deeper

- [Markin and your data science team](https://markin.ai/compare/markin-and-your-data-science-team), Evidence standards compared.
- [Growth optimization](https://markin.ai/solutions/growth-optimization), Holdouts as default, not exception.

Source: https://markin.ai/glossary/holdout-group