---
title: What is Experiment?
url: https://markin.ai/glossary/experiment
category: Markin vocabulary
---

# Experiment

> An experiment is a controlled release of a candidate action against a randomised holdout, sized in advance so the result can distinguish a real effect from noise. It has a pre-registered metric, a stopping rule and guardrails, and it produces an estimate of incremental revenue rather than a rate.

## Why it matters for ARPU

Reported uplift that was never held out is usually seasonality or selection. Experiments are the only mechanism that converts activity into defensible ARPU growth.

## Related terms

- [Holdout group](https://markin.ai/glossary/holdout-group), A holdout is a randomly selected set of customers deliberately excluded from an action, kept as the counterfactual.
- [Incrementality](https://markin.ai/glossary/incrementality), Incrementality is the portion of an outcome that would not have happened without the action.
- [Statistical power](https://markin.ai/glossary/statistical-power), Statistical power is the probability that a test detects an effect of a given size when that effect is real.
- [Guardrail metric](https://markin.ai/glossary/guardrail-metric), A guardrail metric is a measure an experiment must not damage even if the primary metric improves: unsubscribe rate, complaint volume, margin, support contacts, app uninstalls.

## Go deeper

- [Growth optimization](https://markin.ai/solutions/growth-optimization), Continuous experimentation at decision volume.
- [Markin and your data science team](https://markin.ai/compare/markin-and-your-data-science-team), Experiment throughput compared.

Source: https://markin.ai/glossary/experiment