Glossary of Experimentation and CRO

A method that involves comparing two versions of a page or element to determine which performs best.

A statistical approach that updates the probability of a variant being a winner as data arrives, rather than waiting for a fixed threshold.

A feature that automatically adjusts traffic distribution towards the best performing variant being tested.

An algorithm that progressively allocates more traffic to the best-performing variants being tested, instead of an equal share between all variants.

An error that occurs when the sample of visitors exposed to a test is not representative of the entire audience, thus skewing the results.

A visual representation that shows where visitors click, hover over, or scroll on a page, useful for identifying areas to optimize.

A method that pits the current version of a page (the champion) against one or more new versions (the challengers) to see which performs best.

Segmentation of visitors based on their actions or their journey on the site.

All practices aimed at improving the conversion rate of a website or application.

An interface that allows you to modify the appearance of a page (text, colors, elements) without writing code, in order to create test variants.

Brief display of the original content before its replacement by the tested variant, which may negatively impact the user experience.

The following steps a visitor goes through before making a conversion (e.g., product page, then cart, then payment), useful for identifying where users abandon their purchases.

To wrongly conclude that a variant is winning when the observed difference is due to chance.

Failing to detect a real difference between variants, often due to an insufficient sample size.

A technical mechanism that allows a feature to be enabled or disabled for certain users without redeploying the code, often used to roll out tests gradually.

A testable statement, formulated prior to an experiment, that describes the expected change and its presumed effect on visitor behavior.

A metric chosen to measure the success of a test or experiment, such as conversion rate or average order value.

Range of values ​​within which the true result of a test is estimated to lie, with a certain level of certainty.

Simultaneous testing of several combinations of variables on the same page, to identify the most efficient combination.

Specific action expected from a visitor (purchase, registration, click...), used to measure the performance of a variant.

Page that a visitor arrives on after clicking on a link, often the focus of optimization tests.

Adapting the content displayed to a visitor based on their profile or behavior.

The probability that a test will detect a real difference between variants if it exists; it depends in particular on the sample size.

Random distribution of visitors between the different variants of a test, essential to guarantee reliable results.

Dividing visitors into groups according to common criteria (behavior, source, device...) for targeted analyses or tests.

Continuous activity period of a visitor on a site, often used as a unit of measurement in test analysis.

An indicator to determine whether the observed difference between two variants is reliable and not due to chance.

A type of test that compares two distinct URLs rather than two versions of the same page.

Number of visitors or sessions required for a test to reach reliable statistical significance.

Percentage of visitors who click on an element (button, link, banner) compared to the total number of people who saw it.

Proportion of visitors completing a defined objective action, compared to the total number of visitors.

Percentage of visitors who leave a site after viewing only one page, without further interaction.

A test that compares two identical versions of a page, used to verify that the testing tool and measurement method are working correctly.

In-depth analysis of results after the end of a test, for example by audience segment, to draw further lessons.

Probability of observing a result at least as extreme as the one measured if no real difference existed between the variants; a low value strengthens confidence in the result.

An alternative version of a page or element, tested against the original (the control variant) to measure its impact.