1. How and Why We Prioritize Hypotheses

To maximize the expected improvement of key business metrics, we have to prioritize the hypotheses we formulate.

The Eisenhower Matrix

The Eisenhower Matrix is a way of prioritizing where the problem is assigned two qualities: importance and urgency. Although the matrix was originally designed for prioritizing tasks, it works for hypotheses, too.

Each task enters one of the four quadrants: important and urgent, not important and urgent, important and not urgent, not important and not urgent. Hypotheses from the A quadrant (important and urgent) must be tested first. Then come hypotheses from the B quadrant (important and not urgent).

For hypotheses, but not for tasks, those that are not important (the C and В quadrants) aren't tested at all. Urgent but unimportant hypotheses (the C quadrant) probably have short-term results that won't impact long-term business goals.

WSJF

Weighted shortest job first (WSJF) is a method of prioritizing that allows you to evaluate tasks in more detail.

Formula:

The denominator contains Job Duration, or how long hypothesis testing is expected to take. Cost of Delay in the numerator is the sum of several parameters:

User-Business Value — how much we'll boost a user or business metric if the hypothesis is true

Time Criticality — how urgently the hypothesis must be tested