How every number here is produced
Elasticsearch carries the arm assignment, so it supplies every
ratio.
MySQL carries the billed ledger, so it supplies every
dollar. They meet once, at the end.
- Denominator — requests that came in.
SUM(sample_weight) over both log
streams for the arm, across the whole client. Endpoint rows use the same client-wide arm
total: win_dsp_id exists only on winning documents, so an endpoint's own
weight is a win count, and a win is itself an AdOpt outcome.
- Numerator — money actually billed. Revenue
SUM(cost_w) and cost
SUM(rev_w), both gated on is_impression = 1. (The field names are
swapped: cost_w is the DSP bid we are paid, rev_w is our bid back
to the SSP.)
- Uplift = treatment rate ÷ control rate − 1, computed separately for revenue per
request and profit per request.
- Extra $ = booked $ ×
f·u/(1+f·u), where f is the measured
treatment share of requests. Only that share of traffic ever carried the uplift, so the
book we observe is X·(1+f·u) against an all-control counterfactual X.
- Band is an indicative 95% interval from the sampled document count
(has_bid is a 10% sample), scaled for per-impression price dispersion, and shown on
both revenue and profit. An uplift smaller than its own band is greyed and marked
within noise — its sign carries no information. Rows with a thin control arm are
dimmed; under 30 control documents nothing is reported at all. This is a proxy, not a
significance test: an hourly cluster jackknife is the real instrument.