Metrics and analytics

Subscribers watched, minutes per subscriber, playback error rate: each number your team reports needs one definition everybody uses. A metric definition records the name, the grain, the numerator and denominator expressions, the unit, the aggregation method and the version. The aggregation method is one of Sum, count distinct, ratio of sums, last as of or Median. A definition starts as Draft, becomes Approved once the Data Steward has reviewed it, and is Retired when a new version replaces it.

Observations hold the measured values per source period, with numerator, denominator, unit and value, tied to the snapshot they came from. Results are calculated per period and dimension from approved definitions, with the number of source rows used. A result is Calculated, Suppressed when there is too little data to show, or Invalid with a reason.

• Draft, then approve. The Metrics Analyst drafts the definition and the Data Steward approves it. The reviewer is named on the definition.
• Version, never overwrite. A change to an approved metric is a new version. The old version is Retired, and past results keep their link to the version used.
• Results with their source. Each result names its metric, period, dimension, source row count and the run that produced it.

Agree the first set of metrics and write their numerators and denominators in plain terms before turning them into expressions. Agree when to use a ratio of sums instead of averaging ratios. Agree the threshold below which a result is suppressed. Decide who may approve definitions and how a new version is announced to report readers.

The team wants “minutes watched per active subscriber” for February. The Metrics Analyst drafts it with accepted watched minutes as numerator, distinct active subscribers as denominator, monthly grain and ratio of sums. The Data Steward checks the expressions against the snapshot of February sessions and approves version 1.

The results for February calculate by plan. The Premium result for one small region has fourteen subscribers, below the agreed threshold, and is Suppressed. A result for another region is Invalid because its snapshot was Stale. The analyst opens a follow-up action, the steward recaptures the snapshot and the result recalculates.

• Metric Results by Period — Calculated and suppressed results by metric, dimension and period, with source row count.
• Invalid Metric Results — Results marked Invalid with the reason and the run that produced them.
• Approved Metric Definitions — The definitions in use, with expressions, aggregation method, version and reviewer.

• Metric definition to results: Approve a definition and calculate its results from fresh sources.

Review the data model before defining your first metrics.