True progress is hard; it is always easier to move the goalposts than to score more goals. Human beings are very good at rationalization and self-deception, so adjustments made in ambiguous or complicated situations for varied motivations are not always obviously directly aimed at shifting the reference in order to inflate the metric. On… Continue reading "But we don’t WANT to teach ’em, we want to LEARN ’em…"
December 2015
There’s no such thing as bad publicity
“It’s easier to measure if that change led to increased engagement than to measure if it also made your users hate you” –Benedict Evans https://twitter.com/BenedictEvans/status/673633094113484800
Turtles, questions, and fleas
Metrics are used to answer questions, and questions are like fleas: So, naturalists observe, a flea Has smaller fleas that on him prey; And these have smaller still to bite ’em, And so proceed ad infinitum. (Jonathan Swift, On Poetry: A Rhapsody, 1733) So a change in a single metric should be… Continue reading Turtles, questions, and fleas
Two’s company, three’s a crowd
Counting things is generally much easier than defining sharply the boundary between what is to be counted and what is not. Focus often lands on that problem late, when a decision is to be made based on (or at least informed by) the metric and a shift in the definition could change the outcome. … Continue reading Two’s company, three’s a crowd