davids_admin

If Wells Fargo had set out to write a cautionary tale, a case study of data-driven dysfunction, they could not have succeeded better. The first article posted here showed how creating strong incentives to hit numerical targets drove people over the edge.  Entirely predictable, and straight out of Austin’s Measuring and Managing Performance in Organizations. … Continue reading Lies, damn lies, and data science

Wells Fargo paid $185M in fines and 5,300 people paid with their jobs to bring you this perfect example of metrics dysfunction.   The question that does not seem to have been answered is whether or not the people who set up the system — the people who put 5,300 line workers in the impossible… Continue reading Stagecoach robbery

Careful, thoughtful, painstaking use of metrics; clear aspirational goals; intrinsically motivated people with no desire to game the system:  all together, a big improvement.   “For five years Charlie took it upon himself to create a new workflow system for the tracing center, breaking down each step in the tracing process into equations, doing time-motion… Continue reading Ready, aim, fire! A metrics bulls-eye!

A large and possibly calamitous financial hole for the aged and infirm at the intersection of two metrics tied tightly to incentives.    Flexibility and discretion in the application of rules invites abuse and corruption.  Inflexible application produces unintended consequences and invites efforts to game the rules.   http://www.nytimes.com/2016/08/07/us/politics/new-medicare-law-to-notify-patients-of-loophole-in-nursing-home-coverage.html?smid=go-share

“The main measure of economic activity, GDP, counts housework when it is paid, but excludes it when it is done free of charge. This is an arbitrary distinction, and leads to perverse outcomes. … The usual defence is that measuring unpaid work is hard.”   The perverse outcomes are in many ways visited upon women,… Continue reading It’s not work if you don’t get paid for it

“`The p-value was never intended to be a substitute for scientific reasoning,’ the ASA’s executive director, Ron Wasserstein, said in a press release.”   Note that p-values are metrics, measures of a particular probability under certain assumptions.  As usual, the Goal – Question/Signal – Metric framework is appropriate:  the goal would be to determine, say,… Continue reading What’s the value of this value?

A colleague  mentioned the hazard of biasing a research investigation towards the data that are readily available over the data that are most desirable.   That seemed a good cue to post a link to this old chestnut.   http://quoteinvestigator.com/2013/04/11/better-light/

“How deeply a problem is understood and how concretely it is defined sets an upper bound for the quality of any subsequent solutions.”   https://medium.com/intercom-inside/growth-hacking-is-bullshit-60aae95f9caa

True progress is hard; it is always easier to move the goalposts than to score more goals.  Human beings are very good at rationalization and self-deception, so adjustments made in ambiguous or complicated situations for varied motivations are not always obviously directly aimed at shifting the reference in order to inflate the metric.   On… Continue reading "But we don’t WANT to teach ’em, we want to LEARN ’em…"

“It’s easier to measure if that change led to increased engagement than to measure if it also made your users hate you” –Benedict Evans   https://twitter.com/BenedictEvans/status/673633094113484800