Explaining the Model Is Easy Until Stakeholders Ask “So What?”

Read time: 2.5 minutes

Explaining Model Production Survival

1. Focus decision over model

Only stakeholders are interested in model decisions, not in how it was created.
Instead, say: “This model changes this decision.”

2. Replace AUC by tradeoff

AUC does not provide a decision.
Instead, give an example of tradeoffs in layman's terms (false positive vs. false negative).

3. Use one business outcome metric

Model accuracy will be for the stakeholders not for their customers.
Instead, use one business outcome: cost savings, increased conversion, fraud prevention and time resolution.

4. Required model appears optional

Optional must have no impact.
Instead, include model output to be part of the implementation process and track how often overridden.

5. Explain in less than 30 minutes to build trust

People don't spend long building trust.
Instead, pass 10 minute test: what is the problem, what is the decision, what is the impact, risk.

💡Key Takeaway: 

A model can only be classified as "effective" once your Data Science buddies have given it their stamp of approval and it will be deemed an "effective" model if it helps stakeholders determine what actions, if taken, would lead to a change in the way they perform their job, even if you're not there.