Communicating Uncertainty to Executives

73.1 What executives do with a range (40 min)

Concept. A decision-maker reads a forecast to answer three questions: which way does it go? how much? can it hurt us? A confidence interval answers none of them directly. Translate:

Words alone are dangerous: people read "likely" as anything from 55% to 90%. Numbers alone get skimmed. Give both, using a fixed vocabulary (here the IPCC's calibrated bands), so "likely" always means 66–90%.

Precision is a message too. "$458,929" claims accuracy to the dollar from estimates with 5% standard errors. Round.

73.2 Numbers into sentences (50 min)

Create engine/narrate.py:

"""Turn a simulation result into sentences an executive can read (Day 73)."""


def money(x: float) -> str:
    """$1.2M, $340k, -$5.1k: two or three significant figures is all the precision a forecast deserves."""
    if abs(x) < 0.5:
        return "$0"
    sign = "-" if x < 0 else ""
    x = abs(x)
    if x >= 1e6:
        return f"{sign}${x / 1e6:.1f}M" if x < 1e7 else f"{sign}${x / 1e6:.0f}M"
    if x >= 1e3:
        return f"{sign}${x / 1e3:.1f}k" if x < 1e4 else f"{sign}${x / 1e3:.0f}k"
    return f"{sign}${x:.0f}"


def likelihood(p: float) -> str:
    """Words for probabilities, with the number alongside (words alone are read very differently by
    different people; numbers alone are skimmed). Bands follow the IPCC's calibrated language."""
    for cut, word in [(0.99, "virtually certain"), (0.9, "very likely"), (0.66, "likely"), (0.33, "about as likely as not"),
                      (0.1, "unlikely"), (0.01, "very unlikely")]:
        if p >= cut:
            return word
    return "exceptionally unlikely"


def summary(name: str, r: dict) -> str:
    g = r["change"]["gross_profit"]
    p = r["change"]["prob_profit_up"]
    lines = [f"{name}: gross profit changes by about {money(g['median'])} a year "
             f"(in 9 of 10 plausible outcomes, between {money(g['p5'])} and {money(g['p95'])})."]
    if p is not None:
        lines.append(f"Profit going up is {likelihood(p)} ({p:.0%}).")
    if r["change"]["prob_loss_over_1m"] > 0.01:
        lines.append(f"There is a {r['change']['prob_loss_over_1m']:.0%} chance of losing more than $1M.")
    if r["warnings"]:
        lines.append(f"Caution: {len(r['warnings'])} of the price changes go beyond what the data have seen.")
    return " ".join(lines)

Create day73/exec_summary.py:

# day73/exec_summary.py   (python -m day73.exec_summary)
"""
The same results, written for a CFO: rounded to two significant figures, ranges as 'in 9 of 10 plausible
outcomes', probabilities in words AND numbers, and a warning when the scenario leaves the data.
"""
from engine import Scenario, load_parameters, simulate
from engine.narrate import summary
from engine.presets import PRESETS

params = load_parameters()
for name in ["5% everywhere", "5% in the 'raise' cells", "coupons to the top 50%", "coupons to everyone"]:
    print(summary(name, simulate(params, PRESETS[name])))
    print()
print(summary("10% in every student cell", simulate(params, Scenario(price_change={"student": 0.10}))))

python -m day73.exec_summary

5% everywhere: gross profit changes by about $459k a year (in 9 of 10 plausible outcomes, between $419k and $502k). Profit going up is virtually certain (100%).

coupons to the top 50%: gross profit changes by about $5.6M a year (in 9 of 10 plausible outcomes, between $4.8M and $6.4M). Profit going up is virtually certain (100%).

coupons to everyone: gross profit changes by about -$2.8M a year (in 9 of 10 plausible outcomes, between -$3.9M and -$1.8M). Profit going up is exceptionally unlikely (0%). There is a 100% chance of losing more than $1M.

10% in every student cell: gross profit changes by about -$143k a year (in 9 of 10 plausible outcomes, between -$178k and -$107k). Profit going up is exceptionally unlikely (0%). Caution: 2 of the price changes go beyond what the data have seen.

Every sentence is generated from the same result the charts use. Nobody types a number into a slide, so the memo and the app can't disagree.

Try it. Write the summary for coupons to the top 30% and the top 80%. Which would you recommend, and why is 80% still profitable when 100% isn't? Create day73/try_summary.py.

# day73/try_summary.py   (python -m day73.try_summary)
from engine import Scenario, load_parameters, simulate
from engine.narrate import summary

params = load_parameters()
for share in [0.3, 0.8]:
    print(summary(f"Coupons to the top {share:.0%}", simulate(params, Scenario(coupon_share=share))), "\n")

Coupons to the top 30%: gross profit changes by about $5.7M a year (in 9 of 10 plausible outcomes, between $5.0M and $6.3M). Profit going up is virtually certain (100%).

Coupons to the top 80%: gross profit changes by about $2.4M a year (in 9 of 10 plausible outcomes, between $1.4M and $3.3M). Profit going up is virtually certain (100%).

73.3 Show the distribution (45 min)

A sentence gives three numbers. A picture shows the shape: whether the range is symmetric, whether a scenario straddles zero. Create day73/fan_chart.py:

# day73/fan_chart.py   (python -m day73.fan_chart)
"""
Show the whole distribution, not just a number: for each scenario, the spread of plausible changes in
annual gross profit, with zero marked. The eye sees at once which options can lose money.
"""
import matplotlib.pyplot as plt
import numpy as np

from engine import load_parameters, simulate
from engine.presets import PRESETS

params = load_parameters()
names = ["5% everywhere", "5% in the 'raise' cells", "coupons to the top 50%", "coupons to everyone"]
fig, ax = plt.subplots(figsize=(8, 3.8))
for i, name in enumerate(names):
    r = simulate(params, PRESETS[name].model_copy(update={"draws": 5000}))
    x = np.array(r["draws"]["gross_profit_change"]) / 1e6
    parts = ax.violinplot(x, positions=[i], vert=False, showextrema=False, widths=0.8)
    for b in parts["bodies"]:
        b.set_facecolor("#16a34a" if np.median(x) > 0 else "#dc2626")
        b.set_alpha(0.5)
    ax.plot(np.median(x), i, "o", color="#1e293b")
    ax.text(np.percentile(x, 95) + 0.2, i, f"P(up) {np.mean(x > 0):.0%}", va="center", fontsize=8)
ax.axvline(0, color="grey", lw=0.8)
ax.set_yticks(range(len(names)), names)
ax.set_xlabel("change in annual gross profit ($M), vs doing nothing")
fig.tight_layout()
fig.savefig("figures/day73_distributions.png", dpi=130)
print("Saved figures/day73_distributions.png")

python -m day73.fan_chart

Saved figures/day73_distributions.png

Each violin is one scenario's 2,000 simulated profit changes; the red line is zero. Any violin that crosses the line can lose money, and you see how much of it does. Put this chart next to the sentences, not instead of them.

73.4 Which uncertainty is in the range, and which isn't (40 min)

The range covers estimation uncertainty: the elasticities' standard errors and the coupon curve's bootstrap. It doesn't cover:

• Assumption uncertainty: constant elasticity outside the data, the 15% coupon pull-forward, no interaction between price and coupons. Handle these with sliders (Day 74) and with the warnings: "if pull-forward is 30% rather than 15%, the coupon gain falls to $X".
• Model drift: the world changing after the estimates were made. Days 77–78.

The app's caption says this in one sentence. A range that claims to cover everything is worse than one that says what it leaves out.

73.5 Day 73 mini-project: correlated errors (75 min)

The engine draws each cell's elasticity independently, so 20 cells' errors partly cancel and the total looks precise. But Module 6 (Day 59) found that DML was about 3% too close to zero in every cell. Errors that share a cause don't cancel. The scenario has an elasticity_correlation setting: each draw is the estimate + SE × (√ρ · common shock + √(1−ρ) · own shock). Create day73/correlated_errors.py:

# day73/correlated_errors.py   (python -m day73.correlated_errors)
"""
Mini-project: the simulator draws each cell's elasticity independently, so errors partly cancel across
20 cells and the total looks precise. But if every estimate shares a bias (Module 6, Day 59: DML was ~3%
too close to zero in EVERY cell), errors add up instead. How much wider is the range if they're correlated?
"""
from engine import load_parameters, simulate
from engine.narrate import money
from engine.presets import PRESETS

params = load_parameters()
sc = PRESETS["5% everywhere"]
for rho in [0.0, 0.5, 1.0]:
    r = simulate(params, sc.model_copy(update={"elasticity_correlation": rho}))
    g = r["change"]["gross_profit"]
    print(f"correlation {rho:.1f}: median {money(g['median'])}, 90% range {money(g['p5'])} to {money(g['p95'])} "
          f"(width {money(g['p95'] - g['p5'])})")

python -m day73.correlated_errors

correlation 0.0: median $459k, 90% range $419k to $502k (width $83k)
correlation 0.5: median $456k, 90% range $338k to $579k (width $241k)
correlation 1.0: median $455k, 90% range $292k to $622k (width $330k)

Same median, four times the width. With independent errors the simulator claims ±$40k; if the cells share a bias, ±$165k is honest. We don't know ρ, so the app exposes it as a slider, defaulting to 0 and explained in the caption. In a memo, quote the ρ = 0.5 range: it still shows that profit goes up, and nobody can say you hid the risk.

73.6 Save your work with Git (20 min)

git add engine/narrate.py day73
git commit -m "Day 73: executive summaries, distributions, correlated estimation errors"
git push

Day 73 checklist

• My summaries give a rounded median, a 90% range in words, and a calibrated probability
• The text and the charts come from the same simulation result
• I can say which uncertainties the range covers and which it doesn't
• I know why correlated estimation errors widen the range, and by how much here