Causal Machine Learning & Heterogeneity
Using machine learning to estimate effects in high dimensions and find who responds.
What you'll learn
- 1Days 51–54Orthogonalization, cross-fitting and DoubleML for partially linear, interactive and IV models.
By the end: Students can explain why a good predictive model can give a bad effect estimate, estimate effects by partialling out with cross-fitted machine learning (DML) by hand and with DoubleML, say what a partially linear model's single coefficient means when effects vary, and estimate ATE, ATT and LATE with DoubleML's interactive models.
- 2Days 55–57Meta-learners, causal forests and validating heterogeneous effects.
By the end: Students can estimate conditional average treatment effects (CATEs) with S-, T-, X- and DR-learners and causal forests, turn them into a targeting policy, estimate group effects that aren't shrunk towards the average, and validate a CATE model on a randomised experiment with BLP, uplift (TOC/AUTOC) and calibration checks.
- 3Days 58–60Price elasticities with DML and instruments, by segment and category.
By the end: Students can explain why prices set by a pricing team are endogenous, estimate a price elasticity with fixed effects, an instrument and DML, estimate elasticities for segments and segment × category cells, cross-check two estimators that rest on different assumptions, and turn elasticities and margins into a pricing decision with an honest "test first" category.
Tools
Mini-capstone
Price elasticity engine: which segments, in which categories, can absorb a 5% price rise without losing gross profit?
The ten days
Before you start: module setup and project structure
Course: Applied Causal Inference & Decision Science Audience: Economists and analysts who completed Modules 1–5 Environment: GitHub Codespaces (VS Code in the browser) Time per day: ~5 hours (including breaks and exercises) Goal by end of Day 54: Students can explain why a good predictive model can give a bad effect estimate, estimate effects by partialling out with cross-fitted machine learning (DML) by hand and with DoubleML, say what a partially linear model's single coefficient means when effects vary, and estimate ATE, ATT and LATE with DoubleML's interactive models.
Where we are
Modules 4 and 5 estimated one effect, with a handful of carefully chosen controls. Two things change in Module 6:
- Many features, unknown functional form. A marketing database has dozens of customer features, and nobody knows how they combine. Machine learning is good at that. Days 51–54 show how to use it without breaking causal inference.
- Effects that differ across customers. The real question is usually for whom: who should get the coupon, which segments can absorb a price rise. Days 55–57 estimate and validate heterogeneous effects; Days 58–60 apply all of it to prices.
| Days | Question | Tools |
|---|---|---|
| 51–54 | How do we use ML for one effect, with valid confidence intervals? | DoubleML |
| 55–57 | Who responds, and how do we know the model is right? | EconML: meta-learners, causal forests, DRTester |
| 58–60 | How price-sensitive is each segment, and where can prices rise? | DoubleML, EconML, pyfixest |
Two simulators
| File | What it simulates | Used on |
|---|---|---|
causalkit/coupons.py | 20,000 customers; a targeting model sends $10-off coupons; each customer's true effect is known | Days 51–57 |
causalkit/demand.py | 80 stores × 5 categories × 104 weeks; a pricing team sets prices from costs and its demand forecast | Days 58–60 |
causalkit/coupons.py (provided; click to expand)
"""
NorthMart's targeted coupons (simulated). Provided for Module 6: read the docstrings, not the code,
until the Day 60 capstone is done.
A targeting model sends a $10-off coupon to customers it thinks will respond, mostly deal-seekers and
lapsing customers. Did the coupon raise their spending next month, and for WHOM?
from causalkit.coupons import coupons, truth, X_COLS
df = coupons() # 20,000 customers; coupons targeted by the marketing model
df = coupons(randomized=True) # same customers, coupons assigned by a coin flip (an A/B test)
df = coupons(holdout=True, hidden=1.0)
# half of customers randomly made ineligible; the targeting model also used a real-time
# browsing signal that was never stored (an unobserved confounder)
truth(df) # true ATE, ATT, LATE (for checking only)
Columns an analyst has:
customer_id, 24 features (X_COLS: behaviour, demographics, and 10 columns z1..z10 of pure noise),
eligible (1 = could receive a coupon; random when holdout=True), coupon (the TREATMENT),
spend (next 30 days, $: the OUTCOME)
Columns starting with '_' are the simulator's hidden truth: _tau (each customer's effect), _ps, _complier,
_browsing (the unstored browsing signal).
"""
import numpy as np
import polars as pl
BEHAVIOUR = ["tenure_years", "age", "pre_spend", "visits", "deal_share", "app_user", "email_open",
"basket", "returns_rate", "region_east", "region_west", "region_north", "region_south"]
NOISE = [f"z{i}" for i in range(1, 11)]
X_COLS = BEHAVIOUR + ["lapsing"] + NOISE
def _sigmoid(x):
return 1 / (1 + np.exp(-x))
def _features(n: int, rng: np.random.Generator) -> dict:
region = rng.integers(0, 5, n)
f = {
"tenure_years": np.round(rng.gamma(2.0, 2.0, n), 1),
"age": np.clip(rng.normal(44, 13, n), 18, 85).round(),
"pre_spend": np.round(np.exp(rng.normal(5.6, 0.6, n)), 2),
"visits": rng.poisson(5, n).astype(float),
"deal_share": np.round(rng.beta(2, 5, n), 3),
"app_user": rng.binomial(1, 0.45, n).astype(float),
"email_open": np.round(rng.beta(2, 3, n), 3),
"basket": np.round(np.exp(rng.normal(3.6, 0.35, n)), 2),
"returns_rate": np.round(rng.beta(1, 12, n), 3),
}
for k, name in enumerate(["east", "west", "north", "south"], start=1):
f[f"region_{name}"] = (region == k).astype(float)
f["lapsing"] = (f["visits"] <= 2).astype(float)
for z in NOISE:
f[z] = np.round(rng.normal(0, 1, n), 3)
return f
def cate(f: dict) -> np.ndarray:
"""True effect of the coupon on next-month spend ($) for customers with features f."""
return (-8 + 70 * f["deal_share"] + 10 * f["app_user"] - 12 * (f["pre_spend"] > 500)
+ 6 * np.tanh(f["visits"] - 4) + 0.15 * (f["age"] - 44) * (f["deal_share"] > 0.4))
def coupons(n: int = 20_000, seed: int = 51, randomized: bool = False, holdout: bool = False,
hidden: float = 0.0) -> pl.DataFrame:
"""randomized: coupons by coin flip (p = 0.5) instead of the targeting model.
holdout: half of customers are randomly made ineligible (they can't receive a coupon).
hidden: strength of an unstored browsing signal that drives both targeting and spending."""
rng = np.random.default_rng(seed)
f = _features(n, rng)
browsing = rng.normal(0, 1, n)
lp = np.log(f["pre_spend"] / 300)
g = (90 + 75 * lp + 18 * lp**2 + 12 * np.sqrt(f["visits"]) * (1 + f["app_user"]) + 60 * f["email_open"] ** 2
+ 15 * np.sin(f["age"] / 8) + 0.6 * f["basket"] - 40 * f["returns_rate"] - 25 * f["lapsing"]
+ 8 * f["region_west"] + 30 * hidden * browsing)
tau = cate(f)
score = (-1.1 + 4.0 * (f["deal_share"] - 0.28) + 1.3 * f["lapsing"] + 0.7 * f["app_user"] - 0.9 * lp
+ 0.8 * np.sin(3 * f["email_open"]) - 0.3 * (f["tenure_years"] > 6) + 1.2 * hidden * browsing)
ps = np.full(n, 0.5) if randomized else np.clip(_sigmoid(score), 0.03, 0.97)
targeted = rng.random(n) < ps
eligible = rng.random(n) < 0.5 if holdout else np.ones(n, bool)
coupon = targeted & eligible
spend = np.maximum(g + tau * coupon + rng.normal(0, 35, n), 0)
cols = {"customer_id": [f"C{i:05d}" for i in range(1, n + 1)]}
cols |= {k: f[k] for k in X_COLS}
cols |= {"eligible": eligible.astype(np.int64), "coupon": coupon.astype(np.int64), "spend": np.round(spend, 2),
"_tau": tau, "_ps": ps, "_complier": targeted, "_browsing": browsing}
return pl.DataFrame(cols)
def truth(df: pl.DataFrame) -> dict:
tau = df["_tau"].to_numpy()
d = df["coupon"].to_numpy() == 1
out = {"ATE": round(float(tau.mean()), 2), "ATT": round(float(tau[d].mean()), 2)}
if df["eligible"].mean() < 1:
out["LATE"] = round(float(tau[df["_complier"].to_numpy()].mean()), 2)
return out
def public(df: pl.DataFrame) -> pl.DataFrame:
return df.select([c for c in df.columns if not c.startswith("_")])
Module 6 also has a small shared file, so every day uses the same machine-learning settings. Create causalkit/learners.py:
"""Learners shared by Module 6 (Days 51-57): one place to change them."""
import lightgbm as lgb
def reg(**kw):
"""Gradient-boosted regression trees with settings that work well on the coupon data."""
return lgb.LGBMRegressor(**({"n_estimators": 300, "learning_rate": 0.05, "num_leaves": 15,
"min_child_samples": 40, "verbose": -1, "random_state": 0} | kw))
def clf(**kw):
"""Gradient-boosted classification trees (for propensity scores)."""
return lgb.LGBMClassifier(**({"n_estimators": 300, "learning_rate": 0.05, "num_leaves": 15,
"min_child_samples": 40, "verbose": -1, "random_state": 0} | kw))
def variance_weighted(tau, ps):
"""What a partially linear model targets when effects vary: CATEs weighted by e(X)(1 - e(X))."""
w = ps * (1 - ps)
return float((tau * w).sum() / w.sum())
New packages
Add to requirements.txt, then pip install -r requirements.txt:
doubleml==0.11.4
econml==0.17.0
lightgbm==4.7.0
mkdir -p day51 day52 day53 day54 day55 day56 day57 day58 day59 day60
Add one line to .gitignore (Days 55–60 save predictions between scripts):
data/*.npz
Some scripts in this module take several minutes; the notes say which. Start them, then read the next section while they run.