Instrumental Variables: Intuition and 2SLS

37.1 When selection on observables fails (40 min)

The brand team has a worry: some customers simply like NorthMart. They join the programme and they spend more, and the data has no column for "likes NorthMart". In DAG terms (Day 14), brand affinity is an unmeasured confounder: an open backdoor path that no adjustment can close.

World B (hidden=1.0) builds that in. Create day37/selection_fails.py:

# day37/selection_fails.py   (python -m day37.selection_fails)
"""
Same customers, one change: an UNOBSERVED trait (brand affinity) now makes customers both more
likely to join and bigger spenders. Everything from Days 31-36 assumes no such trait exists.
"""
from causalkit.estimators import aipw, g_computation
from causalkit.loyalty import public, truth, world

for label, h in [("selection on observables holds", 0.0), ("hidden brand affinity", 1.0)]:
    full = world(hidden=h)
    df = public(full).to_pandas()
    t, g, a = truth(full), g_computation(df), aipw(df)
    print(f"{label}:")
    print(f"   g-computation ATE {g['ATE']:6.1f}   AIPW ATE {a['ATE']:6.1f} (SE {a['ATE_se']:.1f})   truth {t['ATE']:6.1f}")
    print(f"   g-computation ATT {g['ATT']:6.1f}   AIPW ATT {a['ATT']:6.1f} (SE {a['ATT_se']:.1f})   truth {t['ATT']:6.1f}")

python -m day37.selection_fails

selection on observables holds:
   g-computation ATE  156.8   AIPW ATE  157.8 (SE 2.1)   truth  158.4
   g-computation ATT  183.9   AIPW ATT  184.6 (SE 2.7)   truth  186.0
hidden brand affinity:
   g-computation ATE  230.6   AIPW ATE  232.5 (SE 2.4)   truth  158.4
   g-computation ATT  255.9   AIPW ATT  256.3 (SE 3.3)   truth  183.4

In world B, every Days 31–36 method overstates the effect by $72–74, with a standard error of $2.4. That's the most dangerous output in applied work: precise and wrong. Nothing in the data warns you: balance, overlap and specification tables all look the same as in world A.

37.2 An instrument: the random invitation (50 min)

Last year, NorthMart sent half of its customers, chosen at random, a $20 sign-up offer. An instrument Z is a variable that:

Draw it as a DAG: invited → joined → spend, with affinity → joined and affinity → spend. There is no arrow from affinity to invited (random) and none from invited to spend (exclusion). Any difference in spending between invited and non-invited customers must therefore travel through joining.

37.3 The Wald estimator and the LATE (45 min)

Concept. Two randomised comparisons:

• First stage: invited vs not, in joining rates.
• Intention-to-treat (ITT): invited vs not, in spending. This is a clean experimental effect, of the invitation.

If the invitation raised joining by 16 percentage points and spending by $20 per customer, then those extra 16% of joiners must account for the whole $20: the effect per extra joiner is $20 / 0.16. That ratio is the Wald estimator.

Who are those 16%? Compliers: customers who join if invited and don't if not. The Wald estimator gives the effect for them: the local average treatment effect (LATE) (Imbens & Angrist, 1994). Customers who join regardless (always-takers) or never join (never-takers) contribute nothing, because the invitation didn't change what they did.

Create day37/wald.py:

# day37/wald.py   (python -m day37.wald)
"""
The invitation was RANDOM: half of customers got a $20 sign-up offer. It can't be related to brand
affinity. Use it as an instrument: the effect of joining = (effect of the invitation on spend) /
(effect of the invitation on joining).
"""
import polars as pl

from causalkit.loyalty import public, truth, world

full = world(hidden=1.0)
df = public(full)
t = truth(full)


def mean(col: str, invited: int) -> float:
    return df.filter(pl.col("invited") == invited)[col].mean()


itt = mean("spend", 1) - mean("spend", 0)
first_stage = mean("joined", 1) - mean("joined", 0)
print(f"Joined: {mean('joined', 1):.1%} if invited, {mean('joined', 0):.1%} if not -> first stage {first_stage:.3f}")
print(f"Spend:  ${mean('spend', 1):,.0f} if invited, ${mean('spend', 0):,.0f} if not -> ITT ${itt:.1f}")
print(f"Wald estimate = ITT / first stage = {itt / first_stage:.1f}")
print(f"Truth: LATE {t['LATE']:.1f} (compliers are {t['share_compliers']:.1%} of customers);"
      f" ATE {t['ATE']:.1f}; ATT {t['ATT']:.1f}")

python -m day37.wald

Joined: 56.3% if invited, 40.2% if not -> first stage 0.161
Spend:  $1,056 if invited, $1,035 if not -> ITT $20.4
Wald estimate = ITT / first stage = 126.7
Truth: LATE 153.6 (compliers are 15.9% of customers); ATE 158.4; ATT 183.4

$127 against a true LATE of $154. Far closer than AIPW's $232, but is $27 a lot? We need a standard error.

37.4 Two-stage least squares (45 min)

Concept. 2SLS gives the same estimate, plus standard errors and room for covariates:

1. Stage 1: regress joined on invited (and covariates); keep the predicted values. They contain only the variation in joining caused by the random invitation.
2. Stage 2: regress spend on predicted joining (and the same covariates).

Type IV_CONTROLS and two_sls into estimators.py. Create day37/two_sls.py:

# day37/two_sls.py   (python -m day37.two_sls)
"""
Two-stage least squares (2SLS), by hand and with linearmodels.
Stage 1: predict joining from the invitation. Stage 2: regress spend on PREDICTED joining.
"""
import statsmodels.formula.api as smf

from causalkit.estimators import two_sls
from causalkit.loyalty import public, truth, world

full = world(hidden=1.0)
df = public(full).to_pandas()
t = truth(full)

# By hand: right estimate, WRONG standard error (stage 2 ignores that stage 1 was estimated)
df["joined_hat"] = smf.ols("joined ~ invited", df).fit().fittedvalues
by_hand = smf.ols("spend ~ joined_hat", df).fit()
print(f"2SLS by hand:          {by_hand.params['joined_hat']:7.1f}  (SE {by_hand.bse['joined_hat']:.1f}  <- not valid)")

iv = two_sls(df, controls=None)
print(f"2SLS, linearmodels:    {iv.params['joined']:7.1f}  (SE {iv.std_errors['joined']:.1f})")
print(f"First-stage F statistic: {iv.first_stage.diagnostics['f.stat'].iloc[0]:.0f}")
print(f"Truth: LATE {t['LATE']:.1f}")

python -m day37.two_sls

2SLS by hand:            126.7  (SE 65.9  <- not valid)
2SLS, linearmodels:      126.7  (SE 63.8)
First-stage F statistic: 532
Truth: LATE 153.6

• Doing it by hand gives the right estimate but the wrong standard error: the second-stage regression treats predicted joining as data, not as an estimate. Always use an IV routine (linearmodels.iv.IV2SLS).
• The SE is $64. The truth is well within one SE. IV removed the $74 bias, but the price is a standard error more than 25 times larger than AIPW's. Only 16% of customers' behaviour carries the signal.
• First-stage F = 532: the instrument is strong (Day 38).

37.5 Day 37 mini-project: IV with covariates (70 min)

The invitation is random, so covariates aren't needed for identification. But spend is very predictable from past spend, so covariates should remove noise, just as CUPED did on Day 25. Create day37/iv_with_covariates.py:

# day37/iv_with_covariates.py   (python -m day37.iv_with_covariates)
"""
Mini-project: the invitation is random, so covariates aren't needed for identification.
But spend is very predictable from past spend, so covariates remove noise, exactly like CUPED (Day 25).
Compare OLS, AIPW and 2SLS with and without covariates, against the truths they target.
"""
import polars as pl
import statsmodels.formula.api as smf

from causalkit.estimators import OUTCOME_FORMULA, aipw, two_sls
from causalkit.loyalty import public, truth, world

full = world(hidden=1.0)
df = public(full).to_pandas()
t = truth(full)

ols = smf.ols(OUTCOME_FORMULA.replace("spend ~", "spend ~ joined +"), df).fit(cov_type="HC2")
a = aipw(df)
iv0, iv1 = two_sls(df, controls=None), two_sls(df)
rows = [
    {"estimator": "OLS with flexible controls", "estimate": ols.params["joined"], "se": ols.bse["joined"],
     "targets": "(a weighted average)", "truth": None},
    {"estimator": "AIPW", "estimate": a["ATE"], "se": a["ATE_se"], "targets": "ATE", "truth": t["ATE"]},
    {"estimator": "2SLS, no covariates", "estimate": iv0.params["joined"], "se": iv0.std_errors["joined"],
     "targets": "LATE", "truth": t["LATE"]},
    {"estimator": "2SLS with covariates", "estimate": iv1.params["joined"], "se": iv1.std_errors["joined"],
     "targets": "LATE", "truth": t["LATE"]},
]
table = pl.DataFrame(rows).with_columns(pl.col("estimate", "se", "truth").round(1))
with pl.Config(fmt_str_lengths=30, tbl_width_chars=110):
    print(table)

python -m day37.iv_with_covariates

│ estimator                  ┆ estimate ┆ se   ┆ targets              ┆ truth │
│ OLS with flexible controls ┆ 226.1    ┆ 2.4  ┆ (a weighted average) ┆ null  │
│ AIPW                       ┆ 232.5    ┆ 2.4  ┆ ATE                  ┆ 158.4 │
│ 2SLS, no covariates        ┆ 126.7    ┆ 63.8 ┆ LATE                 ┆ 153.6 │
│ 2SLS with covariates       ┆ 141.3    ┆ 13.5 ┆ LATE                 ┆ 153.6 │

Covariates cut the 2SLS standard error from $64 to $13.5, a fivefold gain, equivalent to having 22 times as many customers. The 95% CI [$115, $168] now excludes AIPW's $232: in this world, the data can reject the selection-on-observables answer.

Try it. Run AIPW and 2SLS (with covariates) in world A, where there is no hidden confounder. Do they agree? Create day37/ols_vs_iv_world_a.py.

# day37/ols_vs_iv_world_a.py   (python -m day37.ols_vs_iv_world_a)
from causalkit.estimators import aipw, two_sls
from causalkit.loyalty import public, truth, world

full = world(hidden=0.0)
df = public(full).to_pandas()
t, a, iv = truth(full), aipw(df), two_sls(df)
print(f"AIPW ATE {a['ATE']:.1f} (SE {a['ATE_se']:.1f}) vs truth {t['ATE']:.1f}")
print(f"2SLS     {iv.params['joined']:.1f} (SE {iv.std_errors['joined']:.1f}) vs LATE {t['LATE']:.1f}")

AIPW ATE 157.8 (SE 2.1) vs truth 158.4
2SLS     151.9 (SE 9.6) vs LATE 156.0

37.6 Save your work with Git (20 min)

git add .
git commit -m "Day 37: hidden confounding, Wald estimator, 2SLS, IV with covariates"
git push

Day 37 checklist

• I can recognise when an unmeasured confounder makes all of Days 31–36 fail
• I can state the four instrument conditions and say which are checkable
• I can compute the Wald estimator from a first stage and an ITT
• I can explain what a LATE is and who the compliers are
• I run 2SLS with an IV routine and use covariates to cut its standard error