Home Energy Audit Homework: Solutions
Economics of Energy and the Environment
Econ 3391.01
Boston College
2026-10-22
Topic: How do we measure the benefits of energy efficiency policies?
Key Concepts (from lecture):
Methods Covered:
What is a home energy audit? (e.g., a Mass Save Home Energy Assessment)
Policy Goal: Reduce information barriers to energy efficiency adoption
This homework: Unlike Mass Save, audits cost $250, and a household decides whether to get one.
We have an unusual dataset with parallel universe observations:
Variables:
y10: Spending WITHOUT audit (\(Y_{i0}\))y11: Spending WITH audit (\(Y_{i1}\))ey10, ey11: Expected spendinghassle: Audit hassle cost (\(\xi\) from lecture)y00: Pre-period spending, unused)Example: Household 1
Key Point: We observe both potential outcomes for each household!
Only observe:
y10 if household didn’t get audity11 if household did get auditThis is the Fundamental Problem of Causal Inference from lecture. But with this special data, we can compute the true savings for different groups of households and compare them to various estimation methods. Seeing what true answer should be allows us to compare that to what we get using econometrics on real world data.
| household | y00 | y10 | y11 | ey10 | ey11 | hassle |
|---|---|---|---|---|---|---|
| 1 | 1251.248 | 1874.361 | 1548.779 | 1843.381 | 1486.276 | 99.83278 |
| 2 | 1418.414 | 2129.128 | 1786.575 | 2151.938 | 1830.218 | 70.99882 |
| 3 | 1549.773 | 2099.195 | 1646.176 | 2165.068 | 1648.257 | 111.12599 |
| 4 | 1255.664 | 1922.626 | 1548.693 | 1963.576 | 1570.199 | 101.88632 |
Dataset: 10,000 households
Central Question
How much do audits reduce household energy bills?
Whose causal effect do we want?
Sign convention: Lecture reports \(Y_{i1} - Y_{i0}\) (negative = savings). Here savings \(= Y_{i0} - Y_{i1}\) (positive = savings).
Question: If we could force everyone to get an audit, what would be the average savings?
Calculation: \(\text{Savings} = y_{10} - y_{11}\)
This is the Average Treatment Effect (ATE).
Answer
Average savings across all households if audited: $301.96
This is the true population average treatment effect.
Treatment effects are heterogeneous: some households save a lot, some very little.
Setup (same model as “Who selects into audits?” in lecture):
Decision Rule: Get an audit (\(A_i = 1\)) if: \[\text{Expected Savings} > \$250 + \text{Hassle Cost}\]
Calculate: Net savings = \((\text{ey}_{10} - \text{ey}_{11}) - \text{hassle} - 250\)
# Calculate expected savings and net savings
audit_data <- audit_data %>%
mutate(
exp_savings = ey10 - ey11,
net_savings = exp_savings - hassle - 250,
D_optin = if_else(net_savings > 0, 1, 0)
)
# Count opt-ins
optin_count <- sum(audit_data$D_optin)
# Average treatment effect by group
ate_by_group <- audit_data %>%
group_by(D_optin) %>%
summarise(
n = n(),
avg_savings = mean(savings),
avg_exp_savings = mean(exp_savings),
avg_net_savings = mean(net_savings),
.groups = "drop"
)
ATT <- ate_by_group$avg_savings[ate_by_group$D_optin == 1]| Group | N | Avg Savings | Avg Expected Savings | Avg Net Savings |
|---|---|---|---|---|
| Don't opt-in | 6488 | 254.56 | 233.61 | -120.35 |
| Opt-in | 3512 | 389.53 | 429.77 | 87.58 |
Selection into Treatment
Why? People who benefit more are more likely to opt in!
The average savings for the opt-in group is the average treatment effect on the treated (ATT).
The tempting approach: Compare people who got audits to those who didn’t
# Create observed outcome variable
audit_data <- audit_data %>%
mutate(y1obs = if_else(D_optin == 1, y11, y10))
# Calculate means by group
cross_section <- audit_data %>%
group_by(D_optin) %>%
summarise(
mean_y10 = mean(y10),
mean_y11 = mean(y11),
mean_y1obs = mean(y1obs),
.groups = "drop"
)
# Naive estimate: non-audited minus audited spending
est_xsection <- cross_section$mean_y1obs[1] - cross_section$mean_y1obs[2]| Group | Mean Y10 | Mean Y11 | Mean Observed Y |
|---|---|---|---|
| Don't opt-in | 1979 | 1724 | 1979 |
| Opt-in | 2034 | 1645 | 1645 |
Naive difference: $334 vs. True ATE: $302 vs. True ATT: $390
In savings terms: naive comparison = ATT − selection bias.
\[ \begin{aligned} \underbrace{E[Y_{i0}\mid A_i=0] - E[Y_{i1}\mid A_i=1]}_{\text{naive}} &= \underbrace{E[Y_{i0} - Y_{i1}\mid A_i=1]}_{\text{ATT}} \\ &\quad - \underbrace{\left(E[Y_{i0}\mid A_i=1] - E[Y_{i0}\mid A_i=0]\right)}_{\text{selection bias}} \end{aligned} \]
Our special data reveal every piece
Opt-ins use more energy without audits, biasing naive savings downward.
With a single audit dummy, OLS gives the same biased answer:
\[Y_i^{\text{obs}} = \alpha + \beta A_i + \varepsilon_i\]
Here \(A_i = 1\) if household \(i\) opts in (D_optin).
| term | estimate | std.error | statistic | p.value |
|---|---|---|---|---|
| (Intercept) | 1979.0 | 2.1 | 935.6 | 0 |
| D_optin | -334.3 | 3.6 | -93.7 | 0 |
D_optin = −(naive difference)Setup:
Key: Randomization makes the selection bias term zero (in expectation)
# Randomize treatment assignment
audit_data <- audit_data %>%
mutate(irand = runif(n())) %>%
arrange(irand) %>%
mutate(D_treat = if_else(row_number() <= 1000, 1, 0))
# Calculate means for treatment and control
rct_means <- audit_data %>%
group_by(D_treat) %>%
summarise(
mean_y10 = mean(y10),
mean_y11 = mean(y11),
.groups = "drop"
)
# Estimate from experiment: control Y0 minus treated Y1
est_experiment <- rct_means$mean_y10[1] - rct_means$mean_y11[2]| Group | Mean Y10 | Mean Y11 |
|---|---|---|
| Control (no audit) | 1998 | 1696 |
| Treatment (forced audit) | 2003 | 1699 |
Result
Very close! Small difference due to sampling variation.
Mean Y10: Both groups would spend about the same without an audit: no selection bias.
Problem: Can’t actually force people to get audits!
New Setup:
Key Change: Randomize the offer (\(Z_i\)), not the audit itself (\(A_i\)). This is the random encouragement design from lecture.
# Calculate net savings with RCT (free audits for treatment group)
audit_data <- audit_data %>%
mutate(
net_savings_rct = if_else(D_treat == 1,
net_savings + 250, # No cost for treatment
net_savings), # Still costs $250 for control
D_optin_rct = if_else(net_savings_rct > 0, 1, 0)
)
# New opt-ins
new_optins <- sum(audit_data$D_optin_rct == 1 & audit_data$D_optin == 0)
total_optin_rct <- sum(audit_data$D_optin_rct)Takeup
The free offer increases participation by 17%!
Definition: Compare outcomes based on random assignment (\(Z_i\)), not actual behavior (\(A_i\))
Why? Preserves randomization even with imperfect compliance
# Create observed outcome under RCT
audit_data <- audit_data %>%
mutate(y1obs_rct = if_else(D_optin_rct == 1, y11, y10))
# Calculate ITT
itt_means <- audit_data %>%
group_by(D_treat) %>%
summarise(mean_y1obs_rct = mean(y1obs_rct), .groups = "drop")
# ITT estimate
est_itt <- itt_means$mean_y1obs_rct[1] - itt_means$mean_y1obs_rct[2]| Group | Mean Electricity Spending |
|---|---|
| Not offered | 1861 |
| Offered free audit | 1704 |
Much Smaller Effect!
Many households in both groups get audits. The offer changes behavior only for some.
Three types of households (as in lecture):
Key insight: The encouragement only changes behavior of compliers!
# Identify compliers
audit_data <- audit_data %>%
mutate(compliers = if_else(D_optin_rct == 1 & D_optin == 0, 1, 0))
# Compliance rates by treatment group
compliance_table <- audit_data %>%
group_by(D_treat) %>%
summarise(
n = n(),
pct_optin_rct = mean(D_optin_rct) * 100,
pct_optin_original = mean(D_optin) * 100,
pct_compliers = mean(compliers) * 100,
.groups = "drop"
)
# Complier share and LATE
comply_share <- mean(audit_data$compliers[audit_data$D_treat == 1])
est_late <- mean(audit_data$savings[audit_data$compliers == 1])
never_share <- mean(audit_data$D_optin_rct[audit_data$D_treat == 1] == 0)
always_share <- mean(audit_data$D_optin[audit_data$D_treat == 1])| Group | N | % Opt-in (RCT) | % Opt-in (at $250) | % Compliers |
|---|---|---|---|---|
| Control | 9000 | 35.0 | 35.0 | 0 |
| Treatment | 1000 | 95.8 | 35.8 | 60 |
Key Numbers
Shares of the treatment group:
Compliers (free audit only):
Average savings: $268
Always takers (pay $250):
Average savings: $390
Households with highest benefits adopt first!
Relationship (from lecture): \[\text{ITT} = \text{Share of compliers} \times \text{LATE}\]
Intuition:
The Math Checks Out (up to sampling variation)
Not exact: the always takers in the control group are a different random sample than those in the treatment group.
We used unobserved hassle costs and expectations to identify compliers. A regulator can observe:
\[\text{LATE} = \frac{E[Y|Z_i=1]-E[Y|Z_i=0]}{Pr(A_i=1|Z_i=1)-Pr(A_i=1|Z_i=0)}\]
ATE ($302):
Use if you can:
Problem: Infeasible and ignores heterogeneity
LATE ($268):
Use for:
Advantage: Estimates effect on those you’ll actually influence
Main Takeaways
Heterogeneous treatment effects: Benefits vary across households
Selection matters: Those who benefit most adopt first
Declining returns: Each additional adopter has lower savings
Cost-benefit analysis: Optimal policy stops before 100% adoption
Target the margin: Focus on compliers, not always takers
| Method | Question | Estimate | Notes |
|---|---|---|---|
| True ATE | 1 | 302 | Everyone |
| True ATT | 2 | 390 | Voluntary adopters |
| Naive comparison | 3 | 334 | Selection bias |
| RCT - Forced | 4 | 299 | None (forced) |
| Intent-to-Treat | 6 | 157 | Diluted by always/never takers |
| LATE | 7 | 268 | Compliers only |
Naive comparisons mix the causal effect with selection bias
Randomization eliminates selection bias (when compliance is perfect)
ITT preserves randomization with imperfect compliance
LATE identifies effect on marginal participants, and can be recovered as ITT / share of compliers
Choose estimand based on policy: What question are you trying to answer?
The Big Picture
Energy efficiency programs face fundamental challenges:
Good policy evaluation requires understanding who is affected by your intervention!
Fowlie, Greenstone & Wolfram: The Weatherization Assistance Program in Michigan
Hassle costs (\(\xi\)) capture the time and effort of undergoing an audit.
Econ 3391.01 - Energy Economics
