{"schemaVersion":2,"courseId":"logit","module":{"id":"logit-why-lpm-fails","title":"1 · Why a linear model fails for 0/1 outcomes","summary":"When y is 0 or 1, fitting a straight line (the Linear Probability Model) breaks down: it predicts probabilities outside [0,1], has built-in heteroskedasticity, and imposes a constant effect that cannot be globally true.","stages":[{"type":"read","title":"Binary outcomes and the urge to run OLS","html":"

Modelling a yes/no outcome

Many outcomes are binary: a loan defaults or not, a patient recovers or not, a customer buys or not. We code these as y ∈ {0,1}. The natural quantity of interest is the probability that y = 1 given some features x:

p(x) = P(y = 1 | x)

A tempting first move is to just run OLS of y on x. That is called the Linear Probability Model (LPM):

P(y = 1 | x) = β₀ + β₁·x₁ + β₂·x₂ + …

It is fast, the coefficients read directly as \"a one-unit rise in x changes the probability by β\", and for x near the data centre it is often a decent approximation. But it has three structural problems that get worse near the edges.

Key idea. A probability must live in [0,1] and the way x moves it cannot be a straight line forever — eventually the line must bend, or it will predict nonsense.
","id":"logit-why-lpm-fails-01","skillIds":["logit.lpm-limits"],"estimatedMinutes":6,"difficulty":"applied","prerequisiteStageIds":[]},{"type":"read","title":"Three things that go wrong","html":"

1. Predictions escape [0,1]

A line has no ceiling or floor. Push x far enough and ŷ goes below 0 or above 1 — an impossible probability. You cannot interpret \"there is a 130% chance of default.\"

2. The errors are heteroskedastic by construction

If y is Bernoulli with success probability p, its variance is p·(1−p). That depends on x, so the OLS homoskedasticity assumption is automatically violated and the default standard errors are wrong.

3. A constant marginal effect is implausible

LPM says one extra unit of x always changes the probability by the same β. But moving someone from a 50% chance to 60% is easy, while moving them from 98% to 100% should be nearly impossible. The real effect of x must shrink near 0 and 1.

Key idea. We want an S-shaped curve that is squeezed into (0,1): steep in the middle, flat at the extremes. That is exactly what logit and probit deliver.
","id":"logit-why-lpm-fails-02","skillIds":["logit.binary-links"],"estimatedMinutes":6,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-01"]},{"type":"code","title":"Watch the LPM predict impossible probabilities","note":"We simulate a true logistic process, fit OLS (the LPM), and print how many fitted 'probabilities' fall outside [0,1]. The scatter shows the flat line cutting through 0 and 1.","code":"import numpy as np, pandas as pd, statsmodels.formula.api as smf\nimport matplotlib.pyplot as plt\nrng = np.random.default_rng(0)\nn = 600\nx = rng.normal(0, 1.6, n)\neta = -0.4 + 2.2*x # true log-odds\np = 1/(1+np.exp(-eta)) # true probability\ny = rng.binomial(1, p)\ndf = pd.DataFrame({'y': y, 'x': x})\n\nlpm = smf.ols('y ~ x', df).fit()\nphat = lpm.predict(df)\nout = ((phat < 0) | (phat > 1)).sum()\nprint('LPM fitted prob range: [%.3f, %.3f]' % (phat.min(), phat.max()))\nprint('Fitted values outside [0,1]: %d of %d (%.1f%%)' % (out, n, 100*out/n))\n\nxs = np.linspace(x.min(), x.max(), 100)\nplt.figure(figsize=(6,3.7))\nplt.scatter(x, y, s=10, color='#da9100', alpha=.35, label='data (0/1)')\nplt.plot(xs, lpm.params['Intercept']+lpm.params['x']*xs, color='#6f6b57', lw=2, label='LPM line')\nplt.axhline(0, color='#c9c6ac', lw=1); plt.axhline(1, color='#c9c6ac', lw=1)\nplt.ylim(-0.4, 1.4); plt.xlabel('x'); plt.ylabel('y / predicted prob')\nplt.title('LPM: %d impossible predictions outside [0,1]' % out)\nplt.legend(loc='best', fontsize=8)","id":"logit-why-lpm-fails-03","skillIds":["logit.lpm-limits"],"estimatedMinutes":10,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-02"]},{"type":"interactive","title":"How far out of bounds does the line go?","note":"Increase the true slope or shift the intercept. The steeper and more off-centre the relationship, the further the straight LPM line pokes past 0 and 1. The title reports how many fitted probabilities are impossible.","params":[{"name":"slope","label":"True log-odds slope","min":0.2,"max":5,"step":0.1,"value":2.2},{"name":"intercept","label":"True log-odds intercept","min":-3,"max":3,"step":0.1,"value":-0.4}],"template":"import numpy as np, pandas as pd, statsmodels.formula.api as smf\nimport matplotlib.pyplot as plt\nrng = np.random.default_rng(0)\nn = 500\nx = rng.normal(0, 1.6, n)\neta = {{intercept}} + {{slope}}*x\np = 1/(1+np.exp(-eta))\ny = rng.binomial(1, p)\ndf = pd.DataFrame({'y': y, 'x': x})\nlpm = smf.ols('y ~ x', df).fit()\nphat = lpm.predict(df)\nout = int(((phat < 0) | (phat > 1)).sum())\nxs = np.linspace(x.min(), x.max(), 100)\nplt.figure(figsize=(6,3.7))\nplt.scatter(x, y, s=10, color='#da9100', alpha=.3)\nplt.plot(xs, lpm.params['Intercept']+lpm.params['x']*xs, color='#6f6b57', lw=2)\nplt.axhline(0, color='#c9c6ac', lw=1); plt.axhline(1, color='#c9c6ac', lw=1)\nplt.ylim(-0.5, 1.5); plt.xlabel('x'); plt.ylabel('predicted prob')\nplt.title('LPM impossible predictions: %d of %d' % (out, n))","id":"logit-why-lpm-fails-04","skillIds":["logit.binary-links"],"estimatedMinutes":8,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-03"]},{"type":"quiz","title":"Check","prompt":"Which problem is NOT a structural flaw of the Linear Probability Model for a 0/1 outcome?","choices":["Fitted probabilities can fall below 0 or above 1","The error variance p(1−p) depends on x, so errors are heteroskedastic","It forces a constant marginal effect of x at every probability level","It can only be estimated by maximum likelihood, never by OLS"],"answer":3,"hint":"Think about how the LPM is actually fit versus what logit/probit require.","explain":"The LPM is fit by ordinary least squares — that is its whole appeal. The genuine flaws are out-of-range predictions, built-in heteroskedasticity, and a constant (rather than diminishing) marginal effect.","id":"logit-why-lpm-fails-05","skillIds":["logit.lpm-limits"],"estimatedMinutes":5,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-04"],"variantId":"logit-why-lpm-fails-05-core"},{"type":"quiz","title":"Check","prompt":"Why should the effect of x on P(y=1) shrink as the probability approaches 0 or 1?","choices":["Because the sample size falls at the extremes","Because a probability is bounded in [0,1], so there is little room left to move","Because OLS standard errors grow at the extremes","Because x stops varying at the extremes"],"answer":1,"hint":"You cannot push a probability above 1.","explain":"Near the floor or ceiling there is almost no room left, so the same change in x can only nudge the probability a tiny amount — implying an S-shaped curve, not a straight line.","id":"logit-why-lpm-fails-06","skillIds":["logit.binary-links"],"estimatedMinutes":5,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-05"],"variantId":"logit-why-lpm-fails-06-core"},{"type":"truefalse","title":"Does a straight line have a constant effect?","prompt":"You just saw the Linear Probability Model fit a straight line to a 0/1 outcome. Claim: because the line is straight, a one-unit rise in x changes P(y=1) by the same amount whether you start at p=0.05 or at p=0.95.","answer":true,"hint":"A straight line has one slope everywhere — that is the whole point of the module. Is that constant effect realistic near the 0 and 1 boundaries?","explain":"This is the constant-marginal-effect flaw of the LPM. The claim is literally TRUE of the straight line — its slope is the same everywhere — and that is exactly why the LPM is wrong for probabilities: near p=0.95 there is almost no room left to move, so a real effect must shrink at the extremes. The fix is an S-curve (logit/probit) whose slope flattens in the tails. Boundary: a constant effect is only defensible over a narrow middle range where the fitted probabilities stay far from 0 and 1.","points":10,"id":"logit-why-lpm-fails-07","skillIds":["logit.lpm-limits"],"estimatedMinutes":3,"difficulty":"applied","prerequisiteStageIds":["logit-why-lpm-fails-06"],"variantId":"logit-why-lpm-fails-07-core"}]}}