Almost every estimator you know secretly says: there is some quantity in the population that should equal zero. GMM makes that statement the centerpiece.
A moment condition is a population statement of the form
E[ g(data, θ) ] = 0
where θ is the parameter we want and g is a function we choose because we believe its expectation is zero at the true θ. The vector g is called the moment function.
Suppose we want the population mean μ of a variable x. The defining property of the mean is that deviations average to zero:
E[ x − μ ] = 0
Here θ = μ and g(x, μ) = x − μ. That single equation is the model.
We cannot compute E[·], but we can compute its sample analog — the average over our n data points. Define the sample moment:
ḡ(θ) = (1/n) Σᵢ g(xᵢ, θ)
The method of moments simply sets ḡ(θ) = 0 and solves. For the mean, (1/n)Σ(xᵢ − μ) = 0 gives μ̂ = x̄. We just re-derived the sample average from a moment condition.
Count things. Let θ have k parameters and let g produce m moment equations.
The over-identified case is where GMM earns its name. We can't make every moment zero, so instead we make them collectively small. We minimize a weighted sum of squared moments:
θ̂ = argminθ ḡ(θ)′ · W · ḡ(θ)
where W is a positive-definite weighting matrix that says how much each moment (and each combination of moments) matters. This scalar quantity is the GMM objective.
Notice this nests the method of moments: if you can drive ḡ to zero, then ḡ′Wḡ = 0 for any W, so the GMM solution coincides with the exact solution.
","id":"gmm-moments-02","skillIds":["gmm.gmm-identification"],"estimatedMinutes":6,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-01"]},{"type":"code","title":"Hand-roll GMM for a mean","note":"Estimate μ from the single moment E[x−μ]=0 by literally minimizing the GMM objective with scipy. With one moment and one parameter (m=k=1) the objective is driven to zero and μ̂ equals the sample mean.","code":"import numpy as np\nfrom scipy.optimize import minimize\n\nrng = np.random.default_rng(0)\nn = 400\nx = rng.normal(5.0, 2.0, n) # true mean 5\n\n# moment function g(x, mu) = x - mu -> sample moment is its average\ndef gbar(mu):\n return np.array([np.mean(x - mu)])\n\n# GMM objective J(mu) = gbar' W gbar, here W = 1 (scalar, m=1)\ndef objective(p):\n g = gbar(p[0])\n return float(g @ g)\n\nres = minimize(objective, x0=[0.0])\nmu_hat = res.x[0]\n\nprint(f'sample mean x̄ = {x.mean():.6f}')\nprint(f'GMM μ̂ (minimizer) = {mu_hat:.6f}')\nprint(f'objective at optimum = {objective([mu_hat]):.3e} (≈ 0, just-identified)')\nprint(f'gbar at optimum = {gbar(mu_hat)[0]:.3e}')","id":"gmm-moments-03","skillIds":["gmm.moment-conditions"],"estimatedMinutes":10,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-02"]},{"type":"interactive","title":"Watch the GMM objective J(μ)","note":"The objective ḡ(μ)′ḡ(μ) is a parabola in μ that bottoms out exactly at the sample mean. Drag the candidate μ and read off J. The orange marker is your guess; the dashed line is where J is minimized (the sample mean).","params":[{"name":"mu","label":"Candidate μ","min":0,"max":10,"step":0.1,"value":2}],"template":"import numpy as np, matplotlib.pyplot as plt\nrng = np.random.default_rng(0)\nn = 400\nx = rng.normal(5.0, 2.0, n)\ndef J(mu):\n g = np.mean(x - mu)\n return g*g\ngrid = np.linspace(0,10,200)\nvals = [J(m) for m in grid]\nmu_star = x.mean()\nmu = {{mu}}\nplt.figure(figsize=(6,3.7))\nplt.plot(grid, vals, color='#af983f', lw=2, label='J(μ)=ḡ(μ)²')\nplt.axvline(mu_star, color='#6f6b57', ls='--', lw=1.5, label=f'min at x̄={mu_star:.2f}')\nplt.scatter([mu],[J(mu)], color='#da9100', s=70, zorder=5, label='your μ')\nplt.xlabel('candidate μ'); plt.ylabel('GMM objective J')\nplt.legend(fontsize=8)\nplt.title(f'μ={mu:.2f} → J={J(mu):.4f} (min J at μ={mu_star:.2f})')","id":"gmm-moments-04","skillIds":["gmm.gmm-identification"],"estimatedMinutes":8,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-03"]},{"type":"quiz","title":"Check: counting moments","prompt":"You have a model with 2 parameters and you write down 5 moment conditions. This model is:","choices":["under-identified, so GMM cannot be used","just-identified, so the weighting matrix is irrelevant","over-identified, so GMM minimizes a weighted sum of moments and W matters","exactly-identified only if the moments are independent"],"answer":2,"hint":"Compare m (number of moments) with k (number of parameters).","explain":"With m = 5 moments and k = 2 parameters, m > k: the model is over-identified. You generally can't zero out all moments at once, so GMM minimizes ḡ′Wḡ and the choice of W changes the estimate.","id":"gmm-moments-05","skillIds":["gmm.moment-conditions"],"estimatedMinutes":5,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-04"],"variantId":"gmm-moments-05-core"},{"type":"quiz","title":"Check: what is a moment condition?","prompt":"The moment condition E[x − μ] = 0 is used to estimate μ. What does the sample analog (1/n)Σ(xᵢ − μ̂) = 0 give you?","choices":["the sample variance","the sample mean x̄","the median of x","an arbitrary value depending on W"],"answer":1,"hint":"Solve (1/n)Σ(xᵢ − μ̂) = 0 for μ̂.","explain":"Setting the sample moment to zero gives Σxᵢ = nμ̂, i.e. μ̂ = x̄. The method-of-moments estimator of the mean is just the sample average.","id":"gmm-moments-06","skillIds":["gmm.gmm-identification"],"estimatedMinutes":5,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-05"],"variantId":"gmm-moments-06-core"},{"type":"fillblank","title":"Counting the surplus","prompt":"When you write down more moment conditions than parameters (m > k), the model is said to be ___.","lead":"You just saw a model with 5 moment conditions for 2 parameters — there are more equations than unknowns.","accept":["over-identified","overidentified","over identified","over-id","overid","over- identified"],"hint":"It's the opposite of 'just-identified' (m = k). The prefix means you have a surplus.","explain":"The 'extra moments are wasted' instinct, named: a model with m > k is over-identified, and those surplus moments are not wasted — they give you a testable restriction (the J-test) and, if valid, extra efficiency. The tempting belief is that you should always reduce to m = k so every moment binds exactly; that throws away the over-identifying information. Boundary: exactly m = k is just-identified (moments hit zero exactly, W is irrelevant); m < k is under-identified and not estimable.","points":15,"id":"gmm-moments-07","skillIds":["gmm.moment-conditions"],"estimatedMinutes":4,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-06"],"variantId":"gmm-moments-07-core"},{"type":"numeric","title":"How many over-identifying restrictions?","prompt":"A GMM model has m = 5 moment conditions and k = 2 parameters. How many over-identifying restrictions does it have? (enter a whole number)","answer":3,"tol":0.5,"hint":"Over-identifying restrictions = surplus moments = m − k.","explain":"The 'restrictions = number of moments' trap: the count of over-identifying restrictions is m − k = 5 − 2 = 3, not 5. The tempting answer 5 counts all moments; but k of them are 'spent' pinning down the k parameters, leaving only m − k as independent checks. Boundary: when m = k the count is 0 — a just-identified model has nothing left over to test, which is exactly why no J-test exists there.","points":20,"id":"gmm-moments-08","skillIds":["gmm.gmm-identification"],"estimatedMinutes":5,"difficulty":"advanced","prerequisiteStageIds":["gmm-moments-07"],"variantId":"gmm-moments-08-core"}]}}