Abstract
We study the asymptotic behavior of the least squares estimators when the model is possibly misspecified. We consider the setting where we wish to estimate an unknown function f∗:(0,1)d→R from observations (X,Y),(X1,Y1),⋯,(Xn,Yn); our estimator ^gn is the minimizer of ∑ni=1(Yi−g(Xi))2/n over g∈G for some set of functions G. We provide sufficient conditions on the metric entropy of G, under which ^gn converges to g∗ as n→∞, where g∗ is the minimizer of ∥g−f∗∥≜E(g(X)−f∗(X))2 over g∈G. As corollaries of our theorem, we establish ∥^gn−g∗∥→0 as n→∞ when G is the set of monotone functions or the set of convex functions. We also make a connection between the convergence rate of ∥^gn−g∗∥ and the metric entropy of G. As special cases of our finding, we compute the convergence rate of ∥^gn−g∗∥2 when G is the set of bounded monotone functions or the set of bounded convex functions.