Highly Adaptive Lasso (HAL)

Abstract

In this chapter, we define a general nonparametric estimator of a d-variate function valued parameter ψ ₀. This parameter is defined as a minimizer of an expectation of a loss function L(ψ)(O) that is guaranteed to converge to the true ψ ₀ at a rate faster than n ^−1∕4, for all dimensions d: $\sqrt{ d_{0}(\psi _{n},\psi _{0})} = O_{P}(n^{-1/4-\alpha (d)/8})$, where d ₀(ψ, ψ ₀) = P ₀ L(ψ) − P ₀ L(ψ ₀) is the loss-based dissimilarity. This is a remarkable result because this rate does not depend on the underlying smoothness of ψ ₀. For example, ψ ₀ can be a function that is discontinuous at many points or nondifferentiable. The only assumption we need to assume is that ψ ₀ is right-continuous with left-hand limits, and has a finite variation norm, so that ψ ₀ generates a measure (just as a cumulative distribution function generates a measure on the Euclidean space).