The Hessian Screening Rule
Predictor screening rules, which discard predictors from the design matrix before fitting a model, have had considerable impact on the speed with which l1-regularized regression problems, such as the lasso, can be solved. Current state-of-the-art screening rules, however, have difficulties in dealing with highly-correlated predictors, often becoming too conservative. In this paper, we present a new screening rule to deal with this issue: the Hessian Screening Rule. The rule uses second-order information from the model to provide more accurate screening as well as higher-quality warm starts. The proposed rule outperforms all studied alternatives on data sets with high correlation for both l1-regularized least-squares (the lasso) and logistic regression. It also performs best overall on the real data sets that we examine.
Citation
@misc{larsson2022,
author = {Johan Larsson and Jonas Wallin},
publisher = {arXiv},
title = {The {Hessian} Screening Rule},
number = {arXiv:2104.13026},
date = {2022-10-04},
url = {http://arxiv.org/abs/2104.13026},
doi = {10.48550/arXiv.2104.13026},
langid = {en},
abstract = {Predictor screening rules, which discard predictors from
the design matrix before fitting a model, have had considerable
impact on the speed with which l1-regularized regression problems,
such as the lasso, can be solved. Current state-of-the-art screening
rules, however, have difficulties in dealing with highly-correlated
predictors, often becoming too conservative. In this paper, we
present a new screening rule to deal with this issue: the Hessian
Screening Rule. The rule uses second-order information from the
model to provide more accurate screening as well as higher-quality
warm starts. The proposed rule outperforms all studied alternatives
on data sets with high correlation for both l1-regularized
least-squares (the lasso) and logistic regression. It also performs
best overall on the real data sets that we examine.}
}