The Hessian Screening Rule

Lasso
Screening Rules
Authors

Johan Larsson

Jonas Wallin

Published

4 October 2022

Details

arXiv, arXiv:2104.13026

Links
Abstract

Predictor screening rules, which discard predictors from the design matrix before fitting a model, have had considerable impact on the speed with which l1-regularized regression problems, such as the lasso, can be solved. Current state-of-the-art screening rules, however, have difficulties in dealing with highly-correlated predictors, often becoming too conservative. In this paper, we present a new screening rule to deal with this issue: the Hessian Screening Rule. The rule uses second-order information from the model to provide more accurate screening as well as higher-quality warm starts. The proposed rule outperforms all studied alternatives on data sets with high correlation for both l1-regularized least-squares (the lasso) and logistic regression. It also performs best overall on the real data sets that we examine.

 

Citation

BibTeX citation:
@misc{larsson2022,
  author = {Johan Larsson and Jonas Wallin},
  publisher = {arXiv},
  title = {The {Hessian} Screening Rule},
  number = {arXiv:2104.13026},
  date = {2022-10-04},
  url = {http://arxiv.org/abs/2104.13026},
  doi = {10.48550/arXiv.2104.13026},
  langid = {en},
  abstract = {Predictor screening rules, which discard predictors from
    the design matrix before fitting a model, have had considerable
    impact on the speed with which l1-regularized regression problems,
    such as the lasso, can be solved. Current state-of-the-art screening
    rules, however, have difficulties in dealing with highly-correlated
    predictors, often becoming too conservative. In this paper, we
    present a new screening rule to deal with this issue: the Hessian
    Screening Rule. The rule uses second-order information from the
    model to provide more accurate screening as well as higher-quality
    warm starts. The proposed rule outperforms all studied alternatives
    on data sets with high correlation for both l1-regularized
    least-squares (the lasso) and logistic regression. It also performs
    best overall on the real data sets that we examine.}
}
For attribution, please cite this work as:
Johan Larsson, and Jonas Wallin. 2022. “The Hessian Screening Rule.” arXiv. https://doi.org/10.48550/arXiv.2104.13026.