Choosing parameters for regularized regression

Along with some colleagues, I’m working on an automated parameter and feature selection tuner for machine learning. Currently, we’re searching for ranges of parameters that have been “successful.” I recently did a deep dive into LASSO, RIDGE, or Elastic-Net regularized regression, where you have to pick a lambda value (or a range of them to search over). There are some recommendations on Data Science blogs (like Jason Brownlee’s) on parameter ranges to search.

However, some of the creators of these methods wrote an R package called “glmnet” that picks the range of parameters based on the data you’re working with. If you aren’t using R, though, they describe the method in Section 2.5 of this paper. AND, there’s a great stack overflow discussion. Pay close attention to that second-most-upvoted answer if you’re doing logistic regression.