GLM with Elastic Net Regularization Regression Learner
Source:R/LearnerRegrGlmnet.R
mlr_learners_regr.glmnet.RdGeneralized linear models with elastic net regularization.
Calls glmnet::glmnet() from package glmnet.
The default for hyperparameter family is set to "gaussian".
Details
Caution: This learner is different to learners calling glmnet::cv.glmnet()
in that it does not use the internal optimization of parameter lambda.
Instead, lambda needs to be tuned by the user (e.g., via mlr3tuning).
When lambda is tuned, the glmnet will be trained for each tuning iteration.
While fitting the whole path of lambdas would be more efficient, as is done
by default in glmnet::glmnet(), tuning/selecting the parameter at prediction time
(using parameter s) is currently not supported in mlr3
(at least not in efficient manner).
Tuning the s parameter is, therefore, currently discouraged.
When the data are i.i.d. and efficiency is key, we recommend using the respective
auto-tuning counterparts in mlr_learners_classif.cv_glmnet() or
mlr_learners_regr.cv_glmnet().
However, in some situations this is not applicable, usually when data are
imbalanced or not i.i.d. (longitudinal, time-series) and tuning requires
custom resampling strategies (blocked design, stratification).
Dictionary
This mlr3::Learner can be instantiated via the dictionary mlr3::mlr_learners or with the associated sugar function mlr3::lrn():
Meta Information
Task type: “regr”
Predict Types: “response”
Feature Types: “logical”, “integer”, “numeric”
Required Packages: mlr3, mlr3learners, glmnet
Parameters
| Id | Type | Default | Levels | Range |
| alignment | character | lambda | lambda, fraction | - |
| alpha | numeric | 1 | \([0, 1]\) | |
| big | numeric | 9.9e+35 | \((-\infty, \infty)\) | |
| devmax | numeric | 0.999 | \([0, 1]\) | |
| dfmax | integer | - | \([0, \infty)\) | |
| eps | numeric | 1e-06 | \([0, 1]\) | |
| epsnr | numeric | 1e-08 | \([0, 1]\) | |
| exact | logical | FALSE | TRUE, FALSE | - |
| exclude | integer | - | \([1, \infty)\) | |
| exmx | numeric | 250 | \((-\infty, \infty)\) | |
| family | character | gaussian | gaussian, poisson | - |
| fdev | numeric | 1e-05 | \([0, 1]\) | |
| gamma | numeric | 1 | \((-\infty, \infty)\) | |
| grouped | logical | TRUE | TRUE, FALSE | - |
| intercept | logical | TRUE | TRUE, FALSE | - |
| keep | logical | FALSE | TRUE, FALSE | - |
| lambda | untyped | - | - | |
| lambda.min.ratio | numeric | - | \([0, 1]\) | |
| lower.limits | untyped | - | - | |
| maxit | integer | 100000 | \([1, \infty)\) | |
| mnlam | integer | 5 | \([1, \infty)\) | |
| mxit | integer | 100 | \([1, \infty)\) | |
| mxitnr | integer | 25 | \([1, \infty)\) | |
| newoffset | untyped | - | - | |
| nlambda | integer | 100 | \([1, \infty)\) | |
| offset | untyped | NULL | - | |
| parallel | logical | FALSE | TRUE, FALSE | - |
| penalty.factor | untyped | - | - | |
| pmax | integer | - | \([0, \infty)\) | |
| pmin | numeric | 1e-09 | \([0, 1]\) | |
| prec | numeric | 1e-10 | \((-\infty, \infty)\) | |
| relax | logical | FALSE | TRUE, FALSE | - |
| s | numeric | 0.01 | \([0, \infty)\) | |
| standardize | logical | TRUE | TRUE, FALSE | - |
| standardize.response | logical | FALSE | TRUE, FALSE | - |
| thresh | numeric | 1e-07 | \([0, \infty)\) | |
| trace.it | integer | 0 | \([0, 1]\) | |
| type.gaussian | character | - | covariance, naive | - |
| type.logistic | character | - | Newton, modified.Newton | - |
| type.multinomial | character | - | ungrouped, grouped | - |
| upper.limits | untyped | - | - |
References
Friedman J, Hastie T, Tibshirani R (2010). “Regularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software, 33(1), 1–22. doi:10.18637/jss.v033.i01 .
See also
Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter2/data_and_basic_modeling.html#sec-learners
Package mlr3extralearners for more learners.
as.data.table(mlr_learners)for a table of available Learners in the running session (depending on the loaded packages).mlr3pipelines to combine learners with pre- and postprocessing steps.
Extension packages for additional task types:
mlr3proba for probabilistic supervised regression and survival analysis.
mlr3cluster for unsupervised clustering.
mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.
Other Learner:
mlr_learners_classif.cv_glmnet,
mlr_learners_classif.glmnet,
mlr_learners_classif.kknn,
mlr_learners_classif.lda,
mlr_learners_classif.log_reg,
mlr_learners_classif.multinom,
mlr_learners_classif.naive_bayes,
mlr_learners_classif.nnet,
mlr_learners_classif.qda,
mlr_learners_classif.ranger,
mlr_learners_classif.svm,
mlr_learners_classif.xgboost,
mlr_learners_regr.cv_glmnet,
mlr_learners_regr.kknn,
mlr_learners_regr.km,
mlr_learners_regr.lm,
mlr_learners_regr.nnet,
mlr_learners_regr.ranger,
mlr_learners_regr.svm,
mlr_learners_regr.xgboost
Super classes
mlr3::Learner -> mlr3::LearnerRegr -> LearnerRegrGlmnet
Methods
Method selected_features()
Returns the set of selected features as reported by glmnet::predict.glmnet()
with type set to "nonzero".
Arguments
lambda(
numeric(1))
Customlambda, defaults to the active lambda depending on parameter set.
Returns
(character()) of feature names.
Examples
if (requireNamespace("glmnet", quietly = TRUE)) {
# Define the Learner and set parameter values
learner = lrn("regr.glmnet")
print(learner)
# Define a Task
task = tsk("mtcars")
# Create train and test set
ids = partition(task)
# Train the learner on the training ids
learner$train(task, row_ids = ids$train)
# print the model
print(learner$model)
# importance method
if("importance" %in% learner$properties) print(learner$importance)
# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)
# Score the predictions
predictions$score()
}
#> <LearnerRegrGlmnet:regr.glmnet>: GLM with Elastic Net Regularization
#> * Model: -
#> * Parameters: family=gaussian
#> * Packages: mlr3, mlr3learners, glmnet
#> * Predict Types: [response]
#> * Feature Types: logical, integer, numeric
#> * Properties: weights
#>
#> Call: (if (cv) glmnet::cv.glmnet else glmnet::glmnet)(x = data, y = target, family = "gaussian")
#>
#> Df %Dev Lambda
#> 1 0 0.00 5.3800
#> 2 1 12.71 4.9020
#> 3 1 23.27 4.4670
#> 4 2 33.34 4.0700
#> 5 2 41.88 3.7080
#> 6 2 48.98 3.3790
#> 7 2 54.86 3.0790
#> 8 2 59.75 2.8050
#> 9 2 63.81 2.5560
#> 10 2 67.18 2.3290
#> 11 2 69.97 2.1220
#> 12 2 72.29 1.9330
#> 13 2 74.22 1.7620
#> 14 2 75.82 1.6050
#> 15 3 77.23 1.4630
#> 16 3 78.48 1.3330
#> 17 3 79.52 1.2140
#> 18 3 80.38 1.1060
#> 19 3 81.10 1.0080
#> 20 3 81.70 0.9185
#> 21 4 82.34 0.8369
#> 22 4 82.95 0.7626
#> 23 4 83.46 0.6948
#> 24 4 83.88 0.6331
#> 25 4 84.22 0.5769
#> 26 4 84.51 0.5256
#> 27 4 84.75 0.4789
#> 28 4 84.95 0.4364
#> 29 4 85.12 0.3976
#> 30 4 85.25 0.3623
#> 31 4 85.37 0.3301
#> 32 5 85.50 0.3008
#> 33 5 85.61 0.2741
#> 34 5 85.70 0.2497
#> 35 5 85.77 0.2275
#> 36 5 85.84 0.2073
#> 37 5 85.89 0.1889
#> 38 5 85.93 0.1721
#> 39 5 85.97 0.1568
#> 40 5 86.00 0.1429
#> 41 6 86.02 0.1302
#> 42 6 86.05 0.1186
#> 43 6 86.07 0.1081
#> 44 6 86.09 0.0985
#> 45 6 86.11 0.0897
#> 46 6 86.12 0.0818
#> 47 8 86.24 0.0745
#> 48 8 86.42 0.0679
#> 49 8 86.56 0.0619
#> 50 8 86.68 0.0564
#> 51 8 86.78 0.0514
#> 52 8 86.86 0.0468
#> 53 8 86.93 0.0426
#> 54 8 86.99 0.0389
#> 55 9 87.04 0.0354
#> 56 9 87.08 0.0322
#> 57 9 87.11 0.0294
#> 58 9 87.14 0.0268
#> 59 9 87.17 0.0244
#> 60 9 87.19 0.0222
#> 61 9 87.20 0.0203
#> 62 9 87.22 0.0185
#> 63 9 87.23 0.0168
#> 64 9 87.24 0.0153
#> 65 9 87.24 0.0140
#> 66 9 87.25 0.0127
#> 67 9 87.25 0.0116
#> 68 10 87.26 0.0106
#> 69 10 87.27 0.0096
#> 70 10 87.27 0.0088
#> 71 10 87.28 0.0080
#> 72 10 87.28 0.0073
#> 73 10 87.28 0.0066
#> 74 10 87.28 0.0060
#> 75 10 87.29 0.0055
#> 76 10 87.29 0.0050
#> Warning: Multiple lambdas have been fit. Lambda will be set to 0.01 (see parameter 's').
#> regr.mse
#> 10.7196