GLM with Elastic Net Regularization Classification Learner
Source:R/LearnerClassifCVGlmnet.R
mlr_learners_classif.cv_glmnet.RdGeneralized linear models with elastic net regularization.
Calls glmnet::cv.glmnet() from package glmnet.
The default for hyperparameter family is set to "binomial" or "multinomial",
depending on the number of classes.
Offset
If a Task contains a column with the offset role, it is automatically incorporated during training via the offset argument in glmnet::glmnet().
During prediction, the offset column from the test set is used only if use_pred_offset = TRUE (default), passed via the newoffset argument in glmnet::predict.glmnet().
Otherwise, if the user sets use_pred_offset = FALSE, a zero offset is applied, effectively disabling the offset adjustment during prediction.
Dictionary
This mlr3::Learner can be instantiated via the dictionary mlr3::mlr_learners or with the associated sugar function mlr3::lrn():
Meta Information
Task type: “classif”
Predict Types: “response”, “prob”
Feature Types: “logical”, “integer”, “numeric”
Required Packages: mlr3, mlr3learners, glmnet
Parameters
| Id | Type | Default | Levels | Range |
| alignment | character | lambda | lambda, fraction | - |
| alpha | numeric | 1 | \([0, 1]\) | |
| big | numeric | 9.9e+35 | \((-\infty, \infty)\) | |
| devmax | numeric | 0.999 | \([0, 1]\) | |
| dfmax | integer | - | \([0, \infty)\) | |
| epsnr | numeric | 1e-08 | \([0, 1]\) | |
| eps | numeric | 1e-06 | \([0, 1]\) | |
| exclude | integer | - | \([1, \infty)\) | |
| exmx | numeric | 250 | \((-\infty, \infty)\) | |
| fdev | numeric | 1e-05 | \([0, 1]\) | |
| foldid | untyped | NULL | - | |
| gamma | untyped | - | - | |
| grouped | logical | TRUE | TRUE, FALSE | - |
| intercept | logical | TRUE | TRUE, FALSE | - |
| keep | logical | FALSE | TRUE, FALSE | - |
| lambda.min.ratio | numeric | - | \([0, 1]\) | |
| lambda | untyped | - | - | |
| lower.limits | untyped | - | - | |
| maxit | integer | 100000 | \([1, \infty)\) | |
| mnlam | integer | 5 | \([1, \infty)\) | |
| mxitnr | integer | 25 | \([1, \infty)\) | |
| mxit | integer | 100 | \([1, \infty)\) | |
| nfolds | integer | 10 | \([3, \infty)\) | |
| nlambda | integer | 100 | \([1, \infty)\) | |
| use_pred_offset | logical | TRUE | TRUE, FALSE | - |
| parallel | logical | FALSE | TRUE, FALSE | - |
| penalty.factor | untyped | - | - | |
| pmax | integer | - | \([0, \infty)\) | |
| pmin | numeric | 1e-09 | \([0, 1]\) | |
| prec | numeric | 1e-10 | \((-\infty, \infty)\) | |
| predict.gamma | numeric | gamma.1se | \((-\infty, \infty)\) | |
| relax | logical | FALSE | TRUE, FALSE | - |
| s | numeric | lambda.1se | \([0, \infty)\) | |
| standardize | logical | TRUE | TRUE, FALSE | - |
| standardize.response | logical | FALSE | TRUE, FALSE | - |
| thresh | numeric | 1e-07 | \([0, \infty)\) | |
| trace.it | integer | 0 | \([0, 1]\) | |
| type.gaussian | character | - | covariance, naive | - |
| type.logistic | character | - | Newton, modified.Newton | - |
| type.measure | character | deviance | deviance, class, auc, mse, mae | - |
| type.multinomial | character | - | ungrouped, grouped | - |
| upper.limits | untyped | - | - |
Internal Encoding
Starting with mlr3 v0.5.0, the order of class labels is reversed prior to
model fitting to comply to the stats::glm() convention that the negative class is provided
as the first factor level.
References
Friedman J, Hastie T, Tibshirani R (2010). “Regularization Paths for Generalized Linear Models via Coordinate Descent.” Journal of Statistical Software, 33(1), 1–22. doi:10.18637/jss.v033.i01 .
See also
Chapter in the mlr3book: https://mlr3book.mlr-org.com/chapters/chapter2/data_and_basic_modeling.html#sec-learners
Package mlr3extralearners for more learners.
as.data.table(mlr_learners)for a table of available Learners in the running session (depending on the loaded packages).mlr3pipelines to combine learners with pre- and postprocessing steps.
Extension packages for additional task types:
mlr3proba for probabilistic supervised regression and survival analysis.
mlr3cluster for unsupervised clustering.
mlr3tuning for tuning of hyperparameters, mlr3tuningspaces for established default tuning spaces.
Other Learner:
mlr_learners_classif.glmnet,
mlr_learners_classif.kknn,
mlr_learners_classif.lda,
mlr_learners_classif.log_reg,
mlr_learners_classif.multinom,
mlr_learners_classif.naive_bayes,
mlr_learners_classif.nnet,
mlr_learners_classif.qda,
mlr_learners_classif.ranger,
mlr_learners_classif.svm,
mlr_learners_classif.xgboost,
mlr_learners_regr.cv_glmnet,
mlr_learners_regr.glmnet,
mlr_learners_regr.kknn,
mlr_learners_regr.km,
mlr_learners_regr.lm,
mlr_learners_regr.nnet,
mlr_learners_regr.ranger,
mlr_learners_regr.svm,
mlr_learners_regr.xgboost
Super classes
mlr3::Learner -> mlr3::LearnerClassif -> LearnerClassifCVGlmnet
Methods
Method selected_features()
Returns the set of selected features as reported by glmnet::predict.glmnet()
with type set to "nonzero".
Arguments
lambda(
numeric(1))
Customlambda, defaults to the active lambda depending on parameter set.
Returns
(character()) of feature names.
Examples
if (requireNamespace("glmnet", quietly = TRUE)) {
# Define the Learner and set parameter values
learner = lrn("classif.cv_glmnet")
print(learner)
# Define a Task
task = tsk("sonar")
# Create train and test set
ids = partition(task)
# Train the learner on the training ids
learner$train(task, row_ids = ids$train)
# print the model
print(learner$model)
# importance method
if("importance" %in% learner$properties) print(learner$importance)
# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)
# Score the predictions
predictions$score()
}
#> <LearnerClassifCVGlmnet:classif.cv_glmnet>: GLM with Elastic Net Regularization
#> * Model: -
#> * Parameters: use_pred_offset=TRUE
#> * Packages: mlr3, mlr3learners, glmnet
#> * Predict Types: [response], prob
#> * Feature Types: logical, integer, numeric
#> * Properties: multiclass, offset, selected_features, twoclass, weights
#>
#> Call: (if (cv) glmnet::cv.glmnet else glmnet::glmnet)(x = data, y = target, family = "binomial")
#>
#> Measure: Binomial Deviance
#>
#> Lambda Index Measure SE Nonzero
#> min 0.03602 22 0.9852 0.09482 15
#> 1se 0.08322 13 1.0594 0.05488 6
#> classif.ce
#> 0.2753623