Model Selection

Estimators that derive from _OrthoLearner fit first stage nuisance models on different folds of the data and then fit a final model. In many cases it will make sense to perform model selection over a number of first-stage models, and the library facilitates this by allowing a flexible specification of the first-stage models, as any of the following:

• An sklearn-compatible estimator

  • If the estimator is a known class that performs its own hyperparameter selection via cross-validation (such as LassoCV), then this will be done once and then the selected hyperparameters will be used when cross-fitting on each fold

  • If a custom class is used, then it should support a fit method and either a predict method if the target is continuous or predict_proba if the target is discrete.

• One of the following strings; the exact set of models supported by each of these keywords may vary depending on the version of our package:

  "linear"

  Selects over linear models regularized by L1 or L2 norm

  "poly"

  Selects over regularized linear models with polynomial features of different degrees

  "forest"

  Selects over random forest models

  "gbf"

  Selects over gradient boosting models

  "nnet"

  Selects over neural network models

  "automl"

  Selects over all of the above (note that this will be potentially time consuming)

• A list of any of the above

• An implementation of ModelSelector, which is a class that supports a two-stage model selection and fitting process (this is used internally by our library and is not generally intended to be used directly by end users).

Most subclasses also use the string “auto”` as a special default value to automatically select a model from an appropriate smaller subset of models than would be generated by “automl”.