econml.score.RScorer
- class econml.score.RScorer(*, model_y, model_t, discrete_treatment=False, categories='auto', cv=2, mc_iters=None, mc_agg='mean', random_state=None)[source]
Bases:
objectScorer based on the RLearner loss. Fits residual models at fit time and calculates residuals of the evaluation data in a cross-fitting manner:
Yres = Y - E[Y|X, W] Tres = T - E[T|X, W]
Then for any given cate model calculates the loss:
loss(cate) = E_n[(Yres - <cate(X), Tres>)^2]
Also calculates a baseline loss based on a constant treatment effect model, i.e.:
base_loss = min_{theta} E_n[(Yres - <theta, Tres>)^2]
Returns an analogue of the R-square score for regression:
score = 1 - loss(cate) / base_loss
This corresponds to the extra variance of the outcome explained by introducing heterogeneity in the effect as captured by the cate model, as opposed to always predicting a constant effect. A negative score, means that the cate model performs even worse than a constant effect model and hints at overfitting during training of the cate model.
This method was also advocated in recent work of [Schuleretal2018] when compared among several alternatives for causal model selection and introduced in the work of [NieWager2017].
- Parameters
model_y (estimator) – The estimator for fitting the response to the features. Must implement fit and predict methods.
model_t (estimator) – The estimator for fitting the treatment to the features. Must implement fit and predict methods.
discrete_treatment (bool, default
False) – Whether the treatment values should be treated as categorical, rather than continuous, quantitiescategories (‘auto’ or list, default ‘auto’) – The categories to use when encoding discrete treatments (or ‘auto’ to use the unique sorted values). The first category will be treated as the control treatment.
cv (int, cross-validation generator or an iterable, default 2) – Determines the cross-validation splitting strategy. Possible inputs for cv are:
None, to use the default 3-fold cross-validation,
integer, to specify the number of folds.
An iterable yielding (train, test) splits as arrays of indices.
For integer/None inputs, if the treatment is discrete
StratifiedKFoldis used, else,KFoldis used (with a random shuffle in either case).Unless an iterable is used, we call split(concat[W, X], T) to generate the splits. If all W, X are None, then we call split(ones((T.shape[0], 1)), T).
mc_iters (int, optional) – The number of times to rerun the first stage models to reduce the variance of the nuisances.
mc_agg ({‘mean’, ‘median’}, default ‘mean’) – How to aggregate the nuisance value for each sample across the mc_iters monte carlo iterations of cross-fitting.
random_state (int, RandomState instance, or None, default None) – If int, random_state is the seed used by the random number generator; If
RandomStateinstance, random_state is the random number generator; If None, the random number generator is theRandomStateinstance used bynp.random.
References
- NieWager2017
X. Nie and S. Wager. Quasi-Oracle Estimation of Heterogeneous Treatment Effects. arXiv preprint arXiv:1712.04912, 2017. https://arxiv.org/pdf/1712.04912.pdf
- Schuleretal2018
Alejandro Schuler, Michael Baiocchi, Robert Tibshirani, Nigam Shah. “A comparison of methods for model selection when estimating individual treatment effects.” Arxiv, 2018 https://arxiv.org/pdf/1804.05146.pdf
- __init__(*, model_y, model_t, discrete_treatment=False, categories='auto', cv=2, mc_iters=None, mc_agg='mean', random_state=None)[source]
Methods
__init__(*, model_y, model_t[, ...])best_model(cate_models[, return_scores])Chooses the best among a list of models
ensemble(cate_models[, eta, return_scores])Ensembles a list of models based on their performance
fit(y, T[, X, W, sample_weight, groups])- Parameters
Y ((n × d_y) matrix or vector of length n) -- Outcomes for each sample
score(cate_model)- Parameters
cate_model (instance of fitted BaseCateEstimator)
- best_model(cate_models, return_scores=False)[source]
Chooses the best among a list of models
- Parameters
cate_models (list of instance of fitted BaseCateEstimator)
return_scores (bool, default False) – Whether to return the list scores of each model
- Returns
best_model (instance of fitted BaseCateEstimator) – The model that achieves the best score
best_score (double) – The score of the best model
scores (list of double) – The list of scores for each of the input models. Returned only if return_scores=True.
- ensemble(cate_models, eta=1000.0, return_scores=False)[source]
Ensembles a list of models based on their performance
- Parameters
cate_models (list of instance of fitted BaseCateEstimator)
eta (double, default 1000) – The soft-max parameter for the ensemble
return_scores (bool, default False) – Whether to return the list scores of each model
- Returns
ensemble_model (instance of fitted EnsembleCateEstimator) – A fitted ensemble cate model that calculates effects based on a weighted version of the input cate models, weighted by a softmax of their score performance
ensemble_score (double) – The score of the ensemble model
scores (list of double) – The list of scores for each of the input models. Returned only if return_scores=True.
- fit(y, T, X=None, W=None, sample_weight=None, groups=None)[source]
- Parameters
Y ((n × d_y) matrix or vector of length n) – Outcomes for each sample
T ((n × dₜ) matrix or vector of length n) – Treatments for each sample
X ((n × dₓ) matrix, optional) – Features for each sample
W ((n × d_w) matrix, optional) – Controls for each sample
sample_weight ((n,) vector, optional) – Weights for each row
groups ((n,) vector, optional) – All rows corresponding to the same group will be kept together during splitting. If groups is not None, the cv argument passed to this class’s initializer must support a ‘groups’ argument to its split method.
- Return type
self