
Methods for developing and validating CATE scores
November 20, 2026
Source:vignettes/Theoretical-details.Rmd
Theoretical-details.RmdThis vignette introduces conditional average treatment effects and CATE scores for binary, count and survival outcomes. The detailed derivations below focus on count and survival outcomes; the binary CATE example illustrates binary score fitting and validation. For population-average effects, see Methods for estimating average treatment effects.
What is a CATE score?
A conditional average treatment effect (CATE) compares outcomes under
two treatments among patients with the same measured baseline
covariates. For a binary outcome, precmed uses the
conditional risk difference: predicted risk under treatment 1 minus
predicted risk under treatment 0. For a count outcome, the score is a
conditional log rate ratio. For survival, it is a conditional log ratio
of restricted mean time lost up to the chosen horizon. Ratios compare
treatment 1 with treatment 0. These are averages conditional on measured
characteristics, not the unobservable effect for an individual
patient.
A CATE score is an estimate of such a conditional contrast. It can be used to rank patients, with the favorable direction depending on the outcome and the treatment contrast. A causal interpretation requires assumptions such as consistency, no unmeasured treatment–outcome confounding given the measured covariates, and adequate treatment overlap, as well as the assumptions of the chosen estimator. Survival analyses also require appropriate censoring assumptions and support at the selected horizon.
How are the scores fitted?
A T-learner fits an outcome model separately in each
treatment arm and contrasts their predictions at the same covariate
values. For a binary outcome, separate logistic outcome models give a
fitted risk-difference score by subtracting the predicted event risk
under treatment 0 from that under treatment 1. This score uses the
outcome predictions alone. For binary outcomes, catefit()
combines outcome predictions and propensity scores from the treatment
model to obtain doubly robust estimates of the average risk difference
within each score-defined subgroup.
Use outcome_model to specify baseline covariates and
terms that let each arm-specific model adequately estimate the
conditional outcome and capture variation in the treatment contrast. For
binary outcomes, the risk difference can vary with baseline risk even
when the treatment odds ratio is constant (Hoogland et al. 2021). A causal interpretation
of the T-learner score in observational data requires consistency,
treatment overlap, and a set of covariates in the outcome models that
suffices to control treatment–outcome confounding. Chapter 17 discusses
CATE prediction and its identification assumptions (Hoogland et al. 2026). The fitted scores are
fixed before subgroup estimation, so propensity-score adjustment at that
stage cannot repair bias in the scores due to omitted effect modifiers,
misspecified outcome models, or unmeasured confounding.
An S-learner, by contrast, fits a single outcome
model that includes treatment as a predictor, and compares predictions
under the two treatment values. The S-learner is described here for
context; it is not a selectable CATE score method in the current
precmed interface.
The package also implements two-regression and contrast-regression scores. These use cross-fitted initial outcome predictions and a treatment model that estimates propensity scores, followed by different procedures for estimating the score. Their doubly robust properties depend on the estimator’s assumptions and are not the same as a guarantee for a direct T-learner score. For survival outcomes, the censoring model supplies probabilities used for IPCW; treatment-model double robustness does not protect against a misspecified censoring model. The count and survival sections below describe the methods in more detail.
Why validate a score?
catefit() fits scores and estimates effects in
score-defined subgroups using the same data. Those subgroup estimates
are useful for exploring a fit, but they do not independently validate
it. catecv() fits scores on training splits and evaluates
score-defined groups on held-out splits, providing an internal
assessment. Comparing or selecting methods using those results still
benefits from an independent assessment. See the validation section
below and the binary, count and survival CATE examples for the available
procedures.
Background and further reading
A CATE describes the average treatment effect among patients with the same measured baseline characteristics; it does not identify an individual patient’s unobservable treatment effect. Chapter 17 of Debray et al. (2026) introduces the principles of treatment effect prediction, model development in randomized and non-randomized data, and the assumptions needed to identify CATEs (Hoogland et al. 2026).
Chapter 5 introduces marginal and conditional causal effects (Nguyen et al. 2026).
The primary methodological reference for the two-regression and contrast-regression estimators is Yadlowsky et al. (2021). For practical guidance on interpreting and evaluating the resulting scores, the book provides two complementary chapters:
- Chapter 19: Validation of prediction models for patient outcomes and individualized treatment effect discusses validation strategies, discrimination, calibration, and decision accuracy (Thao et al. 2026).
-
Chapter 20: Visualization and interpretation of
individualized treatment rule results discusses treatment
rules, value functions, validation curves, and responder diagnostics,
including examples using
precmed(Jiang et al. 2026).
Development of CATE scores for count outcomes
For count outcomes, the CATE score is expressed as a log-rate ratio conditional on baseline covariates. It can be estimated with boosting, Poisson regression, negative binomial regression, or the doubly robust two-regression and contrast-regression estimators (Yadlowsky et al. 2021). The two-regression approach models the potential outcomes separately by treatment group, whereas contrast regression directly models the treatment contrast.
Assume that the following data are recorded for each of observations:
- is a binary treatment taking value 0 or 1.
- is a vector of baseline covariates.
- is a count outcome.
- is the exposure time during which is recorded. It can vary across observations.
The objective is to estimate the ratio-based CATE score defined as
where is the potential outcome if the patient received the treatment . is interpreted as the individualized log-rate ratio of treatment 1 over treatment 0 conditional on the baseline covariates.
The package offers 5 methods to estimate the CATE score: Poisson regression, negative binomial regression, boosting, two regressions, and contrast regression.
Poisson
Estimate the conditional mean outcome given baseline covariates and log-transformed exposure time as the offset separately in each treatment group (i.e., for where is the with an intercept) with Poisson regression. Denote the prediction for one time unit as .
The CATE score with Poisson is the plug-in estimator
Negative binomial
Follow the same step as in Poisson but replace Poisson regression with negative binomial regression in step 1.
Boosting
Gradient boosting builds an additive prediction model using sequentially fitted learners (Friedman 2001).
-
Estimate the conditional mean outcome given baseline covariates and log-transformed exposure time as the offset separately in each treatment group (i.e., for ) with Poisson-based gradient boosting regression method. Denote the prediction for one time unit as .
- The base learners are regression trees with depth specified with the
argument
tree.depthincatecvcount()andcatefitcount(). Default is 2. - The number of trees in boosting is selected via cross-validation
with a maximum number of trees specificed with the argument
n.trees.boostingincatecvcount()andcatefitcount(). Default is 200.
- The base learners are regression trees with depth specified with the
argument
The CATE score with boosting is the plug-in estimator
Two regressions
The procedure adapts the ratio-based estimator of Yadlowsky et al. (2021) to count outcomes. Split the observations into treatment-stratified folds . For each fold, fit the initial outcome and treatment models on and predict the held-out observations in . Pool these out-of-fold predictions, denoting each patient’s arm-specific predicted rate by and propensity for the observed arm by .
For each arm , fit a weighted Poisson regression of the observed rate on and , with weight . Predict the resulting mean for every observation, and fit a second unweighted Poisson regression of on . Denote the resulting coefficients by . The score is
Repeat the procedure times with new random fold assignments and average the coefficient differences. These are repeated cross-fitting splits, not bootstrap samples.
Contrast regression
The contrast estimator uses the same held-out predictions and repeated fold assignments (Yadlowsky et al. 2021). Write , for the out-of-fold propensity for treatment 1, and for the out-of-fold rate predictions. The fitted score solves
The final score is . The coefficients are averaged across converged repetitions; if none converged, the coefficients from all repetitions are averaged. The equation is solved by Newton–Raphson, with numerical minimization as a fallback.
Development of CATE scores for survival outcomes
For survival outcomes, the CATE score is expressed as a log-ratio of restricted mean time lost conditional on baseline covariates. It can be estimated with random forest, boosting, Poisson regression, or adaptations of the doubly robust two-regression and contrast-regression approaches (Yadlowsky et al. 2021). Chapter 20 of the book also discusses restricted mean time lost as an outcome for ratio-based treatment effects (Jiang et al. 2026). For restricted mean event time regression using inverse probability of censoring weighting (IPCW), see Tian et al. (2014); the procedures below adapt these ideas to the RMTL treatment-effect setting.
Assume that the following data are recorded for each of observations:
- is a binary treatment taking value 0 or 1.
- is a vector of baseline covariates.
- is a survival time.
- is a censoring time.
- is the minimum between and .
- is an indicator taking value 1 if and 0 otherwise.
- is a truncation time for restricted mean time lost.
The objective is to estimate the ratio-based CATE score defined as
where is the potential survival time if the patient received the treatment . is interpreted as the individualized logarithm restricted mean time lost (RMTL) ratio of treatment 1 over treatment 0 conditional on the baseline covariates.
The package offers 5 methods to estimate the CATE score: Poisson regression, random forest, boosting, two regressions, and contrast regression.
Poisson
Estimate the conditional mean RMTL given baseline covariates separately in each treatment group (i.e., for where is the with an intercept) with Poisson regression weighted with IPCW. Denote the prediction as .
The CATE score with Poisson is the plug-in estimator
Boosting
Within each treatment arm, the implementation fits IPCW-weighted
Gaussian gradient boosting to the restricted survival time divided by
,
followed by a weighted binomial GAM calibration of the predictions and
covariates. It converts the resulting predicted restricted mean survival
time to RMTL and takes the log ratio. Gradient boosting is described by
Friedman (2001). tree.depth
controls the boosting trees; n.trees.boosting is the
maximum number of trees considered for cross-validation.
Random forest
Random survival forests extend random forests to right-censored data (Ishwaran et al. 2008). A forest is fitted separately in each treatment arm. The predicted survival curve is integrated to to estimate RMST; subtracting that integral from gives predicted RMTL . The score is
n.trees.rf specifies the number of trees (default
1,000); it is not selected by cross-validation. Optional forest settings
can be supplied through rf.control, for example
list(mtry = 3, nodesize = 15). tree.depth
applies to boosting; use rf.control$nodedepth to set a
forest depth limit.
Two regressions
This procedure adapts the ratio-based two-regression approach of Yadlowsky et al. (2021) to survival outcomes, using IPCW for censoring (Tian et al. 2014). Let and . The censoring model is fitted on the full data used for the fit. The weight is
normalized to have mean one, where
is the estimated probability of remaining uncensored. In
catefit(), the censoring model defaults to the
outcome-model covariates and treatment. Use censoring_model
to specify a different formula.
Split the data into treatment-stratified folds. Fit arm-specific initial survival outcome models and a treatment model on each fold’s complement and predict its held-out observations. The initial model may use random survival forests, weighted Gaussian boosting followed by GAM calibration, or weighted logistic regression. Pool the out-of-fold predictions, cap predicted restricted survival time at , and convert it to predicted RMTL .
For each arm , fit a Poisson regression of on and , weighted by , where is the out-of-fold propensity for that arm. Predict the fitted means for all observations and regress them on with an unweighted Poisson regression. Denoting the resulting coefficients by , the score is .
The process is repeated for random fold assignments and the coefficient differences are averaged. These are repeated cross-fitting splits, not bootstrap samples.
Contrast regression
Use the same IPCW and out-of-fold predictions as above. The contrast-regression approach adapts Yadlowsky et al. (2021) to censored survival outcomes. Let , let denote the predicted RMTL divided by , and write . With the out-of-fold propensity for treatment 1, the implementation solves
The final score is . The coefficients are averaged across converged repetitions; if none converged, the coefficients from all repetitions are averaged. Newton–Raphson is used with numerical minimization as a fallback.
Validation of CATE scores
The following validation curves and ABC statistics apply to count and
survival CATE scores supported by catecv(). They assess
ranking of treatment benefit and are distinct from the bootstrap
inference described in the ATE methods
vignette.
Count outcomes
The ABC statistic summarizes the area between the validation curve and the overall ATE reference line (Zhao et al. 2013; Yadlowsky et al. 2021). Chapter 20 provides worked examples of these curves and their interpretation as responder diagnostics (Jiang et al. 2026). Validation curves and ABC assess how well a score ranks patients by treatment benefit. They do not, by themselves, establish calibration of the predicted CATEs or the clinical value of a treatment rule; Chapter 19 discusses these complementary aspects of validation (Thao et al. 2026).
For a single CV iteration, it is implemented in the training and validation sets separately as following:
Step 1. Calculate the ATE in the training or validation sets.
Step 2. Calculate the ATE in 100 nested subgroups
based on the estimated CATE score and derive the corresponding
validation curve. Subgroups are defined with 100 equally-spaced
proportions from min(prop.cutoff) to
max(prop.cutoff) to ensure that enough data points are
available to build the validation curve.
Step 3. The ABC is calculated with
auc() from utility.R using the natural cubic
spline interpolation, which calculates the area between the horizontal
line with y-intercept at the ATE calculated in step 1
over the range
[min(prop.cutoff),max(prop.cutoff)]
(y) and the validation curve calculated in step
2 (x).
The function plot() allows the user to combine
validation curves from 2 or more CV iterations (i.e.,
cv.n > 1). There are 2 ways to combine the validation
curves:
The option
combine=“median” takes the median of the ATEs across all CV iterations in step 1 and the median of the ATEs in the 100 nested subgroups in step 2.The option
combine=“mean” takes the mean of the ATEs across all CV iterations in step 1 and the mean of the ATEs in the 100 nested subgroups in step 2.
In either case, the ABC calculations are carried out as in step 3 with the resulting x and y.
The figure below explains how the ABC is calculated with a simple schema. The ABC calculations are such that larger positive ABC values always indicate more treatment effect heterogeneity. This is implemented by considering separately the cases when larger or smaller outcomes are preferred.
If larger count outcomes are preferred (
higher.y = TRUE, e.g., positive events like number of motor developmental milestones reached), a validation curve above the ATE line and decreasing towards that line is synonym with treatment effect heterogeneity (top left figure). However, sections of the validation curve below the ATE line indicate inability to capture treatment effect heterogeneity (bottom left figure). Hence, the ABC is defined by subtracting the areas below the ATE line (red) from the areas above the ATE line (green) such that larger positive ABC are preferred.If smaller count outcomes are preferred (
higher.y = FALSE, e.g., negative events like number of relapses), a validation curve below the ATE line and increasing towards that line is synonym with treatment effect heterogeneity (top right figure). However, sections of the validation curve above the ATE line indicate inability to capture treatment effect heterogeneity (bottom right figure). Hence, the ABC is defined by subtracting the areas above the ATE line (red) from the areas below the ATE line (green) such that larger positive ABC are preferred.

ABC calculation examples in relation with higher.y
argument in catecv() and catefit(). Validation
curves are represented with a blue line and the dashed line is the
ATE
Survival outcomes
The ABC statistic summarizes the area between the validation curve and the overall ATE reference line (Zhao et al. 2013; Yadlowsky et al. 2021). Chapter 20 provides worked examples of these curves and their interpretation as responder diagnostics (Jiang et al. 2026). For a single CV iteration and a certain CATE score method, it is implemented as following in the training and validation sets separately:
Step 1. Calculate the ATE in the training or validation sets.
Step 2. Calculate the ATE in 100 nested subgroups
and derive the corresponding validation curve. Subgroups are defined
with 100 equally-spaced proportions from min(prop.cutoff)
to max(prop.cutoff) to ensure that enough data points are
available to build the validation curve.
Step 3. The ABC is calculated with
auc() from utility.R using the natural cubic
spline interpolation, which calculates the area between the horizontal
line with y-intercept at the ATE calculated in step 1
(y) and the validation curve calculated in step
2 (x) over the range [min(prop.cutoff),
max(prop.cutoff)].
The function plot() allows the user combining validation
curves from 2 or more CV iterations (i.e., cv.n > 1).
There are 2 ways to combine the validation curves:
The option
combine ="median"takes the median of the ATEs across all CV iterations in step 1 and the median of the ATEs in the 100 nested subgroups in step 2.The option
combine ="mean"takes the mean of the ATEs across all CV iterations in step 1 and the mean of the ATEs in the 100 nested subgroups in step 2.
In either case, the ABC calculations are carried out as in step 3 with the resulting x and y.
The figure below explains how the ABC is calculated with a simple schema. The ABC calculations are such that larger positive ABC values always indicate more treatment effect heterogeneity. This is implemented by considering separately the cases when larger or smaller outcomes are preferred.
If smaller survival outcomes are preferred (
higher.y = FALSE, e.g., time to a positive event like recovery), a validation curve above the ATE line which decreases towards it is synonym with treatment effect heterogeneity (top left figure). However, sections of the validation curve below the ATE line indicate inability to capture treatment effect heterogeneity (bottom left figure). Hence, the ABC is defined by subtracting the areas below the ATE line from the areas above the ATE line such that larger positive ABC are preferred.If larger survival outcomes are preferred (
higher.y = TRUE, e.g., time to a negative event like symptom onset), a validation curve (in blue) below the ATE line (dashed) which increases towards it is synonym with treatment effect heterogeneity (top right figure). However, sections of the validation curve above the ATE line indicate inability to capture treatment effect heterogeneity (bottom right figure). Hence, the ABC is defined by subtracting the areas above the ATE line (in red) from the areas below the ATE line (in green) such that larger positive ABC are preferred.

ABC calculation examples in relation with higher.y
argument in catecv() and catefit(). Validation
curves are represented with a blue line and the dashed line is the
ATE.
