Targeting Predictors in Random Forest Regression

TY - JOUR

T1 - Targeting Predictors in Random Forest Regression

AU - Andersen, Daniel Borup

AU - Christensen, Bent Jesper

AU - Mühlbach, Nicolaj Adam Søndergaard

AU - Nielsen, Mikkel Slot

PY - 2023/4

Y1 - 2023/4

N2 - Random forest regression (RF) is an extremely popular tool for the analysis of highdimensional data. Nonetheless, its benefits may be lessened in sparse settings due to weak predictors, and a pre-estimation dimension reduction (targeting) step is required. We show that proper targeting controls the probability of placing splits along strong predictors, thus providing an important complement to RF’s feature sampling. This is supported by simulations using representative finite samples. Moreover, we quantify the immediate gain from targeting in terms of increased strength of individual trees. Macroeconomic and financial applications show that the bias-variance trade-off implied by targeting, due to increased correlation among trees in the forest, is balanced at a medium degree of targeting, selecting the best 5–30% of commonly applied predictors.Improvements in predictive accuracy of targeted RF relative to ordinary RF are considerable, up to 21%, occurring both in recessions and expansions, particularly at long horizons.

AB - Random forest regression (RF) is an extremely popular tool for the analysis of highdimensional data. Nonetheless, its benefits may be lessened in sparse settings due to weak predictors, and a pre-estimation dimension reduction (targeting) step is required. We show that proper targeting controls the probability of placing splits along strong predictors, thus providing an important complement to RF’s feature sampling. This is supported by simulations using representative finite samples. Moreover, we quantify the immediate gain from targeting in terms of increased strength of individual trees. Macroeconomic and financial applications show that the bias-variance trade-off implied by targeting, due to increased correlation among trees in the forest, is balanced at a medium degree of targeting, selecting the best 5–30% of commonly applied predictors.Improvements in predictive accuracy of targeted RF relative to ordinary RF are considerable, up to 21%, occurring both in recessions and expansions, particularly at long horizons.

KW - Random forests

KW - Targeted predictors

KW - High-dimensional forecasting

KW - Weak predictors

KW - Variable selection

U2 - 10.1016/j.ijforecast.2022.02.010

DO - 10.1016/j.ijforecast.2022.02.010

M3 - Journal article

SN - 0169-2070

VL - 39

SP - 841

EP - 868

JO - International Journal of Forecasting

JF - International Journal of Forecasting

IS - 2

ER -