This work offers insight into the effectiveness of probabilistic models, specifically those based on ensemble approximations, in predicting adverse side effects following radiotherapy for prostate cancer. We trained a random forest model on radiomic features from 134 T2-weighted Magnetic Resonance (MRI) images of the prostate gland to identify patients experiencing acute or chronic rectal and urinary toxicity (AU-ROC ranging from 61.4% for endorectal coil acquisitions to 70.8% for the full dataset). We evaluated the reliability of the predictions using an ensemble approximation of simplified random forests obtained by an adaptive procedure of random subsampling of the training data. We used this reliability score to define a not-confident class and then recompute performance metrics more in accordance with a probabilistic approach. The outcomes we obtained (up to 7.9% increase in accuracy) indicate the approximated probabilistic models pledge more reliable predictions, thus being suitable for further investigation.
