Metrics#

We provide easy to use metrics (the same style and args as sklearn.metrics) for binary classification, multiclass classification, multilabel classification. For applicable tasks, we provide the relevant metrics for model calibration, as well as those for prediction set evaluation. Among these we also provide metrics related to uncertainty quantification, for model calibration, as well as metrics that measure the quality of prediction sets We also provide other metrics specically for healthcare tasks, such as drug drug interaction (DDI) rate. For synthetic (generative) EHR data, we provide privacy, utility, and statistical fidelity metrics.

Expected Calibration Error (ECE) is available for binary and multiclass predictions through pyhealth.metrics.calibration.ece_confidence_binary() and pyhealth.metrics.calibration.ece_confidence_multiclass(). The API reference includes the ECE definition and formula used by these helpers.