Metrics

We provide easy to use metrics (the same style and args as sklearn.metrics) for binary classification, multiclass classification, multilabel classification. For applicable tasks, we provide the relevant metrics for model calibration, as well as those for prediction set evaluation. Among these we also provide metrics related to uncertainty quantification, for model calibration, as well as metrics that measure the quality of prediction sets We also provide other metrics specically for healthcare tasks, such as drug drug interaction (DDI) rate.

• pyhealth.metrics.multiclass
  • multiclass_metrics_fn()
• pyhealth.metrics.multilabel
  • multilabel_metrics_fn()
• pyhealth.metrics.binary
  • binary_metrics_fn()
• [core] calibration
  • ece_confidence_multiclass()
  • ece_confidence_binary()
  • ece_classwise()
  • brier_top1()
• [core] prediction_set
  • size()
  • rejection_rate()
  • miscoverage_ps()
  • error_ps()
  • miscoverage_overall_ps()
  • error_overall_ps()
• pyhealth.metrics.fairness
  • fairness_metrics_fn()
  • disparate_impact()
  • statistical_parity_difference()
  • sensitive_attributes_from_patient_ids()