pyhealth.metrics.multilabel#
- pyhealth.metrics.multilabel.multilabel_metrics_fn(y_true, y_prob, metrics=None, threshold=0.3, y_predset=None)[source]#
Computes metrics for multilabel classification.
User can specify which metrics to compute by passing a list of metric names. The accepted metric names are:
roc_auc_micro: area under the receiver operating characteristic curve, micro averaged
roc_auc_macro: area under the receiver operating characteristic curve, macro averaged
roc_auc_weighted: area under the receiver operating characteristic curve, weighted averaged
roc_auc_samples: area under the receiver operating characteristic curve, samples averaged
pr_auc_micro: area under the precision recall curve, micro averaged
pr_auc_macro: area under the precision recall curve, macro averaged
pr_auc_weighted: area under the precision recall curve, weighted averaged
pr_auc_samples: area under the precision recall curve, samples averaged
accuracy: accuracy score
f1_micro: f1 score, micro averaged
f1_macro: f1 score, macro averaged
f1_weighted: f1 score, weighted averaged
f1_samples: f1 score, samples averaged
precision_micro: precision score, micro averaged
precision_macro: precision score, macro averaged
precision_weighted: precision score, weighted averaged
precision_samples: precision score, samples averaged
recall_micro: recall score, micro averaged
recall_macro: recall score, macro averaged
recall_weighted: recall score, weighted averaged
recall_samples: recall score, samples averaged
jaccard_micro: Jaccard similarity coefficient score, micro averaged
jaccard_macro: Jaccard similarity coefficient score, macro averaged
jaccard_weighted: Jaccard similarity coefficient score, weighted averaged
jaccard_samples: Jaccard similarity coefficient score, samples averaged
ddi: drug-drug interaction score (specifically for drug-related tasks, such as drug recommendation)
hamming_loss: Hamming loss
cwECE: classwise ECE (with 20 equal-width bins). Check
pyhealth.metrics.calibration.ece_classwise().cwECE_adapt: classwise adaptive ECE (with 20 equal-size bins). Check
pyhealth.metrics.calibration.ece_classwise().
- The following metrics related to the prediction sets are accepted as well, but will be ignored if y_predset is None:
fp: Number of false positives.
tp: Number of true positives.
If no metrics are specified, pr_auc_samples is computed by default.
This function calls sklearn.metrics functions to compute the metrics. For more information on the metrics, please refer to the documentation of the corresponding sklearn.metrics functions.
- Parameters:
y_true (
ndarray) – True target values of shape (n_samples, n_labels).y_prob (
ndarray) – Predicted probabilities of shape (n_samples, n_labels).metrics (
Optional[List[str]]) – List of metrics to compute. Default is [“pr_auc_samples”].threshold (
float) – Threshold to binarize the predicted probabilities. Default is 0.5.
- Return type:
- Returns:
- Dictionary of metrics whose keys are the metric names and values are
the metric values.
Examples
>>> from pyhealth.metrics import multilabel_metrics_fn >>> y_true = np.array([[0, 1, 1], [1, 0, 1]]) >>> y_prob = np.array([[0.1, 0.9, 0.8], [0.05, 0.95, 0.6]]) >>> multilabel_metrics_fn(y_true, y_prob, metrics=["accuracy"]) {'accuracy': 0.5}
Metric order#
multilabel_metrics_fn walks the metrics list one entry at a time, and
no entry changes the arrays the others read, so the order of the list does not
change the numbers. metrics=["ddi", "jaccard_samples"] and
metrics=["jaccard_samples", "ddi"] return the same values.
"ddi" loads ddi_adj.npy from the PyHealth cache directory. GAMENet,
MICRON, MoleRec and SafeDrug write that file when the model is built, so
"ddi" is available once one of those models has been created. All four
write the same file, so it holds the matrix of whichever model was built last.