pyhealth.models.AdaCare#

The separate callable AdaCareLayer and the complete AdaCare model.

class pyhealth.models.AdaCareLayer(input_dim, hidden_dim=128, kernel_size=2, kernel_num=64, r_v=4, r_c=4, activation='sigmoid', rnn_type='gru', dropout=0.5)[source]#

Bases: Module

AdaCare layer.

Paper: Liantao Ma et al. Adacare: Explainable clinical health status representation learning via scale-adaptive feature extraction and recalibration. AAAI 2020.

This layer is used in the AdaCare model. But it can also be used as a standalone layer.

Parameters:

  • input_dim (int) – the input feature size.

  • hidden_dim (int) – the hidden dimension of the GRU layer. Default is 128.

  • kernel_size (int) – the kernel size of the causal convolution layer. Default is 2.

  • kernel_num (int) – the kernel number of the causal convolution layer. Default is 64.

  • r_v (int) – the number of the reduction rate for the original feature calibration. Default is 4.

  • r_c (int) – the number of the reduction rate for the convolutional feature recalibration. Default is 4.

  • activation (str) – the activation function for the recalibration layer (sigmoid, sparsemax, softmax). Default is “sigmoid”.

  • dropout (float) – dropout rate. Default is 0.5.

Examples

>>> from pyhealth.models import AdaCareLayer
>>> input = torch.randn(3, 128, 64)  # [batch size, sequence len, feature_size]
>>> layer = AdaCareLayer(64)
>>> c, _, inputatt, convatt = layer(input)
>>> c.shape
torch.Size([3, 64])
forward(x, mask=None)[source]#

Forward propagation.

Parameters:

  • x (tensor) – a tensor of shape [batch size, sequence len, input_dim].

  • mask (Optional[tensor]) – an optional tensor of shape [batch size, sequence len], where 1 indicates valid and 0 indicates invalid.

Returns:

a tensor of shape [batch size, input_dim] representing the

patient embedding.

output: a tensor of shape [batch size, sequence_len, input_dim] representing the patient embedding at each time step. inputatt: a tensor of shape [batch size, sequence_len, input_dim] representing the feature importance of the input. convatt: a tensor of shape [batch size, sequence_len, 3 * kernel_num] representing the feature importance of the convolutional features.

Return type:

last_output

training: bool#
class pyhealth.models.AdaCare(dataset, feature_keys, label_key, mode, use_embedding, embedding_dim=128, hidden_dim=128, **kwargs)[source]#

Bases: BaseModel

AdaCare model.

Paper: Liantao Ma et al. Adacare: Explainable clinical health status representation learning via scale-adaptive feature extraction and recalibration. AAAI 2020.

Note

We use separate AdaCare layers for different feature_keys. Currently, we automatically support different input formats:

  • code based input (need to use the embedding table later)

  • float/int based value input

Since the AdaCare model calibrate the original features to provide interpretability, we do not recommend use embeddings for the input features. We follow the current convention for the AdaCare model:

  • case 1. [code1, code2, code3, …]

    • we will assume the code follows the order; our model will encode

    each code into a vector and apply AdaCare on the code level

  • case 2. [[code1, code2]] or [[code1, code2], [code3, code4, code5], …]

    • we will assume the inner bracket follows the order; our model first

    use the embedding table to encode each code into a vector and then use average/mean pooling to get one vector for one inner bracket; then use AdaCare one the braket level

  • case 3. [[1.5, 2.0, 0.0]] or [[1.5, 2.0, 0.0], [8, 1.2, 4.5], …]

    • this case only makes sense when each inner bracket has the same length;

    we assume each dimension has the same meaning; we run AdaCare directly on the inner bracket level, similar to case 1 after embedding table

  • case 4. [[[1.5, 2.0, 0.0]]] or [[[1.5, 2.0, 0.0], [8, 1.2, 4.5]], …]

    • this case only makes sense when each inner bracket has the same length;

    we assume each dimension has the same meaning; we run AdaCare directly on the inner bracket level, similar to case 2 after embedding table

Parameters:

  • dataset (SampleEHRDataset) – the dataset to train the model. It is used to query certain information such as the set of all tokens.

  • feature_keys (List[str]) – list of keys in samples to use as features, e.g. [“conditions”, “procedures”].

  • label_key (str) – key in samples to use as label (e.g., “drugs”).

  • mode (str) – one of “binary”, “multiclass”, or “multilabel”.

  • use_embedding (List[bool]) – list of bools indicating whether to use embedding for each feature type, e.g. [True, False].

  • embedding_dim (int) – the embedding dimension. Default is 128.

  • hidden_dim (int) – the hidden dimension. Default is 128.

  • **kwargs – other parameters for the AdaCare layer.

Examples

>>> from pyhealth.datasets import SampleEHRDataset
>>> samples = [
...         {
...             "patient_id": "patient-0",
...             "visit_id": "visit-0",
...             "list_codes": ["505800458", "50580045810", "50580045811"],  # NDC
...             "list_vectors": [[1.0, 2.55, 3.4], [4.1, 5.5, 6.0]],
...             "list_list_codes": [["A05B", "A05C", "A06A"], ["A11D", "A11E"]],  # ATC-4
...             "list_list_vectors": [
...                 [[1.8, 2.25, 3.41], [4.50, 5.9, 6.0]],
...                 [[7.7, 8.5, 9.4]],
...             ],
...             "label": 1,
...         },
...         {
...             "patient_id": "patient-0",
...             "visit_id": "visit-1",
...             "list_codes": [
...                 "55154191800",
...                 "551541928",
...                 "55154192800",
...                 "705182798",
...                 "70518279800",
...             ],
...             "list_vectors": [[1.4, 3.2, 3.5], [4.1, 5.9, 1.7], [4.5, 5.9, 1.7]],
...             "list_list_codes": [["A04A", "B035", "C129"]],
...             "list_list_vectors": [
...                 [[1.0, 2.8, 3.3], [4.9, 5.0, 6.6], [7.7, 8.4, 1.3], [7.7, 8.4, 1.3]],
...             ],
...             "label": 0,
...         },
...     ]
>>> dataset = SampleEHRDataset(samples=samples, dataset_name="test")
>>>
>>> from pyhealth.models import AdaCare
>>> model = AdaCare(
...         dataset=dataset,
...         feature_keys=[
...             "list_codes",
...             "list_vectors",
...             "list_list_codes",
...             "list_list_vectors",
...         ],
...         label_key="label",
...         use_embedding=[True, False, True, False],
...         mode="binary",
...     )
>>>
>>> from pyhealth.datasets import get_dataloader
>>> train_loader = get_dataloader(dataset, batch_size=2, shuffle=True)
>>> data_batch = next(iter(train_loader))
>>>
>>> ret = model(**data_batch)
>>> print(ret)
{
    'loss': tensor(0.7167, grad_fn=<BinaryCrossEntropyWithLogitsBackward0>),
    'y_prob': tensor([[0.5009], [0.4779]], grad_fn=<SigmoidBackward0>),
    'y_true': tensor([[0.], [1.]]),
    'logit': tensor([[ 0.0036], [-0.0886]], grad_fn=<AddmmBackward0>)
}
forward(**kwargs)[source]#

Forward propagation.

The label kwargs[self.label_key] is a list of labels for each patient.

Parameters:

**kwargs – keyword arguments for the model. The keys must contain all the feature keys and the label key.

Returns:

loss: a scalar tensor representing the loss. feature_importance: a list of tensors with shape (feature_type, batch_size, time_step, features)

representing the feature importance.

conv_feature_importance: a list of tensors with shape (feature_type, batch_size, time_step, 3*kernal_size)

representing the convolutional feature importance.

y_prob: a tensor representing the predicted probabilities. y_true: a tensor representing the true labels.

Return type:

A dictionary with the following keys

training: bool#