pyhealth.models.MLP#

The separate callable MLP model.

class pyhealth.models.MLP(dataset, feature_keys, label_key, mode, pretrained_emb=None, embedding_dim=128, hidden_dim=128, n_layers=2, activation='relu', **kwargs)[source]#

Bases: BaseModel

Multi-layer perceptron model.

This model applies a separate MLP layer for each feature, and then concatenates the final hidden states of each MLP layer. The concatenated hidden states are then fed into a fully connected layer to make predictions.

Note

We use separate MLP layers for different feature_keys. Currentluy, we automatically support different input formats:

  • code based input (need to use the embedding table later)

  • float/int based value input

We follow the current convention for the rnn model:
  • case 1. [code1, code2, code3, …]

    • we will assume the code follows the order; our model will encode

    each code into a vector; we use mean/sum pooling and then MLP

  • case 2. [[code1, code2]] or [[code1, code2], [code3, code4, code5], …]

    • we first use the embedding table to encode each code into a vector

    and then use mean/sum pooling to get one vector for each sample; we then use MLP layers

  • case 3. [1.5, 2.0, 0.0]

    • we run MLP directly

  • case 4. [[1.5, 2.0, 0.0]] or [[1.5, 2.0, 0.0], [8, 1.2, 4.5], …]

    • This case only makes sense when each inner bracket has the same length;

    we assume each dimension has the same meaning; we use mean/sum pooling within each outer bracket and use MLP, similar to case 1 after embedding table

  • case 5. [[[1.5, 2.0, 0.0]]] or [[[1.5, 2.0, 0.0], [8, 1.2, 4.5]], …]

    • This case only makes sense when each inner bracket has the same length;

    we assume each dimension has the same meaning; we use mean/sum pooling within each outer bracket and use MLP, similar to case 2 after embedding table

Parameters:

  • dataset (SampleEHRDataset) – the dataset to train the model. It is used to query certain information such as the set of all tokens.

  • feature_keys (List[str]) – list of keys in samples to use as features, e.g. [“conditions”, “procedures”].

  • label_key (str) – key in samples to use as label (e.g., “drugs”).

  • mode (str) – one of “binary”, “multiclass”, or “multilabel”.

  • embedding_dim (int) – the embedding dimension. Default is 128.

  • hidden_dim (int) – the hidden dimension. Default is 128.

  • n_layers (int) – the number of layers. Default is 2.

  • activation (str) – the activation function. Default is “relu”.

  • **kwargs – other parameters for the RNN layer.

Examples

>>> from pyhealth.datasets import SampleEHRDataset
>>> samples = [
...         {
...             "patient_id": "patient-0",
...             "visit_id": "visit-0",
...             "conditions": ["cond-33", "cond-86", "cond-80"],
...             "procedures": [1.0, 2.0, 3.5, 4],
...             "label": 0,
...         },
...         {
...             "patient_id": "patient-0",
...             "visit_id": "visit-0",
...             "conditions": ["cond-33", "cond-86", "cond-80"],
...             "procedures": [5.0, 2.0, 3.5, 4],
...             "label": 1,
...         },
...     ]
>>> dataset = SampleEHRDataset(samples=samples, dataset_name="test")
>>>
>>> from pyhealth.models import MLP
>>> model = MLP(
...         dataset=dataset,
...         feature_keys=["conditions", "procedures"],
...         label_key="label",
...         mode="binary",
...     )
>>>
>>> from pyhealth.datasets import get_dataloader
>>> train_loader = get_dataloader(dataset, batch_size=2, shuffle=True)
>>> data_batch = next(iter(train_loader))
>>>
>>> ret = model(**data_batch)
>>> print(ret)
{
    'loss': tensor(0.6659, grad_fn=<BinaryCrossEntropyWithLogitsBackward0>),
    'y_prob': tensor([[0.5680],
                    [0.5352]], grad_fn=<SigmoidBackward0>),
    'y_true': tensor([[1.],
                    [0.]]),
    'logit': tensor([[0.2736],
                    [0.1411]], grad_fn=<AddmmBackward0>)
}
>>>
static mean_pooling(x, mask)[source]#

Mean pooling over the middle dimension of the tensor. :param x: tensor of shape (batch_size, seq_len, embedding_dim) :param mask: tensor of shape (batch_size, seq_len)

Returns:

tensor of shape (batch_size, embedding_dim)

Return type:

x

Examples

>>> x.shape
[128, 5, 32]
>>> mean_pooling(x).shape
[128, 32]
static sum_pooling(x)[source]#

Mean pooling over the middle dimension of the tensor. :param x: tensor of shape (batch_size, seq_len, embedding_dim) :param mask: tensor of shape (batch_size, seq_len)

Returns:

tensor of shape (batch_size, embedding_dim)

Return type:

x

Examples

>>> x.shape
[128, 5, 32]
>>> sum_pooling(x).shape
[128, 32]
forward(**kwargs)[source]#

Forward propagation.

The label kwargs[self.label_key] is a list of labels for each patient.

Parameters:

**kwargs – keyword arguments for the model. The keys must contain all the feature keys and the label key.

Returns:

loss: a scalar tensor representing the loss. y_prob: a tensor representing the predicted probabilities. y_true: a tensor representing the true labels.

Return type:

A dictionary with the following keys

training: bool#