pyhealth.processors.TensorProcessor#
Processor for tensor data.
size() returns the width of the feature’s last dimension seen in fit()
(1 for scalars), or None before fitting.
Custom tensor processors#
Subclass TensorProcessor to transform numeric features, for example to select
columns, impute or scale. Models such as MLP size their input layer from the
already-processed samples, falling back to size(), and never call
process() on a sample a second time, so process() does not need to be
idempotent. If process() changes the width, override size() to return the
output width. See examples/custom_tensor_processor.py.
In PyHealth 2.0.2 and earlier, EmbeddingModel called process() again on
already-processed samples to infer the width, which broke processors that are not
idempotent.
- class pyhealth.processors.TensorProcessor(dtype=torch.float32, spatial_dims=None, normalization=None, missing_indicators=False, impute_strategy=None)[source]#
Bases:
FeatureProcessorFeature processor for converting numerical lists to tensors.
- Input:
List of numbers (int/float) or nested lists of numbers
- Processing:
Convert input directly to torch.Tensor using torch.tensor()
Optionally, along the last dimension: normalization with statistics learned in
fit()from the observed (non-NaN) values, missing-value indicators (1 where the value is NaN) appended, and NaN filled with 0, in that order. After"zscore"normalization, 0 is the training mean.
- Output:
torch.Tensor with appropriate shape and dtype
Examples
>>> from pyhealth.processors import TensorProcessor >>> processor = TensorProcessor() >>> processor.fit([{"x": [1.0, 2.0, 3.0]}], "x") >>> processor.size() 3 >>> processor.process([4.0, 5.0, 6.0]) tensor([4., 5., 6.])
Initialize the TensorProcessor.
- Parameters:
dtype (
dtype) – The desired torch data type for the output tensor. Default is torch.float32.spatial_dims (
Optional[tuple[bool,...]]) – Tuple of booleans indicating which dimensions are spatial. If None, defaults to all False. Default is None.normalization (
Optional[Literal[‘zscore’, ‘minmax’]]) –None,"zscore"(zero mean, unit variance) or"minmax"(scaled to [0, 1] on the fitted range), per entry of the last dimension. Default is None.missing_indicators (
bool) – Whether to append a missing-value indicator per entry of the last dimension. Default is False.impute_strategy (
Optional[Literal[‘zero’]]) –None(keep NaN) or"zero". Default is None.
- __init__(dtype=torch.float32, spatial_dims=None, normalization=None, missing_indicators=False, impute_strategy=None)[source]#
Initialize the TensorProcessor.
- Parameters:
dtype (
dtype) – The desired torch data type for the output tensor. Default is torch.float32.spatial_dims (
Optional[tuple[bool,...]]) – Tuple of booleans indicating which dimensions are spatial. If None, defaults to all False. Default is None.normalization (
Optional[Literal[‘zscore’, ‘minmax’]]) –None,"zscore"(zero mean, unit variance) or"minmax"(scaled to [0, 1] on the fitted range), per entry of the last dimension. Default is None.missing_indicators (
bool) – Whether to append a missing-value indicator per entry of the last dimension. Default is False.impute_strategy (
Optional[Literal[‘zero’]]) –None(keep NaN) or"zero". Default is None.
- process(value)[source]#
Process a numerical value or list into a torch.Tensor.
- Parameters:
value (
Any) – Input value (list of numbers or nested lists)- Returns:
Processed tensor
- Return type:
torch.Tensor
- size()[source]#
Get the feature size of the processor.
- Return type:
Optional[int]- Returns:
The width of the last dimension (1 for scalars) seen in
fit(), doubled with missing-value indicators, or None before fitting.
- is_token()[source]#
Whether the output tensor represents discrete token indices, inferred from dtype.
- Return type:
- Returns:
True if dtype is integer (discrete tokens), False if floating point (continuous).
- schema()[source]#
Returns the schema of the processed feature. For a processor that emits a single tensor, this should just return [“value”]. For a processor that emits a tuple of tensors, this should return a tuple of the same length as the tuple, with the semantic name of each tensor, such as [“time”, “value”], [“value”, “mask”], etc.
- Typical semantic names include:
“value”: the main processed tensor output of the processor
“time”: the time tensor output of the processor (mostly for StageNet)
“mask”: the mask tensor output of the processor (if applicable)
- dim()[source]#
Number of dimensions for the output tensor.
- Return type:
- Returns:
(n_dim,). With missing-value indicators, a scalar input becomes a vector of 2, so n_dim is at least 1.
- Raises:
NotImplementedError – If n_dim was not provided and fit() was not called.
- load(path)#
Optional: Load processor state from disk.
- save(path)#
Optional: Save processor state to disk.