gyoza.modelling package

Submodules

gyoza.modelling.flow_layers module

class gyoza.modelling.flow_layers.ActivationNormalization(*args, **kwargs)[source]

Bases: FlowLayer

Applies a trainable location- and scale-transformation of the following form to its inputs:

\[\hat{y} = \frac{x - l}{s}, \quad l \in \mathbb{R}, \quad s > 0,\]

where \(l\) and \(s\) are the location and scale parameters. For each dimension covered by the axes argument, a separate location and scale parameter is learned. For example, suppose you construct this layer with axes == [1,2] and shape == [width, height]. If you then provide inputs of shape [batch-size, width, height, channel-count] to this layer’s call() method, one location and one scale parameter will be learned for each pixel position (i.e. 2 * width * height many parameters). The above location- and scale-transformation will then sweep across all other axes of the input (here: batch-size and channel-count) during the call routine.

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer.

References:

  • Sankar, A., Keicher, M., Eisawy, R., Parida, A., Pfister, F., Kim, S., T. & Navab, N. (2021) “GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images” arXiv:2103.10868

  • Esser, P., Rombach, R., & Ommer, B. (2020). “A Disentangling Invertible Interpretation Network for Explaining Latent Representations.” arXiv:2004.13166

class _PositiveConstraint_[source]

Bases: Constraint

Constraint enforcing strictly positive scale parameters.

_location_

The value by which each data point shall be translated.

_prepare_variables_for_computation_(x: Tensor | KerasTensor) → Tuple[Variable, Variable][source]

Prepares the variables for computation with data. This ensures variable shapes are compatible with x.

Parameters:

x (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The input to be passed through call().

Returns:

  • location (tensorflow.Variable): The _location_ attribute shaped to fit x.

  • scale (tensorflow.Variable): The _scale_ attribute shaped to fit x.

_scale_

The value by which each data point shall be scaled. This variable is constrained to be positive.

call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Executes the operation of this layer in the forward direction.

Parameters:

inputs (Union[tf.Tensor, tf.keras.KerasTensor]) – The data to be tranformed. Assumed to be of shape [batch-size, …], where … is the shape of a single instance assumed to have at least one axis.

Returns:

  • y_hat (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The output of the transformation of shape [batch-size, …].

  • jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The Jacobian determinant on logarithmic scale of shape [batch-size].

compute_jacobian_determinant(x: Tensor | KerasTensor) → Tensor[source]

Computes the Jacobian determinant of this layer’s transformation on logarithmic scale. This is computed as the negative sum of ln(scale) over all scale parameters held by this layer, multiplied by the number of remaining dimensions located on the other axes of x. This function supports symbolic execution.

Parameters:

x (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The input to this layer.

Returns:

jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) - The Jacobian determinant on logarithmic scale of shape [batch-size].

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this layer in the inverse direction. It is thus the counterpart to call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The outputs of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The reconstructed input to call().

class gyoza.modelling.flow_layers.AdditiveCoupling(*args, **kwargs)[source]

Bases: Coupling

This coupling layer implements an additive coupling law of the form \(f(x_2, g(x_1)) = x_2 + g(x_1)\), where \(g\) is the compute_coupling_parameters function provided as argument when constructing an instance of this class. For details on the encapsulating theory refer to Coupling or the paper by Dinh, Sohl-Dickstein and Bengio (2016) referenced below.

References:

  • Dinh, L., Sohl-Dickstein, J. & Bengio, S. (2016). “Density estimation using Real NVP” arXiv:1605.08803

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See superclass FlowLayer.

  • compute_coupling_parameters (Union[tensorflow.keras.Layer, tensorflow.keras.Model]) – See base class Coupling.

  • mask (Mask) – See base class Coupling.

_couple_(inputs: Tensor | KerasTensor, coupling_parameters: Tensor | KerasTensor) → Tensor[source]

This function implements an invertible coupling law \(f\) for inputs and coupling_parameters. Subclasses must implement this method according to the coupling law they provide. There is no need to apply the negative mask to the inputs here, since this is done afterwards by call().

Parameters:

  • inputs (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The inputs to call(). This can be treated as \(x_2\) in the formulas given in the Coupling description, although the negative mask will only be applied by call() after _couple_() returned.

  • coupling_parameters (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor, List[tensorflow.Tensor], List[tensorflow.keras.KerasTensor]]) – The coupling parameters that shall be used to transform inputs, corresponding to \(g(x_1)\) in the formulas given in the Coupling description.

Returns:

outputs (tensorflow.Tensor) - The coupled tensor. The negative mask will be applied to this output by call() to obtain \(y_2\) in the formulas given in the Coupling description.

_decouple_(outputs: Tensor, coupling_parameters: Tensor) → Tensor[source]

This function is the inverse of _couple_(). Subclasses must implement this method according to the coupling law they proovide. There is no need to apply the negative mask here, since this is done afterwards by invert(). This method only has to work in eager mode, thus not in symbolic mode, since it is only called from invert().

Parameters:

  • outputs (tensorflow.Tensor) – The outputs of call(), i.e. \(y\). This can be treated as \(y_2\) in the formulas given in the Coupling description, although the negative mask will only be applied by invert() after _decouple_() returned.

  • coupling_parameters (Union[tensorflow.Tensor, List[tensorflow.Tensor]]) – The coupling parameters corresponding to \(g(x_1)\) in the formulas given in the Coupling description.

Returns:

reconstructed_inputs (tensorflow.Tensor) - The decoupled tensor corresponding to \(x_2\) in the formulas given in the Coupling description. The negative mask will be applied to this outout by invert() to obtain said \(x_2\).

compute_jacobian_determinant(x: Tensor | KerasTensor) → Tensor[source]

Computes the Jacobian determinant of this layer’s transformation on logarithmic scale. This is simply zero for each instance since additive couplings are volume-preserving.

Parameters:

x (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The input to this layer.

Returns:

jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) - The Jacobian determinant on logarithmic scale of shape [batch-size].

class gyoza.modelling.flow_layers.CheckerBoardPermutation(*args, **kwargs)[source]

Bases: Permutation

A Permutation subclass that swaps the entries of its inputs as inspired by the checkerboard pattern. Swapping is done in a way that preserves adjacency of among the swapped entries as much as possible. Important: The checkerboard pattern is usually defined on a matrix, i.e. 2 axes. Yet, here it is possible to specify any number of axes. Note: If the total number of elements in the to-be-permuted vector is odd, then the last element will remain in place.

Parameters:

  • axes (List[int]) – See base class FlowLayer.

  • shape (List[int]) – See base class FlowLayer.

classmethod from_config(config: Dict[str, Any]) → CheckerBoardPermutation[source]

Loads a CheckerBoardPermutation from its configuration dictionary. The dictionary is required to have been obtained via get_config().

Parameters:

  • cls (FlowLayerType) – The type of this layer (i.e. CheckerBoardPermutation).

  • config (Dict[str, Any]) – The dictionary containing the configuration of the checkerboard permutation.

Returns:

instance (CheckerBoardPermutation) - The constructed instance.

static generate_rope_indices(shape: List[int]) → Generator[int, None, None][source]

Generates indices to traverse a tensor of given shape. The traversal follows a rope fitted along the axes (0 to len(shape)-1) by prioritizing later axes before earlier axes.

Parameters:

shape (List[int]) – The shape of the tensor to be traversed.

Yield:

current_indices (List[int]) - The indices pointing to the current cell, one index per axis.

static is_end_of_axis(index: int, limit: int, direction: int) → bool[source]

Determines whether an index iterated in direction is at the end of a given axis.

Parameters:

  • index (int) – The index to be checked.

  • limit (int) – The number of elements along the axis. An index is considered to be at the end if it is equal to limit-1 and direction == 1.

  • direction (int) – The direction in which the index is iterated. A value of 1 indicates incremental, -1 indicates decremental.

Returns:

is_end_of_axis (bool) - An indicator for whether the endpoint has been reached.

class gyoza.modelling.flow_layers.Coupling(*args, **kwargs)[source]

Bases: FlowLayer

This subclass of FlowLayer couples the input provided to its call() method with itself.

Theoretical Background:

The call() method implements the following formulas:

\begin{align*} x_1 &= w * x \\ x_2 &= (1-w) * x \\ y_1 &= x_1 \\ y_2 &= f(x_2, g(x_1)) \\ y &= y_1 + y_2, \end{align*}

where \(x\), \(x_1\), \(x_2\), \(w\), \(y_1\), \(y_2\) and \(y\) all have the same shape. While the \(x\)’s and the \(y\)’s are all real-valued, the mask \(w\) is binary. As can be seen from the formulas, \(w\) is used to separate the inputs \(x\) into two halves which are then saved to \(x_1\) and \(x_2\). While half of the output entries in \(y\) are directly set to \(x_1\) via \(y_1\), the main contribution of this layer is in the computation of the other half of the output via \(y_2\). That is, the function \(g\) is used to compute the so-called coupling parameters which then feed into the coupling law \(f\) to compute \(y_2\). The computation of coupling parameters can be an arbitrarily complicated and not necessarily invertible mapping, here executed by the tensorflow.keras.Layer or even tensorflow.keras.Model called compute_coupling_parameters. The coupling parameters can be a tensor of any shape (or even a list of tensors), as long as they are compatible with the coupling law \(f\). The coupling law \(f\) is a simple combination of \(x_2\) and the coupling parameters, e.g. a sum or a product that has to be trivially invertible and have a tractable Jacobian determinant.

Usage Instructions:

To couple your data, you need to instantiate a subclass of Coupling that implements your preferred coupling law \(f\), e.g. AdditiveCoupling. You also need to provide a suitable mask \(w\) from the masks module to allow the coupling layer to filter out half of the entries of your data tensor and couple them with the other half. It is important to specify the same shape and axes for both the coupling layer and the mask so that they can work together correctly.

You will also need to provide a suitable compute_coupling_parameters layer/ model that computes the coupling parameters. The coupling layer’s call() method will take its inputs, use the mask to cancel out half of the entries and then feed the result (which has the same shape as inputs) to your compute_coupling_parameters layer/ model. Read the description of your chosen coupling layer’s subclass to see what requirements it has on the outputs of compute_coupling_parameters. Simple Coupling subclasses like AdditiveCoupling will expect the output of compute_coupling_parameters to have the same shape as inputs which allows for simple element-wise addition or multiplication with inputs before the negative mask will keep only the entries relevant to form \(y_2\). But, more complex coupling layers might have different requirements.

It is important to realize that the halving of the inputs will effectively lead the coupling layer to couple only half of the input dimensions while leaving the other half unchanged. To allow all input dimensions to be transformed by coupling layers, it is common practice to succeed the coupling layer with a Permutation that swaps the positions of the dimensions of the two halves. Importantly, choose a permutation that is equivalent to the mask of your choice, e.g. a CheckerBoardPermutation for a CheckerBoardMask. Then, apply another coupling layer with the same mask, followed by another permutation of same kind to move all dimensions back to their original positions. This results in a complete coupling block that can be repeated several times in a normalizing flow to allow for increasingly complex transformations of all input dimensions.

Subclass Instructions:

Any subclasses of this class must implement

  • _couple_ to implement the usually trivial coupling law \(f\) used in call(), e.g. addition or multiplication.

  • _decouple_ to implement the inverse of the coupling law \(f\) used in invert().

  • compute_log_jacobian_determinant to compute the logarithmic Jacobian determinant of the coupling law \(f\).

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer.

  • compute_coupling_parameters (Union[tensorflow.keras.Layer, tensorflow.keras.Model]) – A tensorflow.keras.Layer or even tensorflow.keras.Model that maps its input tensor to coupling parameters. This model can be arbitrarily complex and does not need to be invertible, but its outputs must be compatible with the coupling law (\(f\)) implemented by the specific Coupling subclass.

  • mask (Mask) – The mask used to keep one half of the data while cancelling the other half. The mask is evaluated in its positive configuration (i.e. 1s select entries, 0s cancel entries, see call()) to select \(x_1\) and in its negative configuration (i.e. 1s and 0s get flipped) to select \(x_2\).

References:

  • Dinh, L., Sohl-Dickstein, J. & Bengio, S. (2016). “Density estimation using Real NVP” arXiv:1605.08803

  • Dinh, L., Krueger, D. & Bengio, Y. (2015) “NICE: NON-LINEAR INDEPENDENT COMPONENTS ESTIMATION” arXiv:1410.8516

_compute_coupling_parameters_

(Union[tensorflow.keras.Layer, tensorflow.keras.Model]) A keras model or layer that maps its input tensor to coupling parameters. This layer/ model can be arbitrarily complex and does not need to be invertible, but its outputs must be compatible with the coupling law (\(f\)) implemented by the specific Coupling subclass.

_couple_(inputs: Tensor | KerasTensor, coupling_parameters: Tensor | KerasTensor | List[Tensor] | List[KerasTensor]) → Tensor[source]

This function implements an invertible coupling law \(f\) for inputs and coupling_parameters. Subclasses must implement this method according to the coupling law they provide. There is no need to apply the negative mask to the inputs here, since this is done afterwards by call().

Parameters:

  • inputs (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The inputs to call(). This can be treated as \(x_2\) in the formulas given in the Coupling description, although the negative mask will only be applied by call() after _couple_() returned.

  • coupling_parameters (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor, List[tensorflow.Tensor], List[tensorflow.keras.KerasTensor]]) – The coupling parameters that shall be used to transform inputs, corresponding to \(g(x_1)\) in the formulas given in the Coupling description.

Returns:

outputs (tensorflow.Tensor) - The coupled tensor. The negative mask will be applied to this output by call() to obtain \(y_2\) in the formulas given in the Coupling description.

_decouple_(outputs: Tensor, coupling_parameters: Tensor | List[Tensor]) → Tensor[source]

This function is the inverse of _couple_(). Subclasses must implement this method according to the coupling law they proovide. There is no need to apply the negative mask here, since this is done afterwards by invert(). This method only has to work in eager mode, thus not in symbolic mode, since it is only called from invert().

Parameters:

  • outputs (tensorflow.Tensor) – The outputs of call(), i.e. \(y\). This can be treated as \(y_2\) in the formulas given in the Coupling description, although the negative mask will only be applied by invert() after _decouple_() returned.

  • coupling_parameters (Union[tensorflow.Tensor, List[tensorflow.Tensor]]) – The coupling parameters corresponding to \(g(x_1)\) in the formulas given in the Coupling description.

Returns:

reconstructed_inputs (tensorflow.Tensor) - The decoupled tensor corresponding to \(x_2\) in the formulas given in the Coupling description. The negative mask will be applied to this outout by invert() to obtain said \(x_2\).

_mask_

(Mask) - The mask used to keep one half of the data while cancelling out the other half.

build(input_shape: TensorShape | Tuple | List) → None[source]
call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Executes the operation of this layer in the forward direction.

Parameters:

inputs (Union[tf.Tensor, tf.keras.KerasTensor]) – The data to be tranformed. Assumed to be of shape [batch-size, …], where … is the shape of a single instance assumed to have at least one axis.

Returns:

  • y_hat (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The output of the transformation of shape [batch-size, …].

  • jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The Jacobian determinant on logarithmic scale of shape [batch-size].

classmethod from_config(config: Dict[str, Any]) → FlowLayerType[source]

Loads a Coupling layer from its configuration dictionary. The dictionary is required to have been obtained via get_config().

Parameters:

  • cls (FlowLayerType) – The specific type of this layer (i.e. Coupling or one of its subclasses).

  • config (Dict[str, Any]) – The dictionary containing the configuration of the coupling layer.

Returns:

instance (FlowLayerType) - The constructed instance.

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding flow layer class’s from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this layer in the inverse direction. It is thus the counterpart to call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The outputs of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The reconstructed input to call().

class gyoza.modelling.flow_layers.DisentanglingFlowModel(*args, **kwargs)[source]

Bases: FlowModel

A FlowModel subclass designed to disentangle latent factors in neural network representations.

This model follows the methodology described in:

  • Esser, P., Rombach, R., & Ommer, B. (2020). “A Disentangling Invertible Interpretation Network for Explaining Latent Representations.” arXiv:2004.13166

Important: The model expects the output of its call() to have shape [batch-size, instance-dimensionality]. For inputs with more axes (e.g. images), you must either flatten them before passing them to this model or include flattening layers in your flow_layers sequence to make sure your flow model’s output enumerates all its dimensions along one axis.

Inference: Use the regular call() on a batch of instances.

Training: The model overrides train_step() to accept pairs of instances (Z_a, Z_b) and a corresponding target tensor y_true. The outputs of both instances, including their Jacobian determinants, are concatenated along axis=-1 and passed as y_pred to the loss function. When calling tensorflow.keras.Model.compile(), you should provide ~gyoza.modelling.losses.SupervisedFactorLoss (or compatible) as the loss.

Requirements for :py:meth:`tensorflow.keras.Model.fit` inputs:

  • x must be a tuple (batch A, batch B) where each batch is valid input to call().

  • y must match the y_pred expected by the loss function assigned during tensorflow.keras.Model.compile().

Note: This model is intended for Keras subclassing workflow (eager or graph mode), not the functional API.

Parameters:

flow_layers (List[FlowLayer]) – List of flow layers for the model.

train_step(data: Tuple[Tuple[Tensor, Tensor], Tensor]) → Dict[str, Tensor][source]
class gyoza.modelling.flow_layers.FlowLayer(*args, **kwargs)[source]

Bases: Layer

Abstract base class for flow layers. Any input to this layer is assumed to have shape along axes as specified during initialization.

Subclass Instructions: Any subclasses of this class must

Parameters:

  • shape (List[int]) – The shape of the input along the specified axes that shall be transformed by this layer. If you have e.g. a tensor [batch-size, width, height, color] and you want this layer to transform along width and height, you enter the shape [width, height]. If you want the layer to operate on the color you provide the shape [color] instead.

  • axes (List[int]) – The axes along which the transformation of this layers shall be performed. In the example for shape on width and height you would enter [1,2] here. In the example for color you would enter [3] here. Although axes are counted starting from zero, it is assumed that axes does not contain the axis 0, i.e. the batch axis. Furthermore, axes are assumed to be ascending but they do not have to be contiguous.

References:

  • Dinh, L., Sohl-Dickstein, J. & Bengio, S. (2016). “Density estimation using Real NVP” arXiv:1605.08803

  • Kingma, D. P. & Dhariwal, P. (2018) “Glow: Generative Flow with Invertible 1x1 Convolutions” arXiv:1807.03039

  • Dinh, L., Krueger, D. & Bengio, Y. (2015) “NICE: NON-LINEAR INDEPENDENT COMPONENTS ESTIMATION” arXiv:1410.8516

  • Sankar, A., Keicher, M., Eisawy, R., Parida, A., Pfister, F., Kim, S., T. & Navab, N. (2021) “GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images” arXiv:2103.10868

  • Meng, C., Song, Y., Song, J. & Ermon, S. (2020) “Gaussianization Flows” arXiv:2003.01941

  • Esser, P., Rombach, R., & Ommer, B. (2020). “A Disentangling Invertible Interpretation Network for Explaining Latent Representations.” arXiv:2004.13166

_axes_

(List[int]) - The axes of transformation. For detail, see constructor of FlowLayer

_shape_

(List[int]) - The shape of the input that shall be transformed by this layer. For detail, see constructor of FlowLayer

call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Executes the operation of this layer in the forward direction.

Parameters:

inputs (Union[tf.Tensor, tf.keras.KerasTensor]) – The data to be tranformed. Assumed to be of shape [batch-size, …], where … is the shape of a single instance assumed to have at least one axis.

Returns:

  • y_hat (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The output of the transformation of shape [batch-size, …].

  • jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The Jacobian determinant on logarithmic scale of shape [batch-size].

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding flow layer class’s from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this layer in the inverse direction. It is thus the counterpart to call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The outputs of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The reconstructed input to call().

class gyoza.modelling.flow_layers.FlowModel(*args, **kwargs)[source]

Bases: Model

A tensorflow.keras.Model subclass for creating flow models. It relies on a list of flow_layers as input which will always be executed in the given sequence. Each such FlowLayer is expected to implement:

  • a call(self, inputs: Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) -> Tuple[Union[tensorflow.Tensor, tensorflow.keras.KerasTensor], Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]] method that works on eager as well as symbolic tensorflow.Tensor instances and outputs (outputs, jacobian_determinant), where outputs is the transformed version of inputs and jacobian_determinant is the Jacobian determinant of the transformation on logarithmic scale.

  • an invert(self, y_hat: tensorflow.Tensor) -> tensorflow.Tensor method that reconstructs the inputs tensor to the call method from its outputs.

  • a build(self, input_shape: tensorflow.TensorShape) method that ensures all model variables are constructed and layer.built == True. This is only needed if the __init__ method does not fulfill these criteria already.

  • serializable behavior via tensorflow.keras.layers.Layer.get_config() and tensorflow.keras.layers.Layer.from_config().

It is possible to add() layers after construction, but this has to happen before the model is built via build() and compiled via tensorflow.keras.Model.compile().

Parameters:

flow_layers (List[py:class:~gyoza.modelling.flow_layers.FlowLayer]) – A list of flow layers.

add(flow_layer: FlowLayer) → None[source]

Adds another flow-layer to the internal list of flow-layers. Important: It is only possible to add layers before this model is built via build() and compiled via tensorflow.keras.Model.compile().

Parameters:

flow_layer (FlowLayer) – A flow layer with the same requirements as assumed by the constructor of FlowModel.

build(input_shape: TensorShape | Tuple | List) → None[source]

Builds the model by calling the build method on all internally managed flow-layers. Assumes that each such flow-layer implements a call(self, inputs: tensorflow.keras.KerasTensor) -> Tuple[tensorflow.keras.KerasTensor, Union[tensorflow.keras.KerasTensor, tensorflow.keras.KerasTensor]] method that works on symbolic tensorflow.keras.KerasTensor instances and outputs (outputs, _), where outputs is the transformed version of input. Also assumes that this call method can be called once the layer is built to pass a symbolic input to the next layer in sequence.

Parameters:

input_shape (Union[tensorflow.TensorShape, tuple, list]) – The shape of the input to the model. The input is assumed to be a single tensorflow.Tensor (i.e. not a collection) and its shape to be provided here shall be structured as [batch-size, …], where … is any instance shape consistent with what the FlowLayer instances of this model support.

call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Calls the call() method of each of the internally maintained FlowLayer instances in the sequence in which they were added to this model. Assumes that each flow-layer implements a call(self, inputs: Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) -> Tuple[Union[tensorflow.Tensor, tensorflow.keras.KerasTensor], Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]] method that works on eager as well as symbolic tensorflow.Tensor instances and outputs (outputs, jacobian_determinant), where outputs is the transformed version of inputs and jacobian_determinant is the Jacobian determinant of the transformation on logarithmic scale.

Parameters:

inputs (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The data to be tranformed. Assumed to be eager numeric or symbolic and of shape [batch-size, …], where … is any instance shape with at least 1 axis consistent with what the FlowLayer instances of this model support.

Returns:

(outputs, jacobian_determinant) (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor], Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]]) - The outputs of the transformation of shape [batch-size, …] and the Jacobian determinant on logarithmic scale of shape [batch-size].

classmethod from_config(config: Dict[str, Any]) → FlowModelType[source]

Loads a flow model from its configuration dictionary. The dictionary is required to have been obtained via get_config().

Parameters:

  • cls (FlowModelType) – The specific type of the flow model to be constructed.

  • config (Dict[str, Any]) – The dictionary containing the configuration of the flow model.

Returns:

instance (FlowModelType) - The constructed instance.

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding flow model class’s from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this model in the inverse direction by calling the invert() method on all layers in the internally managed list of FlowLayer instances in reverse order. This method hence assumes that every FlowLayer implements an invert(self, outputs: tf.Tensor) -> tf.Tensor method that reconstructs the input to its call() method. This model’s invert method is thus the counterpart to its call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The data to be transformed. Assumed to be of the same shape as the output of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The output of the transformation of shape [batch-size, …].

class gyoza.modelling.flow_layers.HeavisidePermutation(*args, **kwargs)[source]

Bases: Permutation

A Permutation subclass that swaps the first and second half of its inputs as inspired by the Heaviside function. Important: The Heaviside function is defined on a vector, yet by the requirement of Permutation, inputs to this layer are allowed to have more than one axis in axes. As described in Permutation, an input is first flattened along axes and thus the swopping can be applied. For background information see Permutation. If the number of dimensions to be permuted is odd-length, the permutation will move the first half excluding the middle element.

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer.

classmethod from_config(config: Dict[str, Any]) → HeavisidePermutation[source]

Loads a HeavisidePermutation from its configuration dictionary. The dictionary is required to have been obtained via get_config().

Parameters:

  • cls (FlowLayerType) – The type of this layer (i.e. HeavisidePermutation).

  • config (Dict[str, Any]) – The dictionary containing the configuration of the heaviside permutation.

Returns:

instance (HeavisidePermutation) - The constructed instance.

class gyoza.modelling.flow_layers.Permutation(*args, **kwargs)[source]

Bases: FlowLayer

A FlowLayer subclass that flattens its input along axes, then reorders the dimensions using permutation and reshapes it back to its original shape. This transformation is volume-preserving, meaning it has a Jacobian determinant of 1 (or 0 on logarithmic scale).

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer.

  • permutation (List[int]) – A new order of the indices in the interval [0, product(shape)).

_forward_permutation_

(tensorflow.Variable) - Stores the permutation vector for the forward operation.

_inverse_permutation_

(tensorflow.Variable) - Stores the permutation vector for the inverse operation.

call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Executes the operation of this layer in the forward direction.

Parameters:

inputs (Union[tf.Tensor, tf.keras.KerasTensor]) – The data to be tranformed. Assumed to be of shape [batch-size, …], where … is the shape of a single instance assumed to have at least one axis.

Returns:

  • y_hat (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The output of the transformation of shape [batch-size, …].

  • jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The Jacobian determinant on logarithmic scale of shape [batch-size].

compute_jacobian_determinant(x: Tensor | KerasTensor) → Tensor[source]

Computes the Jacobian determinant of this layer’s transformation on logarithmic scale. This is simply zero since permutations are volume-preserving. This function supports symbolic execution.

Parameters:

x (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The input to this layer.

Returns:

jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) - The Jacobian determinant on logarithmic scale of shape [batch-size].

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding flow layer class’s from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this layer in the inverse direction. It is thus the counterpart to call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The outputs of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The reconstructed input to call().

class gyoza.modelling.flow_layers.Reflection(*args, **kwargs)[source]

Bases: FlowLayer

This layer reflects a data point around reflection_count learnable normals using the Householder transform. In this context, the normal is the unit-length vector orthogonal to the hyperplane of reflection. When axes contains more than a single entry, the input is first flattened along these axes, then reflected and then unflattened to original shape.

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer. Important: These axes are the axes of a tensor shape, e.g. axes 1 and 2 in a tensor of shape [batch-size, width, height] and hence distinct from the learnable reflection axes. The reflection axes are the normals to the hyperplanes of reflection and are learned internally as part of this layer.

  • reflection_count (int) – The number of successive reflections that shall be executed. Expected to be at least 1.

References:

  • Meng, C., Song, Y., Song, J. & Ermon, S. (2020) “Gaussianization Flows” arXiv:2003.01941

_inverse_mode_

(bool) - Indicates whether the reflections shall be executed in reversed order (True) or forward order (False).

_reflect_(x: Tensor) → Tensor[source]

This function executes all the reflections of self in a sequence by applying successive Householder reflections defined by the _reflection_normals_. This method provides the backward reflection if self._inverse_mode == True and forward otherwise.

Parameters:

x (tensorflow.Tensor) – The flattened data of shape […, dimension count], where dimension count is the product of the _shape_ as specified during initialization of self. It is assumed that all axes except for _axes_ (again, see initialization of self) are moved to … in the aforementioned shape of x.

Returns:

x_new (tensorflow.Tensor) - The rotated version of x with same shape.

_reflection_normals_

(tensorflow.Tensor) - These are the axes along which an instance is reflected. Shape == [reflection count, dimension count] where dimension count is the product of the shape of the input instance along self._axes_.

call(inputs: Tensor | KerasTensor) → Tuple[Tensor | KerasTensor, Tensor | KerasTensor][source]

Executes the operation of this layer in the forward direction.

Parameters:

inputs (Union[tf.Tensor, tf.keras.KerasTensor]) – The data to be tranformed. Assumed to be of shape [batch-size, …], where … is the shape of a single instance assumed to have at least one axis.

Returns:

  • y_hat (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The output of the transformation of shape [batch-size, …].

  • jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]): The Jacobian determinant on logarithmic scale of shape [batch-size].

compute_jacobian_determinant(x: Tensor | KerasTensor) → Tensor[source]

Computes the Jacobian determinant of this layer’s transformation on logarithmic scale. This is simply zero since reflections are volume-preserving. This function supports symbolic execution.

Parameters:

x (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – The input to this layer.

Returns:

jacobian_determinant (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) - The Jacobian determinant on logarithmic scale of shape [batch-size].

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding flow layer class’s from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

invert(outputs: Tensor) → Tensor[source]

Executes the operation of this layer in the inverse direction. It is thus the counterpart to call(). Note: This method is only intended for use in eager mode.

Parameters:

outputs (tensorflow.Tensor) – The outputs of call().

Returns:

reconstructed_inputs (tensorflow.Tensor) - The reconstructed input to call().

class gyoza.modelling.flow_layers.ShufflePermutation(*args, **kwargs)[source]

Bases: Permutation

A Permutation subclass that shuffles its inputs along the specified axes. The permutation used for shuffling is randomly chosen once during initialization. Thereafter it is saved as a private attribute. Shuffling is thus deterministic. Important: The shuffle function is defined on a vector, yet by the requirement of Permutation, inputs to this layer are allowed to have more than one axis in axes. As described in Permutation, inputs are first flattened along axes and thus the shuffling can be applied. For background information see Permutation.

Parameters:

  • shape (List[int]) – See base class FlowLayer.

  • axes (List[int]) – See base class FlowLayer.

classmethod from_config(config: Dict[str, Any]) → ShufflePermutation[source]

Loads a ShufflePermutation from its configuration dictionary. The dictionary is required to have been obtained via get_config().

Parameters:

  • cls (FlowLayerType) – The type of this layer (i.e. ShufflePermutation).

  • config (Dict[str, Any]) – The dictionary containing the configuration of the shuffle permutation.

Returns:

instance (ShufflePermutation) - The constructed instance.

gyoza.modelling.losses module

class gyoza.modelling.losses.SupervisedFactorLoss(dimensions_per_factor: List[int], *args)[source]

Bases: Loss

This loss can be used to incentivize the entries of the output vector of a FlowModel to be arranged according to semantic factors of the data with multivariate normal distribution. It implements the following formula:

\[\mathcal{L} = \sum_{F=1}^{K} \mathbb{E}_{(z^a,z^b) \sim p(z^a, z^b | F) } l(z^a, z^b | F)\]
\begin{eqnarray} l(z^a,z^b | F) &= \frac{1}{2} \sum_{k=0}^{K} ||T(z^a)_k||^2 - log|T'(z^a)| \\ &+ \frac{1}{2} \sum_{k \neq F} ||T(z^b)_k||^2 - log|T'(z^b)| \\ &+ \frac{1}{2} \frac{||T'(z^b)_F - \sigma_{ab} T(z^a)_F||^2}{1-\sigma_{ab}^2}, \end{eqnarray}

where \(T(z)\) is the model whose loss shall be computed, \(z^a\), \(z^b\) are instances passed trough \(T\), \(T'(z^a)\) is the Jacobian of \(T\) and \(\sigma_{ab}\) is the clustering strength of instances (see below). The factors can be thought of as independent components. A factor \(k\) spreading across \(N_k\) entries of the output vector is incentivised by this loss to represent the similarity of two inputs \(z^a\) and \(z^b\) along one and only one concept. For instance, factors can represent color, texture, size, animal species, or material. The loss expects training instances to come in pairs \(z^a\) and \(z^b\). The correlation of the instances of a pair should be captured by \(\sigma_{ab}\) (alias for y_true in call) such that the corresponding factor can capture the underlying concept of similarity. Yet, the association shall be (on average) close to zero for all other factors.

Parameters:

dimensions_per_factor (List[int]) – A list of integers that enumerates the number of dimensions (entries in a vector) of the factors thought to underly the representation of \(z^{\sim}\). These shall include the residual factor at index 0 which collect all variation not captured by the true factors. The sum of all entries is assumed to be equal to the number of dimensions in \(z^{\sim}\).

References:

  • Esser, P., Rombach, R., & Ommer, B. (2020). “A Disentangling Invertible Interpretation Network for Explaining Latent Representations.” arXiv:2004.13166

call(y_true: Tensor | KerasTensor, y_pred: Tensor | KerasTensor) → Tensor | KerasTensor[source]

Computes the loss.

Parameters:

  • y_true (tensorflow.Tensor) – A matrix of shape [batch-size, factor-count], that indicates for each pair in the batch and each factor, to what extent the two instances from z_tilde_a and z_tilde_b are correlated along this factor. This correlation is assumed to be in the range [0,1]. The residual factor (located at index 0) is typically not the same for any two instances and thus usually stores a zero in this array.

  • y_pred (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) – A concatenation along the dimension axis for the tensors [z_tilde_a, z_tilde_b, j_a, j_b]. Shape == [batch-size, 2*dimensionality of z-tilde + 2]

  • z_tilde_a: The output of model T on the first input of the pair (z^a, z^b). Shape == [batch size, dimension count] where dimension count is the number of dimensions in the flattened output of T.

  • z_tilde_b: The output of model T on the second input of the pair (z^a, z^b). Shape == [batch size, dimension count] where dimension count is the number of dimensions in the flattened output of T.

  • j_a: The jacobian determinant on logarithmic scale of T at z^a. Shape == [batch size,1]

  • j_b: The jacobian determinant on logarithmic scale of T at z^b. Shape == [batch size,1]

Returns:

loss (Union[tensorflow.Tensor, tensorflow.keras.KerasTensor]) - A single value indicating the amount of error the model makes in factoring its inputs.

gyoza.modelling.masks module

class gyoza.modelling.masks.CheckerBoardMask(*args, **kwargs)[source]

Bases: Mask

A mask of ones and zeros arranged in a checkerboard .

Parameters:

  • axes (List[int]) – The axes along which the checkerboard pattern shall be applied. Assumed to be consecutive indices, e.g. [2,3] or [3,4].

  • shape (List[int]) – The shape of the mask along axes, e.g. 64*32 if an input \(x\) has shape [10,3,64,32] and axes == [2,3].

classmethod from_config(config) → MaskType[source]

Loads a gyoza.modelling.masks.Mask from its configuration dictionary. The dictionary is required to have been obtained via the corresponding mask’s gyoza.modelling.masks.Mask.get_config().

Parameters:

  • cls (MaskType) – The specific type of the mask to be constructed.

  • config (Dict[str, Any]) – The dictionary containing the configuration of the mask.

Returns:

instance (MaskType) - The constructed mask instance.

class gyoza.modelling.masks.HeavisideMask(*args, **kwargs)[source]

Bases: Mask

Applies a Heaviside function to its input \(x\), e.g. 0001111. Important: The Heaviside function is defined on a vector, yet by the requirement of Mask, inputs \(x\) to this layer are allowed to have more than one axis in axes. As described in Mask, an input \(x\) is first flattened along axes and thus Heaviside can be applied. For background information see Mask.

Parameters:

  • shape (List[int]) – See base class Mask.

  • axes (List[int]) – See base class Mask.

classmethod from_config(config: Dict[str, Any]) → MaskType[source]

Loads a gyoza.modelling.masks.Mask from its configuration dictionary. The dictionary is required to have been obtained via the corresponding mask’s gyoza.modelling.masks.Mask.get_config().

Parameters:

  • cls (MaskType) – The specific type of the mask to be constructed.

  • config (Dict[str, Any]) – The dictionary containing the configuration of the mask.

Returns:

instance (MaskType) - The constructed mask instance.

class gyoza.modelling.masks.Mask(*args, **kwargs)[source]

Bases: Layer

This class can be used to let half of the elements of a tensor \(x\) pass through while cancelling out the other half. An input \(x\) to this layer is expected to have shape along axes. It is then flattened along these axes and the mask is applied. Thereafter, \(x\) is unflattened and returned.

Parameters:

  • axes (List[int]) – The axes along which the selection shall be applied.

  • shape (List[int]) – The shape that input \(x\) to this layer has along axes.

  • mask (tensorflow.Tensor) – The mask to be applied to data passing through this layer. Its shape is expected to be a vector with product(shape) many entries.

_axes_

(List[int]) - The axes along which the selection shall be applied.

_mask_

(tensorflow.Tensor) - The mask to be applied to input :math:`x to this layer.

_shape_

(List[int]) - The shape that input \(x\) to this layer has along gyoza.modelling.masks.Mask.axes.

arrange(x: Tensor) → Tensor[source]

Arranges x into a vector such that all elements set to 0 by gyoza.modelling.masks.Mask.mask() are enumerated first and all elements that passed through the mask are enumerated last.

Parameters:

x (tensorflow.Tensor) – The data to be arranged. The shape is assumed to be compatible with gyoza.modelling.masks.Mask.mask().

Returns:

x_flat (tensorflow.Tensor) - The arranged version of x whose shape is flattened along the first axis of attribute _axes_.

call(inputs: Tensor, is_positive: bool = True) → Tensor[source]

Applies the binary mask of self to inputs.

Parameters:

  • inputs (tensorflow.Tensor) – The data to be masked. The expected shape of inputs depends on the axis and shape specified during initialization.

  • is_positive (bool, optional, default == True) – Indicates whether the positive or negative mask version shall be applied. For the positive configuration, 1s let entries pass while 0 cancel them out. For the negative configuration, roles are switched.

Returns:

x_masked (tensorflow.Tensor) - The masked data of same shape as inputs.

classmethod from_config(config: Dict[str, Any]) → MaskType[source]

Loads a gyoza.modelling.masks.Mask from its configuration dictionary. The dictionary is required to have been obtained via the corresponding mask’s gyoza.modelling.masks.Mask.get_config().

Parameters:

  • cls (MaskType) – The specific type of the mask to be constructed.

  • config (Dict[str, Any]) – The dictionary containing the configuration of the mask.

Returns:

instance (MaskType) - The constructed mask instance.

get_config() → Dict[str, Any][source]

Converts self to a configuration dictionary that can be used to reconstruct it via the corresponding mask class’s gyoza.modelling.masks.Mask.from_config().

Returns:

config (Dict[str, Any]) - The configuration dictionary.

re_arrange(x_new: Tensor) → Tensor[source]

This function is the inverse of gyoza.modelling.masks.Mask.arrange().

Parameters:

x_new – The output of gyoza.modelling.masks.Mask.arrange().

Returns:

x (tensorflow.Tensor) - The input to gyoza.modelling.masks.Mask.arrange().

Module contents