odak.learn.models
odak.learn.models
Provides necessary definitions for components used in machine learning and deep learning.
channel_gate
¶
Bases: Module
Channel attention module with various pooling strategies. This class is heavily inspired https://github.com/Jongchan/attention-module/commit/e4ee180f1335c09db14d39a65d97c8ca3d1f7b16 (MIT License).
Source code in odak/learn/models/components.py
__init__(gate_channels, reduction_ratio=16, pool_types=['avg', 'max'])
¶
Initializes the channel gate module.
Parameters:
-
gate_channels
–Number of channels of the input feature map.
-
reduction_ratio
(int
, default:16
) –Reduction ratio for the intermediate layer.
-
pool_types
–List of pooling operations to apply.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the ChannelGate module.
Applies channel-wise attention to the input tensor.
Parameters:
-
x
–Input tensor to the ChannelGate module.
Returns:
-
output
(tensor
) –Output tensor after applying channel attention.
Source code in odak/learn/models/components.py
convolution_layer
¶
Bases: Module
A convolution layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, output_channels=2, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A convolutional layer class.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
, default:2
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
convolutional_block_attention
¶
Bases: Module
Convolutional Block Attention Module (CBAM) class. This class is heavily inspired https://github.com/Jongchan/attention-module/commit/e4ee180f1335c09db14d39a65d97c8ca3d1f7b16 (MIT License).
Source code in odak/learn/models/components.py
Flatten
¶
__init__(gate_channels, reduction_ratio=16, pool_types=['avg', 'max'], no_spatial=False)
¶
Initializes the convolutional block attention module.
Parameters:
-
gate_channels
–Number of channels of the input feature map.
-
reduction_ratio
(int
, default:16
) –Reduction ratio for the channel attention.
-
pool_types
–List of pooling operations to apply for channel attention.
-
no_spatial
–If True, spatial attention is not applied.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the convolutional block attention module.
Parameters:
-
x
–Input tensor to the CBAM module.
Returns:
-
x_out
(tensor
) –Output tensor after applying channel and spatial attention.
Source code in odak/learn/models/components.py
double_convolution
¶
Bases: Module
A double convolution layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, mid_channels=None, output_channels=2, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
Double convolution model.
Parameters:
-
input_channels
–Number of input channels.
-
mid_channels
–Number of channels in the hidden layer between two convolutions.
-
output_channels
(int
, default:2
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
downsample_layer
¶
Bases: Module
A downscaling component followed by a double convolution.
Source code in odak/learn/models/components.py
__init__(input_channels, output_channels, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A downscaling component with a double convolution.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–First input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
multi_layer_perceptron
¶
Bases: Module
A multi-layer perceptron model.
Source code in odak/learn/models/models.py
__init__(dimensions, activation=torch.nn.ReLU(), bias=False, model_type='conventional', siren_multiplier=1.0, input_multiplier=None)
¶
Parameters:
-
dimensions
–List of integers representing the dimensions of each layer (e.g., [2, 10, 1], where the first layer has two channels and last one has one channel.).
-
activation
–Nonlinear activation function. Default is `torch.nn.ReLU()`.
-
bias
–If set to True, linear layers will include biases.
-
siren_multiplier
–When using `SIREN` model type, this parameter functions as a hyperparameter. The original SIREN work uses 30. You can bypass this parameter by providing input that are not normalized and larger then one.
-
input_multiplier
–Initial value of the input multiplier before the very first layer.
-
model_type
–Model type: `conventional`, `swish`, `SIREN`, `FILM SIREN`, `Gaussian`. `conventional` refers to a standard multi layer perceptron. For `SIREN,` see: Sitzmann, Vincent, et al. "Implicit neural representations with periodic activation functions." Advances in neural information processing systems 33 (2020): 7462-7473. For `Swish,` see: Ramachandran, Prajit, Barret Zoph, and Quoc V. Le. "Searching for activation functions." arXiv preprint arXiv:1710.05941 (2017). For `FILM SIREN,` see: Chan, Eric R., et al. "pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021. For `Gaussian,` see: Ramasinghe, Sameera, and Simon Lucey. "Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps." In European Conference on Computer Vision, pp. 142-158. Cham: Springer Nature Switzerland, 2022.
Source code in odak/learn/models/models.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/models.py
non_local_layer
¶
Bases: Module
Self-Attention Layer [zi = Wzyi + xi] (non-local block : ref https://arxiv.org/abs/1711.07971)
Source code in odak/learn/models/components.py
__init__(input_channels=1024, bottleneck_channels=512, kernel_size=1, bias=False)
¶
Parameters:
-
input_channels
–Number of input channels.
-
bottleneck_channels
(int
, default:512
) –Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model [zi = Wzyi + xi]
Parameters:
-
x
–First input data.
Returns:
-
z
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
normalization
¶
Bases: Module
A normalization layer.
Source code in odak/learn/models/components.py
__init__(dim=1)
¶
Normalization layer.
Parameters:
-
dim
–Dimension (axis) to normalize.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
positional_encoder
¶
Bases: Module
A positional encoder module.
Source code in odak/learn/models/components.py
__init__(L)
¶
A positional encoder module.
Parameters:
-
L
–Positional encoding level.
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Result of the forward operation
Source code in odak/learn/models/components.py
residual_attention_layer
¶
Bases: Module
A residual block with an attention layer.
Source code in odak/learn/models/components.py
284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 |
|
__init__(input_channels=2, output_channels=2, kernel_size=1, bias=False, activation=torch.nn.ReLU())
¶
An attention layer class.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int or optional
, default:2
) –Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x0, x1)
¶
Forward model.
Parameters:
-
x0
–First input data.
-
x1
–Seconnd input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
residual_layer
¶
Bases: Module
A residual layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, mid_channels=16, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A convolutional layer class.
Parameters:
-
input_channels
–Number of input channels.
-
mid_channels
–Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
spatial_gate
¶
Bases: Module
Spatial attention module that applies a convolution layer after channel pooling. This class is heavily inspired by https://github.com/Jongchan/attention-module/blob/master/MODELS/cbam.py.
Source code in odak/learn/models/components.py
__init__()
¶
Initializes the spatial gate module.
channel_pool(x)
¶
Applies max and average pooling on the channels.
Parameters:
-
x
–Input tensor.
Returns:
-
output
(tensor
) –Output tensor.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the SpatialGate module.
Applies spatial attention to the input tensor.
Parameters:
-
x
–Input tensor to the SpatialGate module.
Returns:
-
scaled_x
(tensor
) –Output tensor after applying spatial attention.
Source code in odak/learn/models/components.py
unet
¶
Bases: Module
A U-Net model, heavily inspired from https://github.com/milesial/Pytorch-UNet/tree/master/unet
and more can be read from Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015.
Source code in odak/learn/models/models.py
99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 |
|
__init__(depth=4, dimensions=64, input_channels=2, output_channels=1, bilinear=False, kernel_size=3, bias=False, activation=torch.nn.ReLU(inplace=True))
¶
U-Net model.
Parameters:
-
depth
–Number of upsampling and downsampling
-
dimensions
–Number of dimensions.
-
input_channels
–Number of input channels.
-
output_channels
–Number of output channels.
-
bilinear
–Uses bilinear upsampling in upsampling layers when set True.
-
bias
–Set True to let convolutional layers learn a bias term.
-
activation
–Non-linear activation layer to be used (e.g., torch.nn.ReLU(), torch.nn.Sigmoid().
Source code in odak/learn/models/models.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/models.py
upsample_layer
¶
Bases: Module
An upsampling convolutional layer.
Source code in odak/learn/models/components.py
509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 |
|
__init__(input_channels, output_channels, kernel_size=3, bias=False, activation=torch.nn.ReLU(), bilinear=True)
¶
A downscaling component with a double convolution.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
-
bilinear
–If set to True, bilinear sampling is used.
Source code in odak/learn/models/components.py
forward(x1, x2)
¶
Forward model.
Parameters:
-
x1
–First input data.
-
x2
–Second input data.
Returns:
-
result
(tensor
) –Result of the forward operation
Source code in odak/learn/models/components.py
gaussian(x, multiplier=1.0)
¶
A Gaussian non-linear activation. For more details: Ramasinghe, Sameera, and Simon Lucey. "Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps." In European Conference on Computer Vision, pp. 142-158. Cham: Springer Nature Switzerland, 2022.
Parameters:
-
x
–Input data.
-
multiplier
–Multiplier.
Returns:
-
result
(float or tensor
) –Ouput data.
Source code in odak/learn/models/components.py
swish(x)
¶
A swish non-linear activation. For more details: https://en.wikipedia.org/wiki/Swish_function
Parameters:
-
x
–Input.
Returns:
-
out
(float or tensor
) –Output.
Source code in odak/learn/models/components.py
channel_gate
¶
Bases: Module
Channel attention module with various pooling strategies. This class is heavily inspired https://github.com/Jongchan/attention-module/commit/e4ee180f1335c09db14d39a65d97c8ca3d1f7b16 (MIT License).
Source code in odak/learn/models/components.py
__init__(gate_channels, reduction_ratio=16, pool_types=['avg', 'max'])
¶
Initializes the channel gate module.
Parameters:
-
gate_channels
–Number of channels of the input feature map.
-
reduction_ratio
(int
, default:16
) –Reduction ratio for the intermediate layer.
-
pool_types
–List of pooling operations to apply.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the ChannelGate module.
Applies channel-wise attention to the input tensor.
Parameters:
-
x
–Input tensor to the ChannelGate module.
Returns:
-
output
(tensor
) –Output tensor after applying channel attention.
Source code in odak/learn/models/components.py
convolution_layer
¶
Bases: Module
A convolution layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, output_channels=2, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A convolutional layer class.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
, default:2
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
convolutional_block_attention
¶
Bases: Module
Convolutional Block Attention Module (CBAM) class. This class is heavily inspired https://github.com/Jongchan/attention-module/commit/e4ee180f1335c09db14d39a65d97c8ca3d1f7b16 (MIT License).
Source code in odak/learn/models/components.py
Flatten
¶
__init__(gate_channels, reduction_ratio=16, pool_types=['avg', 'max'], no_spatial=False)
¶
Initializes the convolutional block attention module.
Parameters:
-
gate_channels
–Number of channels of the input feature map.
-
reduction_ratio
(int
, default:16
) –Reduction ratio for the channel attention.
-
pool_types
–List of pooling operations to apply for channel attention.
-
no_spatial
–If True, spatial attention is not applied.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the convolutional block attention module.
Parameters:
-
x
–Input tensor to the CBAM module.
Returns:
-
x_out
(tensor
) –Output tensor after applying channel and spatial attention.
Source code in odak/learn/models/components.py
double_convolution
¶
Bases: Module
A double convolution layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, mid_channels=None, output_channels=2, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
Double convolution model.
Parameters:
-
input_channels
–Number of input channels.
-
mid_channels
–Number of channels in the hidden layer between two convolutions.
-
output_channels
(int
, default:2
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
downsample_layer
¶
Bases: Module
A downscaling component followed by a double convolution.
Source code in odak/learn/models/components.py
__init__(input_channels, output_channels, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A downscaling component with a double convolution.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–First input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
non_local_layer
¶
Bases: Module
Self-Attention Layer [zi = Wzyi + xi] (non-local block : ref https://arxiv.org/abs/1711.07971)
Source code in odak/learn/models/components.py
__init__(input_channels=1024, bottleneck_channels=512, kernel_size=1, bias=False)
¶
Parameters:
-
input_channels
–Number of input channels.
-
bottleneck_channels
(int
, default:512
) –Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model [zi = Wzyi + xi]
Parameters:
-
x
–First input data.
Returns:
-
z
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
normalization
¶
Bases: Module
A normalization layer.
Source code in odak/learn/models/components.py
__init__(dim=1)
¶
Normalization layer.
Parameters:
-
dim
–Dimension (axis) to normalize.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
positional_encoder
¶
Bases: Module
A positional encoder module.
Source code in odak/learn/models/components.py
__init__(L)
¶
A positional encoder module.
Parameters:
-
L
–Positional encoding level.
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Result of the forward operation
Source code in odak/learn/models/components.py
residual_attention_layer
¶
Bases: Module
A residual block with an attention layer.
Source code in odak/learn/models/components.py
284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 |
|
__init__(input_channels=2, output_channels=2, kernel_size=1, bias=False, activation=torch.nn.ReLU())
¶
An attention layer class.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int or optional
, default:2
) –Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x0, x1)
¶
Forward model.
Parameters:
-
x0
–First input data.
-
x1
–Seconnd input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/components.py
residual_layer
¶
Bases: Module
A residual layer.
Source code in odak/learn/models/components.py
__init__(input_channels=2, mid_channels=16, kernel_size=3, bias=False, activation=torch.nn.ReLU())
¶
A convolutional layer class.
Parameters:
-
input_channels
–Number of input channels.
-
mid_channels
–Number of middle channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
Source code in odak/learn/models/components.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
spatial_gate
¶
Bases: Module
Spatial attention module that applies a convolution layer after channel pooling. This class is heavily inspired by https://github.com/Jongchan/attention-module/blob/master/MODELS/cbam.py.
Source code in odak/learn/models/components.py
__init__()
¶
Initializes the spatial gate module.
channel_pool(x)
¶
Applies max and average pooling on the channels.
Parameters:
-
x
–Input tensor.
Returns:
-
output
(tensor
) –Output tensor.
Source code in odak/learn/models/components.py
forward(x)
¶
Forward pass of the SpatialGate module.
Applies spatial attention to the input tensor.
Parameters:
-
x
–Input tensor to the SpatialGate module.
Returns:
-
scaled_x
(tensor
) –Output tensor after applying spatial attention.
Source code in odak/learn/models/components.py
upsample_layer
¶
Bases: Module
An upsampling convolutional layer.
Source code in odak/learn/models/components.py
509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 |
|
__init__(input_channels, output_channels, kernel_size=3, bias=False, activation=torch.nn.ReLU(), bilinear=True)
¶
A downscaling component with a double convolution.
Parameters:
-
input_channels
–Number of input channels.
-
output_channels
(int
) –Number of output channels.
-
kernel_size
–Kernel size.
-
bias
–Set to True to let convolutional layers have bias term.
-
activation
–Nonlinear activation layer to be used. If None, uses torch.nn.ReLU().
-
bilinear
–If set to True, bilinear sampling is used.
Source code in odak/learn/models/components.py
forward(x1, x2)
¶
Forward model.
Parameters:
-
x1
–First input data.
-
x2
–Second input data.
Returns:
-
result
(tensor
) –Result of the forward operation
Source code in odak/learn/models/components.py
gaussian(x, multiplier=1.0)
¶
A Gaussian non-linear activation. For more details: Ramasinghe, Sameera, and Simon Lucey. "Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps." In European Conference on Computer Vision, pp. 142-158. Cham: Springer Nature Switzerland, 2022.
Parameters:
-
x
–Input data.
-
multiplier
–Multiplier.
Returns:
-
result
(float or tensor
) –Ouput data.
Source code in odak/learn/models/components.py
swish(x)
¶
A swish non-linear activation. For more details: https://en.wikipedia.org/wiki/Swish_function
Parameters:
-
x
–Input.
Returns:
-
out
(float or tensor
) –Output.
Source code in odak/learn/models/components.py
multi_layer_perceptron
¶
Bases: Module
A multi-layer perceptron model.
Source code in odak/learn/models/models.py
__init__(dimensions, activation=torch.nn.ReLU(), bias=False, model_type='conventional', siren_multiplier=1.0, input_multiplier=None)
¶
Parameters:
-
dimensions
–List of integers representing the dimensions of each layer (e.g., [2, 10, 1], where the first layer has two channels and last one has one channel.).
-
activation
–Nonlinear activation function. Default is `torch.nn.ReLU()`.
-
bias
–If set to True, linear layers will include biases.
-
siren_multiplier
–When using `SIREN` model type, this parameter functions as a hyperparameter. The original SIREN work uses 30. You can bypass this parameter by providing input that are not normalized and larger then one.
-
input_multiplier
–Initial value of the input multiplier before the very first layer.
-
model_type
–Model type: `conventional`, `swish`, `SIREN`, `FILM SIREN`, `Gaussian`. `conventional` refers to a standard multi layer perceptron. For `SIREN,` see: Sitzmann, Vincent, et al. "Implicit neural representations with periodic activation functions." Advances in neural information processing systems 33 (2020): 7462-7473. For `Swish,` see: Ramachandran, Prajit, Barret Zoph, and Quoc V. Le. "Searching for activation functions." arXiv preprint arXiv:1710.05941 (2017). For `FILM SIREN,` see: Chan, Eric R., et al. "pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021. For `Gaussian,` see: Ramasinghe, Sameera, and Simon Lucey. "Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps." In European Conference on Computer Vision, pp. 142-158. Cham: Springer Nature Switzerland, 2022.
Source code in odak/learn/models/models.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.
Source code in odak/learn/models/models.py
unet
¶
Bases: Module
A U-Net model, heavily inspired from https://github.com/milesial/Pytorch-UNet/tree/master/unet
and more can be read from Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015.
Source code in odak/learn/models/models.py
99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 |
|
__init__(depth=4, dimensions=64, input_channels=2, output_channels=1, bilinear=False, kernel_size=3, bias=False, activation=torch.nn.ReLU(inplace=True))
¶
U-Net model.
Parameters:
-
depth
–Number of upsampling and downsampling
-
dimensions
–Number of dimensions.
-
input_channels
–Number of input channels.
-
output_channels
–Number of output channels.
-
bilinear
–Uses bilinear upsampling in upsampling layers when set True.
-
bias
–Set True to let convolutional layers learn a bias term.
-
activation
–Non-linear activation layer to be used (e.g., torch.nn.ReLU(), torch.nn.Sigmoid().
Source code in odak/learn/models/models.py
forward(x)
¶
Forward model.
Parameters:
-
x
–Input data.
Returns:
-
result
(tensor
) –Estimated output.