lasagne.layers.cuda_convnet
¶
This module houses layers that require pylearn2 <https://deeplearning.net/software/pylearn2> to work. Its layers are not automatically imported into the lasagne.layers
namespace: To use these layers, you need to import lasagne.layers.cuda_convnet
explicitly.
-
class
lasagne.layers.cuda_convnet.
Conv2DCCLayer
(incoming, num_filters, filter_size, stride=(1, 1), pad=0, untie_biases=False, W=None, b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, dimshuffle=True, flip_filters=False, partial_sum=1, **kwargs)[source]¶ 2D convolutional layer
Performs a 2D convolution on its input and optionally adds a bias and applies an elementwise nonlinearity. This is an alternative implementation which uses the cuda-convnet wrappers from pylearn2:
pylearn2.sandbox.cuda_convnet.filter_acts.FilterActs
.Parameters: incoming : a
Layer
instance or a tupleThe layer feeding into this layer, or the expected input shape. This layer expects a 4D tensor as its input, with shape
(batch_size, num_input_channels, input_rows, input_columns)
. If automatic dimshuffling is disabled (see notes), the shape should be(num_input_channels, input_rows, input_columns, batch_size)
instead (c01b axis order).num_filters : int
The number of learnable convolutional filters this layer has.
filter_size : int or iterable of int
An integer or a 2-element tuple specifying the size of the filters. This layer does not support non-square filters.
stride : int or iterable of int
An integer or a 2-element tuple specifying the stride of the convolution operation. This layer does not support using different strides along both axes.
pad : int, iterable of int, ‘full’, ‘same’ or ‘valid’ (default: 0)
By default, the convolution is only computed where the input and the filter fully overlap (a valid convolution). When
stride=1
, this yields an output that is smaller than the input byfilter_size - 1
. The pad argument allows you to implicitly pad the input with zeros, extending the output size.A single integer results in symmetric zero-padding of the given size on all borders. This layer does not support using different amounts of padding along both axes, but for compatibility to other layers you can still specify the padding as a tuple of two same-valued integers.
'full'
pads with one less than the filter size on both sides. This is equivalent to computing the convolution wherever the input and the filter overlap by at least one position.'same'
pads with half the filter size (rounded down) on both sides. Whenstride=1
this results in an output size equal to the input size. Even filter size is not supported.'valid'
is an alias for0
(no padding / a valid convolution).Note that
'full'
and'same'
can be faster than equivalent integer values due to optimizations by Theano.untie_biases : bool (default: False)
If
False
, the layer will have a bias parameter for each channel, which is shared across all positions in this channel. As a result, the b attribute will be a vector (1D).If
True
, the layer will have separate bias parameters for each position in each channel. As a result, the b attribute will be a 3D tensor.W : Theano shared variable, expression, numpy array or callable
Initial value, expression or initializer for the weights. These should be a 4D tensor with shape
(num_filters, num_input_channels, filter_rows, filter_columns)
. If automatic dimshuffling is disabled (see notes), the shape should be(num_input_channels, input_rows, input_columns, num_filters)
instead (c01b axis order). Seelasagne.utils.create_param()
for more information.b : Theano shared variable, expression, numpy array, callable or
None
Initial value, expression or initializer for the biases. If set to
None
, the layer will have no biases. Otherwise, biases should be a 1D array with shape(num_filters,)
if untied_biases is set toFalse
. If it is set toTrue
, its shape should be(num_filters, output_rows, output_columns)
instead. Seelasagne.utils.create_param()
for more information.nonlinearity : callable or None
The nonlinearity that is applied to the layer activations. If None is provided, the layer will be linear.
dimshuffle : bool (default: True)
If
True
, the layer will automatically apply the necessary dimshuffle operations to deal with the fact that the cuda-convnet implementation uses c01b (batch-size-last) axis order instead of bc01 (batch-size-first), which is the Lasagne/Theano default. This makes the layer interoperable with other Lasagne layers.If
False
, this automatic dimshuffling is disabled and the layer will expect its input and parameters to have c01b axis order. It is up to the user to ensure this.ShuffleBC01ToC01BLayer
andShuffleC01BToBC01Layer
can be used to convert between bc01 and c01b axis order.flip_filters : bool (default: False)
Whether to flip the filters and perform a convolution, or not to flip them and perform a correlation. Flipping adds a bit of overhead, so it is disabled by default. In most cases this does not make a difference anyway because the filters are learnt. However,
flip_filters
should be set toTrue
if weights are loaded into it that were learnt using a regularlasagne.layers.Conv2DLayer
, for example.partial_sum : int or None (default: 1)
This value tunes the trade-off between memory usage and performance. You can specify any positive integer that is a divisor of the output feature map size (i.e. output rows times output columns). Higher values decrease memory usage, but also performance. Specifying 0 or
None
means the highest possible value will be used. The Lasagne default of1
gives the best performance, but also the highest memory usage.More information about this parameter can be found in the cuda-convnet documentation.
**kwargs
Any additional keyword arguments are passed to the Layer superclass.
Notes
The cuda-convnet convolution implementation has several limitations:
- only square filters are supported.
- only identical strides in the horizontal and vertical direction are supported.
- the number of filters must be a multiple of 16.
- the number of input channels must be even, or less than or equal to 3.
- if the gradient w.r.t. the input is to be computed, the number of channels must be divisible by 4.
- performance is optimal when the batch size is a multiple of 128 (but other batch sizes are supported).
- this layer only works on the GPU.
The cuda-convnet convolution implementation uses c01b (batch-size-last) axis order by default. The Theano/Lasagne default is bc01 (batch-size-first). This layer automatically adds the necessary dimshuffle operations for the input and the parameters so that it is interoperable with other layers that assume bc01 axis order. However, these additional dimshuffle operations may sometimes negatively affect performance. For this reason, it is possible to disable them by setting
dimshuffle=False
. In this case, the user is expected to manually ensure that the input and parameters have the correct axis order.ShuffleBC01ToC01BLayer
andShuffleC01BToBC01Layer
can be used to convert between bc01 and c01b axis order.Attributes
W (Theano shared variable or expression) Variable or expression representing the filter weights. b (Theano shared variable or expression) Variable or expression representing the biases.
-
class
lasagne.layers.cuda_convnet.
MaxPool2DCCLayer
(incoming, pool_size, stride=None, ignore_border=False, dimshuffle=True, **kwargs)[source]¶ 2D max-pooling layer
Performs 2D max-pooling over the two trailing axes of a 4D input tensor (or over axis 1 and 2 if
dimshuffle=False
, see notes). This is an alternative implementation which uses the cuda-convnet wrappers from pylearn2:pylearn2.sandbox.cuda_convnet.pool.MaxPool
.Parameters: incoming : a
Layer
instance or tupleThe layer feeding into this layer, or the expected input shape.
pool_size : integer or iterable
The length of the pooling region in each dimension. If an integer, it is promoted to a square pooling region. If an iterable, it should have two elements. This layer does not support non-square pooling regions.
stride : integer, iterable or
None
The strides between sucessive pooling regions in each dimension. If
None
thenstride = pool_size
. This layer does not support using different strides along both axes.pad : integer or iterable (default: 0)
This implementation does not support custom padding, so this argument must always be set to
0
. It exists only to make sure the interface is compatible withlasagne.layers.MaxPool2DLayer
.ignore_border : bool (default: False)
This implementation always includes partial pooling regions, so this argument must always be set to False. It exists only to make sure the interface is compatible with
lasagne.layers.MaxPool2DLayer
.dimshuffle : bool (default: True)
If
True
, the layer will automatically apply the necessary dimshuffle operations to deal with the fact that the cuda-convnet implementation uses c01b (batch-size-last) axis order instead of bc01 (batch-size-first), which is the Lasagne/Theano default. This makes the layer interoperable with other Lasagne layers.If
False
, this automatic dimshuffling is disabled and the layer will expect its input and parameters to have c01b axis order. It is up to the user to ensure this.ShuffleBC01ToC01BLayer
andShuffleC01BToBC01Layer
can be used to convert between bc01 and c01b axis order.**kwargs
Any additional keyword arguments are passed to the
Layer
superclass.Notes
The cuda-convnet max-pooling implementation has several limitations:
- only square pooling regions are supported.
- only identical strides in the horizontal and vertical direction are supported.
- only square inputs are supported. (This limitation does not exist for the convolution implementation.)
- partial pooling regions are always included (
ignore_border
is forced toFalse
). - custom padding is not supported (
pad
is forced to0
). - this layer only works on the GPU.
The cuda-convnet pooling implementation uses c01b (batch-size-last) axis order by default. The Theano/Lasagne default is bc01 (batch-size-first). This layer automatically adds the necessary dimshuffle operations for the input and the parameters so that it is interoperable with other layers that assume bc01 axis order. However, these additional dimshuffle operations may sometimes negatively affect performance. For this reason, it is possible to disable them by setting
dimshuffle=False
. In this case, the user is expected to manually ensure that the input and parameters have the correct axis order.ShuffleBC01ToC01BLayer
andShuffleC01BToBC01Layer
can be used to convert between bc01 and c01b axis order.
-
class
lasagne.layers.cuda_convnet.
ShuffleBC01ToC01BLayer
(incoming, name=None)[source]¶ shuffle 4D input from bc01 (batch-size-first) order to c01b (batch-size-last) order.
This layer can be used for interoperability between c01b and bc01 layers. For example,
MaxPool2DCCLayer
andConv2DCCLayer
operate in c01b mode when they are created withdimshuffle=False
.Parameters: incoming : a
Layer
instance or tupleThe layer feeding into this layer, or the expected input shape.
**kwargs
Any additional keyword arguments are passed to the Layer superclass.
-
lasagne.layers.cuda_convnet.
bc01_to_c01b
[source]¶ alias of
ShuffleBC01ToC01BLayer
-
class
lasagne.layers.cuda_convnet.
ShuffleC01BToBC01Layer
(incoming, name=None)[source]¶ shuffle 4D input from c01b (batch-size-last) order to bc01 (batch-size-first) order.
This layer can be used for interoperability between c01b and bc01 layers. For example,
MaxPool2DCCLayer
andConv2DCCLayer
operate in c01b mode when they are created withdimshuffle=False
.Parameters: incoming : a
Layer
instance or tupleThe layer feeding into this layer, or the expected input shape.
**kwargs
Any additional keyword arguments are passed to the Layer superclass.
-
lasagne.layers.cuda_convnet.
c01b_to_bc01
[source]¶ alias of
ShuffleC01BToBC01Layer
-
class
lasagne.layers.cuda_convnet.
NINLayer_c01b
(incoming, num_units, untie_biases=False, W=lasagne.init.GlorotUniform(), b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, **kwargs)[source]¶ Network-in-network layer with c01b axis ordering.
This is a c01b version of
lasagne.layers.NINLayer
.Parameters: incoming : a
Layer
instance or a tupleThe layer feeding into this layer, or the expected input shape
num_units : int
The number of units of the layer
untie_biases : bool
If
False
, the network has a single bias vector similar to a dense layer. IfTrue
, a separate bias vector is used for each spatial position.W : Theano shared variable, numpy array or callable
An initializer for the weights of the layer. If a shared variable or a numpy array is provided the shape should be (num_units, num_input_channels). See
lasagne.utils.create_param()
for more information.b : Theano shared variable, numpy array, callable or None
An initializer for the biases of the layer. If a shared variable or a numpy array is provided the correct shape is determined by the untie_biases setting. If untie_biases is
False
, then the shape should be(num_units,)
. If untie_biases isTrue
then the shape should be(num_units, rows, columns)
. IfNone
is provided the layer will have no biases. Seelasagne.utils.create_param()
for more information.nonlinearity : callable or None
The nonlinearity that is applied to the layer activations. If None is provided, the layer will be linear.
**kwargs
Any additional keyword arguments are passed to the Layer superclass.