Convolution-like operations with sparse matrix multiplication.
To read about different sparse formats, see U{http://www-users.cs.umn.edu/~saad/software/SPARSKIT/paper.ps}.
@todo: Automatic methods for determining best sparse format?
Build indices for a sparse CSC matrix that could implement A (convolve) B.
This generates a sparse matrix M, which generates a stack of image patches when computing the dot product of M with image patch. Convolution is then simply the dot product of (img x M) and the kernels.
Build a sparse matrix which can be used for performing... * convolution: in this case, the dot product of this matrix with the input images will generate a stack of images patches. Convolution is then a tensordot operation of the filters and the patch stack. * sparse local connections: in this case, the sparse matrix allows us to operate the weight matrix as if it were fully-connected. The structured-dot with the input image gives the output for the following layer.
Parameters: |
|
---|---|
Return type: | tuple(indices, indptr, logical_shape, sp_type, out_img_shp) |
Returns: | the structure of a sparse matrix, and the logical dimensions of the image which will be the result of filtering. |
“images” is assumed to be a matrix of shape batch_size x img_size, where the second dimension represents each image in raster order
Output feature map will have shape:
batch_size x number of kernels * output_size
Note
IMPORTANT: note that this means that each feature map is contiguous in memory.
The memory layout will therefore be: [ <feature_map_0> <feature_map_1> ... <feature_map_n>], where <feature_map> represents a “feature map” in raster order
Note that the concept of feature map doesn’t really apply to sparse filters without weight sharing. Basically, nkern=1 will generate one output img/feature map, nkern=2 a second feature map, etc.
kerns is a 1D tensor, and assume to be of shape:
nkern * N.prod(outshp) x N.prod(kshp)
Each filter is applied seperately to consecutive output pixels.
Parameters: |
|
---|---|
Returns: | out1, symbolic result |
Returns: | out2, logical shape of the output img (nkern,height,width) (after dot product, not of the sparse matrix!) |
Convolution implementation by sparse matrix multiplication.
Note : | For best speed, put the matrix which you expect to be smaller as the ‘kernel’ argument |
---|
“images” is assumed to be a matrix of shape batch_size x img_size, where the second dimension represents each image in raster order
If flatten is “False”, the output feature map will have shape:
batch_size x number of kernels x output_size
If flatten is “True”, the output feature map will have shape:
batch_size x number of kernels * output_size
Note
IMPORTANT: note that this means that each feature map (image generate by each kernel) is contiguous in memory. The memory layout will therefore be: [ <feature_map_0> <feature_map_1> ... <feature_map_n>], where <feature_map> represents a “feature map” in raster order
kerns is a 2D tensor of shape nkern x N.prod(kshp)
Parameters: |
|
---|---|
Returns: | out1, symbolic result |
Returns: | out2, logical shape of the output img (nkern,heigt,width) |
Todo : | test for 1D and think of how to do n-d convolutions |
Implements a max pooling layer
Takes as input a 2D tensor of shape batch_size x img_size and performs max pooling. Max pooling downsamples by taking the max value in a given area, here defined by maxpoolshp. Outputs a 2D tensor of shape batch_size x output_size.
Parameters: |
|
---|---|
Returns: | out1, symbolic result (2D tensor) |
Returns: | out2, logical shape of the output |
Return a sparse matrix having random values from a binomial density having number of experiment n and probability of succes p.
WARNING: This Op is NOT deterministic, as calling it twice with the same inputs will NOT give the same result. This is a violation of Theano’s contract for Ops
Parameters: |
|
---|---|
Returns: | A sparse matrix of integers representing the number of success. |
Return a sparse matrix having random values from a multinomial density having number of experiment n and probability of succes p.
WARNING: This Op is NOT deterministic, as calling it twice with the same inputs will NOT give the same result. This is a violation of Theano’s contract for Ops
Parameters: |
|
---|---|
Returns: | A sparse matrix of random integers from a multinomial density for each row. |
Note : | It will works only if p have csr format. |
Return a sparse having random values from a Poisson density with mean from the input.
WARNING: This Op is NOT deterministic, as calling it twice with the same inputs will NOT give the same result. This is a violation of Theano’s contract for Ops
Parameters: | x – Sparse matrix. |
---|---|
Returns: | A sparse matrix of random integers of a Poisson density with mean of x element wise. |
Attributes: grad_preserves_dense - a boolean flags [default: True]. grad_preserves_dense controls whether gradients with respect to inputs are converted to dense matrices when the corresponding input y is dense (not in a L{SparseVariable} wrapper). This is generally a good idea when L{Dot} is in the middle of a larger graph, because the types of gy will match that of y. This conversion might be inefficient if the gradients are graph outputs though, hence this mask.
@todo: Simplify code by splitting into DotSS and DotSD.
Note : | Because of trickiness of implementing, we assume that the left argument x is SparseVariable (not dense) |
---|
@todo: Verify that output is sufficiently sparse, and raise a warning if it is not @todo: Also determine that we are storing the output in the best storage format?
@todo: Maybe the triple-transposition formulation (when x is dense) is slow. See if there is a direct way to do this.