In the tutorial section, you can find a sparse tutorial.
The sparse submodule is not loaded when we import Theano. You must import theano.sparse to enable it.
The sparse module provides the same functionality as the tensor module. The difference lies under the covers because sparse matrices do not store data in a contiguous array. Note that there are no GPU implementations for sparse matrices in Theano. The sparse module has been used in:
This section tries to explain how information is stored for the two sparse formats of SciPy supported by Theano. There are more formats that can be used with SciPy and some documentation about them may be found here.
Theano supports two compressed sparse formats csc and csr, respectively based on columns and rows. They have both the same attributes: data, indices, indptr and shape.
- The data attribute is a one-dimentionnal ndarray which contains all the non-zero elements of the sparse matrix.
- The indices and indptr attributes are used to store the position of the data in the sparse matrix.
- The shape attribute is exactly the same as the shape attribute of a dense (i.e. generic) matrix. It can be explicitly specified at the creation of a sparse matrix if it cannot be infered from the first three attributes.
In the Compressed Sparse Column format, indices stands for indexes inside the column vectors of the matrix and indptr tells where the column starts in the data and in the indices attributes. indptr can be thought of as giving the slice which must be applied to the other attribute in order to get each column of the matrix. In other words, slice(indptr[i], indptr[i+1]) corresponds to the slice needed to find the i-th column of the matrix in the data and indices fields.
The following example builds a matrix and returns its columns. It prints the i-th column, i.e. a list of indices in the column and their corresponding value in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csc_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 0 0]
[8 0 0]
[0 9 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
In the Compressed Sparse Row format, indices stands for indexes inside the row vectors of the matrix and indptr tells where the row starts in the data and in the indices attributes. indptr can be thought of as giving the slice which must be applied to the other attribute in order to get each row of the matrix. In other words, slice(indptr[i], indptr[i+1]) corresponds to the slice needed to find the i-th row of the matrix in the data and indices fields.
The following example builds a matrix and returns its rows. It prints the i-th row, i.e. a list of indices in the row and their corresponding value in the second list.
>>> data = np.asarray([7, 8, 9])
>>> indices = np.asarray([0, 1, 2])
>>> indptr = np.asarray([0, 2, 3, 3])
>>> m = sp.csr_matrix((data, indices, indptr), shape=(3, 3))
>>> print m.toarray()
[[7 8 0]
[0 0 9]
[0 0 0]]
>>> i = 0
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[0, 1] [7, 8]
>>> i = 1
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[2] [9]
>>> i = 2
>>> print m.indices[m.indptr[i]:m.indptr[i+1]], m.data[m.indptr[i]:m.indptr[i+1]]
[] []
They all have a structured grad.
dot.
- One of the inputs must be sparse, the other sparse or dense.
- The grad implemented is regular.
- No C code for perform and no C code for grad.
- Returns a dense for perform and a dense for grad.
- The first input is sparse, the second can be sparse or dense.
- The grad implemented is structured.
- C code for perform and grad.
- It returns a sparse output if both inputs are sparse and dense one if one of the inputs is dense.
- Returns a sparse grad for sparse inputs and dense grad for dense inputs.
- The first input is sparse, the second can be sparse or dense.
- The grad implemented is regular.
- No C code for perform and no C code for grad.
- Returns a Sparse.
- The gradient returns a Sparse for sparse inputs and by default a dense for dense inputs. The parameter grad_preserves_dense can be set to False to return a sparse grad for dense inputs.
- Both inputs must be dense.
- The grad implemented is structured for p.
- Sample of the dot and sample of the gradient.
- C code for perform but not for grad.
- Returns sparse for perform and grad.
usmm.
- You shouldn’t insert this op yourself!
- There is an optimization that transform a dot to Usmm when possible.
This op is the equivalent of gemm for sparse dot.
There is no grad implemented for this op.
One of the inputs must be sparse, the other sparse or dense.
Returns a dense from perform.
There is no grad implemented for these operations.
They all have a regular grad implemented.
Classes for handling sparse matrices.
To read about different sparse formats, see http://www-users.cs.umn.edu/~saad/software/SPARSKIT/paper.ps
Construct a CSC matrix from the internal representation.
Parameters: |
|
---|---|
Returns: | A sparse matrix having the properties specified by the inputs. |
Note: | The grad method returns a dense vector, so it provides a regular grad. |
Construct a CSR matrix from the internal representation.
Parameters: |
|
---|---|
Returns: | A sparse matrix having the properties specified by the inputs. |
Note: | The grad method returns a dense vector, so it provides a regular grad. |
Add two matrices, at least one of which is sparse.
This method will provide the right op according to the inputs.
Parameters: |
|
---|---|
Returns: | x + y |
Note: | At least one of x and y must be a sparse matrix. |
Note: | The grad will be structured only when one of the variable will be a dense matrix. |
Add two sparse matrices assuming they have the same sparsity pattern.
Parameters: |
|
---|---|
Returns: | The sum of the two sparse matrices element wise. |
Note: | x and y are assumed to have the same sparsity pattern. |
Note: | The grad implemented is structured. |
Wrapper around SparseVariable constructor to construct a Variable with a sparse matrix with the same dtype and format.
Parameters: | x – A sparse matrix. |
---|---|
Returns: | SparseVariable version of x. |
Same as as_sparse_variable but If we can’t make a sparse variable, we try to make a tensor variable. format.
Parameters: | x – A sparse matrix. |
---|---|
Returns: | SparseVariable or TensorVariable version of x. |
Wrapper around SparseVariable constructor to construct a Variable with a sparse matrix with the same dtype and format.
Parameters: | x – A sparse matrix. |
---|---|
Returns: | SparseVariable version of x. |
Cast sparse variable to the desired dtype.
Parameters: |
|
---|---|
Returns: | Same as x but having dtype as dtype. |
Note: | The grad implemented is regular, i.e. not structured. |
Remove explicit zeros from a sparse matrix, and re-sort indices.
CSR column indices are not necessarily sorted. Likewise for CSC row indices. Use clean when sorted indices are required (e.g. when passing data to other libraries) and to ensure there are no zeros in the data.
Parameters: | x – A sparse matrix. |
---|---|
Returns: | The same as x with indices sorted and zeros removed. |
Note: | The grad implemented is regular, i.e. not structured. |
Scale each columns of a sparse matrix by the corresponding element of a dense vector
Parameters: |
|
---|---|
Returns: | A sparse matrix in the same format as x which each column had been multiply by the corresponding element of s. |
Note: | The grad implemented is structured. |
Constructs a sparse matrix out of a list of 2-D matrix rows
Note: | The grad implemented is regular, i.e. not structured. |
---|
Convert a dense matrix to a sparse csc matrix. :param x: A dense matrix. :return: The same as x in a sparse csc matrix format.
return the data field of the sparse variable.
return the indices field of the sparse variable.
return the indptr field of the sparse variable.
Extract all of .data, .indices, .indptr and .shape field.
For specific field, csm_data, csm_indices, csm_indptr and csm_shape are provided.
Parameters: | csm – Sparse matrix in CSR or CSC format. |
---|---|
Returns: | (data, indices, indptr, shape), the properties of csm. |
Note: | The grad implemented is regular, i.e. not structured. infer_shape method is not available for this op. |
return the shape field of the sparse variable.
Convert a dense matrix to a sparse csr matrix. :param x: A dense matrix. :return: The same as x in a sparse csr matrix format.
Convert a sparse matrix to a dense one.
Parameters: | x – A sparse matrix. |
---|---|
Returns: | A dense matrix, the same as x. |
Note: | The grad implementation can be controlled through the constructor via the structured parameter. True will provide a structured grad while False will provide a regular grad. By default, the grad is structured. |
Extract the diagonal of a square sparse matrix as a dense vector.
param x: A square sparse matrix in csc format. return: A dense vector representing the diagonal elements.
Note
The grad implemented is regular, i.e. not structured, since the output is a dense vector.
Operation for efficiently calculating the dot product when one or all operands is sparse. Supported format are CSC and CSR. The output of the operation is dense.
Parameters: |
|
---|---|
Returns: | The dot product x.`y` in a dense format. |
Note: | The grad implemented is regular, i.e. not structured. |
Note: | At least one of x or y must be a sparse matrix. |
Note: | At least one of x or y must be a sparse matrix. |
Note: | When the operation has the form dot(csr_matrix, dense) the gradient of this operation can be performed inplace by UsmmCscDense. This leads to significant speed-ups. |
Re-sort indices of a sparse matrix.
CSR column indices are not necessarily sorted. Likewise for CSC row indices. Use ensure_sorted_indices when sorted indices are required (e.g. when passing data to other libraries).
Parameters: | x – A sparse matrix. |
---|---|
Returns: | The same as x with indices sorted. |
Note: | The grad implemented is regular, i.e. not structured. |
Parameters: |
|
---|---|
Returns: | x == y |
Note: | At least one of x and y must be a sparse matrix. |
Parameters: |
|
---|---|
Returns: | x >= y |
Note: | At least one of x and y must be a sparse matrix. |
Implement a subtensor of sparse variable, returning a sparse matrix.
If you want to take only one element of a sparse matrix see GetItemScalar that returns a tensor scalar.
Note
Subtensor selection always returns a matrix, so indexing with [a:b, c:d] is forced. If one index is a scalar, for instance, x[a:b, c] or x[a, b:c], an error will be raised. Use instead x[a:b, c:c+1] or x[a:a+1, b:c].
The above indexing methods are not supported because the return value would be a sparse matrix rather than a sparse vector, which is a deviation from numpy indexing rule. This decision is made largely to preserve consistency between numpy and theano. This may be revised when sparse vectors are supported.
Parameters: |
|
---|---|
Returns: | The corresponding slice in x. |
Note: | The grad is not implemented for this op. |
Select elements of sparse matrix, returning them in a vector.
Parameters: |
|
---|---|
Returns: | The corresponding elements in x. |
Select row of sparse matrix, returning them as a new sparse matrix.
Parameters: |
|
---|---|
Returns: | The corresponding rows in x. |
Implement a subtensor of a sparse variable that takes two scalars as index and returns a scalar.
If you want to take a slice of a sparse matrix see GetItem2d that returns a sparse matrix.
Parameters: |
|
---|---|
Returns: | The corresponding item in x. |
Note: | The grad is not implemented for this op. |
Parameters: |
|
---|---|
Returns: | x > y |
Note: | At least one of x and y must be a sparse matrix. |
Stack sparse matrices horizontally (column wise).
This wrap the method hstack from scipy.
Parameters: |
|
---|---|
Returns: | The concatenation of the sparse array column wise. |
Note: | The number of line of the sparse matrix must agree. |
Note: | The grad implemented is regular, i.e. not structured. |
Parameters: |
|
---|---|
Returns: | x <= y |
Note: | At least one of x and y must be a sparse matrix. |
Parameters: |
|
---|---|
Returns: | x < y |
Note: | At least one of x and y must be a sparse matrix. |
Multiply elementwise two matrices, at least one of which is sparse.
This method will provide the right op according to the inputs.
Parameters: |
|
---|---|
Returns: | x + y |
Note: | At least one of x and y must be a sparse matrix. |
Note: | The grad is regular, i.e. not structured. |
Multiplication of sparse matrix by a broadcasted dense vector element wise.
Parameters: |
|
---|---|
Return: | The product x * y element wise. |
Note: | The grad implemented is regular, i.e. not structured. |
Return the negation of the sparse matrix.
Parameters: | x – Sparse matrix. |
---|---|
Returns: | -x. |
Note: | The grad is regular, i.e. not structured. |
Parameters: |
|
---|---|
Returns: | x != y |
Note: | At least one of x and y must be a sparse matrix. |
Remove explicit zeros from a sparse matrix.
Parameters: | x – Sparse matrix. |
---|---|
Returns: | Exactly x but with a data attribute exempt of zeros. |
Note: | The grad implemented is regular, i.e. not structured. |
Scale each row of a sparse matrix by the corresponding element of a dense vector
Parameters: |
|
---|---|
Returns: | A sparse matrix in the same format as x which each row had been multiply by the corresponding element of s. |
Note: | The grad implemented is structured. |
Operand for calculating the dot product dot(x, y.T) = z when you only want to calculate a subset of z.
It is equivalent to p o (x . y.T) where o is the element-wise product, x and y operands of the dot product and p is a matrix that contains 1 when the corresponding element of z should be calculated and 0 when it shouldn’t. Note that SamplingDot has a different interface than dot because SamplingDot requires x to be a m`x`k matrix while y is a n`x`k matrix instead of the usual k`x`n matrix.
Note
It will work if the pattern is not binary value, but if the pattern doesn’t have a high sparsity proportion it will be slower then a more optimized dot followed by a normal elemwise multiplication.
Parameters: |
|
---|---|
Returns: | A dense matrix containing the dot product of x by y.T only where p is 1. |
Note: | The grad implemented is regular, i.e. not structured. |
Construct a sparse matrix of ones with the same sparsity pattern.
Parameters: | x – Sparse matrix to take the sparsity pattern. |
---|---|
Returns: | The same as x with data changed for ones. |
Calculate the sum of a sparse matrix along the specified axis.
It operates a reduction along the specified axis. When axis is None, it is applied along all axes.
Parameters: |
|
---|---|
Returns: | The sum of x in a dense format. |
Note: | The grad implementation is controlled with the sparse_grad parameter. True will provide a structured grad and False will provide a regular grad. For both choices, the grad returns a sparse matrix having the same format as x. |
Note: | This op does not return a sparse matrix, but a dense tensor matrix. |
Construct a sparse matrix of zeros.
Parameters: | x – Sparse matrix to take the shape. |
---|---|
Returns: | The same as x with zero entries for all element. |
Types of sparse matrices to use for testing
Return a square sparse (csc) matrix whose diagonal is given by the dense vector argument.
Parameters: | x – Dense vector for the diagonal. |
---|---|
Returns: | A sparse matrix having x as diagonal. |
Note: | The grad implemented is regular, i.e. not structured. |
Structured addition of a sparse matrix and a dense vector. The elements of the vector are only added to the corresponding non-zero elements of the sparse matrix. Therefore, this operation outputs another sparse matrix.
Parameters: |
|
---|---|
Returns: | A sparse matrix containing the addition of the vector to the data of the sparse matrix. |
Note: | The grad implemented is structured since the op is structured. |
Structured Dot is like dot, except that only the gradient wrt non-zero elements of the sparse matrix a are calculated and propagated.
The output is presumed to be a dense matrix, and is represented by a TensorType instance.
Parameters: |
|
---|---|
Returns: | The dot product of a and b. |
Note: | The grad implemented is structured. |
Substact two matrices, at least one of which is sparse.
This method will provide the right op according to the inputs.
Parameters: |
|
---|---|
Returns: | x - y |
Note: | At least one of x and y must be a sparse matrix. |
Note: | The grad will be structured only when one of the variable will be a dense matrix. |
Return the transpose of the sparse matrix.
Parameters: | x – Sparse matrix. |
---|---|
Returns: | x transposed. |
Note: | The returned matrix will not be in the same format. csc matrix will be changed in csr matrix and csr matrix in csc matrix. |
Note: | The grad is regular, i.e. not structured. |
Operation for efficiently calculating the dot product when one or all operands are sparse. Supported formats are CSC and CSR. The output of the operation is sparse.
Parameters: |
|
---|---|
Returns: | The dot product x.`y` in a sparse format. |
Note: |
|
Performs the expression alpha * x y + z.
Parameters: |
|
---|---|
Returns: | The dense matrix resulting from alpha * x y + z. |
Note: | The grad is not implemented for this op. |
Note: | At least one of x or y must be a sparse matrix. |
Wrapper for theano.test.unittest_tools.py:verify_grad wich converts sparse variables back and forth.
Parameters: |
|
---|---|
Returns: | None |
Stack sparse matrices vertically (row wise).
This wrap the method vstack from scipy.
Parameters: |
|
---|---|
Returns: | The concatenation of the sparse array row wise. |
Note: | The number of column of the sparse matrix must agree. |
Note: | The grad implemented is regular, i.e. not structured. |
Return a tuple containing everything needed to perform a test.
If out_dtype is None, theano.config.floatX is used.
Parameters: |
|
---|---|
Returns: | (variable, data) where both variable and data are list. |
Note: | explicit_zero and unsorted_indices was added in Theano 0.6rc4 |