Table Of Contents

Previous topic

sandbox.cuda – List of CUDA GPU Op implemented

Next topic

Optimizations

This Page

sandbox.linalg – Linear Algebra Ops

API

class theano.sandbox.linalg.ops.A_Xinv_b(use_c_code='g++')

Product of form a inv(X) b

class theano.sandbox.linalg.ops.AllocDiag(use_c_code='g++')

Allocates a square matrix with the given vector as its diagonal.

class theano.sandbox.linalg.ops.Cholesky(lower=True)

Return a triangular matrix square root of positive semi-definite x

L = cholesky(X, lower=True) implies dot(L, L.T) == X

class theano.sandbox.linalg.ops.CholeskyGrad(lower=True)
perform(node, inputs, outputs)

Implements the “reverse-mode” gradient [1] for the Cholesky factorization of a positive-definite matrix.

[1]S. P. Smith. “Differentiation of the Cholesky Algorithm”. Journal of Computational and Graphical Statistics, Vol. 4, No. 2 (Jun.,1995), pp. 134-147 http://www.jstor.org/stable/1390762
class theano.sandbox.linalg.ops.Det(use_c_code='g++')

Matrix determinant Input should be a square matrix

class theano.sandbox.linalg.ops.Eig(use_c_code='g++')

Compute the eigenvalues and right eigenvectors of a square array.

props()

Function exposing different properties of each instance of the op.

For the Eig op, there are no properties to be exposed.

class theano.sandbox.linalg.ops.Eigh(UPLO='L')

Return the eigenvalues and eigenvectors of a Hermitian or symmetric matrix.

grad(inputs, g_outputs)

The gradient function should return

\sum_n\left(W_n\frac{\partial\,w_n}
      {\partial a_{ij}} +
\sum_k V_{nk}\frac{\partial\,v_{nk}}
      {\partial a_{ij}}\right),

where [W, V] corresponds to g_outputs, a to inputs, and (w, v)=\mbox{eig}(a).

Analytic formulae for eigensystem gradients are well-known in perturbation theory:

\frac{\partial\,w_n}
{\partial a_{ij}} = v_{in}\,v_{jn}

\frac{\partial\,v_{kn}}
          {\partial a_{ij}} =
\sum_{m\ne n}\frac{v_{km}v_{jn}}{w_n-w_m}

class theano.sandbox.linalg.ops.EighGrad(UPLO='L')

Gradient of an eigensystem of a Hermitian matrix.

perform(node, inputs, outputs)

Implements the “reverse-mode” gradient for the eigensystem of a square matrix.

class theano.sandbox.linalg.ops.ExtractDiag(view=False)

Return the diagonal of a matrix.

perform(node, ins, outs)

For some reason numpy.diag(x) is really slow, so we implemented our own.

class theano.sandbox.linalg.ops.Hint(**kwargs)

Provide arbitrary information to the optimizer

These ops are removed from the graph during canonicalization in order to not interfere with other optimizations. The idea is that prior to canonicalization, one or more Features of the fgraph should register the information contained in any Hint node, and transfer that information out of the graph.

class theano.sandbox.linalg.ops.HintsFeature

FunctionGraph Feature to track matrix properties

This is a similar feature to variable ‘tags’. In fact, tags are one way to provide hints.

This class exists because tags were not documented well, and the semantics of how tag information should be moved around during optimizations was never clearly spelled out.

Hints are assumptions about mathematical properties of variables. If one variable is substituted for another by an optimization, then it means that the assumptions should be transferred to the new variable.

Hints are attached to ‘positions in a graph’ rather than to variables in particular, although Hints are originally attached to a particular positition in a graph via a variable in that original graph.

Examples of hints are: - shape information - matrix properties (e.g. symmetry, psd, banded, diagonal)

Hint information is propagated through the graph similarly to graph optimizations, except that adding a hint does not change the graph. Adding a hint is not something that debugmode will check.

#TODO: should a Hint be an object that can actually evaluate its # truthfulness? # Should the PSD property be an object that can check the # PSD-ness of a variable?

class theano.sandbox.linalg.ops.HintsOptimizer

Optimizer that serves to add HintsFeature as an fgraph feature.

class theano.sandbox.linalg.ops.MatrixInverse

Computes the inverse of a matrix A.

Given a square matrix A, matrix_inverse returns a square matrix A_{inv} such that the dot product A \cdot A_{inv} and A_{inv} \cdot A equals the identity matrix I.

Note :When possible, the call to this op will be optimized to the call of solve.
R_op(inputs, eval_points)

The gradient function should return

\frac{\partial X^{-1}}{\partial X}V,

where V corresponds to g_outputs and X to inputs. Using the matrix cookbook, once can deduce that the relation corresponds to

X^{-1} \cdot V \cdot X^{-1}.

grad(inputs, g_outputs)

The gradient function should return

V\frac{\partial X^{-1}}{\partial X},

where V corresponds to g_outputs and X to inputs. Using the matrix cookbook, once can deduce that the relation corresponds to

(X^{-1} \cdot V^{T} \cdot X^{-1})^T.

props()

Function exposing different properties of each instance of the op.

For the MatrixInverse op, there are no properties to be exposed.

class theano.sandbox.linalg.ops.MatrixPinv

Computes the pseudo-inverse of a matrix A.

The pseudo-inverse of a matrix A, denoted A^+, is defined as: “the matrix that ‘solves’ [the least-squares problem] Ax = b,” i.e., if \bar{x} is said solution, then A^+ is that matrix such that \bar{x} = A^+b.

Note that Ax=AA^+b, so AA^+ is close to the identity matrix. This method is not faster then matrix_inverse. Its strength comes from that it works for non-square matrices. If you have a square matrix though, matrix_inverse can be both more exact and faster to compute. Also this op does not get optimized into a solve op.

props()

Function exposing different properties of each instance of the op.

For the MatrixPinv op, there are no properties to be exposed.

class theano.sandbox.linalg.ops.Solve(A_structure='general', lower=False, overwrite_A=False, overwrite_b=False)

Solve a system of linear equations

theano.sandbox.linalg.ops.diag(x)

Numpy-compatibility method If x is a matrix, return its diagonal. If x is a vector return a matrix with it as its diagonal.

  • This method does not support the k argument that numpy supports.
theano.sandbox.linalg.ops.matrix_dot(*args)

Shorthand for product between several dots

Given N matrices A_0, A_1, .., A_N, matrix_dot will generate the matrix product between all in the given order, namely A_0 \cdot A_1 \cdot A_2 \cdot .. \cdot A_N.

theano.sandbox.linalg.ops.psd(v)

Apply a hint that the variable v is positive semi-definite, i.e. it is a symmetric matrix and x^T A x \ge 0 for any vector x.

theano.sandbox.linalg.ops.spectral_radius_bound(X, log2_exponent)

Returns upper bound on the largest eigenvalue of square symmetrix matrix X.

log2_exponent must be a positive-valued integer. The larger it is, the slower and tighter the bound. Values up to 5 should usually suffice. The algorithm works by multiplying X by itself this many times.

From V.Pan, 1990. “Estimating the Extremal Eigenvalues of a Symmetric Matrix”, Computers Math Applic. Vol 20 n. 2 pp 17-22. Rq: an efficient algorithm, not used here, is defined in this paper.

theano.sandbox.linalg.ops.trace(X)

Returns the sum of diagonal elements of matrix X.