July 19, 2013, Sherbrook, Québec, Canada.
Theano is python software for evaluating complicated array expressions.
What does it do?
- aggressive expression optimizations,
- automatic GPU use,
- symbolic differentiation and R op.
It complements the Python numeric/scientific software stack (e.g. NumPy, SciPy, scikits, matplotlib, PIL.)
Design and feature set has been driven by machine learning research at the University of Montreal (groups of Yoshua Bengio, Pascal Vincent, Aaron Courville and Roland Memisevic) The result is a very good library for doing research in deep learning and neural network training, and a flexible framework for many other models and algorithms in machine learning more generally.
It has proven to be useful for implementing:
- linear and nonlinear neural network classifiers
- convolutional models
- Energy models: RBM, DBN, GRBM, ssRBM, AIS
- Auto-encoders: DAE, CAE
- GP regression
- sparse coding
- recurrent neural networks, echo state, (HMM?)
- online and batch learning and optimization
- Even SVM!
As people’s needs change this list will grow, but Theano is built around vector, matrix, and tensor expressions; there is little reason to use it for calculations on other data structures except. There is also some sparse matrix support.
The structured part of these lab sessions will be a walk-through of the following material. Interleaved with this structured part will be blocks of time for individual or group work. The idea is that you can try out Theano and get help from gurus on hand if you get stuck.