ref: http://rnduja.github.io/2015/10/26/deep_learning_with_torch_step_7_optim/
doc: https://github.com/torch/optim/blob/master/doc/intro.md
Before we implement the gd update step by defining a gradientUpdate function and calling it in a loop.
function gradientUpdate(model, x, y, criterion, learningRate)
local pred = model:forward(x ...ref: http://rnduja.github.io/2015/10/07/deep_learning_with_torch_step_4_nngraph/
doc: https://github.com/torch/nngraph/
The aim of this library is to provide users of nn package with tools to easily create complicated architectures.
luarocks install nngraph
optionally can install graphvis for graph visualization.
From previous posts, to build networks ...
ref: http://rnduja.github.io/2015/10/05/deep_learning_with_torch_step_3_nn_criterions/
doc: https://github.com/torch/nn/blob/master/doc/criterion.md
Criterion: abstract class, given input and target(true label), a Criterion can compute the gradient according to a certain loss function.
important methods:
forward(input, target): compute the ...doc: https://github.com/torch/nn/blob/master/doc/containers.md
ref: http://rnduja.github.io/2015/10/04/deep_learning_with_torch_step_2_nn_containers/
Container, similarly to Module, is the abstract class defining the base methods inherited from concrete containers. Container contains modules (layers).
important methods:
add(module) get(index): get module ...Module is an abstract class which defines fundamental methods necessary for a Layer.
doc: https://github.com/torch/nn/blob/master/doc/module.md
variables in Module:
output: Tensor, the ouput computed from last call of forward(input) gradInput: Tensor, gradient wrt input of module, computed from last ...A Tensor is the fondamental data type in torch, (similar to numpy for tensorflow), it's a potentially multi-dimensional matrix.
See doc: https://github.com/torch/torch7/blob/master/doc/tensor.md
Indicate shape in constructor:
th> x = torch.Tensor(3,4) [0.0000s] th> x 3.7366e ...