上次捣鼓pelican博客系统还是在2012年, 那时farseerfc学长就提供了无私的帮助, 当时非常兴奋写了这么一篇简单的教程.

这两天捣鼓了好久 终于把博客升 …

ref: http://rnduja.github.io/2015/10/26/deep_learning_with_torch_step_7_optim/
doc: https://github.com/torch/optim/blob/master/doc/intro.md

Before we implement the gd update step by defining a gradientUpdate function and calling it in a loop.

function gradientUpdate(model, x, y, criterion, learningRate) 
    local pred = model:forward(x …

ref: http://rnduja.github.io/2015/10/07/deep_learning_with_torch_step_4_nngraph/
doc: https://github.com/torch/nngraph/

The aim of this library is to provide users of nn package with tools to easily create complicated architectures.

luarocks install nngraph

optionally can install graphvis for graph visualization.

From previous posts, to build networks …

ref: http://rnduja.github.io/2015/10/05/deep_learning_with_torch_step_3_nn_criterions/
doc: https://github.com/torch/nn/blob/master/doc/criterion.md

Criterion: abstract class, given input and target(true label), a Criterion can compute the gradient according to a certain loss function.

Criterion class

important methods:

• forward(input, target): compute the …

doc: https://github.com/torch/nn/blob/master/doc/containers.md
ref: http://rnduja.github.io/2015/10/04/deep_learning_with_torch_step_2_nn_containers/

Container, similarly to Module, is the abstract class defining the base methods inherited from concrete containers. Container contains modules (layers).

Container class

important methods:

• add(module)
• get(index): get module …

Module is an abstract class which defines fundamental methods necessary for a Layer.

doc: https://github.com/torch/nn/blob/master/doc/module.md

Module class

variables in Module:

• output: Tensor, the ouput computed from last call of forward(input)
• gradInput: Tensor, gradient wrt input of module, computed from last …

A Tensor is the fondamental data type in torch, (similar to numpy for tensorflow), it's a potentially multi-dimensional matrix.

See doc: https://github.com/torch/torch7/blob/master/doc/tensor.md

basic ops

• Indicate shape in constructor:

  th> x = torch.Tensor(3,4) [0.0000s] th> x 3.7366e+193 …

Intro

ML lang family

statically strongly typed languages

• fisrt-class functions
• type inference
• pattern matching

highlights of ocaml

• safty: static typing, pattern matching
• efficiency: high performance

• expressiveness: functional+type inference+polymorphism

• package manager: opam

• debugger
• profiler
• REPL: "toplevel"
• bytecode compiler: ccamlc
• native compiler: ocamlopt

programming environment fully online: https://try.ocamlpro …

Recap: Functions and Pattern Matching

case classes

ex: json json objects can be seq, num, str, bool,...

⇒ represented as abstract class and case classes.

pattern matching

→ question: what is the type of the {case(key, value)=>"..."} clause?

it is (JBinding => String) type, which is a shorthand for Function1[JBinding, String …

6.1 - Other Collections

so far: only seen List. → more (immutable) collections.

vector

List: is linear -- access to head is faster than middle or end element. Vector: better rand access performance.

represented as very shallow trees(32-split at each node)

Vector support similar operations as List (head, tail,map, fold …