2. Authors
  3. mx

[Structuring Machine Learning Projects] week2. ML Strategy (2)

Category notes deep learning Series Part 9 of «Andrew Ng Deep Learning MOOC»

I-Error Analysis

Carrying out error analysis

"Error analysis": manually examine the mistakes → get insight of what's next.

"ceiling on performance"

example:
cat classification, found some false-positives of dog pictures. → should you try to make ML system better on dog or not ?
→ error analysis:

  • get ~100 false positive examples
  • count how …

[Structuring Machine Learning Projects] week1. ML Strategy (1)

Category notes deep learning Series Part 8 of «Andrew Ng Deep Learning MOOC»

I-Introduction to ML Strategy

Why ML Strategy

A lot of ideas of improving ML performance: strategy on how to choose.

→ how to figure out which ones to pursue and which ones to discard ?

Orthogonalization

How to tune hyperparams & what to expect.

TV tuning example: each knob does only one thing …

[Improving Deep Neural Networks] week2. Optimization algorithms

Category notes deep learning Series Part 6 of «Andrew Ng Deep Learning MOOC»

This week: optimization algos to faster train NN, on large dataset.

Mini-batch gradient descent

batch v.s. mini-batch GD

Compute J on m examples: vectorization, i.e. stacking x(i) y(i) horizontally.
X = [x(1), ..., x(m)]
Y = [y(1), ..., y(m)]
→ still slow or impossible with large m …

[Improving Deep Neural Networks] week3. Hyperparameter tuning, Batch Normalization and Programming Frameworks

Category notes deep learning Series Part 7 of «Andrew Ng Deep Learning MOOC»

Hyperparameter parameters

Tips for hyperparam-tuning.

Tuning process

Many hyperparams to tune, mark importance by colors (red > yellow > purple):

How to select set of values to explore ?

  • Do NOT use grid search (grid of n * n)

— this was OK in pre-DL era.

  • try random values.

reason: difficule to know which hyperparam …

[Improving Deep Neural Networks] week1. Practical aspects of Deep Learning

Category notes deep learning Series Part 5 of «Andrew Ng Deep Learning MOOC»

Setting up your Maching Learning Application

Train / Dev / Test sets

Applied ML: highly iterative process. idea-code-exp loop

splitting data
splitting data in order to speed up the idea-code-exp loop:
*training set / dev(hold-out/cross-validataion) set / test set *

split ratio:

  • with 100~10000 examples: 70/30 or 60/20/20
  • with …

[Neural Networks and Deep Learning] week4. Deep Neural Network

Category notes deep learning Series Part 4 of «Andrew Ng Deep Learning MOOC»

Deep L-layer neural network

Layer counting:

  • input layer is not counted as a layer, "layer 0"
  • last layer (layer L, output layer) is counted.

notation: layer 0 = input layer L = number of layers n^[l] = size of layer l a^[l] = activation of layer l = g[l]( z[l] ) → a …

[Neural Networks and Deep Learning] week3. Shallow Neural Network

Category notes deep learning Series Part 3 of «Andrew Ng Deep Learning MOOC»

Neural Networks Overview

new notation:

  • superscript [i] for quantities in layer i. (compared to superscript (i) for ith training example).
  • subscript i for ith unit in a layer

Neural Network Representation

notation:

  • a^[i]: activation at layer i.
  • input layer: x, layer 0.
  • hidden layer
  • output layer: prediction (yhat)
  • don't …

[Neural Networks and Deep Learning] week2. Neural Networks Basics

Category notes deep learning Series Part 2 of «Andrew Ng Deep Learning MOOC»

This week: logistic regression.

Binary Classification & notation

ex. cat classifier from image image pixels: 64x64x3 ⇒ unroll(flatten) to a feature vector x dim=64x64x3=12288:=n (input dimension)

notation

  • superscript (i) for ith example, e.g. x^(i)
  • superscript [l] for lth layer, e.g. w^[l]
  • m: number of …

[Neural Networks and Deep Learning] week1. Introduction to deep learning

Category notes deep learning Series Part 1 of «Andrew Ng Deep Learning MOOC»

What is a neural network?

Example: housing price prediciton.

Each neuron: ReLU function

Stacking multiple layers of neurons: hidden layers are concepts more general than input layer — found automatically by NN.

Supervised Learning with Neural Networks

supervised learning: during training, always have output corresponding to input.

Different NN types are …