[Sequential Models] week1. Recurrent Neural Networks

Fri, 02 Feb 2018 Category notes deep learning Series Part 14 of «Andrew Ng Deep Learning MOOC»

week1

Created Friday 02 February 2018

Why sequence models

examples of seq data (either input or output):

speech recognition
music generation
sentiment classification
DNA seq analysis
Machine translation
video activity recognition
name entity recognition (NER)

→ in this course: learn models applicable to these different settings.

Notation

motivating example: NER (Each ...

[Convolutional Neural Networks] week4. Special applications: Face recognition & Neural style transfer

Thu, 30 Nov 2017 Category notes deep learning Series Part 13 of «Andrew Ng Deep Learning MOOC»

This week: two special application of ConvNet.

I-Face Recognition

What is face recognition

Face verification & face recognition

verification: input = image and ID → output whether the image and ID are the same.
recognition: database = K persons, input = image → output = ID of the image among the K person or "not recognized".

→ the ...

[Convolutional Neural Networks] week3. Object detection

Mon, 27 Nov 2017 Category notes deep learning Series Part 12 of «Andrew Ng Deep Learning MOOC»

Object Localization

Classification VS. Localization VS. Detection

classification with localization
Apart from softmax output (for classification), add 4 more outputs of bounding box: b_x, b_y, b_h, b_w.

Defining target label y in localization
label format:
P_c indicating if there's any object
bounding box: b_x, b_y, b_h, b_w
class proba ...

[Convolutional Neural Networks] week2. Deep convolutional models: case studies

Wed, 22 Nov 2017 Category notes deep learning Series Part 11 of «Andrew Ng Deep Learning MOOC»

I-Case studies

Why look at case studies?

Good way to get intuition of different component of CNN: case study & reading paper.
Outline

classic networks:
- LeNet-5
- AlexNet
- VGG
ResNet (152-layer NN)
Inception

Classic Networks

LeNet-5(1998)

Goal: recognize hand-written digits.
image → 2 CONV-MEANPOOL layers, all CONV are valid (without padding) → 2 ...

[Convolutional Neural Networks] week1. Foundations of Convolutional Neural Networks

Sun, 19 Nov 2017 Category notes deep learning Series Part 10 of «Andrew Ng Deep Learning MOOC»

Computer Vision

CV with DL: rapid progress in past two years.

CV problems:

image classification
object detection: bounding box of objects
neural style transfer

input features could be very high dimension: e.g. 1000x1000 image → 3 million dim input ⇒ if 1st layer has 1000 hidden units → 3 billion params for ...

[Structuring Machine Learning Projects] week2. ML Strategy (2)

Fri, 17 Nov 2017 Category notes deep learning Series Part 9 of «Andrew Ng Deep Learning MOOC»

I-Error Analysis

Carrying out error analysis

"Error analysis": manually examine the mistakes → get insight of what's next.

"ceiling on performance"

example:
cat classification, found some false-positives of dog pictures. → should you try to make ML system better on dog or not ?
→ error analysis:

get ~100 false positive examples
count ...

[Structuring Machine Learning Projects] week1. ML Strategy (1)

Wed, 15 Nov 2017 Category notes deep learning Series Part 8 of «Andrew Ng Deep Learning MOOC»

I-Introduction to ML Strategy

Why ML Strategy

A lot of ideas of improving ML performance: strategy on how to choose.

→ how to figure out which ones to pursue and which ones to discard ?

Orthogonalization

How to tune hyperparams & what to expect.

TV tuning example: each knob does only one thing ...

[Improving Deep Neural Networks] week2. Optimization algorithms

Mon, 23 Oct 2017 Category notes deep learning Series Part 6 of «Andrew Ng Deep Learning MOOC»

This week: optimization algos to faster train NN, on large dataset.

Mini-batch gradient descent

batch v.s. mini-batch GD

Compute J on m examples: vectorization, i.e. stacking x(i) y(i) horizontally.
X = [x(1), ..., x(m)]
Y = [y(1), ..., y(m)]
→ still slow or impossible with large m ...