"Error analysis": manually examine the mistakes → get insight of what's next.
"ceiling on performance"
example:
cat classification, found some false-positives of dog pictures. → should you try to make ML system better on dog or not ?
→ error analysis:
A lot of ideas of improving ML performance: strategy on how to choose.
→ how to figure out which ones to pursue and which ones to discard ?
How to tune hyperparams & what to expect.
TV tuning example: each knob does only one thing …
This week: optimization algos to faster train NN, on large dataset.
Compute J on m examples: vectorization, i.e. stacking x(i) y(i) horizontally.
X = [x(1), ..., x(m)]
Y = [y(1), ..., y(m)]
→ still slow or impossible with large m …
Tips for hyperparam-tuning.
Many hyperparams to tune, mark importance by colors (red > yellow > purple):
How to select set of values to explore ?
— this was OK in pre-DL era.
reason: difficule to know which hyperparam …
Applied ML: highly iterative process. idea-code-exp loop
splitting data
splitting data in order to speed up the idea-code-exp loop:
*training set / dev(hold-out/cross-validataion) set / test set *
split ratio:
Layer counting:
notation:
layer 0 = input layer
L = number of layers
n^[l] = size of layer l
a^[l] = activation of layer l = g[l]( z[l] ) → a …
new notation:
[i] for quantities in layer i. (compared to superscript (i) for ith training example).i for ith unit in a layernotation:
a^[i]: activation at layer i.This week: logistic regression.
ex. cat classifier from image
image pixels: 64x64x3
⇒ unroll(flatten) to a feature vector x dim=64x64x3=12288:=n (input dimension)
notation
(i) for ith example, e.g. x^(i)[l] for lth layer, e.g. w^[l]m: number of …Example: housing price prediciton.
Each neuron: ReLU function
Stacking multiple layers of neurons: hidden layers are concepts more general than input layer — found automatically by NN.
supervised learning: during training, always have output corresponding to input.
Different NN types are …
Save settings and configurations.
5 different ways of data persistance:
onSavedInstanceState(): store state of views in k-v pairs (Bundles), used when screen rotates / app killed by system, temperary.SharedPreferences: save k-v pairs to a file, can save primitive types.