Linear models

matrix multiplication: fast with GPU
numerically stable
cannot cocatenate linear units → equivalent to one big matrix...

⇒ add non-linear units in between

rectified linear units (RELU)

chain rule: efficient computationally

back propagation

easy to compute the gradient as long as the function Y(X) is made of simple blocks ...

这是udacity上deeplearning的笔记, 做得非常粗糙, 而且这门课也只是介绍性质的...

Softmax function

socres yi ⇒ probabilities pi

property: smaller scores ⇒ less certain about result

Onehot encoding

Cross entropy

measure how well the probability vector S corresponds to the label vector L. ⇒ cross entropy D(S,L)( D>=0, the smaller the better ...

总结一下用python撸codejam时常用的一些库, 并且给一些简单的例子. 发现用python撸codejam非常合适: codejam的时间要求不严格(4/8分钟), 而且程序只要本地运行. 正好可以使用python简洁的语法和丰富的函数库.



from collections import Counter, deque, defaultdict



>>> from itertools import product, combinations   
>>> a = 'ABCD'; b='EFG'   
>>> for p in product(a,b):    
print ...

之前一直用weka的GUI界面做机器学习的任务, 感觉这个软件虽然界面丑, 不过确实是快速开展机器学期的利器. 关于GUI的weka使用以后有时间再写. 今天这篇记录一下最近使用的java版本的weka.

1. Include jars into project

weka官网的下载链接里选择linux版本的weka压缩包即可, 下载以后找到weka.jar文件, 在工程里将其include一下就可以使用了(btw, 现在开始放弃eclipse, 进入IDEA的怀抱了...).

weka的文档在解压缩的文件里有, 另外在线文档在:

about libsvm...

关于libsvm需要有一点特别指出. weka自带的算法里是不包含libsvm的 (有个类似的SMO, 不过还是libsvm久经考验啊...), 需要使用weka的package manager安装. 打开package manager是在weka主界面的菜单里:

在package manager里搜索到libsvm安装即可. 然后(linux下)在主目录可以看到有个wekafiles文件夹, wekafiles/packages/LibSVM/目录下就是libsvm的内容.

需要指出的一点是, 要使用libsvm的话, 需要同时引用两个jar文件, 而且都叫libsvm.jar!!

这两个jar ...

1. Introduction to Intractability

recall model of computation: DFA
a univeral model of computation: turing machine
→ no more powerful model of computation.
Turing machine can compute any function that can be computed by a physically harnessable process of the natural world.

bottom line: turing machine is a simple and universal ...

simplex algo: top 10 algo of the 20th century (ever?).

what is linear programming:
a general problem-solving model that works for:
shortest-path, maxflow, MST, matching, assignment, ...

1. Brewer-'s Problem

toy example: choose products to maximize profit.
feasible region: a convex polygon.

⇒ optimum solution appears at an extreme point.

standard ...


: 文件夹 (可以有多级), 且包含__init__.py文件(每层都要有) 模块: py文件

代码分开放在多个py文件(模块名=文件名). 同名变量互不影响.

模块名冲突: 把同名模块放在不同中.


from math import log
from logging import log as logger

引用时: 使用完整的路径(包+模块名). ex. p1.util.f()


    from cStringIO import ...

Goal: classify problems according to computational requirements.
bad new: for huge number of pbs we don't know...

1. Introduction to Reductions

shifing gears:

  • from individual problems to problem-solving models.
  • from linear/quard to polynomial/exponential pbs
  • from implementation details to conceptual framwork

suppose we could (not) solve pb X ...

函数式编程: 更抽象, 更脱离指令(计算机), 更贴近计算(数学).

  • 不需要变量 (python允许有变量, 所以python非纯函数式)
  • 高阶函数
  • 闭包: 返回函数
  • 匿名函数


  • 变量可以指向函数 f=abs; f(-10)
  • 函数名: 就是指向函数的变量 abs=len
  • 高阶函数: 接收函数作为参数的函数

    def add(x,y,f):
    return f(x)+f(y)
    add(-5, 9, abs)


map()是 Python 内置的高阶函数 ...