0
0
0
1. 云栖社区>
2. 博客>
3. 正文

## 吴恩达机器学习 Coursera 笔记(二) - 单变量线性回归

javaedge 2018-12-19 01:18:22 浏览674

# 1 模型概述 - Model Representation

To establish notation for future use, we’ll use

• x(i)
1. the “input” variables (living area in this example), also called input features, and
• y(i)
1. the “output” or target variable that we are trying to predict (price).

A pair (x(i),y(i)) is called a training example
the dataset that we’ll be using to learn—a list of m training examples (x(i),y(i));i=1,...,m—is called a training set.
the superscript “(i)” in the notation is simply an index into the training set, and has nothing to do with exponentiation

• X
1. the space of input values
• Y
1. the space of output values

In this example

X = Y = R

To describe the supervised learning problem slightly more formally, our goal is,
given a training set, to learn a function h : X → Y so that h(x) is a “good” predictor for the corresponding value of y.
For historical reasons, this function h is called a hypothesis. Seen pictorially, the process is therefore like this

• regression problem
When the target variable that we’re trying to predict is continuous, such as in our housing example
• classification problem
When y can take on only a small number of discrete values (such as if, given the living area, we wanted to predict if a dwelling is a house or an apartment, say)

# 2 代价函数 - Cost Function

We can measure the accuracy of our hypothesis function by using a cost function.
This takes an average difference (actually a fancier version of an average) of all the results of the hypothesis with inputs from x's and the actual output y's.

J(θ0,θ1)=12m∑i=1m(y^i−yi)2=12m∑i=1m(hθ(xi)−yi)2

To break it apart, it is 1/2 x ̄ where x ̄ is the mean of the squares of hθ(xi)−yi , or the difference
between the predicted value and the actual value.
This function is otherwise called theSquared error function, or Mean squared error.
The mean is halved (1/2)as a convenience for the computation of the gradient descent, as the derivative term of the square function will cancel out the 1/2 term.
The following image summarizes what the cost function does:

# 3 代价函数(一)

If we try to think of it in visual terms, our training data set is scattered on the x-y plane.
We are trying to make a straight line (defined by hθ(x)) which passes through these scattered data points.

Our objective is to get the best possible line. The best possible line will be such so that the average squared vertical distances of the scattered points from the line will be the least.

Ideally, the line should pass through all the points of our training data set. In such a case, the value of J(θ0,θ1) will be 0.

The following example shows the ideal situation where we have a cost function of 0.

When θ1=1, we get a slope of 1 which goes through every single data point in our model.
Conversely, when θ1=0.5, we see the vertical distance from our fit to the data points increase.

This increases our cost function to 0.58. Plotting several other points yields to the following graph:
Thus as a goal, we should try to minimize the cost function. In this case, θ1=1 is our global minimum.

# 4 代价函数(二)

• 等高线图是包含许多等高线的图形,双变量函数的等高线在同一条线的所有点处具有恒定值
采用任何颜色并沿着'圆'，可得到相同的成本函数值
• 当θ0= 800且θ1= -0.15时，带圆圈的x显示左侧图形的成本函数的值
取另一个h（x）并绘制其等高线图，可得到以下图表

• 当θ0= 360且θ1= 0时，等高线图中J（θ0，θ1）的值越接近中心，从而降低了成本函数误差
现在给出我们的假设函数略微正斜率可以更好地拟合数据。

# 5 梯度下降 - Gradient Descent

θj:=θj−α∂∂θjJ(θ0,θ1)

where
j=0,1 represents the feature index number.

# 6 梯度下降知识点总结

θ1:=θ1−αddθ1J(θ1)

## Choose Learning Rate α

• 如果J(θ)在下降，但是下降的速度很慢的话，就需要增大学习率α，因为每一步走的都太短了，导致到达最优解的速度下降，即收敛速度下降。

• 因为α大的原因，每次都一步跳过了最优解点，导致距离最优解越来越远，J(θ)不断上升

θ1:=θ1−α∗0

# 7 线性回归的梯度下降

• 梯度下降算法
• 线性回归模型

• 线性假设
• 平方差代价函数

repeat until convergence: {θ0:=θ1:=}θ0−α1m∑i=1m(hθ(xi)−yi)θ1−α1m∑i=1m((hθ(xi)−yi)xi)

derivation of ∂∂θjJ(θ) for a single example :

θ0是一个常数，它将与给定训练集（数据）的θ1和xi，yi 的值同步变化

javaedge
+ 关注