1. 云栖社区>
2. 博客>
3. 正文

## 线性回归之最小二乘法

sea-boat 2017-03-03 10:51:51 浏览716

## 代数推导：

1. 假设拟合直线为y=ax+b
2. 对任意样本点(xi,yi)
3. 误差为e=yi(axi+b)
4. S=ni=1ei2为最小时拟合度最高，即ni=1(yiaxib)2最小。
5. 分别求一阶偏导
Sb=2(i=1nyinbai=1nxi)

Sa=2(i=1nxiyibi=1nxiai=1nxi2)

6.分别让上面两式等于0，并且有nx¯=ni=1xiny¯=ni=1yi

7.得到最终解

a=ni=1(xix¯)(yiy¯)ni=1(xix¯)2

b=y¯ax¯

a=nxiyixiyinxi2(xi)2

b=xi2yixixiyinxi2(xi)2

### 代码实现

import numpy as np
import matplotlib.pyplot as plt

def calcAB(x,y):
n = len(x)
sumX,sumY,sumXY,sumXX =0,0,0,0
for i in range(0,n):
sumX  += x[i]
sumY  += y[i]
sumXX += x[i]*x[i]
sumXY += x[i]*y[i]
a = (n*sumXY -sumX*sumY)/(n*sumXX -sumX*sumX)
b = (sumXX*sumY - sumX*sumXY)/(n*sumXX-sumX*sumX)
return a,b,

xi = [1,2,3,4,5,6,7,8,9,10]
yi = [10,11.5,12,13,14.5,15.5,16.8,17.3,18,18.7]
a,b=calcAB(xi,yi)
print("y = %10.5fx + %10.5f" %(a,b))
x = np.linspace(0,10)
y = a * x + b
plt.plot(x,y)
plt.scatter(xi,yi)
plt.show()

## 矩阵推导

1. 对于y=ax+b转为向量形式
W=[w0w1]X=[1x1]
2. 于是y=w1x1+w0=WTX
3. 损失函数为
L=1ni=1n(yn(WTX)2)=1n(yXW)T(yXW)

最后可化为

1nXTWTXW2nXTWTy+1nyTy
1. 令偏导为0

LW=2nXTXW2nXTy=0

(XTX)1XTXW=(XTX)1XTy

W=(XTX)1XTy

## 代码实现

import numpy as np
import matplotlib.pyplot as plt

x = [1,2,3,4,5,6,7,8,9,10]
y = [10,11.5,12,13,14.5,15.5,16.8,17.3,18,18.7]

A = np.vstack([x,np.ones(len(x))]).T

a,b = np.linalg.lstsq(A,y)[0]
print("y = %10.5fx + %10.5f" %(a,b))
x = np.array(x)
y = np.array(y)

plt.plot(x,y,'o',label='data',markersize=10)
plt.plot(x,a*x+b,'r',label='line')
plt.show()

========广告时间========

=========================