Cours de Régression

Y
X1, ..., XpY
X1, ..., XpfY≈f(X1, ..., Xp)
f
Y X1, ..., Xp
Y = β0+β1X1+... +βpXp+,
β0, β1, ..., βpY
β0, β1, ..., βp
Y X1, ..., Xp
Y
X1, ..., XpXjkjk1×... ×kp
Y
Y X1, ..., Xp
Y
Y X1, ..., Xp
Y X1, ..., Xp
Y
Xj
f(X1, ..., Xp) = E(Y|X1, ..., Xp),
Y 2
E(Y −f(X1, ..., Xp))2(Y,X1, ..., Xp)
f1,X1, ..., Xp
f(X1, ..., Xp) = β0+β1X1+... +βpXp,

X
{yi, xi}i=1,...,n
yi=β0+β1xi+i, i = 1, ..., n.
y= Xβ+
y=
y1
yn
,X = 
1x1
1xn
, β =β0
β1=
1
n
.
xi
iy 
Xβ y X
i
E(i)=0
∀i6=j, cov(i, j) = 0
var(i) = σ2<∞
E() = 0 var() =
σ2I
x y
u=1
nPn
i=1 uiu∈Rn
ρ(x, y) = xy −x y
px2−x2qy2−y2
,
xy =1
nPn
i=1 xiyix2=1
nPn
i=1 x2
iy2=1
nPn
i=1 y2
ixy −x y
x y (xi, yi)i=1,...,n x2−x2
y2−y2x y
−1 1 x y
ρ(x, y) 1 −1 1 −1
y=αxβ

log(x) log(y)y=αeβx
log(y) = log(α) + βlog(x)y= P(x) P
d y 1, x, x2, ..., xd
x y
yi=f(xi) + if
x y ri∈ {1, ..., n}xi
xr1>... >xrnsiyi
ρS(x, y) = ρ(r, s) = rs −r s
pr2−r2ps2−s2.
Y
Xx y r =s r =n−s
r s
x y
yi=β0+β1xi+i,
β0β1
ˆ
β= ( ˆ β0,ˆ β1)>
R(b) = 1
n
n
X
i=1
(yi−b0−b1xi)2, b = (b0, b1)>∈R2.
ˆ
β= ( ˆ β0,ˆ β1)>= arg min
b∈R2R(b)
ˆ
β1=xy −x y
x2−x2ˆ
β0= ¯y−ˆ β1¯x.
ˆ
β β E(yi) = E(β0+β1xi+i) = β0+β1xi
E(y) = β0+β1xE(xy) = β0x+β1x2
E(ˆ β1) = E(xy −x y)
x2−x2=E(xy)−E(x y) x2−x2=β1(x2−x2) x2−x2=β1
E(ˆ β0) = E(y)−E(ˆ β1)x=β0+β1x−β1x=β0.
var(yi) = var(i) = σ2var(y) = σ2/n = cov(yi, y)yi
cov( ˆ β1, y) = cov(xy, y)−xvar(y)
x2−x2=
1
nPn
i=1 xicov(yi, y)−xvar(y)
x2−x2=xσ2−xσ2 n(x2−x2)= 0.