Structure of Examination Paper:
There are Five pages.
There are FOUR questions in total.
There are no appendices.
The mark obtainable for each question is 25.
Instructions to Students :
Answer FOUR questions.
Show clearly all your workings.
In all exercises, dj is the j-th digit of your student number.
1a. Consider the linear regression model Y = Xθ+ε, where θ is an m-vector of unknown parameters, Y is an N-vector of observations, X is a design matrix of size N × m and ε is a vector of normal random errors with mean 0 and covariance matrix σ2IN . Here σ2 is some positive unknown constant and IN is the identity matrix of size N × N. Assume that rank(X) = p ≤ m.
(i) Give a geometrical interpretation of the LSE, the least squares estimator (LSE) of θ.
(ii) Defifine the lasso estimators of θ and discuss a motivation behind these estimators.
(iii) Defifine the ridge estimator of θ and brieflfly discuss its properties.
(iv) Let Z be a vector of size m such that the linear form ZT θ is estimable. Explain the method of constructing confifidence intervals for ZT θ.
(v) Derive the maximum likelihood estimator for σ2.
1b. Assume m = 3, N = 5 and the model yj = θ0 + θ1xj + θ2x2j + εj . The table below displays the design points and the corresponding results:
xj 0 u −u v −v
yj 0 1 −1 1 −1
Here u = d4 + 1 and v = d4 + 2, where d4 is the fourth digit of your student number.
(i) Test the hypothesis that the complete regression model is statistically signifificant on the 95% confifidence level. Write down the corresponding ANOVA table and compute R2.
(ii) Test the hypothesis
on the 95% confifidence level.
To answer this question please use 19.0 as an approximate value for the 0.95-quantile of the F-distribution with 2 and 2 degrees of freedom.
2a. Consider the linear regression model Y = Xθ+ε, where θ is an m-vector of unknown parameters, Y is an N-vector of observations, X is a design matrix of size N × m and ε is a vector of normal random errors with mean 0 and covariance matrix σ2IN .
In a particular application, m = 4, N = 5 and the model is Y = θ1X1 + θ2X2 + θ3X3 + θ4X4 + ε. The table below displays the design points and the corresponding results
X1 u -1 1 0 -1
X2 −v 1 0 1 -1
X3 u−v 0 1 1 -2
X4 u+2v -3 1 -2 1
Y 1 1 1 -1 0