188 views
0 0 votes

Suppose you are interested in predicting a one-dimensional quantitative random variable $Y$ (outcome) in terms of a two-dimensional quantitative random variable $X$ (covariates). You have a learning set $\left(x_1, y_1\right), \ldots,\left(x_3, y_3\right)$ of three observations, given in the table.

$i$$x_i$$y_i$
1$\left[\begin{array}{l}
0 \\
3
\end{array}\right]$
-3
2$\left[\begin{array}{l}
0 \\
3
\end{array}\right]$
0
3$\left[\begin{array}{l}
6 \\
0
\end{array}\right]$
3

Select the one that expresses the regularized empirical risk (mean squared error) for ridge regression for this dataset in terms of $\beta_1$ and $\beta_2$ when $\lambda=1$.

 

  1. $16 \beta_1^2+12 \beta_1+4 \beta_2^2-6 \beta_2+6$
  2. $16 \beta_1^2-12 \beta_1+4 \beta_2^2+6 \beta_2+6$
  3. $16 \beta_1^2+12 \beta_1-4 \beta_2^2-6 \beta_2+6$
  4. $16 \beta_1^2-12 \beta_1-4 \beta_2^2-6 \beta_2+6$

1 Answer

0 0 votes
$\begin{aligned}
&\text { Let } R \text { denote the regularized empirical risk. Then }\\
&\begin{aligned}
R & =\frac{1}{3} \sum_{i=1}^3\left(Y_i-X_i^T \beta\right)^2+\lambda\left(\beta_1^2+\beta_2^2\right) \\
& =\frac{1}{3}\left(\left(-3-3 \beta_2\right)^2+\left(-3 \beta_1\right)^2+\left(3-6 \beta_1\right)^2\right)+\left(\beta_1^2+\beta_2^2\right) \\
& =16 \beta_1^2-12 \beta_1+4 \beta_2^2+6 \beta_2+6
\end{aligned}
\end{aligned}$
Answer:
Position:
Show:

Related questions

0 0 votes
1 1 answer
197
197 views
GO Classes asked Mar 17, 2025
197 views
Suppose you're using $L2$ regularization on a least squares objective. Some value $\lambda^*$ will give you the best test error among all possible $\lambda$. You train yo...
0 0 votes
1 1 answer
227
227 views
GO Classes asked Mar 17, 2025
227 views
We are solving a least-squares linear regression problem without regularization. Suppose that the following twoweight matrices both have the same cost: $W1 = \begin{bmatr...
0 0 votes
1 1 answer
147
147 views
GO Classes asked Mar 17, 2025
147 views
Which of the following statements are true about Lasso and ridge regression?Both ridge regression and Lasso are methods used to reduce overfitting that might occur in sta...
0 0 votes
0 0 answers
127
127 views
GO Classes asked Mar 17, 2025
127 views
Ridge regression can shrink all coefficients to exactly O if the regularization parameter $\lambda$ is large enough.Please enter 1 for True and 0 for False.