337 views
0 0 votes

Consider a linear model $$y_{\text{pred}} = w^T x$$with Mean Absolute Error (MAE) loss and $L_2$ regularization..

Given:
\[
\lambda = 0.20, \quad 
w = \begin{bmatrix} -3 \\ 4 \end{bmatrix}, \quad 
x = \begin{bmatrix} 1 \\ 2 \end{bmatrix},
\]
and $$y_{\text{true}} = x_1 + x_2.$$Find the regularized loss for this instance.

1 Answer

1 1 vote
Step 1: Calculate the predicted value $y_{\text{pred}}$

The predicted value is calculated using the linear model formula $y_{\text{pred}} = \mathbf{w}^T \mathbf{x}$.

$\mathbf{w}^T \mathbf{x} = \begin{bmatrix} -3 & 4 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = (-3 \cdot 1) + (4 \cdot 2) = -3 + 8 = 5$

So, $y_{\text{pred}} = 5$.

 

Step 2: Calculate the true value $y_{\text{true}}$

The true value is given by the formula $y_{\text{true}} = x_1 + x_2$.

$y_{\text{true}} = 1 + 2 = 3$

So, $y_{\text{true}} = 3$.

 

Step 3: Calculate the Mean Absolute Error (MAE) loss

The MAE loss is the absolute difference between the true and predicted values: $\text{MAE} = |y_{\text{true}} - y_{\text{pred}}|$.

$\text{MAE} = |3 - 5| = |-2| = 2$

 

Step 4: Calculate the $L_2$ regularization term

The $L_2$ regularization term is given by $\lambda \cdot \text{L}_2\text{-norm}^2$, where the $\text{L}_2\text{-norm}^2$ of $\mathbf{w}$ is $\sum w_i^2$.

$\text{L}_2\text{-norm}^2 = (-3)^2 + (4)^2 = 9 + 16 = 25$

The regularization term is $\lambda \cdot 25 = 0.20 \cdot 25 = 5$.

 

Step 5: Calculate the total regularized loss

The total regularized loss is the sum of the MAE loss and the $L_2$ regularization term.

$\text{Regularized Loss} = \text{MAE} + \text{Regularization Term} = 2 + 5 = 7$

 

Answer:  The regularized loss for this instance is $\mathbf{7}$.
Answer:
Position:
Show:

Related questions

2 2 votes
2 2 answers
362
362 views
GO Classes asked Feb 16
362 views
Let $M=I_n-\frac{1}{n}\mathbf{1}\mathbf{1}^T$, where $\mathbf{1}=(1,1,\dots,1)^T\in\mathbb{R}^n$.Consider the optimization problem $\max_{x} (x^T A x)$ subject to $x^T x=...
1 1 vote
1 1 answer
329
329 views
GO Classes asked Feb 16
329 views
Let $x_1,x_2,\dots,x_{100}$ be real-valued observations with sample mean$$\bar{x}=\frac{1}{100}\sum_{i=1}^{100}x_i$$The pairwise squared distance summation is $$S=\sum_{i...
1 1 vote
0 0 answers
254
254 views
GO Classes asked Feb 16
254 views
Consider a linear model $f(x)=wx$. Using stochastic gradient descent with learning rate $\lambda=0.10$, if at iteration $i$, $w_i=10.00$, then $w_{i+1}= \_\_\_\_\_$
3 3 votes
1 1 answer
286
286 views
GO Classes asked Feb 16
286 views
Consider a fully connected feedforward neural network with $30$ input neurons, a first hidden layer with $4$ neurons, a second hidden layer with $3$ neurons, and an outpu...