Linear regression | machine learning | Gate Da |

Question

 

 

 

Comments

w1 better coz
w2 fails if negative error

Answer 1

Model 1 / w1 will be a better generalizing on test dataset. Model 2/w2 will fail because of two major problems:

1. Grads will explode / vanish due to presence of square term
2. Cube preserves sign, therefore there will be an absence of lower bound, meaning the optimization can even go to -inf as the lowest value

Comments

w1 generalize better becouse w2 have sum of cube error which can show negative and w2 is non convex

 

Answer 2

option a :

1)for squared loss it will be always >=0

2)for cube loss ,it can go positive or negative so it can go towards -inf

3)so optimization may not have the finite minimum