Recent questions tagged machine-learning

Consider the following loss function:$$L(w)=\sum_{i=1}^n \log \left(1+w^{\top} x_i\right)$$What is $\nabla L(w)$ ? Write down the update step for gradient descent.

Consider the univariate function $f(x)=x^2$. This function has a unique minimum at $x^*=0$. We're using gradient descent (GD) to find this minimum and at time $t$ we arri...

Consider the following loss function on vectors $w \in \mathbb{R}^4$ :$$L(w)=w_1^2+2 w_2^2+w_3^2-2 w_3 w_4+w_4^2+2 w_1-4 w_2+4$$What is $\nabla L(w)$ ?Suppose we use grad...

Suppose we run gradient descent with a fixed learning rate of $\alpha=0.1$ to minimize the 2D function $f(x, y)=5+x^2+y^2+5 x y$.The gradient of this function is$$\nabla_...

Consider applying gradient descent with step size $\alpha=0.5$ to find the $\mathbf{x}$ that minimizes the function $f(\mathbf{x})=f\left(\left(x^{(1)}, x^{(2)}\right)\ri...

Consider the least squares regression model, $\hat{\mathbb{Y}}=\mathbb{X} \theta$. Assume that $\mathbb{X}$ and $\mathbb{Y}$ refer to the design matrix and true response ...

Consider the least squares regression model, $\hat{\mathbb{Y}}=\mathbb{X} \theta$. Assume that $\mathbb{X}$ and $\mathbb{Y}$ refer to the design matrix and true response ...

Consider a simple classification task with features $x_i \in \mathbb{R}^d$ and $k$ classes. Suppose we train a linear classifier to minimize the regularized least squared...

Consider just the general shape of the following plots.For each of the following possible interpretations of the quantities being plotted on the $X$ and $Y$ axes, indicat...

Good practices to avoid overfitting include:  Using a two part cost function which includes a regularizer to penalize model complexity.Using a good optimizer to minimize ...