Recent questions in Machine Learning

Consider the least squares regression model, $\hat{Y} = X\theta$. Assume that $X$ and $Y$ refer to the design matrix and true response vector for our training data.Let $\...

Consider the following dataset for a binary classification problem, where $+$ denotes the positive class and $-$ denotes the negative class.$$\begin{array}{|c|c|c|}\hline...

Given $n$ linearly independent feature vectors in $n$ dimensions, each assigned a label from $\{-1,+1\}$, consider a linear classifier of the form $\operatorname{sign}(w\...

Consider a regression model $h(x)=2x$ with the following loss function:$$L_{\mathrm{OINK}}(\hat{y},y)=\begin{cases}a(\hat{y}-y), & \hat{y}\geq y,\\b(y-\hat{y}), & \hat{y}...

Consider the following regularized linear regression objective function:$$L(w,b)=\sum_{i=1}^{n}(y_i-w^T x_i-b)^2+\lambda w^T D w$$where $D$ is a fixed diagonal matrix cho...

Which of the following statements about $L_1$ and $L_2$ regularization in linear regression are correct?$L1$ regularization tends to produce sparser weight vectors than $...

Consider a binary logistic regression model with labels $Y \in \{0,1\}$. Suppose the class probabilities are obtained from the softmax function using weight vectors $w_0$...

For a hard-margin SVM, the margin is inversely related to $\|w\|$. Suppose two SVM solutions have weight vectors $w_1$ and $w_2$, where$$\left\|w_1\right\|<\left\|w_2\rig...

Consider a least-squares linear regression problem with $X \in \mathbb{R}^{n \times d}$ and $y \in \mathbb{R}^n$. If the least-squares solution is unique, which of the fo...

Suppose a soft-margin SVM is used for a binary classification problem with labels $y_i \in\{-1,+1\}$. Let $\xi_i$ be the slack variable for training point $i$.If an error...

Let $f: \mathbb{R}^d \rightarrow \mathbb{R}$ be a differentiable function. At a point $x$, in which direction does the gradient $\nabla f(x)$ point?Direction of steepest ...

In linear regression, the least-squares prediction is given by$$\hat{y}=X\left(X^T X\right)^{-1} X^T y .$$What does this expression represent geometrically?The projection...

Suppose we are performing linear regression on $n$ training examples. Let $y_i$ be the actual output and $\hat{y}_i$ be the model's prediction for the $i$-th example.The ...

Consider the dataset shown below. A linear regression model $y = \theta x + \theta_0$ is now trained using datasets $A$ and $B$ (combined), and tested on dataset $C$.Base...

Suppose we have $n$ observations $\left(x_{i}, y_{i}\right)$, where $i=1,2, \ldots, n$. We use a linear regression model without an intercept, $\hat{y}=\beta x$.The param...

For the same dataset, consider the following two models:Model A: $\quad y=w_{1}^{2} x+w_{2} x$Model B: $\quad y=w x$Assume that we have unlimited training data. Which of ...

Consider the following two linear regression models, both including an intercept:Model A: $\quad \hat{y}=\theta_{0}+\theta_{1} x$Model B: $\hat{y}=\beta_{0}+\beta_{1} x^{...

Consider the dataset shown below. A linear regression model $y = \theta x + \theta_0$ is trained on dataset $A$ and tested on the disjoint dataset $B$. The performance is...