194 views
0 0 votes
Suppose we have a training dataset $D$ for linear regression that is the union of two (disjoint) datasets $A$ and $B$, where $|A| \gg|B|$. For example, $A$ and $B$ might represent data from two different groups of individuals. Let $\left(x_a^{(i)}, y_a^{(i)}\right)$ denote the $i$-th example in $A$ and $\left(x_b^{(i)}, y_b^{(i)}\right)$ denote the $i$-th example in $B$. The reweighted training loss is defined as

$$
L(w)=\frac{1}{|A|} \sum_{i=1}^{|A|}\left(w x_a^{(i)}-y_a^{(i)}\right)^2+\frac{1}{|B|} \sum_{i=1}^{|B|}\left(w x_b^{(i)}-y_b^{(i)}\right)^2,
$$

where $w$ is the weight parameter for linear regression (in this problem, we will omit the bias term).

Explain why this loss function is more likely to promote more equal treatment of individuals in dataset $A$ and individuals in dataset $B$, compared with running normal linear regression on $D$.

1 Answer

0 0 votes
Solution: Since $1 /|A|$ is much smaller than $1 /|B|$, this objective function gives more importance to the loss on examples in group $B$. Thus, the model is less likely to have much worse accuracy on group $B$ than $A$. In contrast, standard linear regression puts equal importance on each example, so group $A$ as a whole is weighted more heavily.
Position:
Show:

Related questions

1 1 vote
1 1 answer
508
508 views
GO Classes asked Mar 18, 2025
508 views
Consider linear regression and logistic regression. They both use linear functions.They both can be used to solve regression prob-They both use the logistic activation fu...
1 1 vote
1 1 answer
251
251 views
GO Classes asked Mar 13, 2025
251 views
Consider a simple classification task with features $x_i \in \mathbb{R}^d$ and $k$ classes. Suppose we train a linear classifier to minimize the regularized least squared...
1 1 vote
1 1 answer
251
251 views
GO Classes asked Mar 13, 2025
251 views
Consider just the general shape of the following plots.For each of the following possible interpretations of the quantities being plotted on the $X$ and $Y$ axes, indicat...
1 1 vote
0 0 answers
189
189 views
GO Classes asked Mar 13, 2025
189 views
Consider the following training data: xy132130.5Suppose the data comes from a model $y=c x^\beta$ , for unknown constants $c$ and $\beta$. Use least squares linear regres...