ML | DA | Multiclass Classification

Question

Suppose a classifier predicts each possible class with equal probability. If there are 10 classes, what will the cross-entropy error be on a single example?

• A. $− log(10)$
• B. $−0.1 log(1)$
• C. $− log(0.1)$
• D. $−10 log(0.1)$

Answer 1

As $p_{i}$ is 0.1, we can write cross entropy as follows. So answer is C.

 

$L = -\Sigma_{i=1}^{i=10}y_{i}log(p_{i}) = -log(p_{i}) = -log(0.1)$

Answer 2

In this case, since each class is predicted with equal probability, the probability assigned to the true class is 1/10 (assuming there are 10 classes). Therefore, the cross-entropy error (denoted as H) for a single example can be calculated as follows:

H = -log(1/10) = -log(0.1)

The base of the logarithm is typically 2 or e (natural logarithm).

hence answer ‘C’ is the correct answer.