Option B
Note: A simple range measurement would be sufficient to tell whether the addition of similarly signed two numbers causes an overflow. See the last paragraph for a much simpler explanation.
Theory(Stub),
Knowing weighted code representation for identifying 2’s complement would help in identifying whether the number is positive or negative.
$$\begin{array}{|c|c|}\hline-2^{N-1}&2^{N-2}&2^{N-3}&.....&2^0\\\hline\end{array}$$
MSB=1 implies the number is negative and MSB=0 implies it’s positive(considering 0 on the positive spectrum).
In 2’s complement addition, an adder is used irrelevant of the sign of the two numbers, adding numbers with similar signs causes overflow, due to the inability to accommodate bits required to represent positive and negative parts of the number within fixed bit size. And, adding a number with different signs doesn’t cause overflow because they negate each other and therefore could be represented with the same or few number of bits.
Overflow for Positive Numbers,
$$\begin{array}{cccc}0&1&0&1\\0&1&1&1\\\hline\underline{1}&1&0&0\\\hline\end{array}$$
Here, the sign bit(MSB) is occupied by 1(a resultant from the positive component addition), there’s no other way to accommodate this number and maintaining 2’s complement nature without resorting to another bit.
Sign Bit Compaction for negative numbers,
$$\begin{array}{cccc}&1&1&0&1\\&1&1&1&1\\\hline\underline{-1}&\underline{+1}&1&0&0\\\hline&\underline{-1}&1&0&0\\\hline\end{array}\\$$
Here, we consider the addition of the Two MSB of a negative number leading to a larger negative component $-2^N$ but and there’s a positive component $2^{N-1}$ addition of these two would result in $-2^{N-1}$ which could be accommodated in the sign bit or simply omit the leftover carry.
Overflow for Negative Numbers,
$$\begin{array}{ccccc}&1&1&0&0\\&1&0&1&0\\\hline\underline{-1}&0&1&1&0\\\hline\end{array}$$
Unlike the previous example, there’s no way compacting the sign bit hence it, causes a overflow.
4bit 2’s complement range: $[-2^3,2^3-1]=[-8,7]$, convert the above options to decimal numbers and them to see whether they lie within the above range.
In Option B, we note that the addition of $1100(-4)$ and $1010(-6)$ results in $-10$ which is larger than what we could be able to accommodate in this range.
Or, the above case example concerning overflow for negative numbers.
