This is an open question.
We know that for any language $L_p \in P$, its complement $L_p'$ also belongs in $P$, that is, the complexity class $P$ is closed under complements.
Now consider a language $L$ which belongs in $NP$. Two cases arise
- If $L' \in P$, then both $L$ and $L'$ must be in $P$, since $(L')' = L$ and $P$ is closed under complement.
- If $L' \not \in P$, then $L$ cannot be in $P$ either, since if $L$ were in $P$, then $L'$ would also be in $P$. This would mean that we have a language $L$, which is in $NP$, but not in $P$. And that in turn means that $P \neq NP$, which is an open question.
So if we knew the answer to $P \text{ vs } NP$, would we be able to answer this question? In one case, but not the other.
Let's assume that
- $P = NP$. Then if $L \in NP$, then $L$ must also be in $P$, which means that $L' \in P$ as well. In this case, all the given options a, b and c are incorrect.
- $P \neq NP$. Then it could be that $L \in NP$, but $L \not \in P$. Now what about $L'$? $L' \in \text{co-NP}$, but we again don't know if $\text{NP} = \text{co-NP}$, and that is a different open question.
In case the above explanation is confusing, let's look at the options and try to see if they're true or false.
Option (a) says: $\rm L$ belongs to $\rm NP$ does not always imply that $\rm L'$ belongs to $\rm NP$.
If option (a) is correct, it would mean that there exists a language $\rm L \in NP$, such that $\rm L' \not \in NP$. This would mean that $\rm NP \neq co\text-NP$, and we don't know if that is true or not ($\rm NP$ vs $\rm co\text-NP$ is a open question).
Option (b) says: $\rm L$ belongs to $\rm NP$ does not always imply that $\rm L'$ belongs to $\rm P$.
If option (b) is correct, it would mean that there exists a language $\rm L \in NP$, such that $\rm L' \not \in P$. Now if $\rm L$ belonged in $\rm P$ itself, $\rm L'$ would also be in $\rm P$ since $\rm P$ is closed under complement. Thus, our language $\rm L$ must be such that $\rm L \in NP$, but $\rm L \not \in P$, which in turn means that $\rm P \neq NP$, which is again an open problem that we don't know the answer to.
