@jay rathod let us suppose B is correct than
Step1: X adds digital signature to M, which can be done using X's private key, say X_{PR}. So now we have
<M,σ> as <M,(X_{PR},M)>
Step2: X Encrypts <M,σ> (after signing digitally, we are encrypting it to send) using public key of Y, so we have Y_{PU}(M,(X_{PR},M)) then send to Y.
Step 3: At y, Decrypt using X's public key. X_{PU(}Y_{PU}(M,(X_{PR},M))_{)}. which doesn't seems to generate the plain text as order is not correct.
Now instead, If I would have chosen D, then ill first decrypt using Y's private key i.e Y_{PR(}X_{PU(}Y_{PU}(M,(X_{PR},M))) which will give (M,(X_{PR},M)).
Step 4: Now apply X public key to verify if sender's identity is correct or not by extracting out the digital signature and retrieving M, i.e X_{PU}(X_{PR},M)=M
So correct order of operations is in D.