For this we need to find the corresponding characteristic equation which will be cubic :
This is found as :
| A - λ I | = 0
==> (1 - λ)[ (5 - λ)(1 - λ) - 1 ] + 1 [ 3 - 1 + λ ] + 3 [ 1 - 3(5 - λ) ] = 0
==> λ3 - 7λ2 + 36 = 0 which is the required characteristic equation of the given matrix..
Now using relation between roots and coefficients , we have :
Sum of roots taken two at a time = (Coefficient of λ ) / (Coefficient of λ3) = 0
Product of roots = -(Constant term) / (Coefficient of λ3) = -36
But here it is -αβγ = -(-36) = 36
Hence 36 is the correct answer..