M = $\begin{pmatrix} 6&2 \\ -2&-3 \end{pmatrix}$
Characteristic equation is :
$\begin{vmatrix} M - \lambda I \end{vmatrix} = 0$
So after solving characteristic equation will be -
$\lambda ^{2} = 3\lambda + 14I$
Using Caley Hamilton's theorem
$M^{2} = 3M + 14I$
Again $M^{5} = M^{2}\times M^{3}$
$M^{5} = (3M + 14I)\times M^{3}$
$= 3M^{4} + 14M^{3}$
= $3(M^{2})^{2} + 14M\times M^{2}$
$= 3(3M+14I)^{2} + 14M(3M+14I)$
$= 3(9M^{2} + 84M + 196I) + 42M^{2} + 196M$
$= 27M^{2} + 252M + 588I + 42M^{2} + 196M$
$= 69M^{2} + 448M + 588I$
$= 69(3M + 14I) + 448M+ 588I$
$= 207M + 966I + 448M + 588I$
$= 655M + 1554I$
$= 655\times \begin{pmatrix} 6 &2 \\ -2&-3 \end{pmatrix} + 1554\times \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$
$= \begin{pmatrix} 3930 & 1310 \\ -1310 & -1965 \end{pmatrix} + \begin{pmatrix} 1554 & 0\\ 0 & 1554 \end{pmatrix}$
$= \begin{pmatrix} 5484 & 1310\\ -1310 & -411 \end{pmatrix}$
Trace of $M^{5}$ is $5484-411$ $= 5073$