$n^{\log_b a} = n^{\log_2 4} = n^2$
i.e. $f(n) = \Omega \left(n^{\log_b a}\right)$.
Now, to apply Master theorem case 3, we need a positive constant $\epsilon$ such that $f(n) = \Omega \left(n^{\log_b a + \epsilon} \right)$. But here we can't get any such constant and hence we can't apply Master theorem. So, solving by expansion:
$$\begin{align}
T(n) &= 4T \left (\frac{n}{2} \right ) + n^2 \log n\\[1em]
&= 16T\left (\frac{n}{2^2} \right ) + n^2 \log n + n^2 \log \left(\frac{n}{2} \right)\\[1em]
&\quad \vdots\\[1em]
&= 4^{\lg n} + n^2 \left [\log n + \log \left(\frac{n}{2}\right) + \cdots + \log \left(\frac{n}{\large 2^{\lg n} \normalsize}\right) \right ]\\[1em]
&= 4^{\lg n} + n^2 \cdot \log \Biggl ( n \times \frac{n}{2} \times \cdots \times \large \frac{n}{2^{\lg n}} \normalsize \Biggr )\\[1.5em]
&= n^2 + n^2 \cdot \large \log \left ( \frac{n^{\lg n + 1}}{2^{\lg n \cdot (\lg n +1)/2}} \right )\\[1.5em]
&= n^2 + n^2 \cdot \large \log \left ( \frac{n^{\lg n + 1}}{n^{(\lg n +1)/2}} \right )\\[1.5em]
&= n^2 + n^2 \cdot \large \log \left ( n^{(\lg n + 1)/2} \right )\\[1.5em]
&= n^2 + n^2 \cdot \frac{\lg n +1}{2} \cdot \log n\\[1em]
\hline
T(n) &= \Theta\left( n^2 \log^2 n\right)
\end{align}$$
Actually we can even use Extended Master theorem which directly gives the asymptotic bound as $\Theta \left(n^2 \log^2 n \right)$.