There is a concept of Rates of Growth in Calculus.
Definition :- f(x) <<< g(x) as x $\rightarrow$ $\infty$ means growth rate of f(x) is very slow as compared to g(x) when x $\rightarrow$ $\infty$ ie. $\frac{f(x)}{g(x)} \rightarrow 0 \, \, as \, \, x\rightarrow \infty$
It is true when f and g are non-negative.
So, we can say that :-
1) $if \lim_{x\rightarrow \infty } \frac{f(x)}{g(x)} = \infty \, \, then\,\, f(x) >>> g(x)$ ie. growth rate of f(x) is higher than g(x)
(or)
2) $if \lim_{x\rightarrow \infty } \frac{f(x)}{g(x)} = 0 \, \, then \,\,g(x) >>> f(x)$ ie. growth rate of g(x) is higher than f(x)
Based on this fact we can say that :-
Rates of Growth of the following functions is :-
lnx << xp << ex << $e^{x^{2}}$ for p>0
and Rates of Decay should be :-
$\frac{1}{lnx}$ >> $\frac{1}{x^{p}}$ >> e-x >> $e^{-x^{2}}$ , p>0
----------------------------------------------------------------------------------
Now , Rates of growth of Running Time of algorithms is same as order of growth of running time of algorithms. It is already mentioned in Cormen
According to Cormen ,
f(n) is asymptotically larger than g(n) if f(n) = $\omega$ (g(n))
(or)
$if \lim_{n\rightarrow \infty } \frac{f(n)}{g(n)} = \infty \, \, exist \, then\,\, f(n) \,\,becomes\,\, arbitrary \,\, large \,\, as\,\, compared\,\, to\,\, g(n) \,\,as\,\, n \,\,tends \,\,to\,\, infinity $
----------------------------------------------------------------------------------
Now , In this Question ,
On comparing f2 and f3
$\lim_{n \rightarrow \infty } \frac{n^{\frac{3}{2}}}{nlogn} = \lim_{n \rightarrow \infty } \frac{n^{\frac{1}{2}}}{logn} , when \,n \neq \infty \,\,$$Now , using \,L'H\hat{o}pital \, Rule , \lim_{n \rightarrow \infty } \frac{n^{\frac{1}{2}}}{logn} = \infty$
I have taken natural log above because it does not matter and we can convert one base to another and then solve. It will give same answer
So, f2 >>> f3
Now , On comparing f1 and f4 :-
$\lim_{n \rightarrow \infty } \frac{2^{n}}{n^{logn}}$
Since , 2n = (eln2)n = en*ln2
and nlnn = (eln n)ln n = $e^{(ln n)^{2}}$
Since , exponential is an increasing function. So, comparing 2n and nln n is same as comparing n*ln2 and (ln n)2
So, By using $L'H\hat{o}pital's \, Rule $
$\lim_{n\rightarrow \infty } \frac{n*ln2}{(ln n)^{2}} = \infty$
So, n*ln2 > (ln n)2
and $\lim_{n \rightarrow \infty } \frac{2^{n}}{n^{logn}}$ = $\infty$
So, now we can say f1 >>>f4
Now, these 2 comparison of functions are enough to eliminate options in this question.