Here,
$x_{n+1} = cx_{n}^{2}-2 , c > 0$
For stability, We can write non-linear first order recurrence as :- $x_{n} = f(x_{n-1})$ (or) $x_{n+1} = f(x_{n})$
So, $x_{n+1} = cx_{n}^{2}-2 , c > 0$ becomes $x_{n} = cx_{n}^{2}-2 , c > 0$ (or) For simplicity, to find fixed stable points , we can write it as :- $x = cx^{2}-2 , c > 0$
Now, after solving the given equation , we will get :-
$x= \frac{1\pm \sqrt{1+8c}}{2c}$
Now, here we have $2$ fixed points $x_{1}= \frac{1 + \sqrt{1+8c}}{2c}$ and $x_{2}= \frac{1 - \sqrt{1+8c}}{2c}$
Now, We have to check that for which fixed point the given recurrence converges.
As we know if $f' < 0$ at a point or slope is negative then it means function $f$ is strictly decreasing and if $f' > 0$ then it means function $f$ is strictly increasing.
So, if we find $f' < 0$ in a given interval , it means function $f$ is decreasing or converses to a fixed point in the given interval.
So, Now, Since here , $f(x) = cx^{2} - x -2$
So, $f'(x) = 2cx - 1$
Now,
$f'(x_{1}) = 2c * \left ( \frac{1+\sqrt{1+8c}}{2c} \right ) - 1$
So, $f'(x_{1}) \nless 0 , c> 0$
Now,
$f'(x_{2}) = 2c * \left ( \frac{1-\sqrt{1+8c}}{2c} \right ) - 1$
So, $f'(x_{2}) < 0 , c> 0$
So, here local stable point is $\frac{1-\sqrt{1+8c}}{2c}$ for which given recurrence converges.
So, Answer is (B)
Reference :- http://www.ms.uky.edu/~droyster/ma114F16/RecursiveSequences.pdf