Ah! This question was really a good one yesterday. Although it’s actually Θ(n^4), but even O(n^5) is not wrong. @Pranavpurkar Your method is correct if the loops are independent.
But for dependent loops, we need to postmortem the printf statement.
Carefully calculating the no of times printf statement is executed, we’ll see it to be:-
$i=0$ → 0 time
$i=1$ → 1 time
$i=2$ ; $j = 1,2,3,4$ printf runs for : 2+4 = 2(1+2) times
$i=3 $ ; $j = 1,2,3,4,5,6,7,8,9$ printf runs for : 3+6+9 =3(1+2+3) times
Hence, $\sum_{i=1}^{n}(i^3 + i^2) = O(n^4)$
Since there were options yesterday, $O(n^4)$ should have been the correct answer to it.
This q was asked before also: big o - Why is the runtime of this code O(n^5)? - Stack Overflow
Edit: i=0 -> 0 times