I m not exactly clear what u mean here by size of the matrix but no of elements present at any time would be
$\frac{N(N+1)}{2}$-1 if we take starting index to be 1 of i,j.
put N=3 we get 5 elements which are (2,2) (2,3) (3,1) (3,2) (3,3) similarly it holds for N=4 and above !
But above all the matrix size would be N*N