Answer is (A) Commutative but not associative.
$y \oplus x = y^2 + x^2 = x \oplus y$. Hence, commutative.
$ (x \oplus y) \oplus z = (x^2 + y^2) \oplus z = (x^2 + y^2)^2 + z^2$
$ x \oplus (y \oplus z) = x \oplus (y^2 + z^2) = x^2 + (y^2 + z^2)^2$
So, $( (x \oplus y) \oplus z) \neq (x \oplus (y \oplus z))$, hence not associative.