On a set $S$ with $n$ elements how many relations are there?
A relation consists of set of ordered pairs $(a,b)$. Here $a$ can be chosen in $n$ ways and similarly $b$ can be chosen in $n$ ways. So, totally $n^2$ possible ordered pairs are possible for a relation. Now each of these ordered pair can either be present in the relation or not- 2 possibilities for each of the $n^2$ pair. So, total number of possible relations = $$2^{\left(n^2\right)}.$$
Now, for a relation $R$ to be reflexive, ordered pairs $\left\{(a,a) \mid a \in S \right\}$, must be present in $R$. i.e.; the relation set $R$ must have $n$ ordered pairs fixed. So, number of ordered pairs possible is $ n^2 - n$ and hence total number of reflexive relations is equal to
$$ 2^{\left(n^2-n\right)}.$$
Number of irreflexive relations is same as number of reflexive relations. In reflexive relations we always included $n$ ordered pairs for $\left\{(a,a) \mid a \in S \right\}$, while in irreflexive relation we always omit those $n$ ordered pairs. So, the number of ordered pairs to choose for the relation is the same for both reflexive as well as irreflexive relations which is $$2^{\left( n^2-n\right)}.$$
A relation becomes symmetric, if for ordered pair $(a,b)$ in $R$, ordered pair $(b,a)$ is also present in $R$. So, here, the total number of ordered pairs possible is reduced from
$n + n + \dots + n$ to
$n + n-1 + n-2 + \dots +1 = \frac{n (n+1)}{2}$.
So, total number of possible symmetric relations = $$2^{\left( \frac{n (n+1)}{2}\right)}.$$
A relation becomes anti-symmetric if for the ordered pairs $(a,b)$ and $(b,a)$ in $R$, $a=b$. i.e., the pairs $(a,b) $ and $(b,a)$ cannot be simultaneously be in the relation unless $a = b$.
For the $n$ pairs $(a,a)$ in $R$, they can be either present in relation or absent. So, 2 possibilities for each giving $2^n$ possible relations.
Number of pairs $(a,b)$ in $R$ such that $a \neq b $ equals number of ways of selecting 2 numbers from $n$ without repetition, equals $ \frac {n (n-1)}{2}$
Now, for each of these pairs $(a,b)$, there are 3 possibilities-
- $(a,b)$ and $(b,a)$ not in $R$
- $(a,b)$ in $R$ but $(b,a)$ not in $R$
- $(a,b)$ not in $R$ but $(b,a)$ in $R$
So, total number of possibilities for all such pairs = $3^{\left( \frac{n(n-1)}{2}\right)}$.
And total number of anti-symmetric relations on a set of $n$ elements becomes $$2^n \times 3^{\left( \frac{n(n-1)}{2}\right)}.$$
Finally, coming to your question, number of relations that are both irreflexive and anti-symmetric which will be same as the number of relations that are both reflexive and antisymmetric is
$$3^{\left(\frac{n(n-1)}{2}\right)}.$$
We just have to always exclude $n$ pairs being considered for $(a,a)$ while calculating the possible relations for anti-symmetric case.