$N$ is the number of trials required to send a single packet. So I am not too sure if we can just set $N * 0.8 = 100$. Here is how I approached it.
There will be 100 different random variables $X_i$ corresponding to the expected number of transmission attempts of 100 different packets each. Each one will have an independent binomial distribution whose expectation you have to find. The number of trials required to send a single packet can go infinitely long (you might succeed in first attempt, OR you might fail in first and succeed in second, OR you might fail in first two and succeed on the third, and so on).
Once you find $E[X_i]$, by linearity of expectation, you can sum all $E[X_i]$ and get $E[X]$, which is the expected number of transmission attempts for 100 packets.
All $E[X_i] = 5/4$, and $E[X] = 5/4 * 100 = 125$.