How we calculate utilization in case of GBN or SR ARQ ? We simply have a notion of busy transmission time (of sender) and total time, and we divide former by the later to get the utilization factor. Like in Stop and Wait ARQ,
Utilization = $\frac{TT}{RTT+TT}$ where TT = transmission time of one packet and RTT+TT = total time of one cycle. If we look in a different way, in time TT , (Bandwidth*TT) amount of data (or bits) can be in the network circuit (or link). And in case of Stop and wait ARQ in a time span of (RTT+TT) we can send [ Bandwidth*(RTT+TT) ] amount of data (or bits) but instead we have utilized only [ Bandwidth*TT ] bits. so the utilization formula makes sense.
Back to our question, here they have not given any error/flow control policy. We can not assume busy transmission time or idle time like in the previous case. But luckily bandwidth is given as 1GBPS. For maximum through put we assume no stalling in the sender's transmission and for that we have condition,
Time-to-send-entire-window ≤ Time-for-first-ack-to-arrive-back
So in an RTT of 20ms we will send the entire window.
Now, Bandwidth * RTT = $10^{9}*20*10^{-3} = 2*10^{7}$ bits.
Utilization is given = 0.8
So amount of data (or bits) actually in transit on the link = $0.8*2*10^{7} = 1.6*10^{7}$ bits.
Number of packets in window (or window size) = $\frac{1.6*10^{7}}{8*1000} = 2000$
we have neglected the ack bits according to the question