An NFA accepts an infinite language if and only if there is a (directed) path from the initial to an accepting state which contains at least one state more than once.
So, just check for existence of cycle in the graph corresponding to the NFA starting with the start state - BFS can be used - $O(V+E) = O(|Z| n)$ where $Z$ is the alphabet set and $n$ is the no. of states in the NFA.
Since we have an algorithm (no need to worry about the time complexity), our decision problem is answered - "yes" is its answer. Or it is decidable to check if a given regular language is finite. More precisely it is decidable in polynomial time whether a given regular language is finite (the problem is not only decidable but also belongs to complexity class $P$).