This is an amazing question! After reading a lot on math.exchange and especially this paper, https://arxiv.org/pdf/1312.7147.pdf, this is a classic problem called coefficients of Sylvester’s denumerant.
First of here's the expansion from Wolfram so that you can verify.
Here's the shortcut!
So, it's straightforward the coefficient of $x^{100}$ is $10$.
Now let me explain in a simple way how to solve these for three coins. ( Full credit to this paper, https://arxiv.org/pdf/1406.5213.pdf, though note, this paper explains for 4 coins of different denominations.)
Define, $$S = \{i, s, t\} $$ which is a set containing the coins (denominations). For our problem, $i = 10, s = 20, t = 50$
So, for us,
$$S = \{ 10,20,50 \}$$
Let me denote $c_n$ as the coefficient to find for $x^n$ defined in the paper as a recurrence relation,
$$c_n = b_n + c_{n-t}$$
note here, $c_n = 0$ if $n \leq -1.$ ( I proved this by using Point 2, Def 2.2 in Page 3 ), and $c_0 = 1$, just substitute $n=0$ in the relation above.
Now, Theorem 1:
$$(\forall n) b = \left\lfloor\dfrac{n}{s}\right\rfloor + 1 $$
That's it!
Now you can solve any question using this. Let's go with our question and then two examples to clear out.
So, we need to find $c_{100}$. ( here, $s = 20, t = 50$ )
$$c_{100} = b_{100} + c_{100-50} = b_{100} + c_{50}$$
$$b_{100} = \left\lfloor\dfrac{n}{s}\right\rfloor + 1 = \left\lfloor\dfrac{100}{20}\right\rfloor +1 = 5 + 1 = 6$$
So,
$$c_{100} = 6 + c_{50}$$
Now,
$$c_{50} = b_{50} + c_0$$
$$b_{50} = \left\lfloor\dfrac{50}{20}\right\rfloor + 1 = 3$$
So we have now,
$$c_{50} = b_{50} + c_0 = 3 + 1 = 4$$
Thus,
$$c_{100} = 6 + 4$$
$$c_{100} = 10$$
...which is our coefficent!
Let's take another example, find the coeffient of $x^{160}$ in the above expansion. Checking the Wolfram brute force method it's $20$.
Same as usual, ( I'm skipping obvious steps now, it's too long to write now, :P )
$$c_{160} = b_{160} + c_{160-50} = b_{160} + c_{110}$$
We know, $b_{160} = 9$
$$c_{110} = b_{110} + c_{60}$$
Again, $b_{110} = 6$
$$c_{60} = b_{60} + c_{10}$$
Here, $b_{60} = 4$
$$c_{10} = b_{10} + c_{-40}$$
Note, $b_{10} = \left\lfloor\dfrac{10}{20}\right\rfloor + 1 = 0 + 1 = 1$ and $c_{-40} = 0$
So, $c_{10} = 1$
So, $$c_{160} = b_{160} + c_{160-50} = b_{160} + c_{110}$$
$$ = c_{160} = 9 + 6 + 4 + 1$$
$$c_{160} = 20$$
...which fits the brute force method.