Context Switch basically means, taking CPU from one process and giving it to another process (which is in the ready queue and hence already loaded into memory).
Paging is way to manage memory efficiently by bringing into memory those part of the program which is required and keeping the rest of the disk and bring them as per need. Pages of other processes, which were brought into memory earlier might be evicted to make space for new processes.
So in the current situation, during a context switch, the new processes might find that most of its pages have been evicted, which needs to be brought back, which will take time, and hence the time to context switch is increased.
Imagine if there were no paging, then the whole program has to be kept into the memory and during a context switch, the new process will always find its all "parts (pages)" in the memory and hence, this will be very fast but very very very un efficient use of the memory, as we have to keep everything in the memory even if it is not required as of now.
Hope this helps. Please let me know if there are any doubts.
See this answer:
https://stackoverflow.com/a/54057079/2858560