The thing to remember is, at its core, ChatGPT is nothing more than an extremely powerful Markov chain generator. Markov chain generators just look at a body of data and predict what the most likely word to come next is. That is the same idea underlying autocorrect on your phone. ChatGPT has just been fed a giant corpus of data and then weighted it so that certain results are more likely to come up.
We all know how well autocorrect works. That fact, taken with the fact that both Google and Microsoft want to add AI results to their search engines, is nothing short of terrifying.