It is an interesting question, but difficult to tackle in a rigorous way without clear metrics. I have watched Quantum Leap, for example, and I would not classify it as having a cliffhanger finale--rather, it ends on an even note, with no suspense but also no resolution. To my mind, an ending has to incorporate an element of suspense as well as lack of resolution to be a cliffhanger.
My own suspicion is that the prevalence of cliffhangers has a lot to do with fashions among TV shows as well as the evolution of successful TV series into franchises with DVD sales, tie-in merchandising, continuation movies, spin-off series, etc. I don't remember there being a huge vogue for cliffhangers until The X-Files came along in the mid-1990's, and even then that series had a reasonably conventional ending--the cliffhangers tended to occur mainly in two-parters and season finales. The series finale did however leave the door open to a continuation movie (which bombed). Stargate SG-1 in effect had two series finales in its run (first at the end of the eighth season when the Goa'uld story arc came to an end, and finally at the end of the tenth season when the Ori were defeated), and in both cases they were reasonably convincing wrap-ups, but even so there were two continuation movies as well as the spin-off series (Stargate Atlantis and Stargate Universe).
I think an underlying factor could be the use of VCRs for time-shifting (becoming common around 1985), since that takes the scripting tradeoff between multi-episode story arcs and "spectacles of the week" into a new dimension. The "spectacle of the week" format can be useful for pulling in new viewers since those viewers don't feel disadvantaged by not having seen the previous episodes, but such viewers are also easily lost. (Speaking for myself, I am more willing to try to follow series like Prison Break and Supernatural from start to finish, rather than, say, The A-Team or The Rockford Files--not because the latter shows are less entertaining on a per-episode basis, but rather because there is no plot continuity from episode to episode, so I feel I can stop watching at any time.) Multi-episode story arcs are useful for developing a dedicated fan base that is willing to follow the show from week to week, using novel technologies such as VCR recording to avoid missing broadcast episodes. The prevalence of VCR recording (as well as VHS and later DVD boxsets for past seasons) makes it easier for network executives and production companies to develop long-running story arcs without worrying about creating unacceptably high barriers of entry for new fans. If you consider that the age group with the highest susceptibility to becoming fans of a particular TV series is probably located right in the coveted 18-49 demographic, then the financial incentive becomes obvious.