The reason that there are few 55 MPH signs in California is because, by state law, the speed limit on many rural roadways is 55 MPH (unless otherwise posted). Thus, when you see an "END 50 MPH" sign, the implication of that sign is that the speed reverts to 55 MPH. This practice is common on older roadways (say, those built prior to the 1960's) and can even appear on freeways (like on southbound CA 1 at Ocean Street in Santa Cruz).
However, I agree with you that not having 55 MPH signs creates ambiguity for drivers as to what the speed limit truly is. Ambiguity is NOT a good thing for drivers, as it can lead to inattention (while pondering the sign's message), and thus can cause collisions. Yet at the basis of the speed laws in California is a belief that drivers will adjust their speed to the conditions of the road, e.g. if a road has no shoulders, the law assumes that drivers will routinely reduce their speed accordingly. In my experience, this is occurring less and less frequently.
Note: The other two cases in California where an unsigned roadway has an unsigned speed limit are 1) local streets (such as those in neighborhoods), and 2) streets through denser business districts (such as a traditional downtown); in both cases, the speed limits are 25 MPH by state law.