Many of these smaller parks were known as "trolley parks". That is, they were built by trolley companies at the end of streetcar lines so they would get riders on weekends. Where there were once thousands of these, very few trolley parks remain today, though Pittsburgh's Kennywood is one example. As is Dorney Park in Allentown.
One trolley park that closed relatively recently is Willow Grove Park near Philadelphia, which shut down in 1976. It's now a mall by the same name, and I think there's a restored carousel from the original park inside the mall.
For other parks, Ocala, FL had a "wild west"-themed park called Six Gun Territory. It closed in the late '70s, apparently due to the decline in popularity of Westerns. Today it is the "Six Gun Plaza" shopping center.
Hmm, I wonder how many other former amusement parks became shopping centers/malls? Such as Nashville's Opryland, which became Opry Mills. But sometimes they co-existed...
In North Miami Beach, FL, the "Mall at 163rd St." used to have a small amusement park right on the premises, near the Burdine's entrance. This was when back when it was open-air (then called the 163rd St. Shopping Center), and it was only kiddie rides. A train ride circled the park area, which meant that the entrance was "protected" by a crossing gate. This all disappeared in the late '70s, and the mall was enclosed shortly thereafter.
In 1976, in the Omni complex in downtown Atlanta, GA, a hi-rise, indoor amusement park called "The World of Sid and Marty Krofft" opened. The park was, in part, themed around HR Pufnstuf and other TV shows by the puppeteer duo. But I understand the only major rides they had were (1) a carousel with crystal horses, and (2) a dark ride that simulated going through a pinball machine. The park was open for less than 6 months before it folded. Reasons often cited were that it was too pricey for what it was, and that the downtown area was perceived, at least at the time, to be unsafe. Today, the floors occupied by the park are now studios for CNN; the 8-story freestanding escalator used to get to the park's entrance -- I wonder how many people that scared off -- remains in use for studio tours.
Someone else mentioned Old Chicago; I think the only indoor amusement parks that really did well are inside malls. Mall of America near Minneapolis, and West Edmonton are the only ones I'm aware of. I think that Xanadu/American Dream Meadowlands in Northern NJ is supposed to have an amusement element -- this time it's really supposed to be opening next March.