I'm not one to generally consider bridge tolls the same as highway tolls. The sheer volume of traffic through the PANYNJ crossings (bridge or tunnel) necessitates a toll to fund the bridge or tunnel. So, paying the GWB or the Holland or Lincoln tunnels isn't an issue for me. Plus, when as mentioned, the cost of going around the toll generally exceeds the cost of the toll.
And even then, there's only a couple ways from Bronx to Manhattan without a toll. And getting to Staten Island, Brooklyn or Queens all would require a toll bridge or tunnel.
From Chicago, best way is I-94 to IN-49 to US-30 (or I-94 to US 20 to IN-2 to US-20 to US-31 to US-30) and east around Fort Wayne. Depending on desire, either than US-30 to I-71 to I-76 to I-80 OR take US-24 to I-475 to I-75N to I-280 to OH-2 and just slog along the non-freeway wetland area until the freeway starts near Sandusky to I-90 to I-490 to I-77 to I-480 to OH-14 to I-76 to I-80 OR I-90 to I-490 to I-77 to I-480 to US422 to OH-82 to OH-11 to I-80.
Then it's I-80 through Pennsylvania. You can just cross into the NYC on a PANYNJ crossing, or take I-81 to I-84 or US-209 to I-84 and get NY-17 and cross the Thurway on US-6. A little more scenic in that area, cross the toll, and take US-9 south and then cut over to the Taconic, and work your way into Manhattan.
Another option, as noted, would be in Ohio, take either I-71 NB to I-271 NB to I-90 EB or OH-2 EB to I-90 EB and enter PA and take I-86 (also NY-17) until Binghamton and take I-88 East to Albany. if EZ Pass, the stretch of I-90 Thruway is free to the next exit (I think you pay with cash, or get off early and route around it, not difficult), take I-90 across to the Taconic, and take that south.
But, length of the trip and what you want to see, and how much time and money are you willing to spend to avoid the toll roads and/or bridges.