Status of Travel Mapping

[Jim]

The main complication is accounting correctly for concurrencies so segments aren't counted multiple times in summary.  For example, take a segment that carries two interstate routes, a US route, and a state route.  It should count just once in overall and regional stats, once each for system stats for I-, US, and the state's system, and once in each route's stats. 

I believe I have code in place that accounts for these situations, and I've generated regional and system mileage summaries that appear to match CHM's numbers.  For now, they're just printed in an ugly log file.  Before I go further, I'd appreciate some spot checking of these stats:

http://www.teresco.org/~terescoj/travelmapping/logs/clinchableroutes.log

If this looks accurate, I'll apply the same procedures to generate per-user stats, also in ugly log files to start.  If that goes well, all of this information will go into the DB so those writing front-end stuff can include them in web pages.

Edit: More likely, I'll read in the as-yet-ignored-by-me _con.csv files to generate per-route lengths first, so for example we could see I-90's length as a single route across several states.  Then on to per-user stats, then on to getting all of it into the DB.

[mapcat]

The main complication is accounting correctly for concurrencies so segments aren't counted multiple times in summary.  For example, take a segment that carries two interstate routes, a US route, and a state route.  It should count just once in overall and regional stats, once each for system stats for I-, US, and the state's system, and once in each route's stats. 

I believe I have code in place that accounts for these situations, and I've generated regional and system mileage summaries that appear to match CHM's numbers.  For now, they're just printed in an ugly log file.  Before I go further, I'd appreciate some spot checking of these stats:

http://www.teresco.org/~terescoj/travelmapping/logs/clinchableroutes.log

If this looks accurate, I'll apply the same procedures to generate per-user stats, also in ugly log files to start.  If that goes well, all of this information will go into the DB so those writing front-end stuff can include them in web pages.

Edit: More likely, I'll read in the as-yet-ignored-by-me _con.csv files to generate per-route lengths first, so for example we could see I-90's length as a single route across several states.  Then on to per-user stats, then on to getting all of it into the DB.

Looks like this is working.  I spot checked the mileage of US highways per state and each individual Ohio state route and got very accurate results for both sets.

I took CHM data from the tables, which round the mileage to tenths, and your data rounds to hundredths, so rounding was responsible for most of the difference, which came to a maximum 0.05 mi per Ohio route.  For the total mileage of US highways, some states were off by a little more, which I assume is the result of adding multiple routes that are off by a few hundredths.  Texas, for example, has a total of 12,784.39 miles of US highways in your data and 12,784.1 in CHM, for a difference of 0.29 miles.  This is still 99.9977% accurate.  North Carolina was next furthest off (0.18 mi, 99.9965%), and only 9 other states were off by more than 0.1 mi.  Given that there are so many different US highways in those states, the numbers certainly seem acceptable!

[clong]

I would definitely say that is acceptable error. Seems like the computations are working properly.

[Bickendan]

I noticed that I-5 in all three states is reporting high mileage -- should be ~798 for California, 308.5 for Oregon, and 277 for Washington, allowing for the extra distance between the final exit and the next state's border (796, 308, 276). CHM stats show 806, 312, 279.

[mapcat]

I noticed that I-5 in all three states is reporting high mileage -- should be ~798 for California, 308.5 for Oregon, and 277 for Washington, allowing for the extra distance between the final exit and the next state's border (796, 308, 276). CHM stats show 806, 312, 279.

He's checking whether or not the mileages he's calculated match what CHM used (and these do match).  Fixes to the data are coming later.  As for the discrepancy, maybe this is where the 2% fudge factor comes in.

[Jim]

I noticed that I-5 in all three states is reporting high mileage -- should be ~798 for California, 308.5 for Oregon, and 277 for Washington, allowing for the extra distance between the final exit and the next state's border (796, 308, 276). CHM stats show 806, 312, 279.

He's checking whether or not the mileages he's calculated match what CHM used (and these do match).  Fixes to the data are coming later.  As for the discrepancy, maybe this is where the 2% fudge factor comes in.

Exactly - if we're matching CHM that's good for now.

There will be a handful of things that don't match.  For example, I had to remove NM 599 from the usansf system since it's also in usanm, which I hope will be activated fairly soon after we are ready to make data changes.

[mapcat]

There will be a handful of things that don't match.  For example, I had to remove NM 599 from the usansf system since it's also in usanm, which I hope will be activated fairly soon after we are ready to make data changes.

As far as you can tell, which of the in-development systems from CHM are ready to be activated, once it's established that Travel Mapping is working as intended?

[Jim]

There will be a handful of things that don't match.  For example, I had to remove NM 599 from the usansf system since it's also in usanm, which I hope will be activated fairly soon after we are ready to make data changes.

As far as you can tell, which of the in-development systems from CHM are ready to be activated, once it's established that Travel Mapping is working as intended?

I'd say New Mexico, Utah, and Vermont state systems look to be closest, at least in the U.S.

[Jim]

Connected routes are now implemented in python and log files, but not yet in the DB.  For tonight, this means you can now check out roads that are continuous across regions to get total mileages in the

http://www.teresco.org/~terescoj/travelmapping/logs/clinchableroutes.log

file.  Next up, per-user stats in python and log files, then all of the stats into the DB so they can be accessed by web-facing parts of the new system.

Also, there's a DB update running now, likely done by the time almost everyone reads this, that will include all new and updated .list files I've received.

As part of this, I have temporarily commented out 7 in-development systems that had missing connected route files.  For now, you won't see cannf, cansph, usaca, usafl, usaky4, usala1, and usasc in the new HB.  Once we end the data "freeze", those should be able to be created quickly and get them back into the DB, and hence the new HB.

[Duke87]

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.

AL: not officially in development. No significant obstacles to implementation.
AK: officially in development. No significant obstacles to implementation. State of completeness = ?
AR: not officially in development. Obstacle: many discontinuous routes in several segments.
CA: officially in development. No significant obstacles to implementation. Largely done but needs some reworking.
CO: not officially in development. No significant obstacles to implementation.
FL: officially in development. Obstacle: some routes have signed and unsigned portions. State of completeness = ?
GA: not officially in development. No significant obstacles to implementation.
IN: not officially in development. No significant obstacles to implementation.
LA: officially in development for low numbered routes only. Obstacle: lots of minor high numbered routes. State of completeness = ?
MS: not officially in development. No significant obstacles to implementation.
MT: officially in development. No significant obstacles to implementation for primary routes. State of completeness = ?
NM: officially in development. Almost ready to go.
SC: officially in development. No significant obstacles to implementation. State of completeness = ?
TN: not officially in development. Obstacle 1: some routes have signed and unsigned portions. Obstacle 2: secondary and primary systems not easily neatly separated, may have to be treated as all one system like Vermont.
TX: not officially in development, but rough draft of primary system has nonetheless been completed
UT: officially in development. Almost ready to go.
VA: not officially in development. No significant obstacles to implementation for primary routes.
VT: officially in development. Almost ready to go.
WY: not officially in development. No significant obstacles to implementation.

[yakra]

BC: not officially in development.
AB (Primary): officially in development. Call it 83% done.
AB (Secondary): not officially in development, but rough draft of AB500 - AB546 nonetheless completed.
SK (Primary & Secondary): not officially in development.
MB (Primary): not officially in development, but first rough draft has nonetheless been completed. Call it 50% done.
MB (Secondary): not officially in development.
ON (Secondary): not officially in development.
QC: not officially in development.
NL: not officially in development.

[english si]

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.

CO: not officially in development. No significant obstacles to implementation. Draft files made
IN: not officially in development. No significant obstacles to implementation. Draft files made
MT: officially in development. No significant obstacles to implementation for primary routes. State of completeness = awaiting peer review
MT (secondary): not officially in development. No significant obstacles to implementation. IIRC, draft files started
WY: not officially in development. No significant obstacles to implementation. Draft routes made

Other officially indev systems:
CAN "select named freeways" and "select provincial highways" - scrapping them
GBR "A roads (Zone 1)", "A roads (Zone 6)", "A roads (Zone 8)" and "A roads (Zone 9)" - almost ready to go.

And I'll leave my massive Europe phase 3 systems that are unofficially in dev for later. I'm trying to work out which order I'm going to release them, for a start!

I think the priorities go like this:
1) get Travel Mapping fully functional
2) get active systems up to date
3) get those systems listed above in darker green (and MT primary routes) active

Obviously there's overlaps on time lines, and general release won't happen until 1 and 2 have occurred. I'd say that NM, UT and probably VT have very good odds of being activated before that happens. GB A roads a reasonable chance too.

[oscar]

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.

AL: not officially in development. No significant obstacles to implementation.
AK: officially in development. No significant obstacles to implementation. State of completeness = ?
AR: not officially in development. Obstacle: many discontinuous routes in several segments.
CA: officially in development. No significant obstacles to implementation. Largely done but needs some reworking.
CO: not officially in development. No significant obstacles to implementation.
FL: officially in development. Obstacle: some routes have signed and unsigned portions. State of completeness = ?
GA: not officially in development. No significant obstacles to implementation.
IN: not officially in development. No significant obstacles to implementation.
LA: officially in development for low numbered routes only. Obstacle: lots of minor high numbered routes. State of completeness = ?
MS: not officially in development. No significant obstacles to implementation.
MT: officially in development. No significant obstacles to implementation for primary routes. State of completeness = ?
NM: officially in development. Almost ready to go.
SC: officially in development. No significant obstacles to implementation. State of completeness = ?
TN: not officially in development. Obstacle 1: some routes have signed and unsigned portions. Obstacle 2: secondary and primary systems not easily neatly separated, may have to be treated as all one system like Vermont.
TX: not officially in development, but rough draft of primary system has nonetheless been completed
UT: officially in development. Almost ready to go.
VA: not officially in development. No significant obstacles to implementation for primary routes.
VT: officially in development. Almost ready to go.
WY: not officially in development. No significant obstacles to implementation.

MT primary, NM, and UT have gone through peer review, which puts them closest to activation. I'm not sure where VT stands there -- lots of discussion about the system, but I think not the kind of close review of specific routes typical of the peer review process. 

BTW, I think such peer review should be continued, especially without Tim providing a last review (or not, thus holding up the system) before activation.

As for AK, it needs a little more work to get it ready for peer review, as well as a team decision on what to do with two unsigned segments of otherwise signed routes.

For TN, an additional complication is number duplication between the primary and secondary routes, with at least one major route alternating between primary and secondary segments, but other numbers are assigned to completely unrelated primary and secondary routes.

[rickmastfan67]

Other officially indev systems:
CAN "select named freeways" and "select provincial highways" - scrapping them

I don't think the "select named freeways" should be really scrapped, unless we can find a way for the ones in Ontario to be activated somehow.

[Bickendan]

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.


CA: officially in development. No significant obstacles to implementation. Largely done but needs some reworking.

What specifically are we looking at here?
CA 44? Oscar should have a copy that just needs to be modernized.
Pass through all the routes and chopping them up per the stupid discontinuities caused by signage being removed where routes have been turnbacked despite a mandate to maintain guide signage?
Clean up on interstate and US routes, trimming down denser than needed routes?
Exit number conversions on the US routes?

[english si]

I don't think the "select named freeways" should be really scrapped, unless we can find a way for the ones in Ontario to be activated somehow.

They already are, in the canonf system.

[rickmastfan67]

I don't think the "select named freeways" should be really scrapped, unless we can find a way for the ones in Ontario to be activated somehow.

They already are, in the canonf system.

Only the Gardiner and Don Valley, and they aren't really provincial routes.

[yakra]

VT, Froggie peer reviewed for me.
MT, I peer reviewed for Si.

[english si]

MT primary, NM, and UT have gone through peer review, which puts them closest to activation.

MT, I peer reviewed for Si.

I'd forgotten that.

Do we want someone that is neither the author or peer reviewer to check the system over and then activate them?

[Roadgeek Adam]

SK (Primary & Secondary): not officially in development.
MB (Primary): not officially in development, but first rough draft has nonetheless been completed. Call it 50% done.
MB (Secondary): not officially in development.

Would anyone need help with SK and MB, cause I can at least help provide information as needed. A lot of this technical stuff is over me, but I'd happily contribute in any way.

[theFXexpert]

Here is my updated list file for whenever you are able to do another database update. Thanks to everyone for keeping this alive.

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.

FL: officially in development. Obstacle: some routes have signed and unsigned portions. State of completeness = ?

From what I could tell from browsing the old CHM, every road higher than FL802 hasn't been added yet. FL451 hasn't been added yet; FL429 and FL414 need to be modified in that area. FL293 needs to be extended.

[rickmastfan67]

Here is my updated list file for whenever you are able to do another database update. Thanks to everyone for keeping this alive.

Here's a rundown of the remaining states without active SR systems, as far as I'm aware of things. Anyone who knows anything I don't, feel free to chime in.

FL: officially in development. Obstacle: some routes have signed and unsigned portions. State of completeness = ?

From what I could tell from browsing the old CHM, every road higher than FL802 hasn't been added yet. FL451 hasn't been added yet; FL429 and FL414 need to be modified in that area. FL293 needs to be extended.

Yes, I'm aware of these tweaks.  Some of them have been done in my personal copies, others haven't yet.  FL got side tracked when I had to go back and 'update' several US highways in other states that I maintain to the current standards.  Still have to finish a few more TN US highways, and some other updates to active routes, and then I can get back to work on FL's state highways.

[oscar]

What specifically are we looking at here?
CA 44? Oscar should have a copy that just needs to be modernized.
Pass through all the routes and chopping them up per the stupid discontinuities caused by signage being removed where routes have been turnbacked despite a mandate to maintain guide signage?
Clean up on interstate and US routes, trimming down denser than needed routes?
Exit number conversions on the US routes?

The first item remains open for discussion. My inclination would be to chop, but the initial reaction of others on the team is like yours, to leave the routes intact. I can go with that if it remains the consensus. It doesn't hurt that, last time I was out there, many of the relinquished segments still have their old signage, with only a few jurisdictions feeling obligated to erect new signage. I've done almost nothing on the route-chopping, so little lost if we decide to forget it.

Another item is what to do with several routes not yet broken up at ferry crossings. We all know what Tim would've said, but "WWTD?" is no longer determinative. Especially since AASHTO just approved US 10 including a long ferry route across Lake Michigan, we might want to rethink our approach to shorter ferry crossings.

There are other issues, including the others you flagged, we can address later once it comes time to do a new instruction manual (I've volunteered to do that), and otherwise get going on developing California and other new systems. 

[bejacob]

On behalf of those of us who aren't actively involved with this project, I'd like to throw out a few questions to every working on the new travel mapping site.

1. Is the intent to keep the current method of uploading .list files? Have other methods for updating been considered or is that being delayed pending full functionality?
2. When is it appropriate to begin adding routes in states/regions under development? (Based on recent posts, it appears this will be phased in over time).
3. What is the likelihood of current waypoints changing thus requiring revisions to existing .list files?
4. How will we be able to find the correct waypoints for new systems or if existing ones change? (currently, I'm still going to CHM to look at the highway browser).
5. What other pieces of the old CHM site will be incorporated? (I love Jim's new maps). I imagine the highway browser and traveler stats are high on the list of priorities. (Here is one part of CHM that I look at often and would like to see http://cmap.m-plex.com/stat/region.php?u=bejacob&c=usa&rg=oh&du=mi&sort=ra
6. Will concurrencies be automatically updated as they are on CHM now? (I think this is already happening on the maps, but I'm curious how it will show up on the stats--see link in the previous question).
7. Are there things that those of who are not programmers can do to help (other than send Jim .list files and cheer everyone on  )?

I've been using the map Jim produces from my .list file to go back and find errors or missing routes (I'm almost embarrassed at how many I've discovered )

I know there are a great many steps still ahead on this project and I expect many of them will take some time to implement. Right now, I'm thrilled to be able to update my file and see the new maps (something I never got to experience on CHM as I uploaded my first file in November and then all went quiet).

Thanks to everyone involved in development. I don't have the knowledge or expertise to participate fully is all the discussion taking place on this forum, but I am following along in hopes of understanding bits and pieces. I'll chime in from time to time, but otherwise, I'll just keep reading and updating my file as necessary.

Thanks again
-Brian

[sammi]

1. Is the intent to keep the current method of uploading .list files? Have other methods for updating been considered or is that being delayed pending full functionality?

I think I'd rather see everything working first before people can start adding their .list files. I'm definitely looking at direct upload and GitHub as uploading options.

3. What is the likelihood of current waypoints changing thus requiring revisions to existing .list files?
4. How will we be able to find the correct waypoints for new systems or if existing ones change? (currently, I'm still going to CHM to look at the highway browser).

There's a labels_in_use field in the code that updates the database. I'm guessing that's there so that we know not to change labels if they're in use? If that does happen, I'd like to see a changelog for the data that shows which labels have changed.