News:

Thanks to everyone for the feedback on what errors you encountered from the forum database changes made in Fall 2023. Let us know if you discover anymore.

Main Menu

Status of Travel Mapping

Started by Purgatory On Wheels, June 01, 2015, 10:49:44 AM

Previous topic - Next topic

Purgatory On Wheels

Those of us not involved in creating the new CHM/Travel Mapping site would love to hear updates regarding the status of the new site.  What are you currently working on, and when do you expect to have various components complete?

Thanks to all of you who have volunteered to resurrect CHM.


english si

I'm working at bringing files (mostly in Europe) up-to-date. As I have (half an hour ago) finished my last assignment for the summer, I could, in theory, get it done by a couple of weeks time. However, I doubt that urgency is needed, so I'll probably take my time with the remaining few countries (Spain, Germany, Poland, Sweden) and spend time finishing French N roads and other such future systems (and perhaps borders, though they are really boring).

Also (as a reminder to me) US6 in CO needs tweaking after the AASHTO decision.

Jim

Quick status update:

I have python code that can now read in all of the highway data and put it into a database.  .list data next.  It's the first database of any significance that I've designed, so I'm sure I've made some bad decisions.  I hope to learn more about how to do it right as I continue.

My first visible evidence of progress is a very basic highway browser that reads its data from that database.  All it can do is take a highway's internal "root" name, read the points from the database, and put those points in a table and onto a map.  For example, to see I-88 in New York:

http://www.teresco.org/~terescoj/travelmapping/hbtest/?r=ny.i088

Replace the "ny.i088" with your favorite plotted route in the US or Canada (Europe's not in the DB yet) to see others.

Anyone with some HTML/JS/PHP skills is welcome to take this basic model and come up with something more pretty and functional.  There are several ways this could be managed - if interested let me know and we can figure out how you can get the file to populate your own database, or get you some read access to a database I can put somewhere for your development.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Duke87

Quote from: Jim on June 09, 2015, 11:51:59 PM
My first visible evidence of progress is a very basic highway browser that reads its data from that database.  All it can do is take a highway's internal "root" name, read the points from the database, and put those points in a table and onto a map.  For example, to see I-88 in New York:

http://www.teresco.org/~terescoj/travelmapping/hbtest/?r=ny.i088

It might be basic visually but damned if it isn't fully functional. I could map my travels using this no problem.

I actually like the little green d-pad dots more than the google pushpin balloons the old site had. Less visually cluttersome and they get in the way of each other a lot less since they don't stick out from the line as much.
If you always take the same road, you will never see anything new.

english si

Quote from: Duke87 on June 10, 2015, 12:06:38 AMIt might be basic visually but damned if it isn't fully functional. I could map my travels using this no problem.
Extra functionality is intersecting routes and alt labels. The former is very useful, the latter somewhat useful. Oh, and links back to the browser.

Great job!

Jim

Quote from: Duke87 on June 10, 2015, 12:06:38 AM
I actually like the little green d-pad dots more than the google pushpin balloons the old site had. Less visually cluttersome and they get in the way of each other a lot less since they don't stick out from the line as much.

They're tiny versions of a little icon that's supposed to be two roads intersecting (with dashed yellow lines separating lanes).  Admittedly, much of the already limited detail was lost in shrinking down to 16x16 pixels.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

froggie

Jim:  good start.  In addition to the other suggestions, I don't think it's necessary to list the shaping points, so that might be something for whomever follows up to work on in the next rendition.

Jim

All existing systems are now in the database and can be viewed by the draft highway browser.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

sipes23

Quote from: Duke87 on June 10, 2015, 12:06:38 AM
It might be basic visually but damned if it isn't fully functional. I could map my travels using this no problem.

I actually like the little green d-pad dots more than the google pushpin balloons the old site had. Less visually cluttersome and they get in the way of each other a lot less since they don't stick out from the line as much.

Amen and amen. It's not got a spiffy, tricked-out design, but the function is all there. The latter is much more important. And as far as I can see, it's got every highway in the old CHM plus all of the in-development systems too. I'm super delighted to see this. Also, I too like the intersection markers sitting on the intersection.

Jim

A couple quick improvements on the draft highway browser just now.  Hidden waypoints are no longer shown in the table of waypoints, and clicking on a waypoint name in the table centers the map on that point.  The intent is to bring up an "info window" pointing at the marker on the map like the old HB, but that's not happening for some reason.

The version that's up now, minus my mysql username and password, is available in Github in the TravelMapping/Web repository.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Jim

The last few days' efforts in detecting clinched segments and getting them into the DB in a meaningful way have finally paid off tonight with a big enhancement to the draft highway browser.  This included my most complex DB queries yet (though anyone with real DB experience would probably laugh at them).  The HB now takes a second query string parameter - the username of someone whose .list file has been processed into the DB, and will show that user's travels on a given highway.

For example, to see my travels to date on NY 5:

http://www.teresco.org/~terescoj/travelmapping/hbtest/?r=ny.ny005&u=terescoj

A couple things:

- No concurrency detection, yet.  But that is going to be relatively easily taken care of on the Python side, so the DB won't care if you clinched a segment because it was in your list or because some other concurrent segment was, and since the DB won't care, the HB won't care.

- I know it's ugly and not user-friendly and the markers are too big and all of that.

- I'm happy to bring in .list files of others who might want to be able to browse their travels this way and the next time I run a DB update, they'll be there.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Duke87

I'm game. Where should I send my list file to for that?
If you always take the same road, you will never see anything new.

rickmastfan67

Quote from: Jim on June 13, 2015, 10:37:20 PM
For example, to see my travels to date on NY 5:

http://www.teresco.org/~terescoj/travelmapping/hbtest/?r=ny.ny005&u=terescoj

Jim, I just noticed an error in that file.  Have you had a chance to update the reconfiguration of the US-20/NY-5/NY-438 area in the files?  They replaced the separate intersections (w/ one traffic light) into a roundabout for all 3 routes.  That means 'US20_Irv' needs to be merged with 'NY438'.  Updates to all 3 routes are needed.

froggie

James:  as I recall, we're focusing on getting an operational reader/browser/website before we look at route file updates.

Jim:  feel free to use my .list

Purgatory On Wheels

Excellent work, Jim!

Re the waypoint marker size: I don't think they're too big.  Sure, for a route as long as NY5, there are so many that they obscure the route (as did the CHM markers), but for a shorter one, they're not bad at all.

Thanks to all who have volunteered their time so far.  It's great to see that the project will live on.

Jim

Quote from: rickmastfan67 on June 14, 2015, 12:02:20 AM
Jim, I just noticed an error in that file.  Have you had a chance to update the reconfiguration of the US-20/NY-5/NY-438 area in the files?  They replaced the separate intersections (w/ one traffic light) into a roundabout for all 3 routes.  That means 'US20_Irv' needs to be merged with 'NY438'.  Updates to all 3 routes are needed.

Noted in the old CHM forum so I'll remember when the time comes, thanks.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

vdeane

Quote from: Jim on June 13, 2015, 10:37:20 PM
- I'm happy to bring in .list files of others who might want to be able to browse their travels this way and the next time I run a DB update, they'll be there.
Awesome.  It will be great to be able to see the maps and states for individual routes again, as well as the log.  I'm sure there's at least one typo somewhere in my .list file that I never saw.

I liked to it earlier, but here it is again, updated as recently as this morning: http://nysroads.com/files/aaroads/vdeane.list
Please note: All comments here represent my own personal opinion and do not reflect the official position of NYSDOT or its affiliates.

Jim

Today was concurrency day.  Concurrent segments are now detected, and travelers get credit for all concurrent segments if at least one of them is in their .list file.

See what segments were detected as concurrent, and which segments travelers were credited with based on concurrencies in http://www.teresco.org/~terescoj/travelmapping/logs/concurrencies.log.  Every single concurrent segment (the single segment between two consecutive waypoints) is reported, unlike the way the old CHM project joined these together into larger concurrent segments.  Adding that is a low priority, I'd say.

There are now 15 .list files included.  Any of those 15 should be able to see their travels on a single highway with a region, including detected concurrencies by visiting the same URL as in recent days, substituting the traveler name and route name as appropriate.  E.g., http://www.teresco.org/~terescoj/travelmapping/hbtest/?r=ny.ny005&u=terescoj

Also, some missing CA routes were added.

The next step needs to be some efficiency improvements.  All steps are fast except processing .list files (now almost 5 minutes) and loading the DB (about 20 minutes, as it's now just a big old SQL file with 1,214,141 lines.  I have learned of some better ways that I hope to try out the next time I have a chance to put some time into the project).

For now, I continue to welcome reports of anything odd in log files, or any errors you can find in the concurrency detection.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Duke87

Quote from: Jim on June 14, 2015, 10:56:54 PM
Every single concurrent segment (the single segment between two consecutive waypoints) is reported, unlike the way the old CHM project joined these together into larger concurrent segments.  Adding that is a low priority, I'd say.

Indeed. Seems to me at least this is "log for nerds who want to see code line by line", not so much something the average user would need to see. Presumably once stat pages are constructed any autodetected concurrencies would be reflected there, in a more graphical and user friendly way than this.


One thing I do like though is that it's only listing segments the algorithm is adding to a user's stats, not just every concurrency in the users travels, which I believe the old site did. For all my lines of autodetected concurrency, there are conspicuously none in Connecticut - because I have the whole state clinched, so the algorithm has nothing to add!
If you always take the same road, you will never see anything new.

Bickendan

Just updated mine (nailed down the entirety of WA 503 yesterday... save for the Brush Prairie business loop).

Jim

#20
Quote from: Duke87 on June 15, 2015, 01:45:47 AMIndeed. Seems to me at least this is "log for nerds who want to see code line by line", not so much something the average user would need to see. Presumably once stat pages are constructed any autodetected concurrencies would be reflected there, in a more graphical and user friendly way than this.

I'd call it the "log for the programmer who wanted to see exactly what the algorithm was doing."  I'd like to either have something easier to understand in the either in per-user logs or a separate log like this one, and/or a map where one could browse the detected concurrencies, likely with a "u=" query string parameter to restrict to autodetected concurrencies for a given user.  Again, that's for down the road.

After efficiency improvements, I think the priorities should be:

- Nail down data formats and convert (if needed) to them.  I worry about this step until we get a route editor up and running with the new format.  I don't think implementing a new editor is especially difficult, but I'm hoping someone else will take it on.  I can continue on data processing work with the old format, but I do think we should make the switch before starting highway updates and adding new systems.  There's a huge backlog of fixes to existing systems, several in-development systems that are probably close if not ready to activate, and a good number of people itching to contribute new things.  It should be trivial for me to load in data in a new format if and when the format change happens - just a couple lines of the Python code will change.

- More maps.  Both Google Maps and the image files like the old CHM project produced.  I think everything needed to support these is in the DB now, but I'm sure I or others who implement these parts might want additional information or information in more convenient formats.

- Start generating some stats.  I think this is mostly best done on the Python side when a site update is running, when so much information is in convenient to access data structures.  The stats could be displayed in log form to start, then added to the database to be put onto pretty web pages.  I'll probably start with things similar to what we see in the old CHM project, but I think there's all kinds of room to do interesting things here.

- Lots more of the web-facing infrastructure.  I'm generating some pages that have the PHP code that queries the DB and puts up the basics, but I'm hoping others with better web design skills than me will take the lead on this.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Jim

I'm very excited about the newest enhancement to the draft highway browser, one that was always on my wish list in the old CHM project.  Now, if you specify a region and a username, you will get a Google Maps plot of all routes in that region, with the user's travels highlighted.  For example:

http://www.teresco.org/~terescoj/travelmapping/hbtest/?rg=MA&u=terescoj

Of course, you'll need to zoom in a bit to see anything useful with so many waypoint markers and routes plotted in most regions.  It also is using the same code (so the same simple page design) used for viewing a single highway, and doesn't show any stats yet.  But this is an important proof-of-concept that we can provide lots of interesting maps right from the DB.  And it's fun to see my travels in this format, and including those after November 8, 2014.

Those whose list files are in the system, please browse around and look for any problems.  I looked at quite a few regions with my own travels and everything looks good so far, but there are definitely many aspects of this that are lightly-tested at best.  In particular, I am relying on things coming out of the DB in a particular order so any reports of odd-looking routes or incorrect clinches are welcome.  Note that in-development systems will be included on the map, but your clinched segments will not be highlighted, even if they are specified in your .list (for now).

Enjoy!
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

sipes23

#22
Quote from: Jim on June 16, 2015, 06:31:40 PM
I'm very excited about the newest enhancement to the draft highway browser, one that was always on my wish list in the old CHM project.  Now, if you specify a region and a username, you will get a Google Maps plot of all routes in that region, with the user's travels highlighted.

1. That's an awesome feature.
2. That's a killer awesome feature.
3. Thank you.

That said, on my California file, I noticed a gap in the 405 just east of Long Beach. I suspect, but don't know, that it may have something do with the order in which concurrencies are rendered (and whether they are in active vs. in-development systems). Maybe fixable, maybe a bug that can't be fixed. But there it is. Otherwise, like I said, it looks great and I can't find any other weirdness in it.

http://www.teresco.org/~terescoj/travelmapping/hbtest/?rg=CA&u=sipes23

Edit to add: Yeah, same thing in MT too. Where MT 200 runs concurrent with I-90, it's gray. Maybe a layering thing.

Jim

You're welcome!

Quote from: sipes23 on June 16, 2015, 07:50:53 PM
...Maybe a layering thing.

Yes, that's exactly it.  The inactive CA (and MT) systems are being drawn (as unclinched) on top of the clinchable ones (which were drawn as clinched).  I'll see about adding a check for plotting activated systems only.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Purgatory On Wheels

Love this!  Thank you!

I like how concurrent routes are being rendered multiple times (the pink shading looks more vivid where two clinched routes overlap).  Even in cases where my list file only included one of them, the concurrency still shows.

Well done, Jim!



Opinions expressed here on belong solely to the poster and do not represent or reflect the opinions or beliefs of AARoads, its creators and/or associates.