Technical/Design/Implementation Discussions (CHM/Travel Mapping)

Started by Jim, April 04, 2015, 09:50:22 PM

Previous topic - Next topic

yakra

Quote from: english si on July 03, 2015, 04:00:47 PM
Sure, but when some nasty peer-reviewer called Eric comes and asks me to move a point a tiny bit, do I move it on all intersecting routes (often ones that have been peer reviewed and perhaps are active) that were identical or do I simply wait for a concurrency to require the adjustment? ;)
The proper procedure would be to silently tell that blighter to get stuffed. :bigass:

QuoteIt's clear you feel that moving waypoints so that the concurrency works is bad, so (to play devil's advocate) why should I do it when there's no concurrency to require it?
Wait, uhh... that bit confuses me. It's not clear. I don't feel that way.
Unless you're joking and it sailed over my head. Stupid Internets.
FWIW, I did end up changing the point coords on ME196 to match the other two files.
"Officer, I'm always careful to drive the speed limit no matter where I am and that's what I was doin'." Said "No, you weren't," she said, "Yes, I was." He said, "Madam, I just clocked you at 22 MPH," and she said "That's the speed limit," he said "No ma'am, that's the route numbah!"  - Gary Crocker


sammi

It's also just better when you zoom in on the map and everything lines up perfectly. :spin:

Jim

I think part of the problem with making fixes in CHM was the fact that there was just one complete copy of the data, and only one person had the authority to make changes to it.  Other people had copies of just their own parts, and coordinating changes that affected multiple people meant distributed copies being updated, all kinds of problems that the CHM collaborators understand all too well.  I think it will be important to get all those who will be contributing seriously to the highway data to be up to speed on git.  We don't want just anyone to be able to commit/push/whatever changes back in, but we do want people to be able to make the changes, and after a review process that we need to come up with, the changes could either be rejected, sent back for further fixing, or integrated into the master copy.  I really think we're going to be able to manage changes, especially fixes, much more efficiently.  A big part of it is that before if someone made a change that affected usany, for example, it had to go through me.  If someone else changed the official copy, I'd have to update my copy.  With git, that's all manageable.  We merge in the reviewed and approved change, and I pull down the update into my own local working copy.

We'll also be able to have multiple instances of the project very easily.  We could for example have a staging server where we can try out a new system (sort of the highway activation equivalent of a "soft opening" of a store) and see how it all looks and work out some problems.  Since everything is in public repositories, there's nothing stopping an individual from experimenting with a full copy of the project on their own computer/server/whatever.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

Jim

If you're following the GitHub repositories (and who isn't?), you might have seen the current data, now in the new directory organization, has been added to the master copy of the HighwayData repository by Si.  We can start working on fixing up the current data.  If you are and plan to continue to maintain data in some regions, I think it's best to work through GitHub.  I'm hoping someone with some experience in it can work with Si and I to come up with the appropriate workflow and instructions on how to make it happen.  At the moment, Si and I are the only ones who can pull changes into the master, but I hope we'll have more people able to do so once we have the process down.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

oscar

Quote from: Jim on July 06, 2015, 09:40:02 PM
If you're following the GitHub repositories (and who isn't?), you might have seen the current data, now in the new directory organization, has been added to the master copy of the HighwayData repository by Si.  We can start working on fixing up the current data.  If you are and plan to continue to maintain data in some regions, I think it's best to work through GitHub.  I'm hoping someone with some experience in it can work with Si and I to come up with the appropriate workflow and instructions on how to make it happen.  At the moment, Si and I are the only ones who can pull changes into the master, but I hope we'll have more people able to do so once we have the process down.

I will want to make updates, but that'll take a few weeks until i get back home (now in northern Alberta -- yakra asked me to check out some things there, which I hope to take care of tomorrow) to assemble my updates, and also can get the hang of GitHub. I agree that going through GitHub, rather than by e-mails to Jim or Si, makes the most sense.

Part of the process to be worked out (if not already) is creation of an "updates" page, and procedure to make additions to that page. Would be nice to incorporate CHM's update items through mid-2014, so we have in one place all the changes going into the highway data we're using. But that's not high-priority unless we think the CHM site (with its updates lists) is going to wink out on us.

Fortunately, my travel schedule will ease up (both I and my car are already pretty beat up) as Jim starts his own travels, then gears up for the upcoming academic year. So I'll be in a position to pick up some of the slack on the non-technical side.
my Hot Springs and Highways pages, with links to my roads sites:
http://www.alaskaroads.com/home.html

Jim

Regarding finding clinched segments for a user from the DB so they can be colored differently on maps: I decided that rather than sending an email about this to SSOWorld, I'd post here in case it's useful to others.  I'm also very open to suggestions about better ways to do this.

In https://github.com/TravelMapping/Web/blob/master/hbtest/mapview.php, it's the code from lines 196 to 234 that's using SQL query results to build the JS arrays that are used for this. 

The first part builds an array of waypoints, with the waypoint labels and coordinates that exist in the systems or regions being mapped.  Along the way, a second array called newRouteIndices keeps track of the first index into the waypoints array for each route.  There are also arrays built there that track the tier and color for each of those routes.

The second part builds an array of segments by "segmentID" which is just a unique identifier for each segment from the DB.  The key here is that these segments come out of the DB in the same order as the waypoints.  So if the first route being plotted has, say, 27 waypoints, the first 26 segments in the segments array are the 26 segments connecting those 27 waypoints.  Then the next route has 11 waypoints, so the next 10 segments entries connect those 11 waypoints, and so on.  So even though we only get segmentID out of the DB and save only that number in the segments array, from it, we can find the corresponding waypoints at each end of the segment from the waypoints array.

The third part builds an array of clinched segments, again by segmentID.  So when plotting the segments later, we'll check to see if the segmentID is found in the clinched array.  If so, plot it with a vivid color, otherwise with the lighter shade.  We can do this check efficiently as well, since we know the segmentIDs in clinched (when they exist) are in the same order as in the segments array.

So the PHP here builds all of this, and it's the JS code in https://github.com/TravelMapping/Web/blob/master/hbtest/chmviewerfunc3.js that uses it all to plot the maps.  The updateMap function does the work.  JS will execute the PHP-constructed function waypointsFromSQL to populate the arrays as described above.  The logic in play here to use the waypoints, newRouteIndices, segments, and clinched arrays is in updateMap starting at line 637.  I think the logic should be similar for building other kinds of maps, except you won't be making google.maps.Polyline objects, you'll be drawing on your map.

Writing this reminds me of some of the things I hope to enhance on the Google Maps overlay maps implemented in the files described above.  I'd like to be able to have ways to highlight a particular route, to be able to click on a segment and see information about it (what route(s) it carries, clinched status, length, endpoint waypoint names, etc.).  As I mentioned yesterday, that's on hold for a bit.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

bejacob

What steps need to be taken to use the stats/logs and maps that are currently on Jim's test site and create a web based front end?

My latest log looks great (except for a couple minor errors to fix on the next update) http://www.teresco.org/~terescoj/travelmapping/logs/bejacob.log as do the various maps, such as my home state of Ohio http://www.teresco.org/~terescoj/travelmapping/hbtest/mapview.php?rg=OH&u=bejacob.

I've looked at the source code from the CHM web pages and only understand a fraction of it. :confused: I'm sure many many folks working on this have excellent web/programming skills and understand exactly what the code that generates a page like this http://cmap.m-plex.com/stat/region.php?u=bejacob&c=usa&rg=oh&du=mi&sort=ra does.

How easy would it be to create a page like this http://cmap.m-plex.com/stat/traveler.php for the 30+ users that have .list files in the test db?

With the stats/logs available, who is able to take this output and transform it into web pages?

I'm impressed with everything that's happened so far. I don't know how much of it was Jim's work and how much others have contributed, but I thank everyone involved. I've been working with Jim to make sure my maps/list are correct (nearly there). Now I want to see what sort of web interface there will be. I'm sure it will take some time.

I think I speak on behalf of other non-techie folks when I say, keep up the good work. :clap: We're eager to see what comes next.

Brian

SSOWorld

Quote from: bejacob on July 16, 2015, 05:31:15 PM
What steps need to be taken to use the stats/logs and maps that are currently on Jim's test site and create a web based front end?

My latest log looks great (except for a couple minor errors to fix on the next update) http://www.teresco.org/~terescoj/travelmapping/logs/bejacob.log as do the various maps, such as my home state of Ohio http://www.teresco.org/~terescoj/travelmapping/hbtest/mapview.php?rg=OH&u=bejacob.

I've looked at the source code from the CHM web pages and only understand a fraction of it. :confused: I'm sure many many folks working on this have excellent web/programming skills and understand exactly what the code that generates a page like this http://cmap.m-plex.com/stat/region.php?u=bejacob&c=usa&rg=oh&du=mi&sort=ra does.

How easy would it be to create a page like this http://cmap.m-plex.com/stat/traveler.php for the 30+ users that have .list files in the test db?

With the stats/logs available, who is able to take this output and transform it into web pages?

I'm impressed with everything that's happened so far. I don't know how much of it was Jim's work and how much others have contributed, but I thank everyone involved. I've been working with Jim to make sure my maps/list are correct (nearly there). Now I want to see what sort of web interface there will be. I'm sure it will take some time.

I think I speak on behalf of other non-techie folks when I say, keep up the good work. :clap: We're eager to see what comes next.

Brian
These will be implemented down the road but the priority right now is to get the statistical calculations working and the mapping application functional. 

As I mentioned elsewhere, I have a code base started to build the "CHM style" mapping system but that is in the "alpha" stage right now and has many issue that need to be resolved.  It draws the lines and that's it.  The end result ideally will work - at minimum - like on Tim's site. I hope to get back on that next weekend (out of town this weekend).  The maps can be developed in parallel with the stats pages, but the issue is with the current resources, that may not happen yet, but it is on the list.
Scott O.

Not all who wander are lost...
Ah, the open skies, wind at my back, warm sun on my... wait, where the hell am I?!
As a matter of fact, I do own the road.
Raise your what?

Wisconsin - out-multiplexing your state since 1918.

bejacob

Thanks for the answer. I know there is a great deal of effort needed to make these things happen. It's nice to know where things stand.

Jim

(Hello briefly from Thunder Bay, ON.)  One more note: while the information in the log files is or can easily be extended to all of the stats we'd need and more, the logs themselves won't be used for that purpose.  Stats pages should query the DB as needed and generate pages on the fly.  It shouldn't be hard - relatively straightforward web stuff.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

SSOWorld

Quote from: Jim on July 17, 2015, 08:39:12 PM
(Hello briefly from Thunder Bay, ON.)  One more note: while the information in the log files is or can easily be extended to all of the stats we'd need and more, the logs themselves won't be used for that purpose.  Stats pages should query the DB as needed and generate pages on the fly.  It shouldn't be hard - relatively straightforward web stuff.
Agreed. 

and how's the great white north? :sombrero:
Scott O.

Not all who wander are lost...
Ah, the open skies, wind at my back, warm sun on my... wait, where the hell am I?!
As a matter of fact, I do own the road.
Raise your what?

Wisconsin - out-multiplexing your state since 1918.

Jim

Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

oscar

Quote from: Jim on July 17, 2015, 08:48:10 PM
Quote from: SSOWorld on July 17, 2015, 08:43:52 PM
and how's the great white north? :sombrero:

Excellent!  And green.

Hope the wildfire smoke isn't drifting in there from Manitoba. It was worst in northern Alberta and Saskatchewan, where the fires were when I was there last week, but much of the smoke was drifting southeast.
my Hot Springs and Highways pages, with links to my roads sites:
http://www.alaskaroads.com/home.html

Jim

In case anyone's interested in seeing the output produced by the Python site update program, the site update script now captures that program's output and places it with the other log files:

http://tm.teresco.org/logs/read_data.log

The numbers at the start of many lines are elapsed times in seconds from the invocation of the program.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

english si

I was wondering why a route in St Petersburg had so many co-locations, and then I remembered that Oscar copied that route as a place holder for California state routes.

Speaking of co-locations, two things related to them:
1) NMP and the ability to add FPs would be useful
2) In the Highway Browser, intersecting route links would be fantastic.

PS: Thunder Bay is in the north? I'm only a few miles short of nearly 3.5 degrees further north and I'm in the south of my country.  :-P

Jim

Quote from: english si on November 25, 2015, 04:54:05 PM
I was wondering why a route in St Petersburg had so many co-locations, and then I remembered that Oscar copied that route as a place holder for California state routes.

That threw me at first, too.

QuoteSpeaking of co-locations, two things related to them:
1) NMP and the ability to add FPs would be useful
2) In the Highway Browser, intersecting route links would be fantastic.

Both definitely in my plans.
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)

froggie

Jim:  how much of the scripting is being written in Python?  Asking because I'm taking a Python class this semester.

Jim

Quote from: froggie on November 25, 2015, 11:33:33 PM
Jim:  how much of the scripting is being written in Python?  Asking because I'm taking a Python class this semester.

All of the data processing to go from .csv and .wpt data into logs, stats, and a mysql DB instance with that information in a form useful for the web side is in Python.

https://github.com/TravelMapping/DataProcessing/blob/master/siteupdate/python-teresco/read_data.py
Photos I post are my own unless otherwise noted.
Signs: https://www.teresco.org/pics/signs/
Travel Mapping: https://travelmapping.net/user/?u=terescoj
Counties: http://www.mob-rule.com/user/terescoj
Twitter @JimTeresco (roads, travel, skiing, weather, sports)



Opinions expressed here on belong solely to the poster and do not represent or reflect the opinions or beliefs of AARoads, its creators and/or associates.