Custom POI Standards

 

I just looked at the custom POI for Malls of America. When I looked at the cvs spreadsheet I wanted to see where the malls were so I could plan some trips before I downloaded the file. I saw the latitude, longitude, mall name and street address but did not see the city and state. Therefore, without going to another source I have no idea where these are or if they are somewhat close to me.

Don't you think it would be a good idea to develop and communicate some standards that listed all the information necessary to be a complete custome POI? Standards that will tell the developer exactly what information to include in each field. Then, when the custom POI is loaded it will work properly and display addresses and phone numbers all in a uniform manner.

Perhaps, Miss POI has already done this and I am unaware but if not then this might be an opportunity to begin to pull together all custom POI's in a uniform format.

--
I tripped going up the escalator and I fell for an hour and a half!
1 2 3
<<Page 4>>

Bear this in mind too

Now that most POI's listed here can be downloaded in both GARMIN and TOM TOM format care must be taken to keep all relevant info within the first 3 columns.

This is necessary so that the Factory personnel responsible for doing the converting from csv to ov2 format (Miss POI?) - to afford the POI's the Garmin and TomTom download options - doesn't have to pull out any more hair then necessary! smile

Mike

--
Freedom isn't free...thank you veterans! Heard about the tests to detect PANCREATIC CANCER? There aren't any! In Memoriam: #77 NYPD-SCA/Seattle Mike/Joe S./Vinny D./RTC...and God bless Donald Trump!

Subject field is required.

jcavaa wrote:

..I am suggesting using csv files, which are text files, that have a header record that describes the fields of the next records in the file. This makes it into a free format, but ....

Well, that is certainly worth consideration.

I wonder about saying that a standard should be built around CSV, since CSv is currently so nonstandard. While a top line of headers may help, that itself seems to break the CSV design, it would end up getting compiled into the GSPr as an extra POI at a bogus location. Plus there is the issue that for some people a CSV file with lots of data in field 4 contains <br> breaks while other models of Garmin receivers use actual hard CR/LF line breaks (which are much harder to extract subsets from or otherwise process). For these reasons I would shy away from using csv files as a default standard way to store data and favor spreadsheets, a database, or ever a text file that defines fields and their contents. But if a majority of the users here favored doing this type of CVS file over other forms of data storage, I would certainly get behind it, It sure would be better than what we have now.

I'm all for keeping all the data that one can get. The term "required" is hard to use when you are asking people to volunteer to contribute POIs, but consistency and guidelines are important. I sure would ask that if someone compiling a POI has the state information that they keep it, and in the two character upper case postal code format for states in the USA. I would also ask that people who contribute data clean it up a bit; when I processed a bunch of addresses from my state government I saw that people in the government had thrown lots of extra stuff in what was normally a city field, things that I moved to a comment field. So that where I have a city field, it is always a city, not a city and some stray information.

One thing that I did when I compiled information was to keep everything that was there, which included zip codes and the name of the county the office was in. However, that didn't really seem like important information to clutter the CSV file with. So I set up the script that generated a CSV file from the text file to omit the county and zip. But still have that data if anyone should need or want it included in a CSV or GPX file. I expect that I would do the same if I had a list of retail stores with store numbers. If I'm going to a Wal-Mart I don't really need the Wal-Mart store number cluttering my small screen. but I can see that truck drivers or others who work with Wal-Mart may indeed want the corporate store numbers included in their POIs. Since a POI CSV file generated form another form of data with a simple script can format it however desired, it can easily creates POI file with and without store numbers, and there is still only one master set of the data to maintain. This gets a little harder to do if the data is already in csv files, although it is not complete imposable to do if the csv file is done to well defined and consistent standards.

Using CSV file

My intention was not to use the described file format to be used to load into the GPSr. There would always have to be some extraction from the file at the factory to an output csv file for the the specific user. The header record would never get written to the output file. The user should also be able to specify which fields he/she wants to be written to the output records. That way the factory file could have fields for county, store number, etc. But those fields would only be written to the output records if the user requested them. So a output record description could look like:

lat,lon,name,street,city,state,zip,county

or if your device only can handle 4 columns:

lat,lon,name,street+city+state+zip

Here the street,city,state and zip would be compiled into one column4.

With this kind of output record description it would also be easy to the lon field first and then the lat field.

JCA

JCA

Sounds familiar ...

jcavaa wrote:

..I am suggesting using csv files, which are text files, that have a header record that describes the fields of the next records in the file.

EDIT - my post clashed with your latest one.
Please have a re-read of Page 1 of this thread smile

Frovingslosh wrote:

I wonder about saying that a standard should be built around CSV, since CSv is currently so nonstandard. While a top line of headers may help, that itself seems to break the CSV design, it would end up getting compiled into the GSPr as an extra POI at a bogus location.

At best, CSV is certainly a loose standard!

The headers aren't a problem for the majority of users here (ie Garmin) - in that POILoader just ignores them. It does the same with columns beyond 4. There is a more difficulty with the contents of column 4 being interpreted inappropriately (though there are existing tools for dealing with this).

I believe there's a consensus (slowly!) forming, that the data needs to be separated out, one way or another before being uploaded to POI-Factory. The question is, would this be optional or mandatory? I was trying to find a means whereby it could be optional.

The short term issue is that any given user is probably preparing a POI file primarily for their own use; sharing it via POI-Factory being a secondary consideration.

If multi-platform POI creation software existed, it could upload the user's efforts automatically to POI-Factory - and then in return, POI-Factory scripts could send them back a version formatted for their device. I considered providing a free version of GeePeeEx Editor to do this, but it's probably not a practical proposition. (It will never be multi-platform, and there are issues around its use of Google and Yahoo! services, in this context.)

So, as I see it the choices are to either wait for a complete suite of front and back-end software to be available, or to find a means of gradually moving to such a state. Of course, the 2nd option is the more appealing of the two!

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

CSV is certainly a loose standard!

Hornbyp wrote:

At best, CSV is certainly a loose standard!

Hi Phil,

Yes, I think it is too loose to be used as a "standard".

Hornbyp wrote:

The headers aren't a problem for the majority of users here (ie Garmin) - in that POILoader just ignores them.

Oh, that's very interisting. I would not have expected it. How does POI Loader know when line 1 is a header and when it is a line of data? That is, what flags it as a header.

Hornbyp wrote:

There is a more difficulty with the contents of column 4 being interpreted inappropriately (though there are existing tools for dealing with this).

Yea, two problems I see are that field 4 is just a catch-all for any extra data and it can be hard to pick out what you want or need, and also that we just have no standard way to ask all users to express their data. An example I've been using is the many ways that state names are expressed, sometimes in the same POI file.

Hornbyp wrote:

I believe there's a consensus (slowly!) forming, that the data needs to be separated out, one way or another before being uploaded to POI-Factory. The question is, would this be optional or mandatory? I was trying to find a means whereby it could be optional.

I'm hoping that consensus is forming. I certainly saw the need as soon as I first saw this thread (I was slowly realizing the issues even before that). I don't think you can make such a volunteer effort completely mandatory, but without guidelines everyone just does their own thing and building more consistent data files just will not happen.

Hornbyp wrote:

The short term issue is that any given user is probably preparing a POI file primarily for their own use; sharing it via POI-Factory being a secondary consideration.

I don't really think this is so. In cases where a POI is for a city or small locality this is likely the case, but those who build us State or even national POI files are likely motivated to do it well. But even someone building a POI file primarily for their own use might wish for standard guidelines and might benefit from consistent standard techniques used to create POI files.

Hornbyp wrote:

If multi-platform POI creation software existed, it could upload the user's efforts automatically to POI-Factory - and then in return, POI-Factory scripts could send them back a version formatted for their device. I considered providing a free version of GeePeeEx Editor to do this, but it's probably not a practical proposition. (It will never be multi-platform, and there are issues around its use of Google and Yahoo! services, in this context.)

I've only heard good things about your software, but I would hope that standards could be created around simple text files and/or spread sheets built with Excel or the free Open Office. And this is no reason why only one standard could be accepted. If the data is being imported into a database for final POI file generation in any desired format, then the input could be any number of optional formats, including a text file, a spread sheet, an XML/GPX file, or some other well defined formats.

Hornbyp wrote:

So, as I see it the choices are to either wait for a complete suite of front and back-end software to be available, or to find a means of gradually moving to such a state. Of course, the 2nd option is the more appealing of the two!

Well, while waiting to get there I created a very simple AWk script (less than 1 screen full) that could process a text file of labeled fields and output a .CSV file. The script is available to anyone who want to try this approach. I expect that the labels I defined are far from all inclusive and that community feedback would be needed to get a good comprehensive set of labels. Of course, just about everything except the coordinates is optional, if there is no street address or zip code these are just omitted. Even city and state can be omitted, but I would encourage contributors not to leave these out unless these fields didn't apply to the data. My expectation is that eventually I'll be able to import these text files into a database, but for now I can use the technique to produce CSV files directly, and by simple script modifications I can produce alternate format POI files for other GPS models.

Sounds Familiar ...

Hi Phil,

Somehow I must have skipped your post when I was reading through this thread. Or I read it, forgot it and then it submerged from my brain as my idea. At least there is proof you posted it first.

Mostly I'm glad that I found somebody thinking along the same lines as I am.

Since I'm using GeePeeEx, so if I need to open a downloaded CSV file, more often then not I have to do some editing of a header record to define which column data goes where in the GPX file. So the idea of using a header record seems a nice way to handle all the requirement of the different devices and still for the creator be able to supply all the information he/she can provide.

In a few days I will upload a POI file I have created. First I was thinking of only uploading my GPX file, and not create and upload a CSV file, because however I would construct it, somebody would be unhappy with it. But now I think I will create it like in your last example in your post on page 1. And then I will see what kind of feedback I would get, if any.

JCA

.

I wrote:

The headers aren't a problem for the majority of users here (ie Garmin) - in that POILoader just ignores them.

Frovingslosh wrote:

How does POI Loader know when line 1 is a header and when it is a line of data? That is, what flags it as a header.

I don't know. Presumably, just by the fact that it failed to find any 'sensible' data.

Frovingslosh wrote:

Well, while waiting to get there I created a very simple AWk script ... that could process a text file of labeled fields and output a .CSV file.

While agreeing that a file of 'labelled fields' is technically functional, it's a format that's not currently supported by any existing POI Creation software (either Editor or Online source). What would people create such files with? (Surely not free-hand, in notepad?...)

On a different note...
jcavaa wrote:

First I was thinking of only uploading my GPX file...
But now I think I will create it like in your last example in your post on page 1.

Don't forget though, GeePeeEx Editor doesn't currently produce csv files in quite that format. Specifically, it doesn't attempt to produce a 'column 4', that POILoader might be able to turn into something that an arbitary Garmin unit might understand. (After all, it's supposed to be a GPX editor!)

---------------------------------------------------------------------------------

The essence of the proposal, was that if the file you uploaded extended the number of columns beyond the essential first three - and even though the extra columns would have to be annotated with headers - there would always be a 'free-format' column 4.

This column could be just null (,"",) - so at least it wouldn't confuse POILoader, or, it could contain some representation of the rest of the data (for example, concatenated with 0x0Dh,0x0Ah newlines). It could just be a simplified form of the rest of the data - a kind of 'thumbnail' view.

So, if you were a Garmin user who downloaded such a file - even if you did no further processing - POILoader should be able to load it without errors. Your unit may, or may not, be able to render the column 4 data into something readble.

If you were more adventurous, (or shock-horror, didn't have a Garmin!), you could post-process the other columns in the csv file (if present), into a format that your unit could understand. (Obviously, in this case, you would just discard column 4).

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

:)

great job

--
[URL=http://www.speedtest.net][IMG]http://www.speedtest.net/result/693683800.png[/IMG][/URL]

What would people create such files with?

Hornbyp wrote:

While agreeing that a file of 'labeled fields' is technically functional, it's a format that's not currently supported by any existing POI Creation software (either Editor or Online source). What would people create such files with? (Surely not free-hand, in notepad?...)

While text files with labeled fields, or spreadsheets or databases like SQL are not directly supported in any GPS, they are good ways to organize data in well defined ways (a situation that seems to be very lacking in our current CSV based POI files). I would expect that if databases were used then the various CSV and GPX forms of the data (and other formats for other brands of GPSr) would be simple reports done against the database. I'm not sure just how I would do it for a spreadsheet, but I expect we could get some spreadsheets to crank out reports as well, or they could easily be imported into a database and then reports done from there. As to the text file approach, I've already made a couple of text files with labeled fields to contain POI data. I created these files with Editpad lite, which worked pretty well, but any text editor (even humble notepad) should work. I wish I still had a copy of the Brief editor, as it has some macro capability that would make this even easier, but it wasn't all that hard to do with Editpad lite and if you want I'll do the next one in notepad just to see if there are any issues. As to how to make the final csv file, I used a simple script that I wrote in AWK. By simple I mean that it contains about 15 active lines, and a few lines of comments and spaces to help make the text readable. Fits on one screen. A good and absolutely free awk interpreter is available as GAWK from
gnuwin32.sourceforge.net/packages/gawk.htm
and a Google search will find plenty of help and other sources. When I produced my second POI set from a labeled text data file, I only had to change one line in my AWK script, the line that defines what data fields are included in field 4 of the CSV file. I produced a file with <br> type line breaks, but the script can easily be changed to produce actual CR/LF line breaks, and I see no reason it can't crank out GPX files from the same input data file.

If anyone wants my AWK script I'll gladly share it.

Another nice side effect of doing it this way is that I'm free to define my data fields. So while I originally defined two fields for Latitude and Longitude, I soon realized it would be less work for me if I defined one field that accepted Lat&Long together, and in the order that I could copy from websites like Itouchmap and paste directly into the text file. The fact that these fields are in reverse order from Gamnin type coordinates presents no problem, part of the AWK script finds the comma in the coordinates string and splits the two fields out as seperate data items before and after the comma, and a later part of the script writes the fields out in Garmin's desired order, The code to add this to the script once I realized that I wanted to do it took only one extra line.

I'm hoping that we can agree on a database or a spread sheet to import data to and if so I'll modify the script to process my files into a one line per POI format suitable for importing into those forms, but otherwise I'll just keep creating text files and processing them into whatever form that I want with AWK scripts.

In any event, if a database like SQKL is used for the common storage of POI information on POI Factory, then I would not expect individual contributors to submit SQL databases. It would be much simpler to accept data in the submitter's choice of a variety of forms including spreadsheets or text files, and they could all be easily imported into the database. The important concept is that the data be cleanly labeled and delineated, something that is not happening in CVS files now and that I suspect will not happen even if we try to get people to label their CSV files with a top line.

POI files and POI Factory

From the recent posts, it seems to me that you are missing a key property of POI Factory that has enabled the production and sharing of so many files. One way to fix this would be to list files as one of two types i.e. standardized and non standardized. Then you can create a recommended standard and use a database behind it to store and reconfigure data. This would work for all files following the standard, and anyone who has already posted a file could be contacted to provide the standard headers to convert their file into a standardized POI file.
Then, if the user community favors the standard through uploads and download usage, the majority of files would go in that direction and their would be user benefits to creating those files. However, individuals would still have the option of uploading and downloading non standardized files if they don't want to go to the trouble.
The category and label of standardized and non standardized files would alert the user to whether they were interested in downloading the file.

--
Garmin StreetPilot c530, Mapsource

You're getting close

mkahn wrote:

From the recent posts, it seems to me that you are missing a key property of POI Factory that has enabled the production and sharing of so many files. One way to fix this would be to list files as one of two types i.e. standardized and non standardized. Then you can create a recommended standard and use a database behind it to store and reconfigure data. This would work for all files following the standard, and anyone who has already posted a file could be contacted to provide the standard headers to convert their file into a standardized POI file.
Then, if the user community favors the standard through uploads and download usage, the majority of files would go in that direction and their would be user benefits to creating those files. However, individuals would still have the option of uploading and downloading non standardized files if they don't want to go to the trouble.
The category and label of standardized and non standardized files would alert the user to whether they were interested in downloading the file.

We keep throwing around the word 'standard' failing to understand that it has a very specific definition. In the context we are using 'standard' we are talking about a guide rather than a document with hard, fast and explicit rules about what can and cannot be included in a product which is compliant with the standard.

Perhaps the best way to look at a standard would be to think of a set of regulations. With its shalls, musts, mays, and shoulds is much like a set of laws and is not subject to interpretation. A guide on the other hand is just that, a guide on what it describes should contain, but doesn't lay down hard and fast rules.

As an example, a 'Standard' would state the longitude field shall have from 1 to 3 significant digits followed by a decimal point and 6 positions of precision. The number indicating longitude shall be negative for values between 1 and 179 and positive for values between 180 and 359. If this is the standard, then any longitude put into a poi file that didn't have 6 decimal places would not be compliant with the standard.

A guide would state longitude is expressed as a decimal number with 4 to 6 digits of precision. East longitude is indicated as a negative number while West longitude is positive.

--
ɐ‾nsǝɹ Just one click away from the end of the Internet

POILoader

Hornbyp wrote:

Given that Garmin's POILoader ignores column headers and columns beyond col. 4, the bulk of users would have access to a file that could be loaded without modification.

JCA writes:

I tried POILoader 2.5.2 and if I submit a file with a header record, it gives an error message. Error message reads:
"There is a data format error on line 1 of file filename.csv. The data earlier in the file will be written, but no more data can be read from this file."

I tried with the field names quoted and non-quoted with the same result.

So if I was to submit a csv file with a header record, that record would have to stripped out before the file could be used with POILoader.

As I see it the header record is info to be used by some software to create a new file (csv, gpx, ov2 or something else) with some or all of the fields in the following records.

Another strange thing I noticed when testing different things with POILoader was that it skips any blank fields after column 3. So if column 4 is blank (no data or two double-quotes) the column 5 is used, unless column 5 is also blank and so on. I only tried to have column 4 and 5 being blank and have data in column 6 and that data was displayed on my 360.

I will probably for now only submit GPX files and wait for any standard or guide for how to format any CSV file or maybe a completely new file format used to import data into a POI Factory database.

JCA

??

jcavaa wrote:

Hornbyp wrote:

Given that Garmin's POILoader ignores column headers and columns beyond col. 4, the bulk of users would have access to a file that could be loaded without modification.

It's true, I did smile

but then he wrote:

tried POILoader 2.5.2 and if I submit a file with a header record, it gives an error message. Error message reads:
"There is a data format error on line 1 of file filename.csv. The data earlier in the file will be written, but no more data can be read from this file."

I repeated my experiments, using POILoader Versions V2.4.2, V2.5.2 and V2.5.3. They all acted as I previously noted - no error messages, just data that was ignored ...

and he wrote:

Another strange thing I noticed when testing different things with POILoader was that it skips any blank fields after column 3. So if column 4 is blank (no data or two double-quotes) the column 5 is used, unless column 5 is also blank and so on. I only tried to have column 4 and 5 being blank and have data in column 6 and that data was displayed on my 360.

I hadn't noticed that before, but I can confirm this odd behaviour!

Here's my test data:

I picked a gpx file at random from POI-Factory

Montreal, QC - Museums Of Montreal

arrow http://www.poi-factory.com/node/18333

I then used GeePeeEx Editor to produce a multi-column CSV version of it:

arrow http://GeePeeEx.com/MontrealMuseums.csv

Finally, I added a new 'Column 4' to every POI, containing the phrase "Dummy Data". I used Notepad to do this:
arrow http://GeePeeEx.com/MontrealMuseumsMODIFIED.csv

I tried to get Excel to generate a more meaningful 'Column-4', using the 'concatenate' function. It did that part successfully, but the resulting CSV wasn't written correctly - it stopped using "double-quotes" around fields that needed them. (That did cause POILoader some grief!).

I failed miserably in my attempts to repair that file with CSVEd as well sad

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

POILoader problem with header record

Phil,

I tested your MontrealMuseums.csv file and I got the same error message as with my files!!??

I am running POILoader ver 2.5.2 and I am using Express mode. Must say I am a bit more confused than usual now.

In my header records I only had the fields that I was using. So I had the Lon, Lat, Name, Descr, Addr1, City, State, Zip, Phone field names (yes, the long ones generated by GeePeeEx). But that obviously is not what is causing my problem.

JCA

Independent tester required ;-)

jcavaa wrote:

I tested your MontrealMuseums.csv file and I got the same error message as with my files!!??

I just re-tested and all is still OK ?!? - we need someone else to try this!

(I created a temporary directory and downloaded the csv files from the website into it. I then specified that directory as the location of the files to POILoader.
I tried it twice - once writing to a 'Custom Folder' and again writing to a Garmin Device (an SD card pretending to be a Nüvi 760, by virtue of its GarminDevice.xml).

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

POIloader problem

Phil,

I tested creating a new folder with only the MontrealMuseums.csv file and then it works. So the problem occurs when I have other files in the same folder.

Next I tested having just one more file together with the museum file. Then it turns out that 3 of the GPX files (size 24 KB, 59 KB and 146 KB) don't want to coexist. One GPX (23 KB) and another CSV works fine.

Next I test with my test CSV file which is of size 8KB, museum file is 6 KB. Now all except the biggest GPX works fine. I fail to see any logic in this. I'm sure that somehow there must some logic to what is happening.

Just to be sure I should mention that testing with all the files together with my CSV file, if I remove the header record then I don't get the error message. The same goes for the museum file.

At least we know that there is nothing wrong with the museum file.

Somehow I seem to trigger something when I combine a CSV file which has a header record, with a GPX file of a certain minimum size.

I will try to think of any other things I can test.

JCA

.

jcavaa wrote:

I tested creating a new folder with only the MontrealMuseums.csv file and then it works. So the problem occurs when I have other files in the same folder.

I decided to 'throw' a load of other files in the same directory as the MontrealMuseums, but I still can't get it to fail...

(50 odd assorted gpx/csv files = 14000+ POIs. Input files about 9MB, resulting POI.GPI=3MB)

Are you sure one of the other files isn't just 'dodgy'?

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

A newb thought...

Forgive me for 2 things, please- (1)I haven't read ALL of the posts in this thread completely, and (2) I'm a newb to waypoint creation and most things GPS but desperately trying to get up to speed so I can create my own as well as help with this GREAT site.

As far as "Standards" go, I'm seeing that the subject covers a lot of ground and also means slightly different things to different people, so looking at the OP's statement and from my own experience "playing" with some of the files on the site-

Would a good first start maybe be to simply include some type of notation or "flag" (or "grading") system for all of the existing & new files to give an indication of what to expect in them? Not overly complex or hard to tag, but for the "key ingredients" of POI's that set them apart from each other and give a quick indication of the end user that they are most appropriate for.

For example, I see that one big one has been inclusion of phone numbers so maybe a tick box or icon to indicate when a file has none/all/x% of phone numbers in it's records.

Similar thing could be done for addresses. I see that Google's geocoding API has a classification for address precision. Could that "scale" be used (with a page for a key or something) or even automated to tag files as to completeness of location? In fact, I see that there is a precision level for Country and I think there is a means to even return an iso identifier, which would help here in sorting listings.

This wouldn't solve any of the more technical aspects of gpx vs csv, or minimum info inclusion, but it would certainly give the site user some guidance before downloading and opening a file.

Just an observation as a newb POI user as well as (hopefully) a future creator/contributor.

--
It's about the Line- If a line can be drawn between the powers granted and the rights retained, it would seem to be the same thing, whether the latter be secured by declaring that they shall not be abridged, or that the former shall not be extended.

POILoader problem

Phil,

I have now been able to isolate the problem and hopefully you will be able to recreate it.

I have the MontrealMuseums.csv and the gpx file DinersDriveinsDives_RIH_C.gpx from the POI Factory in a subfolder called Food. These are the only files within the topfolder I used.

This will produce the error message on my system.

But if I put the two files in the top folder instead then it works!!

I'm sorry if I'm taking up to much of your time with what seems to be a minor bug in POILoader. I just wanted to find the logic in the error.

JCA

:-(

jcavaa wrote:

I have now been able to isolate the problem and hopefully you will be able to recreate it.

I have the MontrealMuseums.csv and the gpx file DinersDriveinsDives_RIH_C.gpx from the POI Factory in a subfolder called Food.

This will produce the error message on my system.

Yup. I get the same sad

and he wrote:

But if I put the two files in the top folder instead then it works!!

I also found, that if alter the order in which the the files appear in the FOOD directory, I can stop the error. (I renamed DinersDriveinsDives_RIH_C.gpx to ZZDinersDriveinsDives_RIH_C.gpx)

So - coupled with the oddball handling of a 'null' column-4, and (my) inability to produce properly delimited multi-column csv files in Excel - where does this leave the proposal? sad

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

gpx is better

Hornbyp wrote:

So - coupled with the oddball handling of a 'null' column-4, and (my) inability to produce properly delimited multi-column csv files in Excel - where does this leave the proposal?

I'm not clear which proposal you are speaking of here.

Your recent posts have done a lot to explain why gpx files are a much better way to organize poi data, and I'm going to put more effort into my scripts so that I can crank out gpx files as well as csv files. But the average contributor, without being told to buy a tool like GeePeeX isn't likely to be producing gpx files (as evident by the vast majority of POI files being in csv and not gpx format). So it seems to me that your points about gpx data displaying more consistently than csv data makes another argument as to why data should be contributed and stored in a more field consistent form than simple csv files. That way a program can extract known fields cleanly and produce good gpx files for those who appreciate the benefits of them.

baby steps

OK, let me ask a much simpler question, and maybe in doing so show how lacking any standards are and perhaps why we should be giving guidance in the way of standards to contributors.

I've been including telephone numbers in the POIs that I make. Have not done gpx files yet (intend to though), so bluetooth dialing is not an issue, yet. But I find myself wondering if, when I do produce gpx files, bluetooth users will be able to dial with them from the current format I have the data in.

I'm currently using the phone numbers is a format supplied by websites. This is either completely or mainly in "(nnn) nnn-nnnn" format, although I have seen some numbers in nnn-nnn-nnnn format. Without some standard/guideline I can't even know if "(nnn) nnn-nnnn" format is acceptable for bluetooth dialing or if the parenthesis will confuse things. I'm quite willing to format the data properly now as I build files, but I suspect I'll be much less motivated to go back and fix things later if I do the best I can now but can't get a guideline (standard) on something as simple as a phone number.

So my specific question is, what is the required format for telephone numbers to work with Bluetooth. And as bonus questions, does the phone just deal with a (nnn) nnn-nnnn number when it is in the same area code as the number? And what about that damn 1- so often found as a prefix to long distance numbers?

My general question is, why do I even have to ask this? Shouldn't we have some standard information to provide to contributors to provide guidelines on how telephone numbers should be formatted, how State names should be represented, even how the store name should be consistently represented in a POI file?

.

Frovingslosh wrote:

I'm not clear which proposal you are speaking of here.

I was referring to my proposal, of 28th November 2008 (Page 1 of this thread).

To recap my list of requirements:

  1. Ease of creation, using existing software.
  2. Support for multiple target GPSr systems (ie not just Garmin)
  3. Customisable to the user's requirements, using existing software - on multiple platforms; ie not just Windows).
Quote:

So it seems to me that your points about gpx data displaying more consistently than csv data makes another argument as to why data should be contributed and stored in a more field consistent form than simple csv files.

I've been speaking specifically about Garmin POIs though. In this case, the difference comes down to what is possible with Garmin's software, given the two different file types.

The point about using csv files, in the context of a broader standard for this site, was that:-

  1. A wider range of software exists for their creation - although as I delve into Excel's capabilities, I'm becoming somewhat under-whelmed.
  2. They are actually able to convey the same information as an XML-based file.
  3. Most importantly, I believed they offered maximum compatibility, with the way that the majority of the existing users of this site were currently working.
Quote:

OK, let me ask a much simpler question...
...what is the required format for telephone numbers to work with Bluetooth. And as bonus questions, does the phone just deal with a (nnn) nnn-nnnn number when it is in the same area code as the number? And what about that damn 1- so often found as a prefix to long distance numbers?

I don't know the answer - though it's been frequently asked. I *think* Garmin units just pass the numbers through 'as-is' ... and it depends what 'phone understands. The specifics of your area-code question, probably depends on what the 'phone's current network understands.

Quote:

My general question is, why do I even have to ask this?

They are important details - but we seem to be missing the basic framework at the moment.

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

Cell phones in US

Frovingslosh wrote:

So my specific question is, what is the required format for telephone numbers to work with Bluetooth. And as bonus questions, does the phone just deal with a (nnn) nnn-nnnn number when it is in the same area code as the number? And what about that damn 1- so often found as a prefix to long distance numbers?

Cell phones in the US like to dial 10 digits. You don't really have long distance like a land line (i don't want to open that can of worms here please). But what you will find is that a local phone can dial 7 digits and work fine. If that phone moves to a different area code sometimes it does not dial the correct area code when trying to dial its "local" numbers. But out of area phones do need all 10 digits to work.

Basically the safest is to include 10 digits, and they can be (xxx)xxx-xxxx or xxx-xxx-xxxx or xxxxxxxxxx and the phone doesn't care. Do not include the 1 and it should work for everyone.

Daniel

--
Garmin StreetPilot c580 & Nuvi 760 - Member 32160 - Traveling in Kansas

Phone format

My nuvi's phone numbers are precise to region (US 10 digit) and without parens, but with dashes, ie. xxx-xxx-xxxx.

I don't see where the gpx or Garmin gpx extensions (v1, 2, or 3) specify a particular format either.

Seems reasonable to keep some things like phone & postal code relative to the target country of the file.

--
It's about the Line- If a line can be drawn between the powers granted and the rights retained, it would seem to be the same thing, whether the latter be secured by declaring that they shall not be abridged, or that the former shall not be extended.

Phone format

They just added another area code in our area and now we have to use the ten digit number even for local calls.

--
johnm405 660 & MSS&T

Too many people!

johnm405 wrote:

They just added another area code in our area and now we have to use the ten digit number even for local calls.

We've had to do that in our area for about 5 or so years now. sad

In retrospect, I now realize that was a signal to relocate smile

--
It's about the Line- If a line can be drawn between the powers granted and the rights retained, it would seem to be the same thing, whether the latter be secured by declaring that they shall not be abridged, or that the former shall not be extended.

GPX relevant point

As I begin playing with my own Garmin and Magellan units and creating files for them, I noticed a subtle but important thing about GPX files, which I think is relevant to the site's decision on POI standards.

I posted my note regarding "v3" of Garmin's gpx extensions under the thread http://www.poi-factory.com/node/1669

Basically it's a reminder that theoretically a GPX unit and/or software should make use of the "base" GPX v1.1 tags for sure, but doesn't technically have to make use of any extensions from another mfr.

Like I said in the other post, I don't have enough experience with all of the mfrs that can use GPX, so I may be crying "wolf" here a little. I'm posting this because of my "past life" experiences in working with different but similar SGML and XML documents.

--
It's about the Line- If a line can be drawn between the powers granted and the rights retained, it would seem to be the same thing, whether the latter be secured by declaring that they shall not be abridged, or that the former shall not be extended.

Time to think out of the box.

Frovingslosh wrote:

But the average contributor, without being told to buy a tool like GeePeeX isn't likely to be producing gpx files (as evident by the vast majority of POI files being in csv and not gpx format). So it seems to me that your points about gpx data displaying more consistently than csv data makes another argument as to why data should be contributed and stored in a more field consistent form than simple csv files. That way a program can extract known fields cleanly and produce good gpx files for those who appreciate the benefits of them.

Your right, we always seem to be confronted with that reality. I think in the end we will end up storing this in a database, and extracting the data we want int the format we want. That is where this will end up in the future anyhow.

Maybe we need to think out of the box a bit here. We don't all need to have the same software on our PC, we just need set up the software in one central site and access it over the web much like we do here when we type messages to each other.

The database would be stored on the web site, and we would use a web browser to enter wayponts into the database. Organizing those waypoints is the key to making this work, but in a simplified way they could be set up like to do it now, with a group of waypoints for specific place to eat, like all McDonalds in Wisconsin, or best fish fries in Milwaukee.

With some collaboration for different users the McDonalds could be expanded to include all states. Then the POIs could be downloaded to a POI file of your choice, and include the states your interested in.

We just need someone to build the database, build the web site and then manage it. Done right, I think it will grow faster than your expectations.
I wish I could do it myself, but I don't have the expertise and the time to get it done.

--
Nuvi 265WT & Edge 705

Subject field is required. If only POIs imposed some standards.

plemirande wrote:

Maybe we need to think out of the box a bit here. We don't all need to have the same software on our PC, we just need set up the software in one central site....

Absolutely correct. I'm not at all advocating that everyone have a database on their system, The POi factory should keep the database and generate the POI files (which are reports in the database sense) as needed.

To get there I think we ned two things: define the data fields and define suitable ways to submit it to the database.

As far as defining the data, this would pretty much be based on what fields people want to see in POI files (as well as the required coordinates). And it should define the form the data takes. For example, I would propose that we need a State field in most POI files and that when present it should contain the 2 letter capitalized postal abbreviation of the state. I've seen too many different forms of state name in the POI files for standardized database work (some quite creative, and oftem omitted completely). Without establishing a simple standard you can't expect everyone to format data in the same way. I found myself wondering how to format telephone numbers in a POI that I was creating. Do I put parenthesis around the area code, or does that cause dialing problems? There is just no good place that I can look for simple standards that would answer questions like this.

As for submitting data, I would propose that it could be submitted in a number of ways. I've already built a text file format that labels every field, and written a simple script to convert it into a .csv file (the script is about 20 lines and fits on one screen). The script is easily modified to prduce other forms of csv or gpx files, to alter what data is in the file, or to produce a delimited file suitable for import into a spreadsheet or database. I would also expect that the POI factory would accept spreadsheets (Excel or the free OpenOffice) with clearly defined fields. Spreadsheets where actually my first impulse of how to approach this data organization issue, but I realized that I could generate csv files from the formatted text file approach directly by a script, without waiting until the spreadsheet or database work were done. It would only take a few minutes to write a script to comvert my current text form files and import them into aspreadsheets.

I think the thing we really need to do first is define what fields are needed for POI files and what fields are desirable to have, and give good guidelines on how to format any field. The more fields that we have well defined, even when completely optional (such as store hours), the cleaner the data will be. The fewer defined fields we have, the more data will just get thrown into a "comment" field, or discarded completely. You can always build a "catch all field" by combining other well defined fields, but you can't cleanly take apart a "catch all field" into logical fields. (I've looked at POIs where there was data in field 4 that in some cases I couldn't even make sense of.)

So much talk about naught...

It is impossible to satisfy everyone...
Columns 1 & 2 have to be as they are...
Column 3 Should ID the location in some manner and it will be impossible to satisfy everyone... Normally if I am looking for a "Big Mac" I want the closest McD's I don't care for more of an ID than McDonalds if it says McDonalds Hollywood great...
Column 4 I put the complete address and phone number, either with or without line breaks, but this is just the way I do it..

--
It is terrible to speak well and be wrong. -Sophocles snɥɔnıɥdoɐ aka ʎɹɐƃ

I'm thinking we automate the whole process

Frovingslosh wrote:

As for submitting data, I would propose that it could be submitted in a number of ways.

I agree we need to keep the batch file input process so people can still import large data files which they have messaged into a meaning POI file.

For the most part, I'm thinking of automating the complete process. Back in the late 1980s I ran a small rummage sale on a local bulletin board message service similar to this forum. People would send me a list of items they wanted to sell. I would compose one list made up of all the items people had submitted to be sold. I would post a message containing that list, stating that the time that bidding on these items would end.

Bids were placed on these items by typing in a message with the item number followed by the price as the standard bidding process. If the message was time stamped before the bidding end time, then the bid counted. Any messages after that time did not count. We had a lot fun trying to get that last bid in before bidding ended which was higher than the last bid.

I did all the work manually. It was very popular, and grew quickly. Soon, I could not keep up with it. At the time, we knew it was a great idea, just didn't know how great it was.

Move up to 1995 when a 28-year-old software developer Pierre Omidyar wrote the code to automate the whole online rummage sale process into what we know today as ebay. It always was a brilliant idea that people found very useful, the trick was to design a process which could handle the volume.

Somewhere on ebays site must be a large database to house all those items and all the bids for those items. Some show up to bid on items, while some show up to post more items for biding.

We can do the same here. Enter the waypoints on line, using the web site as our editor. The software will then standardize the data in the correct field and in the correct format as it save it to the database.

Some will show up to enter their waypoints, while some will show up to download waypoints. Then all the standard data formatting is handled internally by the web site. The batch process can be as simple as listing standard header fields, then you can use those heard fields to define the data your uploading to the site by placing a header line as the first line of your data.

--
Nuvi 265WT & Edge 705

It is impossible to satisfy everyone

aophiuchus wrote:

It is impossible to satisfy everyone...

But that's not a reason to not try to do things well. Look at the original post that started this thread, the poster was looking at a POI file and couldn't even determine the State that the POIs were in. I agree, some people will always be sloppy and lazy. But that is no reason to not have standards and guidelines, and more people will drift towards sloppy and laze without standards and guidelines. And I'm not trying to fault tho contributor of the original file that started this discussion, without any guidelines he or she contributed something that seemed good to them. But it would be nice to suggest some standards, not rigid rules but guidelines that people can work towards.

aophiuchus wrote:

Columns 1 & 2 have to be as they are...

Actually, I don't even know what that means. When I set out to create POI files I wanted to know how many decimal places of accuracy to include, but couldn't find any standard. Many POIs use 5 places after the decimal, but my geocoding source supplied 6 and I used that. If I were writing a spec I would clearly spell out how many places were expected, what range of places was good, and to not bother exceeding some range (likely 6 places but that's just my opinion). I would also spell out clearly the order of the fields and the decimal form, since gps coordinates come in many formats and usually present the coordinates in the opposite order to the Garmin approach.

aophiuchus wrote:

Column 3 Should ID the location in some manner and it will be impossible to satisfy everyone...

So because we can't satisfy everyone we shouldn't try to give guidelines? But what goes here isn't even the first question, which still is "how to standardize the data so that it could be cleanly organized in a database". That would include a standard name for the store (I've seen POI files that have had a half dozen or more forms of the same store name in them), as well as related information (locality, city, store number and so on). From there field 3 of a csv fie can be built up however one wants, just the store name for someone who just wants to see the name, the city or locality for someone who wants more (very handy if trying to ID a bad POI and they are all just marked with the store name) and while not useful for most users, the store number for delivery people or others traveling to many chain locations. Once the data is well organized this can be done to suit everyone, but if you start with the "you can't please everyone" approach then you're already blocked from doing it well.

aophiuchus wrote:

Column 4 I put the complete address and phone number, either with or without line breaks, but this is just the way I do it..

Column 4 is a hodgepodge of whatever anyone throws into it, and I can show you csv poi files that I don't think anyone can give consistent rules for determining what part is what. What I'm suggesting is that we define some things cleanly (eave as simple as what form to express the state in). Once there are cleanly defined fields for things line a city, state, phone number, hours of opperation, and so on, them those wanting to use csv format can build field 4 from whatever data they wish. But to suggest that field 4 as we currently have it is in any way a standard and a way to well organize data seems to exhibit a lack of understanding of the problem.

Enter the waypoints on line

plemirande wrote:

We can do the same here. Enter the waypoints on line, using the web site as our editor. The software will then standardize the data in the correct field and in the correct format as it save it to the database.

I have done a lot of professional database work and designed some pretty serious systems. From that background I can well see the many problems with the current state of POI files.

On-line input is another of the ways that have been mentioned previously in this thread as a way to input data. But I certainly would not want to see form based POI input as the only or even primary way to submit data. I would think that it is better to let people submit spreadsheets, text files, gpx files or other forms of data. If someone is willing to script a web based data input system, great, there is no reason to not allow this approach also, but it would not be a way that I would want imposed on me. The nice thing about importing data into a databae is that once the fields are well defined and organized, you can accept data in a varity of formats.

But before we can build a data input screen, we need to build a database. And before we can build a database, we need to spec out the fields that it will contain. I can do that (have done some preliminary work), but I'm relatively new to POI files and I certainly expect that others can make important contributions that, in my limited experience, I would miss or overlook.

At her request I did send my preliminary spec to Miss POI. I was told that there was a meeting planned to discuss it, and didn't post the spec here while I was waiting for feedback. But I think it's time to find what I wrote up, post it here in a separate post, and ask people to point out to me what I've done wrong or overlooked completely.

Enter the waypoints your way, get POIs your way

Frovingslosh wrote:

I have done a lot of professional database work and designed some pretty serious systems. From that background I can well see the many problems with the current state of POI files.

You of all people, with your extensive experience, I expected most to understand my point. The historical POI files are history. Let them stay there. They still work in GPS units.

Frovingslosh wrote:

On-line input is another of the ways that have been mentioned previously in this thread as a way to input data. But I certainly would not want to see form based POI input as the only or even primary way to submit data. I would think that it is better to let people submit spreadsheets, text files, gpx files or other forms of data. If someone is willing to script a web based data input system, great, there is no reason to not allow this approach also, but it would not be a way that I would want imposed on me. The nice thing about importing data into a databae is that once the fields are well defined and organized, you can accept data in a varity of formats.

I agree, importing files is the way we do it now, and that needs to be retained, but currently that is what is imposed on us. Some times your doing waypoints one at a time, while others are taking a list of addresses and converting them to POIs.
Both online and file import are useful.

Frovingslosh wrote:

But before we can build a data input screen, we need to build a database. And before we can build a database, we need to spec out the fields that it will contain.

I understand that. I did have database training which I never put to use and forgot 90% of it, so I take my hat off to you as the expert. Listen, that's why I'm talking about it here with you.

Before you spec out the fields for a database you need to analyze of what we are currently doing. Yes we are uploading POI files, but our objective is not to upload files, our objective is to get POI files into our GPSs that work efficiently.

Currently we upload files in the format which POI Loader understand. We have no other choice because those file are download unchanged from what we provided.

When you put a database in the middle, you are no longer restricted by the format understood by POI Loader, Garmin, TomTom, GeePeeEx, or any one else. All the complexities of the output file are not there to restrict the input file. We can deal with many of those complexities later when we build an output file from the database.

You can put the file in any format you want, which can and should include a file readable by POI Loader.

Now for the output file. Why not let me pick what I want in column 3 and 4 when I create the file from your database? I'm all for standards, but why do you have to tell me what appears on my screen, when we have the technology to dynamically create working POI files?

--
Nuvi 265WT & Edge 705

I expected most to understand my point.

plemirande wrote:

You of all people, with your extensive experience, I expected most to understand my point. The historical POI files are history. Let them stay there. They still work in GPS units.

Have you read the whole thread? I'm not clear on why you feel I think any differently than you do on this. Sure, there is no "need" to change any existing PIO factory POI files, although some maintainers of existing POIs may indeed want to put in the little work that it should take to get them into a database suitable format (gpx files should be very easy to convert, although the lack of standards does complicate even that).

plemirande wrote:

When you put a database in the middle, you are no longer restricted by the format understood by POI Loader, Garmin, TomTom, GeePeeEx, or any one else. All the complexities of the output file are not there to restrict the input file. We can deal with many of those complexities later when we build an output file from the database.

Absoultely. In fact, if you look at the North Carolina DMV file or the Pgiily Wiggly file, those look like normal CSV files (are in fact normal CSV files, but they were machine generated). I didn't build the database yet, but as proof of concept I built the field labeled text file that I would use as import to a database, and then wrote the script to produce the output files directly from the text file, bypassing the database step. I can easily modofy the script to produce other formats of CSV or gpx files. The TomTom format seems to be binary rather than text, so it would take a little more work to make a script that cranks out that, but it certainly can be done.

As I expect you already see, the benefits not only include standard formatted data (like the state always being available to be in there and being in a standard form so users can easily extract a particular state if desired), and greater flexibility in the final POI files produced (I could even envision a screen that lets you specify just want you want in your POI file, some people may indeed want the WalMart store numbers after the store name, but it would just clutter the screen for me. Answer a set of questions about what you want and the database could produce a custom report, which is your customized POI file.) But there is another benefit: One update to the database corrects all forms of the POI files that the POI factory offers, no need to go back and edit CSV, GPX, OVwhatever files and so on.

plemirande wrote:

Now for the output file. Why not let me pick what I want in column 3 and 4 when I create the file from your database? I'm all for standards, but why do you have to tell me what appears on my screen, when we have the technology to dynamically create working POI files?

We are in complete agreement here, that's what I already suggested in a previous post in this thread. Some users may indeed want the store number in field 3 of a chain for reasons that make sense for them. I don't, would select to add other information, like a local community name, or a city, or if that were not available at least a state (but to avoid clutter not all 3). There is a lot of stuff that could be included or omitted in field 4 (if you look at my DMV example you see that I have some things in the data that I left out of the CSV - like the county and the zip code. I didn't see any reason to include this and clutter field 4, but it was in the original data and I didn't see any reason to discard it. If someone wants a POI file that includes this I can very very easily create it by editing just one line in the script that generates the POI file from the input text file (and I would envision this being completely automated in a final version of this approach, although that needs to wait until after the database is formalized.)

An example - State LIne Crossings

johnm405 wrote:

If I remember this was mentioned once before but with all the state roads that cross state lines besides the interstate I think it could be a nightmare for one persone to do. Maybe if people from each state want to take it on but I also think it would abe like the time zone change poi very hard to get started and maintaine except for the interstate crossings.

Here is an example of a POI file that would be considerably easier to build if online editing were available.

--
Nuvi 265WT & Edge 705

Abort, Fail or Retry

The two of you are trying to build a product with no foundation.

The intent of this thread is not to develop a database or method of creating POI, but is to determine what must be present before you can build a POI file. You talk about all these elements such as name, coordinate, description, but you haven't defined what the element is or even what components make up the element and the dependencies inherent in whatever definition you use.

To use your state line example. What is a waypoint? What constitutes a waypoint? What makes one waypoint different from another? How is a waypoint structured? Until you both operate from the same book, on the same page, with the same letters in the same position, your definition will never agree with mine because there is NO STANDARD.

--
ɐ‾nsǝɹ Just one click away from the end of the Internet

proposal for standard fields

Here is what I'm offering as a starting point for standardizing the fields. I complete expect I've missed things and this can be improved on. This is only a very minor first step, you can't build a database or even a good spread sheet system until you have considered what fields of data you need (at least you shouldn't try to do it).

POI data file spec, draft 1a
---------------------------------------------------------------------------------------------------------------------
NOTICE: This is a very rough draft of a proposed specification for a database for POI locations. It is only a
rough draft at this point. NO ONE SHOULD BUILD A POI DATA SET TO THIS SPEFICATION. The only reason this proposal
is being released at this time is to solicit comments.
---------------------------------------------------------------------------------------------------------------------
FIELD: POI SET NAME
A name for the POI set, as it will be shown on the POIFactory pages. Examples: "Dunkin Donuts - USA and Canada" , "Dive Shops and Marinas USA", "Golf Driving Ranges in USA", "2 meter Ham Radio repeaters". This field should be exactly the same for each item in a POI set, and this would be key in allowing all POIs to be saved in one master database. It is still not clear how we might deal with All of the small local POI files that exist now, but I'm expecting that they will be combined under master categories. For example we have many Golf Course POI files that cover small areas. I think it would be best to have a master category called "Golf Courses" and combine them. Individual areas can still be filtered out by the State or even City field. This does lead to issues about how to determine which areas are represented in the database and which are not though.
This is a required field.

FIELD: DISPLAY NAME
The name of the POI that will be displayed on the GPS, Such as Wendy's, WalMart, Bob's Dive Shop, State Police HQ and so on. In many cases this name will be consistent through the set, such as when doing Store Chains. In other cases it will be different for different POIs (such as POI files for mom and pop restaurants, Colleges, and so on. If doing a chain that has different types of stores, such as WalMart and Super-Walmart (with grocery stores), please use the more distinctive name, but take care to be consistent in the names, so that someone could filter for just the "Super" stores if they wish. City or State information should NOT be included here, as a final Display name will likely be created by appending the City field to the name (or State field if a city or locality field is not available)
See also the visible comment field.
This is a required field.

FIELD: STREET ADDRESS
If available. Desirable but completely optional.

FIELD: CITY
The name of the city, town or other community that the POI is in. IN most cases the name that is displayed on the GPS will include the city (just seeing a list of POIs marked WalMart is much less useful than seeing "WalMart Muncie", for example.
This field is not required but highly encouraged.

FIELD: STATE
The official postal code abbreviation of the state. This should always be in all upper case. This code is key to breaking down USA or USA and Canada POI files to smaller local files (for people who only want to know about the Starbucks in their own state or states they plan to travel to, for example. Canada entries should use their province. I see we have some POIs that cover Europe and other areas, and I'm not certain how to treat them. For now I propose that the master database focus on USA and Canada locations, and if points outside this area are added this field be left blank, but I expect that others have dealt with this issue and there is a known good system for organizing the data.
This field should be considered extremely encouraged for any new data and should be a input whenever possible if anyone is updating POIs from current data that lacks it.

FIELD: COUNTY
A field allocated to hold the county (or parish) name. Quite frankly this seems unnecessary and will be left blank for most POIs; I have not yet seen a POI file that includes county information. However, in many states a lot of things are grouped under the county government system (polling places and early voting places, for example). Also, a state is likely too large of an area for one person to make a comprehensive POI sets for many types of information (such as school zones) and most towns are too small for the POI file to have wide appeal. One or more counties seems likely to be a good way to focus such efforts. and, let me say this again:
The county field is completely option, can be ignored and in most cases will be ignored.

FIELD: LOCALITY:
I see this as a refinement on the city field. Some City fields may not convey enough information. Such as "Starbucks, Seattle". In such cases a locality field would much better help refine a POI from a list of POIs on a GPS, such as seeing "Starbucks Fisherman's Wharf" . The logic that I anticipate, unless overridden by the person who creates the final POI .csv file from the data, would be to combine the display name the locality if one exists, or the display name with the city as long as it exists if there is no locality, or failing this the display name and the county, or failing this the display name and the state, or finally just the display name.

FIELD: ZIP CODE
If available. Desirable but completely optional.

FIELD: STORE NUMBER
Many chains assign their locations "store numbers" and these are sometimes in our POI files. This field would hold a store number if known. These numbers should only be taken from official chain lists or left blank. They should never be assigned by the person creating the list, even if he or she believes there are just a small number of locations and wishes to distinguish them.
In most cases for most users, seeing the store number on a list of POIs is pretty meaningless. For example, when picking a WalMart from a list of POIs I don't care if it is store 1015 0r 1019, I care what city or locality it is in. So I propose that store numbers not be included in normal POI csv files, but that there be an option available to include them when available in the final .csv files for those who may have use of them (delivery men, corporate people who travel to the stores and so on). For most users this information only clutters their already small screen.
This field is optional and will often be blank, but if building a POI file from corporate lists contributors are strongly encouraged to enter the store numbers when available.

FIELD: PHONE NUMBER
Phone number of the location if available. Looking at existing POIs I can't determine the required format, we are pretty inconsistent here. Someone please spell out how a bluetooth enabled GPS needs to see this number (for example, do we need or not need parens around the area code, do we use dash or space as group separators and so on.

FIELD: FAX NUMBER
I'm not sure we need it and most POI don't have a FAX number included, but at least one does, so I may as well include a field for it. I expect it will mostly be left blank but if a contributor wishes to include it, why not?
This field is optional.

FIELD: HOURS OF OPERATION
I've seen people asking for hours of operation of some POIs, although I have yet to come across such a file. I'm including this field for those who wish to use it. It's not clear to me how to standardize the way the information is entered here, it seems it may have to be at the contributor's discretion. For example it might be "7AM-11PM M-F 9AM-9PM Sat" or just "Closed Mondays". It might be desirable to standardize if the day or time come first, and to urge using day of week abreviations rather than fully spelling out the day name. An important issue here is to be concise, GPS display space is limited. It may not be reasonable to try to enter complicated seasonal schedules here.
This field is optional.

FIELD: EMAIL
I'm including it here because while inspecting some POIs I found email addresses thrown in. I'm not sure we even want email addresses, but if people do, there is little impact on the data to include a field for them. I expect that most contributors will leave this blank. It will take a little thought to determine where to include this in the resulting .csv files, I doubt we want to clutter a map screen with it, but at least for the GPSs that gracefully handle lots of fields, we would have the data.

FIELD: VISABLE COMMENT
A comment that helps refine the POI data and will be included in the displayed data if present. For example "Playland" for McD's with a Playland or "fuel" for a rest area with fuel. This field should not reflect information already in the store name; for example if you are building a list of WalMarts and Super WalMarts, don't include "with grocery" here as the store name already implies that. Be concise.
In some cases either the name field or the visible comment field might be used to convey special information about a store (such as a Taco Bell that is also a Kentucky Fried Chicken combo store). In such cases the contributor should use either the display name or the visible comment to let the user know he isn't looking for a normal Taco Bell, but should not use both fields for this. Be consistent. It is suggested that in this case the visible comment field say something like "KFC combo" or "with KFC". And again try to be consistent and use the exact same term if there are other similar stores.
This field is optional and may be left blank.

FIELD: LATITUDE
Latitude stored in decimal degrees format (ex: 40.123456) It is unclear how many decimal places are needed after the decimal point. I suggest 6, the output as shown by Google Earth. 5 seems to be common in POI factory POIs. Allowing for 6 when only 5 (or less) are available will cause no problem, unused digits will be assumed to be zeros. Note that Longitude comes first in Garmin .csv Poi files, but I'm listing Latitude first here as it is the coordinate I most often find given first by most sources, including Google Earth)
For some POIs this may only reflect an approximate location, such as the ham radio repeater POIs that do not attempt to give an exact antenna location, just a city location.
If you feel that I'm wrong about the order of this and the next field, please say so. It is the opposite of the order in the Garmin POI files and that could be an issue for some people. Order may be important in some input form, in other like headered spreadsheets or field labeled text files it is completely at the contributor's discression.
This is a required field. Some form of imput may all it to be included with the next field as a coordinate pair to simplify geocoding.

FIELD: LONGITUDE
Longitude stored in decimal degrees format (ex: -75.123456) It is unclear how many decimal places are needed after the decimal point. I suggest 6, the output as shown by Google Earth. 5 seems to be common in POI factory POIs. Allowing for 6 when only 5 (or less) are available will cause no problem, unused digits will be assumed to be zeros.
For some POIs this may only reflect an approximate location, such as the ham radio repeater POIs that do not attempt to give an exact antenna location, just a city location.
This is a required field. Some form of imput may all it to be included with the previous field as a coordinate pair to simplify geocoding.

FIELD: PROXIMITY
Used for setting a Proximity alert for some POIs such as red light cameras and speed traps. Not used for most POIs. The assumed unit of measure, at least for POIs in the USA is feet, not meters. 1 mile = 5280 feet (there seems to be a difference of opinion on this conversion rate in the forums, so it seems best to state it here).
Values used here should be determined by the contributor, understanding that in most cases for red light cameras an alert should not be given until at least the driver is past the previous intersection. And for alerts such as speed traps it's better to give the user plenty of warning, both to allow time to decelerate and to allow for long range speed guns or law enforcement officers who vary their position somewhat.
This field is optional, but it use may be required by some types of POI.

FIELD: SPEED
A speed limit which may trigger an alert for a POI. This may be used for some POI files such as speed cameras or school zones or Interstate Highway Speed Limits. Not used for most POIs. The assumed unit of measure, at least for POIs in the USA is mph, not km/h.
This field is optional, not required for most POIs, may be used for certain types of POI. Users can over ride this setting when using Garmin's POI loader in manual mode, but should expect to get some reasonable value when the .csv file is installed in automatic mode. It is suggested that for speed limits and school zones the alert is set to the legal limit (no fudge factor for "the cops don't ticket you unless you are going 5 miles over"), and for speed traps and speed cameras it be even a couple of miles under the limit (to allow for slight variations in speed while approaching the POI). Users who desire other setting can manually load the file, edit the file, or we could even create a csv file generation option that lets the user adjust the setting to his own liking, as long as our base values are standardized and reflect the local limits.

NOTE: I had written the above when first trying to lay out a spec, but on further inspaction of POI data I'm not sure if it could easily be included in POI files. The only good way that I can see so far is to produce multiple files with different speed limits in their name. But perhaps there is a way to use this in gpx files, or someone with better understanding of the speed limit feature in POI files can offer some guidance.

FIELD: CONTRIBUTOR
POI-Factory user name (or other unique identifier as desired) to note who contributed this POI. Should help the site overlords determine who is contributing. Also if a pattern of problems is observed (such as transposing lat and long) it may help ID who should be notified of the issue. Initially all pois contributed as a set will have the same contributor entry. Additions will show who made the addition. In cases where data is imported from existing POI csv files, the original contributor, if known, should be given credit. If extensive work is done to upgrade existing records (such as another contributor going through a POI file of only Lat and Long and the same store name for each and every location and adding City State and phone numbers), the contributor should reflect both members, "Smith" might become "Smith / Jones42" for example. The contributor field is not intended to be included in generated CSV files, it's only for use by the maintainers.
This field should be considered required for all new contributions, may be empty if an old POI file is brought into the database and the contributor or contributors are not known, and is not needed for csv generation. If the field is empty when a new contribution is made the person who accepts the contribution should propagate the contributor's ID through this column.

FIELD: DATE ADDED
The date the record was added to the POI database. In general this field should be the same for all records in a newly contributed file, and as records are added those records may reflect a new date. This field is for internal maintainers benefit and is not expected to be used in CSV file generation.
The contributor is encouraged to fill in this field, but the database overlords can easily propagate the date that they accept the contribution through the file. Any individual records added or edited should reflect the date of last change.

FIELD: INTERNAL COMMENT
Large space for internal comments used by the data maintainers. For example, someone may want to make a comment that a red light camera POI is incorrect and there is no camera there, or that some other aspect of the data is in contention. This information is not expected to make it into the csv files, but may be helpful in maintaining the data and avoid repeating errors. See also the next field.
This field is completely optional, and will usually be empty when a record is first created (and hopefully long after that)

FIELD: ACTIVE
A logical field that indicates if the record is "active" or not. All records that should be included in a CSV file should have this field true. If a POI turns out to be bad, it is suggested that this field be set to false rather than delete the record, and in most cases a comment be put in the internal comment field. This should help prevent the POI from being added again the next time someone gets their hands on old outdated store lists or other inaccurate data. Only records where "active=true" are used when generating .csv files.
This field is required, should be set to "true" for all new POI records (unless the contributor has reason to make a small number false), and is set to false if a POI is bad.

---------------------------------------------------------------------------------------------------------------------
At a minimum a set of POIs could be contributed with a similar level of effort as required now. Someone contributing "Huge Pot Holes in Pennsylvania", for example would only need to take a database template and fill in the lat and long for each pothole. The POI SET NAME would all be set to "Huge Pot Holes in Pennsylvania" (or maybe just "Huge Pot Holes for the USA" if we got contributors from other states) and the DISPLAY NAME could, if he wished, be all set to "Pot Hole". The state would all be set to PA and, if the user didn't want to fill in city or locality he could skip them, although it sure would be nice if he could supply the city name. Street address, store number, phone number, fax, hours of operation, zip code and email can all be skipped (or street and /or zip could be provider if the contributor is bored). Visible comment can be used to provide extra data (depth of really deep holes, for example, or "avoid right lane eastbound") but can be left blank if contributor is the quiet type. Speed alert can be skipped, a proximity alert might be neat, would not be mandated though. The nice thing here is that the contributor doesn't have to work out much for proximity alert syntax, he just puts in a value (say 1320 for 1/4 mile, maybe a bit more for freeway and interstate potholes) and his work is done by the code that creates the .csv files from the data. That's pretty much it. The date and contributor fields would be set by whoever accepts the data if the contributor leaves them blank, and all records would be made active(true). From this data set we could easily generate any POI data file for any Garmin family, a Tom Tom or any other format any other brand of GPS needs. And if things changed (the unlikely event the road crews fixed a hole, or the likely event that there were more holes tomorrow), one change to the database could be propagated to all of the different make/model POI files automatically.

A bit much?

Looking at the OP's issue & the 189 posts in 60 days, I have to think this is getting a little bit over engineered.

It's good to have a standard, but perhaps a minimum standard would suffice? If you look at the mall file that brought the subject on in the first place we are talking about 20 poi's I believe. I think Miss Poi went through & added the state info that was missing, but in reality if the file only had lat, lon, and name, the state was discernable (and probably the street as well). So, what was not met in this case was the OP's expectation of completeness of the description field which would have made it easy to sort by state.

Not all users may have such an issue with not being able to see the state listed as text if they have the lat & lon, but others might. One thing that NO user can currently do is determine prior to download just how descriptive the text in a file is.

To that end, simply ensuring that a file has the 4 items in the GPX spec, lat, lon, name, and description but while ensuring that the description describes textually to the resolution of the lat & lon seems appropriate to me. Filling in the description text from the lat & lon can be automated (reverse geocoded).

And as an aid to the user, indicating the address resolution (house/street or less) and the percentage of 10 digit (or whatever we decide is spec for ROW locale) phone numbers per poi on the download page. Both of which can be done programmatically.

If we start with just this, any enhancements in a text description can be added and indicated as users request. All of the field & database items seem to be an issue more relevant to the webmasters, yes?

--
It's about the Line- If a line can be drawn between the powers granted and the rights retained, it would seem to be the same thing, whether the latter be secured by declaring that they shall not be abridged, or that the former shall not be extended.

How am I doing so far?

a_user wrote:

The two of you are trying to build a product with no foundation.

The intent of this thread is not to develop a database or method of creating POI, but is to determine what must be present before you can build a POI file. You talk about all these elements such as name, coordinate, description, but you haven't defined what the element is or even what components make up the element and the dependencies inherent in whatever definition you use.

To use your state line example. What is a waypoint? What constitutes a waypoint? What makes one waypoint different from another? How is a waypoint structured? Until you both operate from the same book, on the same page, with the same letters in the same position, your definition will never agree with mine because there is NO STANDARD.

A way point is a logical stake in the ground, which a GPS will take you to, or more recently, alert you to as you pass by. Location makes one way point different from the other. The structure of a waypoint is a point.

--
Nuvi 265WT & Edge 705

Well said JD4x4.A minimum

Well said JD4x4.A minimum standard is all that is needed.With all the above fields to complete you would wear your mouse out copying and pasting.Right now using excel with a few strokes of copying and pasting I can put complete information in column d.Using Google my maps is even easier.

--
Charlie. Nuvi 265 WT and Nuvi 2597 LMT. Android Here WeGo - Offline Maps & GPS.

It is what it is.

JD4x4 wrote:

Looking at the OP's issue & the 189 posts in 60 days, I have to think this is getting a little bit over engineered.

That was my opinion for a while, but the more I learn and understand this the more I realize that all POI files are not the same. Lets' look at OP's statement:

bpa5152 wrote:

I just looked at the custom POI for Malls of America. When I looked at the cvs spreadsheet I wanted to see where the malls were so I could plan some trips before I downloaded the file. I saw the latitude, longitude, mall name and street address but did not see the city and state. Therefore, without going to another source I have no idea where these are or if they are somewhat close to me.

Notice he did not request the country be listed for each waypoint. If this were Malls of Wisconsin, then he would have been happy with the descriptions. So if you combine a file fore each state into one large file, you need to change the description in the process to make it work, yet its the same data.

Change the file to a hotel, and now you want a phone number so you can call ahead. Change the file to a campground and now you want to know if they can handle RVs.

When looking for a hamberger joint, some just want to know which is closest, others want one which is on their route and convenient.

I don't understand why the Amtrak Stations POI starts each description with 'Amtrak -' followed by the city.

I do understand why the city is never listed in the descriptions of the Milwaukee Attractions POI.

AZ Major Lakes file lists the name of the lake followed by a phone number. I would hesitate to dial that number because I'm not real sure who it is I would be calling.

The Big 12 Football Stadiums file description lists the name of the field, but not the name of the school. Am I supposed to know which team plays at Memorial Stadium? As it turn out, three schools named their stadium Memorial stadium.

NFL Stadiums file describes the field name followed by the team name. That's good enough for me, I know the states for each team. A few weeks ago I had forgotten the Cardinals were in AZ, but this week I know what state their from. Did you know the Cardinals started out as the Chicago Cardinals?

Many files list the address in the description, but since I have lat and lon I don't use the address.

Most restaurant files simply restate the restaurant name in the description, like 'Boston Market' or 'Cracker Barrel.' Somehow I understand that.

The Lexus Dealers file is provided by state and the dealer names are something like 'Lexus of Brookfield.' That's nice, but it wouldn't work for Burger King.

There is one file called Beaches. The name of every beach ends with the work beach, except for Tunnel Park.

The Cabela's file lists city and state in the description, but most states only have one Cabela's.

The wineries file will always be my favorite, with each description simply listed as winery.

--
Nuvi 265WT & Edge 705

Why the Amtrak Stations POI starts each with 'Amtrak - "

plemirande wrote:

I don't understand why the Amtrak Stations POI starts each description with 'Amtrak -' followed by the city.

This may not be a bad thing. If you're looking up a location in a POI file then you likely already know that you are looking at Amtrak locations. But imagine that you have some reason to set a proximity alert on your Amtrak POIs (or any other set of POIs). Then when the proximity alert pops up you sure want to know that you are near "Amtrak- Dayton" and not just see "Dayton" pop up in the proximity alert with no clue as to what triggered it.

Somehow I understand that

plemirande wrote:

Most restaurant files simply restate the restaurant name in the description, like 'Boston Market' or 'Cracker Barrel.' Somehow I understand that.

Well, this one sentence seems to talk about 2 different things. I'm all for the restaurant name being consistent in the data (I cleaned up the Dunkin Donuts CSV file that had "Dunkin Donuts", "Dunkin' Donuts" and even "Donuts Dunkin" in it, and I thanked the maintainer of the Wendy's file for it's recent cleanup and move towards consistency, there were previously a half dozen or more forms of the restaurant name in that file.

But field 3 of a CSV file isn't exactly always just the store name, and making it only the store name can cause a problem if you need to identify a POI because of bad information, duplicate, store closing or any other reason. When your GPS just identifies the bad location it sent you to as "Boston Market", it can be hard to determine which POI in the file you want to report. If some more unique information such as street, town, or even store number is added to the name to make up field 3 then it is much easier to determine which entry in the POI file needs corrected.

Good point, but all of those were my attempt at humor.

Frovingslosh wrote:
plemirande wrote:

I don't understand why the Amtrak Stations POI starts each description with 'Amtrak -' followed by the city.

This may not be a bad thing. If you're looking up a location in a POI file then you likely already know that you are looking at Amtrak locations. But imagine that you have some reason to set a proximity alert on your Amtrak POIs (on any other set of POIs). Then when the proximity alert pops up you sure want to know that you are near "Amtrak- Dayton" and not just see "Dayton" pop up in the proximity alert with no clue as to what triggered it.

Presenting the opportunity to park my car and take the train instead.

Seriously though, your comment supports my point. The description may depend on what kind of waypoint we are marking, and who is using them. I would never set an alert on an Amtrak terminal, but I may want to be alerted if I'm passing close to a Culvers, because a chocolate malt can make the trip seem shorter, and I don't need to know any more than C U L V E R S.

--
Nuvi 265WT & Edge 705

I don't need to know any more

plemirande wrote:

but I may want to be alerted if I'm passing close to a Culvers, because a chocolate malt can make the trip seem shorter, and I don't need to know any more than C U L V E R S.

You might only care about the name if it is right there in front of you. But what if there were no store at that location, and there was no other location information in the file (because after all, you only care about the name). How would that one bad location be found and fixed with no other information? After all, it's not easy to get the coordinates out when looking at the POI on the screen. And even if you do track down which POI it is, what do you do, just delete it? Maybe it was in there to represent an actual store, but someone typed one of the coordinates wrong. If you just delete the POI because you have no other information to find the correct coordinates, then you may miss out on your sweet cold treat when you are in proximity of the actual location, because just having the name doesn't give enough information to make a proper correction.

This discussion is pointless...

Frovingslosh wrote:
aophiuchus wrote:

Columns 1 & 2 have to be as they are...

Frovingslosh wrote:

Actually, I don't even know what that means. When I set out to create POI files I wanted to know how many decimal places of accuracy to include, but couldn't find any standard.

That means columns 1 & 2 have to have the coordinates. (As they already do, if not the poi won't work in the gps) The number of decimal points equals how many places it takes to define the position. If you don't know this then possibly you shouldn't be submitting poi's.

aophiuchus wrote:

Column 3 Should ID the location in some manner and it will be impossible to satisfy everyone...

Frovingslosh wrote:

So because we can't satisfy everyone we shouldn't try to give guidelines?

What I am saying is there are always exceptions, A "standard" can be made that it must contain a name and a city. What if it isn't in a city? My point is that it should and does contain an ID whatever it may be. i.e. Wal-Marts, McDonalds, Yosemite Natl Park etc

aophiuchus wrote:

Column 4 I put the complete address and phone number, either with or without line breaks, but this is just the way I do it..

Frovingslosh wrote:

Column 4 is a hodgepodge of whatever anyone throws into it, and I can show you csv poi files that I don't think anyone can give consistent rules for determining what part is what.

Here I am the one who doesn't understand what you mean, but regardless, here again you can't have an iron clad "standard". Depending on the poi it may lend itself to have an address and phone number. It also may not have an address or phone number. It may just be a location. i.e. a historical sight... So it has to be the poi maker's decision what should or should not be put in column 4, even if you cansider it to be a hodgepodge.

This post can go on ad nauseum,(which it has) and I doubt if "the standard" will be defined any closer than what you, I, and many others have already stated.

I'm sure Miss Poi and company will come up with a set of "standards" and that will be that, regardless of how many posts continue after this one, but there won't be anymore by me... Maybe. rolleyes

--
It is terrible to speak well and be wrong. -Sophocles snɥɔnıɥdoɐ aka ʎɹɐƃ

aophiuchus: This discussion is pointless

aophiuchus wrote:

This discussion is pointless...

Yes, particularly when there are people who just want to oppose it and not offer anything positive.

There are a lot of things posted in this forum that I have no interest in. I don't inject myself in the conversation and tell the people how uninterested in it I am, I just skip by it and read other things. But that's just me.

I lied...

Frovingslosh wrote:
aophiuchus wrote:

This discussion is pointless...

Yes, particularly when there are people who just want to oppose it and not offer anything positive.

There are a lot of things posted in this forum that I have no interest in. I don't inject myself in the conversation and tell the people how uninterested in it I am, I just skip by it and read other things. But that's just me.

Poor choice of words on my part, The discussion had merit but after four pages ..... I'll butt out... sorry if I offended you. BTW I thought I did offer positive input, but that's just me... rolleyes

--
It is terrible to speak well and be wrong. -Sophocles snɥɔnıɥdoɐ aka ʎɹɐƃ
1 2 3
<<Page 4>>