Dismanteling Individual state files when there is a combined file.....

I agree. We don't necessarily need individual files for each state or province when we're dealing with a national presence. Most of the time the files are very easy to edit so if you don't need partions of the country, just remove them.

I will only update the US Ford Dealers file from here on out.

Great Idea!

That is great, the state is fine to start but once the usa is up it is much easier. I like what mahoney does, he contact each state and ask about update.

How can I get an individiaul state's stores out of the combined nation-wide file?

You can use Extra POI Editor (a menu option- File, Export by, State), GeePeeEx (sort & delete), Excel (or another spreadsheet) or my Excel add-in for gpx (sort & delete), etc.

Maybe we should try to invoke a standard that combine file include the 2 char state code in the name (column c) so that those who wish to split the file can easily sort and cut/paste.

For Example (name portion):

Chipotle_IL - Barrington Road
Chipotle_IL - Beverly
Chipotle_NE - 72nd Crossing
Chipotle_NE - Bakers Square

It depends on the file and how it is put together. Many can be sorted which would group all the locations by the sort value. As CSV files have no standard for laying out the data in Column 4, you may have to do some reformatting within the column before it can be easily done.

Depending on lenght of name you have in column c.Adding anything much more then the state with it will cause the announced arrival to clip off some of the voice message.

I vote for the Name and two letter state. the remainder of the info can be in col d.

I prefer additional info to avoid duplicate names and to help in deciding which location I want without hitting the more button (as min. as possible)....

(EX) The GPS may list them in order of distance but I may choose one location over another due to traffic, parking, neighborhood (crime area), etc.

There is a standard, but it doesn't cover the content. The standard is for the longitude to be in the first column, the latitude to be in the second, the assigned name for the two points in the third. The fourth field or column is optional and contains any comments or description the creator wishes to add.

So, the only two fields (columns) that have specific requirements are the lat/lon. The other fields are "free form" in that a person can put whatever information they wish. Any 'standard' would be unenforceable and be by agreement only.

And before anyone starts a discussion on a "standard" for the information in a CSV file I want to remind people this discussion went on for a couple of months early last year before it was killed by Miss POI as nothing was being accomplished.

I can see where this will make your job administering the files easer. However I do have a couple of questions:

1. How can I tell if the state I'm interested in has been updated? I don't want to download a large US file to find out that a McDonalds on the other side of the country as changed it's phone number.

2. This is going to make some files extremely large, especially GPX files. Are you really sure that national files are going to be a benefit to the average user? Setting up 8 to 10 regional files might be a better idea.

As an example, I have three files that contain railroad grade crossings. I very rarely travel outside of a three state area so I would have to download RR crossings for the entire USofA and then either have a huge file or spend time editing out what I don't want or need.

I personally vote to leave things as they are and let me download as much or as little as I need.

Thank you for allowing me to rant.

Yes larger files will be broken down by region. Please see this POI Package as an example: http://www.poi-factory.com/node/10677

Miss POI

Ok, maybe "standards" was the wrong word since I do not want to force people to complie but I still feel some "guidelines/agreements" will be beneficial for the community.

That was the part we couldn't agree on. Some of us - me included - wanted certain items included in the fields, other felt it was too structured and would keep others from submitting files.

Yes, 'Standards' are a tough nut to crack! There was a big thread on just that subject that's now closed.

Not to reopen it here, but my observation is:
Starting with the two available 'standards' we all currently have & use (csv and gpx), there might even be as much as a 50/50 split in opinion on which is preferred!

Personally, I prefer gpx since it splits everything down to the most used elements, BUT there are just as many times that I prefer or need the tools available for editing in csv as there are editing in gpx.

And at the end of the day, if you have the right 'tools', the only info that really ISN'T 'embedded' in a lat/lon pair is someone's a) name, and b) additional relevant description or commentary.

Maybe we just need to come to a consensus on the easiest/best tools/method to extract address/city/state info from csv lat/lon, and leave csv name & description for agreed upon added value that's relevant to the 'type' of POI it is or it's typical usage?

Where in GeePeeEx do you find "sort & delete"?

I use GPSBabel to extract what I want from large POI files automatically. You can get arc files for US states and counties here:
http://sourceforge.net/projects/gpsbabel/files/

For example, here's how I break down the CO-OP Network ATMs POI file (http://www.poi-factory.com/node/12841) to include only locations in or near the borders of Idaho, Washington, and Oregon:


# Extract locations within Idaho
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x polygon,file="polygons/ID.txt" \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_ID.csv"


# Extract locations within 1 km of Idaho border
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x arc,file="polygons/ID.txt",distance=1k \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_ID-arc.csv"


# Extract locations within Oregon
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x polygon,file="polygons/OR.txt" \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_OR.csv"


# Extract locations within 1 km of Oregon border
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x arc,file="polygons/OR.txt",distance=1k \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_OR-arc.csv"


# Extract locations within Washington
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x polygon,file="polygons/WA.txt" \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_WA.csv"


# Extract locations within 1 km of Washington border
gpsbabel -i garmin_poi \
-f "inbox/CO-OP_ATMs_USA.csv" \
-x arc,file="polygons/WA.txt",distance=1k \
-o garmin_poi \
-F "filtered/CO-OP_ATMs_WA-arc.csv"


# Combine all the intermediate files, and filter out duplicates
gpsbabel -i garmin_poi \
-f "filtered/CO-OP_ATMs_ID.csv" -f "filtered/CO-OP_ATMs_ID-arc.csv" -f "filtered/CO-OP_ATMs_OR.csv" -f "filtered/CO-OP_ATMs_OR-arc.csv" -f "filtered/CO-OP_ATMs_WA.csv" -f "filtered/CO-OP_ATMs_WA-arc.csv" \
-x duplicate,location \
-o garmin_poi \
-F "inbox/CO-OP_ATMs_NW.csv"


I actually automate this process with a Makefile, and run it from a cron job on a Linux box, but these are the commands it executes.

I meant sort, then delete (or export/save) like Excel. But, my bad anyway .. I thought you could right-click on State and a sort option came up, but it's only available for the POI name. Since alphabetical is the only option, GeePeeEx wasn't a good suggestion.

Yeah, that's what I thought, like I've never made a mistake! If she's going to do this, and it looks like she will, then I vote for putting the two-letter abbreviation in the entry name.

For Magellan POI Editor, unique names are needed in column c. The proposal for only the name and 2 letter state won't work. POI Editor will drop all but the 1st entry for each state.

of a few files, and when I travel, after I decide on a particular file, I first want to know the city and after that, the state. That is why my format in column C is Name, City and State(Abbreviated). If this format does not suit Magellan users, maybe a third category could be added to the download page that now includes Garmin and Tom Tom.

In column c I put abbreviation for state (so users can pull out the state they want) and the name.I have tested several times putting the city with it.Not all cases but depending on the length of the name some of the arrival message gets clipped off.

This subject has been discussed many times in the past, just to fall by the wayside.
I personally feel this should be between Maryann and Jon to make the decision since they are our leaders and it's their house.
And it's really up to them anyway!

I don't see the point in boogering up a POI name with encoded stuff, just to continue to fit the csv format. I use gpx files and will only provide gpx files. Any csv file only from this site needs extra effort on my part to reformat it and convert it to an acceptible gpx format. Those with non-Garmin brands can run conversion programs. The Garmin gpx format has specific fields for address, state, postcode, phone, URL and descriptions. With Extra_POI_Editor, most conversions and state sorting can be effortlessly done. EPE can output Garmin gpx, TomTom, Magellan, Excel and others. I think we are limiting ourselves with csv files that need constant attention with differently formatted encoding to squeeze data in that has a home in gpx format.

I'll vote for putting any location 'helper' text into csv col 4 since that's really what it's there for (comment or description), most s/w allows more text there, and it's really a trivial effort to parse any of the columns if a spreadsheet (common denominator for 99.9% of users??) is used.

The only caveat I'd add to that is PLEASE use consistent delimiter chrs (NOT a space) between info that already has a gpx definition such as street, city, state, zip, phone, etc. The most problematic parsing is when there are spaces between state/province and post codes and/or within either of those 'fields' themselves. It's ok to maybe use a dash and an underscore, etc on the same line, but just be consistent in which 'fields' they delimit (at least in the same file, if not site-wide!).

Like it or not gpx IS the de-facto gps data transfer 'standard' even though some mfrs have software that allows reading & converting csv formats. While I admit that I mostly find it easier to manipulate a csv loaded into a spreadsheet format, I'll still vote for GPX being the Poi-Factory default file format since it already 'classifies' the data into the 'fields' that it really is, and gives better potential unit features.

I've been working off & on to convert my Excel GPX add-in utilites to Open Office Calc but it's a bit complex for some features and will take more time than I have at the moment. But, I could certainly create an Open Office Calc add-in that puts some of the commonly used parsing, sorting, search & replace functions under a Calc menu add-in if anyone thinks it would be a help. Let me know the specifics of what you'd like to see a spreadsheet do and I'll give it a shot.

Dave your being a little selfish by saying let non Garmin users Convert?
This a mixed GPS community, Just seems like your more concerned about yourself than the betterment of the community as a whole.
so your extra effort is appreciated by everyone else that uses whatever gps they may have purchased.

From a gpx file that contains nicely formatted fields, any format can be output with fields as the user wants. That's not being selfish. Making users with GPS units that do support additional fields suffer due to the csv lowest common denominator seems to follow that behavior.

As there is an easy, free program, EPE, that provides most any format, what is wrong with providing better information? It would allow anyone access to google maps of each POI, allow for linking to URLs, sorting and exporting by state or category and searching for and optionally removing duplicates. It would cut down on support time by those supporting a POI file. It allows for geocoding addresses. It will even display photos of POIs.

The only negative to that would be increased file size that could be remedied by making all files zip files. Mac and PC users both support that. Step by step instructions can be posted to not leave inexperienced users behind.

I am not convinced that the GPX format should be the ONLY standard for our community.

Those who can NOT use the GPX format will need to convert the file using EPE on each file they download. This could be very time consuming depending on their skill level or the number of files.

We can not forget this is not a GARMIN exclusive site.

It is nice to have a csv file on the site then be able to convert to a gpx if you feel a need to.Most users start off using some notepad,excel and etc before making a gpx file.There is room and need for both formats.I prefer GP when I want to build the alert distance in and bypass the manual mode for poiloader.Users who have Blue-tooth want the ease of being able to use dialable phone numbers.

it's not the format - CSV vs GPX that's the issue, it is about the content and how that content is presented in whichever format is used.

As I've stated before, the CSV or Comma Separated Variable format has 3 defined fields. Those are latitude and longitude in decimal degrees and name. If latitude or longitude is the first field varies between units, but all agree the coordinates must be the first 2. All also agree the only other required field is a name assigned for the coordinate points. Magellan requires each name be unique so the name of a chain can't be the only identifier and then their loader doesn't load the fourth field if present. It's the optional 4th field where the problem lies and the unique name requirement that affects the 3rd field. The fourth field is undefined except to call it 'Description or Comments.'

The GPX format requires each item or field be defined so we have a descriptor that states the value identified is the latitude or POI Name. We can also define a field as a description or comment and a host of other designations. The problem is that GPX files are not easy to work with as they use specific notation to identify the data.

Like most other maintainers on this site I keep all my files in spreadsheets. The difference is my spreadsheets aren't 4 columns they can be up to 12 or 15 columns depending on the information I want included in my file. In order to manipulate my input data into a file that can be processed by my GPS i have to use an editor. The editor takes the data from my spreadsheet and assembles it into 2 files, one a GPX that contains much more data and information and a CSV that provides the basic coordinates, name and a description.

I have started playing with some of the other features available with GPX such as varying the font used and adding color. I haven't tried seeing if it can be incorporated into a CSV but I suspect not.

- download and take a look
http://www.datafilehost.com/download-2b3681fa.html

Who cannot use gpx (to even at least filter to csv)? I admit my car gps's are Garmin but I also have a Magellan and both companies have s/w that reads gpx to transfer to my units. Is it TomTom, or are we talking about Linux and/or Mac users?

Not being rhetorical here .. just wasn't aware some could not use gpx at all.

..While on the subject, is there also any brand and/or OS where there isn't a means to use a 4 column csv?

I think I recall reading a post here that it was not so much the GPX format but the file size of the GPX file. I could be wrong.....

Some older units have very limited space.

It's not the input, it's the output GPI file. There is little difference in the size of the GPI file POI Loader writes.

You can also do it with GPSbabel if you're not scared of some work.

I created a file I call "ne-corners" which contains a series of points making a circle around New England (first and last entry are the same in this file) and when I want to get only the entries near me I can run the gpx file through a polygon filter and it only passes points within the polygon.

Works very well.

Mary Ann I have a combined Royal Canadian Legion File ready to upload with @ 1500 entries. Is this a good file size or is it better to keep the smaller Provincial files.
TIA Doug

If that's the case then the other alternative is something like the universal CSV which has lots of columns defined (in the first row) and you can run your choice of "compilers" (don't know of a better word) to turn it into the format you desire.

IMNSHO, the default storage should be as much info as anyone can use and it is up to us (the users) to filter out only what we need.

The alternative is all files, regardless of purpose read as "xx.xxxx,yyy.yyyy,WP####" and that's it.

However, storing all the info allows everyone to create what they want out of the files.

I don't know what "EPE" is but there are simple tools to allow the conversion of data. One of the best is GPSBabel which allows many conversions from/to with options.

I see your point but would like to point out that a file, GPX, unicsv, other, that has *more* information than you want is easy to make useful where a file format that has *less* information than is needed is useless.

Someone, I forgot who right now, said:

The nice thing about standards are that you have so many to choose from.

I also prefer to have more information than less. However, we should consider the fact that many of those who create content for this site do not have the requisite knowledge yet to handle the more complex structures.

In order to get the most people involved, then it is a good idea to let a person create the most basic of files. others can then encourage the addition of information (perhaps by offering to do the work). This way, the site gets a new file, a member learns something, and I think everyone benefits. This might be described as a case of our site having more files with less than maximum information versus having less file containing every detail. I vote for more files.

If you have not tried EPE, then you are missing something truly remarkable. Go get it at http://turboccc.wikispaces.com/Extra_POI_Editor#toc5

Just install and run it. First thing, click the "Add New POI" button on the top left panel. Enter an address, like your house. Then click the "Geocoding" button to see the address on a map. Grab the red marker and move it to some intersection nearby - using the "x" crosshair for accuracy. Then click the "Grab" button on the top of the screen to obtain the lat/lon of that intersection.

If you were going to save this file, you would need to give the intersection a name and change the original address to something better, of course.

You might note that this is a great way to get a POI near your house so you can run 'simulations" to test out a new icon or sound file.

Not everyone limits themselves to Me$$yWindows machines.

I have no use for them. As such, anything posted specific to that operating system is just noise to me.

I prefer more than less.

Here is a point of view from myself as a fairly new used to all this technical GPS stuff....

Charlied has helped me so much more than I can ever thank in for in the past few months with my GPS. Im computer literate and know my way around, I wouldn't call myself a "basic" user! But having these files state by state can help the novice user to upload these files to their GPS! I have an Uncle that is older and does a bit of traveling, but he doesn't need to load all the "McDonlads" in the 50 states when he is traveling to 3 states. So basically 47 states are useless and waste of space! Another problem is not everyone has Excel so theirfore editing this files can't be done easily. The content and the people that put mass's amount of time in on this site are great, but sometimes it seems way too geared to "smarter" people. And some people on here are way to quick to pull the trigger and execute someone that isn't of the same knowlege caliber. And some of the programs to download to help you accomplish what you want to do, aren't always the most user friendly. I love files and are more likely to download them if they are state by state vs 50 states..... JMO, take it for what it's worth

Well said.

I joined this forum because I liked the poi's. I stayed because it has become a "new" home. I agree with a lot of what is being said. My experience is that there is a lot of info that can be put in the poi's. There are individuals that want very little and individuals that want it all. You can't please everyone. What I would like to see in the poi's is an address and phone number as absolute minimum. I don't consider a poi that has the name listed 3 times as an adeguate poi. An alert for a motel is a nuisance to me (others would prefer it). There are new members that are trying to create poi's. As members we need to help them to at least get the basic info included.
I have had to extract state files from motel listings and then update just my state because the poi is lacking in many areas. Contacting the maintainer to offer advice and assistance to correct and update the file met with either a "no response" or a "I will look at it".
We all need to be aware of all of the various degrees of expertice of the members. Some have very little knowledge while others are writing there own code.

We are getting away from individual state files because it causes too much work for the file maintainers. You should only have to update one file not 50.

Miss POI

I can agree with both sides, but I also understand where Miss POI is also coming from. From her standpoint, the work to put one file on the server is virtually the same, no matter the size of the file. If a combined file has 30 times the size of a single state file, the workload is still the same for our webmistress. She just doesn't have to repeat the process 49 more times.

For the end user it can mean extra work using either a spreadsheet or text editor program for CSV files, but the number of users concerned about having extraneous locations is quite small in comparison. (Now, I'm one of those that do edit my files and strip out locations and information I don't feel are necessary for my use. But then again, I'm proficient in using tools that will manipulate both CSV and GPX files.)

I do not understand the reasons for asking for a "state" file.

If the request is driven by concern for space on the GPS, my question then becomes why? How much space is left on the GPS? Do you know? What is the difference in space for 50 states versus 1 state?

As a newcomer, I want to create a file to share. I am thinking of the Days Inn, something I will use a few times. I want to at least include a city and a phone number in column D. The project is too big for me to tackle, as is. If I just uploaded one state, I could gain experience on a file of manageable size. Edit: Space is not an issue

I phrased my question poorly. I was interested in why someone would want POI-factory to maintain state files.

i think what you are doing is a good way to learn about POI files - it is managable, as you say.

Maybe you could take on a task of creating a file of twenty-four hour pharmacies in your area. The drugstore chains do a good job of giving you locations with certain attributes within at least a 50 mile radius. To cover the area, you would include Walgreens, CVS, Rite-Aid, to name a few. You would get experience and others in your area would get a useful file. Hope you will consider it.

I am only speaking for myself on this issue. I would prefer state files. I do understand Miss POI's problem, state files require 50 times more work on her part than national files. As for the maintainer keeping the files up to date, state files should be easier as there are fewer entries to sort through.

I travel in a three state area for the most part. I am interested in Ohio (where I live), Indiana (where my sister lives) and Michigan (some really pretty country). I am not interested in the other 47 states. If I keep 20 or 30 POI files on my GPS, an extra 47 states will amount to a LOT more data for me to download, store backups of and keep on my GPS.

I'm retired so editing the files to pull those three state's data out is possible for me. It would be time consuming and I can understand why someone who works full time might decide that it isn't worth his or her time. In fact, I might decide to do without instead of investing the time and effort.

But this site belongs to Miss POI, not us. So her decision is law here and I do understand her problem.

By the way, if you folks don't have Microsoft Office or the stand alone version of Excel, there is a very good free substitute. Do a search for "Open Office", download it and install it. It will open and save MS Office files and it looks and feels a lot like the MS Office programs.