new & interesting places for your GPS

CSV vs GPX formats, with questions

 

Ok, to start with, I've been doing software dev for 30 yrs, so I understand these formats. And I write small personal conversion apps to play with conversions between the formats. My personal preference, due to portability, editability, etc, is GPX, but I've noticed some things while converting files and seeing how the Nuvi handles them.

When using CSV files, you basically get NAME and COMMENT fields. They both support line breaks, which allows for greater formatting of the data. If you line break the name and comment fields, output looks something like this:

Company
Branch
Address
City, ST 77777
555-555-5555

In the POI display list, the Nuvi will combine the Company/Branch, which helps in identification. However, the built-in POI's show up differently in the display list. They incorporate a second line which shows the address in a smaller font. This is a very desirable feature.

If I use the GPX format, the Nuvi will display the NAME and ADDRESS information just like built-in POI's, so that's another reason to use this format. But in the case of a file like Starbucks, for instance, the NAME's will be changed in the Nuvi as follows:

Starbucks
Starbucks1
Starbucks2
etc

This is because the GPX format requires a unique name for the waypoint. This makes this format unusable for non-unique waypoints. I could append some other details to the name to make it unique, but I've read there's a 39 char limit, and that might make for odd situations. The built-in POIs for Starbucks don't have this limitation, so I'm trying to find out how to model my files so that the output in the display list looks like that.

Is there a set of GPX tags and/or extensions that can do that? The list of Favorites doesn't allow this rule to be broken, so I'm guessing not. With that in mind, is there a way to use the CSV format to get the address to show along with the waypoint in the list?

Thanks!
David

--
nüvi 200

maybe the previous post was

maybe the previous post was too wordy confused My wife always says she just wanted to know the time, not how to build the clock.

So, is there any way, with either CSV or GPX, to have a list of POIs displayed that have the same name, but uniquely identified by address or some other field?

This doesn't appear to be supported by GPX. With that in mind, how can I use a CSV file and have the address show up in the display list?

--
nüvi 200

Fascinating

That's a fascinating insight. I just tried it myself and you're right - duplicate names in GPX files are changed, but CSV files are left alone.

It looks to me like it first tries removing all the spaces. If I have "Test GPX" and "Test GPX" in a GPX file, the second one gets changed to "TestGPX". But if I have two labeled "TestGPX" the second one is changed to "TestGPX1".

By looking at the GPI file, I can see that it's the POI Loader doing it - not the Nuvi. The strings in the GPI file have already been changed.

This has got to be a bug in Garmin POI Loader. I knew the Garmins do this with Favorites so each has a unique name, and the favorites are added via GPX. My guess is that they have some boilerplate code for reading GPX, and it automatically does the disambiguation.

Just for grins, I changed the "1" at the end of "TestGPX1" to a space in the GPI file (using a hex editor), and that worked, in the sense that the two names display the same on the unit. In the GPI file it looks to me like Garmin is using Pascal-style strings, with a 16-bit size followed by ANSI text. They're not zero-terminated. So shortening them by stuffing in a zero is not possible.

So I suppose one of us could hack something up like this:

- Add all the names that should be the same but append a number, surrounded by a unique identifier unlikely to be present in any string, like "####111%%%%"

- Write a utility that converts the text in the GPI file that starts "####" and ends "%%%%" to spaces. There might even be some kind of binary grep-style utility that would do this already.

The alternative, it seems to me, is to write a new GPI generator, which sounds like a lot of work.

In any case, someone should report this as a bug to Garmin. Something tells me it's not going to be high on the list to fix, but it's worth sending the email anyway.

As for CSV, it's just limited. You can have name and description, and that's it. No phone number, address, special formatting, etc. GPX is much more flexible.

Nice catch!

Don

A hack

I tried my hack, and it seems to work.

I found a search and replace program that will search and replace binary files. It warns you that if you do replacements in binary files it can mess up checksums, but apparently GPI files don't have any checksums.

This is the link

http://www.inforapid.de/html/srdownload.htm

It's free for personal use.

So I created some test POIs with an appended string "QQQ0001ZZZ" on one and "QQQ0002ZZZ" on the other. Then I searched in POI.GPI for "QQQ[:digit:][:digit:][:digit:][:digit:]ZZZ" and replaced it with 10 spaces.

Presto - duplicate names, generated from GPX, with addresses and dialable phone numbers.

Not exactly the most user-friendly approach, and having 10 spaces at the end of the name is inelegant, but it's nice to know it works.

Probably the string could be shorter. Even "Q0001Z" is likely to be unique, or something like "$0001" at the end of the name.

It's just a proof of concept. And obviously, the real answer is to get POI Loader fixed.

I'll try sending a mail to tech support and see if it makes it to an engineer.

Don

I've started going down the

I've started going down the same path with name extensions, but this is great info.

You mention the GPI file. So does that mean you add the extension, download to the unit, and then modify the resulting GPI file?

--
nüvi 200

Correct

Yes, you go ahead and let POI Loader create the GPI file, then copy the POI.GPI file back to the computer, and do the search-and-replace on it. Then copy the modified file back onto the Garmin. You may want to change the name of the GPI file so it won't get deleted next time you load more POIs onto the unit, but that's optional.

GPI is a binary file, so doing a text search and replace is at least slightly dangerous. As long as you ensure that the string you're searching for is unusual enough that it's unlikely to show up accidentally, it should work OK.

Don

A word of caution...

dmunsil wrote:

GPI is a binary file, so doing a text search and replace is at least slightly dangerous.

I would be inclined to test a modified .gpi file on an SD card first.

Corrupt/Unreadable/Unusual! POIs are one of the causes of the infamous "Loading maps..." hang.
(Which you can't get out of by any normal means sad )

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

And I thought I was geeky

And I thought I was geeky for doing my POI stuff with emacs instead of excel smile

My issue with GPX vs CSV is that if you want a description... i.e. text not aligned with a formal 'field', it doesn't show (unless I did something wrong). There is a field in GPX, but after a test load to my nuvi 680, none of the info in the description were visible.

So that said, I think I actually prefer the csv version for heavily descriptive stuff.

I usually convert my csv to

I usually convert my csv to gpx and edit my gpx in mapsource.

!

Slythex wrote:

And I thought I was geeky for doing my POI stuff with emacs instead of excel smile

Geeky? ...no. Masochistic? ... Yes! wink

and he wrote:

My issue with GPX vs CSV is that if you want a description... it doesn't show (unless I did something wrong).

No, nothing wrong Grasshopper - you're just not fully conversant in the ways of 'The Garmin'! But, it can be done...

he additionally wrote:

There is a field in GPX, but after a test load to my nuvi 680, none of the info in the description were visible.

Have look at the .gpx files that GeePeeEx Editor creates for the Nüvi 6xx series.

then, in a moment of madness, he wrote:

So that said, I think I actually prefer the csv version for heavily descriptive stuff.

All that happens with csv input, is that POI Loader populates the description field and nothing else. In the absence of any address/contact information, this gets displayed on Page 1 of your Nüvi, instead of Page 2 where it rightfully belongs.

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

Which is fine for unique

Which is fine for unique waypoints. But the case I'm presenting here is for a list such as Starbucks, or McDonalds. Unless you have some means of uniquely identifying them, you can't use GPX (or MapSource) as the POIs will get #'s appended to them.

This doesn't happen in CSV files, but they have the drawback of NOT showing address info on the second line in the POI display list.

Even the .gpi solution has the drawback (as mentioned) that POIs will have trailing spaces. And, as pointed out, it's rarely a good practice to modify a "results" file smile

--
nüvi 200

Good advice

Hornbyp wrote:

I would be inclined to test a modified .gpi file on an SD card first.

Corrupt/Unreadable/Unusual! POIs are one of the causes of the infamous "Loading maps..." hang.
(Which you can't get out of by any normal means sad )

That's really good advice. I vaguely thought, "I'll just copy over this modified GPI file, and if it doesn't work, I can just delete it." But of course, I can't delete it if the thing's wedged.

From now on, all my tests will be on SD cards.

Phil, do you happen to have any contacts at Garmin? Have you had any luck reporting bugs? I noticed your posts on the Garmin Developers forum saying that no one from Garmin seems to actually be involved there.

And many thanks for building GeePeeEx. I ended up writing my own code for generating the GPX files because I had some special needs, but I did look at the GeePeeEx output to get some ideas.

...I think I owe you $18. smile

Don

.

dmunsil wrote:

I vaguely thought, "I'll just copy over this modified GPI file, and if it doesn't work, I can just delete it." But of course, I can't delete it if the thing's wedged.

Isn't it peculiar that you can't just fire it up in mass-storage-mode, without it doing the full blown initialisation?

One thing that might interest you: there is now a GPI reading/writing module for gpsbabel. It's not complete - but the source code is available. You might be able to figure out more about the .gpi structure from it.

and he wrote:

Phil, do you happen to have any contacts at Garmin? Have you had any luck reporting bugs?

I've got email addresses for some people - but I don't know how influential they are. I have no problems reporting bugs! - but it seems very haphazard as to when or if they get fixed...
...the support team in the UK (sorry, I mean Garmin Europe!) are very friendly and helpful, but how much they know (or are allowed to know!) is open for debate.

(I once had the misfortune to work for the UK arm of an Australian company - so been there, got the T-shirt sad )

--
------------------------ Phil Hornby, Stockport, England ----------------------               http://GeePeeEx.com - Garmin POI Creation made easy           »      

desc field

Quote:

Have look at the .gpx files that GeePeeEx Editor creates for the Nüvi 6xx series.

That's exactly what I did. Converted everything manually, which didn't load right. Did a single waypoint conversion with GeePeeExx to see if I missed something and to compare. The single waypoint .gpx file:
a- it loaded, but never saw the desc field
b- I wasn't easily able to find out where I screwed up in my full POI file (probably the equivalent of missing a semicolon or mismatched parens or somethign)

I may play with it more when I get around to it. Looks like there's some cool stuff in the gpx spec, although not sure how relevant some are for the typical car usage (most seem to be for geocaching and the like).

sponsored links