Data Error in current OFFBEAT TOURIST ATTRACTIONS

See the discussion here:
http://www.poi-factory.com/node/32912

Don't quite understand what you mean....

Anyway... I opened up the file and line 6231 has partial data in the file... but it is not a complete line...
Haven't looked at this before... so I spent some time looking at other CSV files... and they all end without a partially filled line...

So.....

I edited that file, removing the partial data from line 6231, Saved it as a new file, then deleted the file in question and re-named the edited file to the name I use for the 3 files: offbeat

With this fix.... it worked... No error messages with this file and POI Loader V2.5.4...

Now I don't know if there was more data needed in this file, and can't compare to prior versions because I deleted the prior version file...

But......

That's how I fixed the downloaded file....

(Made it look like other CSV files)

!

The fix at that time was to download the gpx file and convert it to a csv but looks like the gpx and csv file was updated since I last looked at it.

So now there is a different error.

Maybe downloading the gpx and converting it will still work with the new error.

I've never tried using or converting a GPX file...

I don't know the total lines in prior version (do you have it?)... I'd like to know if that was less than 6131..... ???

Why not just load the .gpx file with POILoader if it works and the CSV file doesn't?

The number of lines was discussed in the link above:

I have downloaded the new gpx file and will convert it with EPE. Will let you know what I find in a few minutes.

Note that the csv file is the one readily seen. One has to scroll down to find the gpx version and it is called the "Raw data".

Apparently because of that most people are downloading the csv file.

There have been more & more occurrences of truncated csv files on the site. The first that I'm aware of was with my American Legion file about 1 1/2 months ago (still not fixed).

The practical solution is to use the gpx version when it exists. POILoader reads & compiles them the same as it does a csv file.

If for some reason you just can't live with gpx files, Extra Poi Editor will read a gpx file and give you the option to save it as a csv.

Ok I converted the gpx file and the line count is 6235

The count stated is 6234 (http://www.poi-factory.com/node/2661)

POI Loader 2.6.1 successfully installed all 6235 using my csv.

Wrong! downloaded CSV file still at 6231, and that line is still a partial line

I am talking about the csv file I converted, not the one on the site. I am not the maintainer of any files on the site.

I was trying to answer your question about how many lines there should be.

The Site Has Issues

There have been more & more occurrences of truncated csv files on the site. The first that I'm aware of was with my American Legion file about 1 1/2 months ago (still not fixed).

The practical solution is to use the gpx version when it exists. POILoader reads & compiles them the same as it does a csv file.

If for some reason you just can't live with gpx files, Extra Poi Editor will read a gpx file and give you the option to save it as a csv.

(Yes, I've emailed Jon & MaryAnn)

I understood fully what you said. I was just trying to continue my discussion with the OP, to answer his questions, and to clarify which csv I was talking about.

BTW If we cannot trust the csv file, why should we trust the gpx file?

OK I understand... The GPX file shows more lines than the CSV file does....
Is this for everyone (site issue?) or me (Download issue?)
I'm using Firefox 4.

Didn't you see JD4x4's two posts? "The files are truncated on the site."

Sorry JD, maybe you need to post that a third time.

..At least in the case of some files it's a site problem that affects everyone (I think!).

That's certainly the case with my American Legion file & it appears to me to be the case with the Offbeat file as well.

There was some discussion that FireFox was causing the problems, but I ruled that out by getting the same upload/download/truncated file problem with IE, FireFox 3x, Firefox4x.

The American Legion CSV file as downloaded from this site has 2 lines with problems.
The line entry with Long -87.740894 Lat 43 has no entry in col's 3 and 4.
Col 2 should be: Lat 43.75828
Col 3 should be: Am. Leg. WI Post 555
Col 4 should be: 1230 N. 26th St. Sheboygan WI 53081 Phone: Sheboygan Memorial

The line entry with Long -102.049166 Lat 48.692732 has a line break beginning in Col 4 (). If you remove the line break and any resulting leading spaces, you should be okay. These 2 corrections should make the CSV file usable.

Personnaly, I double check every file I download just to be safe. All these people supporting this site do one heck of a job and I appreciate their efforts. My hat is off to each and every one of them for their efforts as well as Miss POI. A great site!

FYI: If you download the GPX file and save it as a 4-column CSV file, you will find the second example above has the error in column 4 showing a leading vertical bar (piping symbol "|") instead of the break (). Again, remove that symbol and delete the leading space(s) and you should have a usable file.
FYI-2: Often times simply cliking on a CSV file to load it into Excel causes errors. Try opening Excel with a blank sheet. Then use the Data tab at the top to Import the file as a comma delimited file. You will often find a better setup load this way.
Hope this helped you with the American Legion file.

Can someone please send me the corrected file so I can get it up on the site?

Miss POI

Thanks, & check your mail.

It's all there in the original uploaded csv.

Nope. The break is there for unit display formatting, and the original csv works fine with POILoader.

I check them on download as well as before upload. The extra pipe chr likely slipped in when I decided to replace them with <br> because I thought it more universal. But, both originals give the desired effect on all my units, with no issues after compiling.

Thanks, but I'm hoping MaryAnn & Jon let me (and us?) know what it turns out to be for sure.

The maintainer sent me an updated file. Can you guys check and make sure that it is fixed now?

http://www.poi-factory.com/node/21753

Miss POI

Still truncates on line 6064 for me.

Same here, line 6064 shows only the coordinates: -87.740894 43

Ditto

I'm seeing the same thing with this American Legion file that I found this morning with the Offbeat Tourist Attraction file....

part of a line (in the case of the Legion file, line number 6064, in the case of the Offbeat Tourist file, line 6231) has data...

Now as a point of reference, this morning I also downloaded a new file Ford dealers US as a CSV file.... and that file end at line 2047.. and I had no issues with that file...

Maybe it has something to do with filesize?

Again, I'm using Firefox 4

I just Logged in here with IE9, instead of Firefox 4, and downloaded the American Legion file... Still ends at 6064, as it did with Firefox 4.

And later tried same thing with Offbeat Tourist file...
That, too is still ending at 6231 with partial line

So it must not be a Firefox thing...

I use Opera 11, definitely not a browser issue.

I use Google Chrome and the file is still truncated at line 6064.

See if it is fixed now, Trippin08 has sent me another try at fixing it.

Miss POI

No, still truncates at line 6064.

And, although line 6064 now has a column 3 & 4, it's out of order compared to my original so I have no idea which 95 records are missing.

Once this file is corrected, don't forget about Offbeat Tourist attractions! I'll feel bad if I hijacked this thread only to get my file fixed & not the other!

JD says it's still broke

I wonder if this problem might be tied to the download filter that JM installed in late January to handle the CSV file parsing problems introduced in POI Loader 2.6.0? Is that filter still in place? If so it might be something for JM to look into. Here is a Jan. 27, 2011 quote from JM from another thread:

I wondered that myself. When looking at the downloaded AmLeg file I see that col 4 entries are enclosed in double quotes, but they are not in my original uploaded csv (which compiles in POILoader v2.6.1 without errors). Also, I never did find the pipe character that trippin08 said was in the file. Not in the original or any of my truncated downloads. ??

Time to get rid of it (the parser) and just tell everyone to use anything except POILoader v2.6.0?

Try deleting the offending line(s) and retyping it/them. Sometimes screwed up special characters cause problems. For instance, a double-quote may appear to be correct but it really has some weird ascii code. If If that's the problem, if you delete the entire line and retype it correctly, that'll fix it.

I just downloaded it on poi loader 2.5.4 and it shows the same error.

Miss poi:
Examining the American Legion file now is ok, but the Offbeat Tourist Attraction file still has a partial line at line 6231
Jan

I had a 3 or 4 csv files in January that were being truncated during the POI Loader 2.6.0 error time period. I was going to post on them but was reading the posts already going back and forth on them and figured it was being handled and would eventually be figured out. I just deleted the partial files and was going to get them later. I also think this is still happening with some files.

Not sure which files they were though, I was just scanning POI files and looking for subjects that peaked my interest.

There may be no download or POILoader error, but the American Legion file is still not complete. There are 6159 entries in the original upload, just as there are in the GPX version. The csv download is still truncated at 6064 entries.

And yes, I think there are more truncated csv files but this is the only one I can comment on in depth, as I'm the maintainer. Hopefully once the problem with this one is sorted out the rest can be fixed.

Sorry about that.... By "OK" I mean that there was no partial line entered on the last line... I had no way of determining it was truncated.
Is there a way of determining how many lines is supposed to be in a file, for if we knew that, we could easily spot a truncated file...

Jan

Just below the download button(s) on each page-
Includes xxxx locations in the following areas:
Where xxxx is the number of POI's/Lines

I wasn't sure if that information was relevant to the CSV files or not... Thanks for that info...

I'm not sure of this but I believe that line count is computed from the downloaded file. It that file is truncated then the line count would be for the number of good lines.

The AmLeg file isn't a good one to look at, but the "Includes xxx.." figure DOES indicate the number of pois in the file as on the server.

Have a look at the Offbeat Attractions page- "Includes 6234 locations.." and the downloaded file yields 6231 lines, the last of which is truncated, for a usable total of 6230 lines/pois.

The Am Leg page previously correctly reported 6159 entries. At some point Miss POI and trippin08 'fixed' the broken download file (6064 entries) by replacing it with a shortened file.

The complete, correct original csv file (which I emailed to Miss POI) has 6159 lines as does the gpx version.

I've had only 1 email from Miss POI or Jon and I'm assuming they are otherwise engaged in something important. I'm also assuming this is a temporary fix of some sort.

Thanks JD4x4, I stand corrected. I was thinking that the line count was showing the same number of lines as the server was sending out. But the problem must be between the file server and the downloading user. That's what I get for assuming!

On my end i do not see any issues with both files, I even loaded both to the GPS with no issues,,

not sure what is going on. i looked at all the lines and no problems so not sure what is going on..

Bob

I am thinking this may be the right thing to do.

I introduced a quick fix back in late January to alter CSV files as they were downloaded in order to work around the severe issues in POI Loader v2.6.0, which many new visitors were starting to use.

http://www.poi-factory.com/node/31969?page=2#comment-226201

Now that 2.6.1 has been out for a while, going back to sending the original files without fixes should eliminate a lot of these odd truncation issues.

I will look into this.

JM

Sounds like a good plan to me. And heck, if it makes things worse.. blame it on me, I'm ok with that.

You never get blamed for trying to move forward!

I don't think POILOADER version has anything to do with it...

I just downloaded the May 4th Offbeat Tourist attractions file, using Firefox 4.0.1 and immediately opened it in excel...

The file still ends with a partial line at line 6231.

All I did was just download it... and then open it in Excel...

I just tried IE9 64 bit... and it did the same thing...