Forum: WHALER
  ContinuousWave
  Whaler
  Moderated Discussion Areas
  ContinuousWave: Small Boat Electrical
  HDS: Export Waypoints, Routes, and Trails

Post New Topic  Post Reply
search | FAQ | profile | register | author help

Author Topic:   HDS: Export Waypoints, Routes, and Trails
jimh posted 05-02-2012 09:16 AM ET (US)   Profile for jimh   Send Email to jimh  
In the HDS operating system, in the PAGE selection, there is a UTILITIES listing with an option called "Files". The Files page permits file manipulation and export of data. The user can choose to export the stored data "Waypoints, Routes, and Trails." The usual method is to save the data to a file, however there is another option permitting export of the data to a serial port.

I could not find any explanation of how this worked in the Lowrance literature, so I investigated the process. My findings are included in my hypertext document on the HDS at

http://continuouswave.com/whaler/reference/HDS_Menu.html#FILES

For the technical curious, I will give more details of the investigation process.

To see what data the HDS would output to its serial ports in the export process, I attached my laptop to one of the HDS serial ports and monitored the data. Since the Apple MacBook Pro laptop does not have a serial port, I used my Keyspan HS-19 Serial-to-USB adaptor to provide the serial port. I connected the Keyspan to the HDS using my Universal NMEA-0183 interface method, previously described in

http://continuouswave.com/whaler/reference/UniversalNMEA0183.html

To see the data coming from the serial port, I used the Mac OS terminal application. In a terminal window, I ran the UNIX program screen. Before running screen I first found the serial port device in the terminal:

bigDog:~ jimh$ ls /dev/tty.*
/dev/tty.Bluetooth-Modem /dev/tty.KeySerial1 /dev/tty.USA19H1a2P1.1
/dev/tty.Bluetooth-PDA-Sync /dev/tty.Moto-Dial-UpNetworkingG-1

The HS-19 shows up as "/dev/tty.KeySerial1". I knew the baud rate would be 4800 coming from the HDS-8. Now I called screen:

bigDog:~ jimh$ screen /dev/tty.KeySerial1 4800

With screen running, I typed an escape command sequence into the screen window to initiate logging the screen data to a file. The escape command is C-a H (i.e., Control-A followed by Shift H). This initiates logging of the screen to a file screenlog.0 in your user home directory.

Then on the HDS-8, I highlighted the option for NMEA0183 and hit ENTER. The HDS-8 began to send its Waypoints, Routes, and Trials to its serial ports. To get a clean capture of the data, I turned off all other data being sent. The normal configuration for my HDS-8 is to send GPS data on the serial ports (to the DSC radio that is normally attached).

The HDS sent about 1,300 lines of data to its serial ports. When the HDS was finished, I sent the C-a H command to screen to turn off the logging to the log file, then exited screen (with c-a c-\, i.e. Control-A Control-backslash). The data was sent in a format resembling the NMEA $GPWPL and $GPRTE, but the identifiers used were $CPWPL and $CPRTE. I presume that $CP probably is intended to mean "chart plotter" as the source instead of $GP for "global positioning (device)" as the source.

The exported data did not contain any trail information, as I had earlier just deleted all the trail points from the HDS. There were over 30,000 trail points, so it is perhaps a good outcome that I did not try to export them.

jimh posted 05-02-2012 09:20 AM ET (US)     Profile for jimh  Send Email to jimh     
Here is an excerpt showing the data and its format. First a few waypoints:

$CPWPL,4320.4341,N,08226.7870,W,REGINA W*3D
$CPWPL,4320.4597,N,08226.8978,W,REGINA *41
$CPWPL,4323.9542,N,08226.5241,W,NORTH ST*35
$CPWPL,4316.0082,N,08227.8918,W,SPORT *3A
$CPWPL,4325.8402,N,08231.8710,W,P0RT SAN*4F
$CPWPL,4315.7561,N,08230.6083,W,LEXINGTO*55

Notice that the waypoint names are truncated to eight characters. Next some routes:

$CPRTE,5,1,c,POINTE A,REC333 ,REC334 ,REC335 ,REC336 ,REC337 ,REC338 *54
$CPRTE,5,2,c,POINTE A,REC339 ,SUBM12 ,SUBM13 ,REC340 ,REC341 ,REC342 ,30.021 *62
$CPRTE,5,3,c,POINTE A,REC343 ,REC344 ,REC345 ,REC346 ,REC347 ,12.029 ,REC348 *6C
$CPRTE,5,4,c,POINTE A,REC349 ,6.0F14 ,REC349 ,REC348 ,12.029 ,REC347 ,REC346 *61
$CPRTE,5,5,c,POINTE A,REC345 ,REC344 ,REC343 ,REC350 ,REC351 ,SEEBUO ,RECO33 *07

The route information appears to rely on the route points being provided in the waypoints data. For example, the above route references a route point "REC348." Sure enough, in the waypoint export there is a listing for that reference:

$CPWPL,4535.6807,N,08023.1368,W,REC348  *3D

jimh posted 05-02-2012 09:28 AM ET (US)     Profile for jimh  Send Email to jimh     
For reference, here are some useful pointers to more information about using MacOS to run a serial port with screen, details of how to capture the data in screen to a log file, and the data format of NMEA $GPWPL and $GPRTE.

Serial Communication in OSX Terminal
http://www.tigoe.net/pcomp/resources/archives/avr/000749.shtml

screen(1) Mac OS X Manual Page
https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ ManPages/man1/screen.1.html

NMEA Sentence Information
http://home.mira.net/~gnb/gps/nmea.html#gpwpl
http://home.mira.net/~gnb/gps/nmea.html#gprte

jimh posted 05-02-2012 09:32 AM ET (US)     Profile for jimh  Send Email to jimh     
By the way, the reason I wrote this article was so that two weeks from now I could remember how I did this.
jimh posted 04-16-2013 10:01 AM ET (US)     Profile for jimh  Send Email to jimh     
Exporting the geo-position data from an HDS (or other device) via a serial data connection and sending the data in NMEA-0183 sentences can be a useful method to export data from devices which do not have memory card slots. When the data is sent from the device via its serial port, it can be captured as a text file by an attached computer. The captured text file will contain the data in NMEA format, as shown above.
jimh posted 04-16-2013 04:44 PM ET (US)     Profile for jimh  Send Email to jimh     
Digging into the notion of exporting data from a Lowrance chart plotter using the NMEA port and having the waypoint data sent as NMEA-0183 sentences, I have found a bit of a snag in the stream. There is a problem.

The data sent on the NMEA-0183 port looks like this (as I have shown above)

$CPWPL,4320.4341,N,08226.7870,W,REGINA W*3D

There is a big problem with this data: it is not in a standard NMEA-0183 sentence for a WAYPOINT. The first problem is the start of the sentence,

$CPWPL

which in standard form should be $GPWPL. I though I might be able to just edit the file and change $CPWPL to $GPWPL and make things work. That didn't fix the problem; there was more that was wrong. It just happens that the Lowrance sentence gives too much data for the latitude and longitude values, using four decimal places. The NMEA-0183 sentence only goes to three decimal places. That means the values in the exported data have to be truncated or rounded. For example:

4320.4341,N,08226.7870,W

becomes

4320.434,N,08226.787,W

Now the original exported data from the HDS can be seen in standard NMEA format:

$GPWPL,4320.434,N,08226.787,W,REGINA W*3D

There is one more problem: the checksum (or last two fields) will be wrong. You cannot send this data to another NMEA device and have it work. The checksum that is sent would indicate an error. However, you can convert this data to .GPX using GPSBABEL. Once in .GPX you can convert back to NMEA, if desired. Here is how that process looks:

Edited NMEA from HDS:

$GPWPL,4320.434,N,08226.787,W,REGINA W*3D

lop off the checksum, which is not need by GPSBABEL for conversion, apparently, and you have

$GPWPL,4320.434,N,08226.787,W,REGINA W

Use that as a file input to GPSBABEL and export to GPX. That produces a file like this:

<?xml version="1.0" encoding="UTF-8"?>
<gpx
version="1.0"
creator="GPSBabel - http://www.gpsbabel.org"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.topografix.com/GPX/1/0"
xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
<time>2013-04-16T20:24:30Z</time>
<bounds minlat="43.340566667" minlon="-82.446450000" maxlat="43.340566667" maxlon="-82.446450000"/>
<wpt lat="43.340566667" lon="-82.446450000">
<name>REGINA W</name>
<cmt>REGINA W</cmt>
<desc>REGINA W</desc>
</wpt>
</gpx>

Then use that file as the input for GPSBABEL and convert back to NMEA:

$GPWPL,4320.434,N,08226.787,W,REGN W*30

This export-->convert--->reconvert process also gives a good check on the overall accuracy of the entire process. We are back with the same NMEA code as we started with, although the description field became shorter and changed slightly.

I was hoping that this method could be used to get data off older Lowrance chart plotters that did not have a memory card slot and could not export the data to a file. But the format in which the Lowrance chart plotters seem to be sending the data on the NMEA port is not quite the proper one for NMEA waypoint data. As a result, GPSBABEL is not able to convert the data to a file. You have to (rather tediously) edit the exported NMEA data when it is saved as a file. You have to change the first field to $GPWPL, which could be done with a find and replace edit, but the other changes are not as easy. You need to truncate or round the position latitude and longitude to one less decimal place, and that will be hard to do with a text editor.

The best method might be to connect the two devices, if they are Lowrance devices and both can read the odd format for waypoints, and interface their NMEA-0183 serial ports. Send the data from one device; the other one ought to be listening for it and add it to its library of waypoints automatically. I will have to test that step next.

jimh posted 04-16-2013 10:37 PM ET (US)     Profile for jimh  Send Email to jimh     
The format for the $GPWPL sentence is to give the latitude and longitude in degrees and decimal minutes, so we have

DDMM.MMM for latitude, and
DDDMM.MMM for longitude

This gives the location to 0.001-minute of precision, which is to a precision of six-feet in latitude. If the location were given to 0.0001-minute of precision, that would be a precision of about seven inches.

On some further testing, I found that GPSBABEL can handle NMEA-0183 sentences in which the position is given to four or even five decimal places. On import, GPSBABEL does not seem to care if the NMEA-0183 data field has more than three digits in the decimal minutes. This means you could use the data from the HDS as it comes, with four decimal places of precision, and the only element to be changed would be the initial sentence identifier, $CPWPL changing to $GPWPL. If this is done, the checksum has to be deleted. Apparently GPSBABEL ignores the NMEA data if the checksum field is wrong. Of course, the checksum is wrong if you change the header. Curiously, if there is no checksum field at all, GPSBABEL will convert the data, even if there are too many digits after the decimal point. This means that the raw HDS data could be used as long as the $CPWPL heading is changed to $GPWPL and the trailing checksum values, that is, the asterisk and the two digits that follow it, are removed.

When exporting to NMEA-0183, GPSBABEL seems to always export to the standard three places of decimal minute precision.

jimh posted 04-17-2013 11:04 AM ET (US)     Profile for jimh  Send Email to jimh     
With a good text editor, it is simple to replace the header

$CPWPL,

with the proper header

$GPWPL,

but stripping the checksum will require using a regular expression search. You can delete the last three characters of every line, which will be the asterisk and the two checksum digits, using the regular expression search

...$

and replacing those instances with nothing. This will enable a long list of waypoints from the HDS in its curious NMEA code to be converted to a format that GPSBABEL can use as an input, and from GPSBABEL you can output the waypoints in any number of other formats.

jimh posted 04-19-2013 09:26 AM ET (US)     Profile for jimh  Send Email to jimh     
A better regular expression to identify the checksum for an editor to remove is

\*[0-9A-F][0-9A-F]

jimh posted 04-19-2013 09:46 AM ET (US)     Profile for jimh  Send Email to jimh     
Re the regular expression used above, the explanation of its form is as follows:

The "\" is an escape character which means the following character is to be interpreted as a literal character, not a meta-character.

The "*" or asterisk is the character we want to find; it marks the beginning of the checksum data in the NMEA-0183 sentence.

The "[0-9A-F]" indicates that the next character is found by the definition of the expresssion inside the square brackets. Since we know that the checksum is a hexadecimal number, we know that it will consists of some combination of the digits zero through nine and the capital letters A through F. That is what the expression inside the square brackets defines. (Also, this expression is a shorter way of writing [0123456789ABCDEF].) We know there are two characters like this, so the second "[0-9A-F]" expression says to look for a second one.

The total expression says to look for an asterisk and two following characters which are some combination of zero through nine or capital A through capital F. That is a very good definition of what the checksum is going to look like, and this helps to avoid accidentally deleting some other characters in the text that might unexpectedly meet other, less stringent, descriptions.

(Again, I am writing this so I will remember what I did two weeks from now.)

Post New Topic  Post Reply
Hop to:


Contact Us | RETURN to ContinuousWave Top Page

Powered by: Ultimate Bulletin Board, Freeware Version 2000
Purchase our Licensed Version- which adds many more features!
© Infopop Corporation (formerly Madrona Park, Inc.), 1998 - 2000.