It’s not difficult to convert coordinates

Every year, the speck season kicks off right in the bays just out from Empire and Buras.

Capt. Paul:

I was wondering how to convert lat/lon coordinates. A source of information on current rig locations in the Gulf of Mexico list their locations in the following strange format as I have listed below. I would like to be able to convert into a ddd mm.mmm format that I use in my GPS unit.

The site that I have is for the Deepwater Nautilus. It is as follows:
Water Depth: 3128
Latitude: 28.1605779576543
Longitude: -89.2156562232999

What’s with all the numbers? I guess it doesn’t get much more precise than that. But what is it? How do I use the numbers? My Garmin GPS 76 didn’t like all of the extras places that are after the decimal point.

Can you help?

Capt. Paul’s Response:

The position you have is stated in degrees and 10-trillionth of a degree. It is 28º north latitude degrees and 89º west longitude degrees.

One degree of latitude equals about 60 nautical miles, so one 10-trillionth of a degree equals about .000000036455669183662 feet. This calibration is way, way beyond the capabilities of the normal consumer GPS receiver.

One of the features of the Garmin GPS 76 is the amount of position or location formats that are available. One of the lat/lon choices in the GPS 76 allows you to select a position format (location) in all degrees. The Garmin programming choice allows you to select 100-thousandth of a degree (H,DDD.ddddd). This format will indicate a change of 3.64566918 feet for every digit increase or decrease in the last place after the decimal point (.00001 to .00002). Most manufacturers of consumer GPS units list a specification for their units that is within 12 feet of accuracy with the WAAS feature activated. Garmin lists the accuracy of the GPS 76 as within 3 meters 95 percent typical with DGPS WAAS corrections.

Evidently Garmin feels that the h,DDD.ddddd format is sufficient for a consumer unit, as such a unit may not always be able to distinguish a value that is about one-tenth of the 3.64566918 feet (3.64566918 X 0.1 = .364566 feet or 4.374+ inches) difference in positions. I totally agree.

There are two ways that you can enter the published position format into your unit.

First, go to the MENU page and select SETUP, then highlight LOCATION, down scroll to the LOCATION FORMAT icon and press enter. Then scroll up or down until you reach the format of H,DDD.ddddd. Now enter your published coordinates rounding off to the fifth place after the decimal place.

After saving the position by highlighting the OK icon, go back to the Main Menu, select SETUP and LOCATION again, and change the format back to the position format that you normally use (h,DDD,MM.mmm).

The GPS unit will do the figuring for you and convert the all degree entry to one-thousandths of a minute (h,DDD,MM.mmm).

Another way is for you to do the math and make the conversions yourself. To do so, remember that there are 360 degrees in a circle, 60 minutes in a degree and 60 seconds in a minute.

To do the conversion, simply multiply the decimal part of the degrees by 60. You can then round off to three places after the decimal and enter that value into your GPS, using the h,DDD,MM.mmm format. So the latitude of 28.1605779576543º becomes about 28º 09.6346677459258’ N (.1605779576543 x 60 ) and the longitude of –89.2156562232999º becomes W 89º 12.939373397994’ (.2156562232999 x 60). Both ddd,mmm’s are rounded off as shown. Good luck in finding a calculator that can handle such large numbers. Most will accommodate only part of the decimal figure, but I wanted to show you how to carry out the 10-trillionth figures.

After getting your answer, round off the converted format to three places after the decimal place and enter in your unit as a H,DDD,MM.mmm position or location format. This format (h,DDD,mm.mmm) will amount to about a 6.2 feet change in latitude for each digit change in the third place after the decimal.

An important note: All U.S. Department of the Interior, Gulf of Mexico Mineral Management Services positions on the leased blocks, production platforms and wells are determined by using NAD 27 datum. If your position was a result of the official government registry, it probably will be listed using that datum.

So to take full advantage of the accuracy of your WAAS GPS unit, change the datum format to NAD27 (CONUS) before entering the position into your unit.

Once entered and SAVED, you can again go back to the datum that you normally use in the area. Your Garmin GPS 76 unit will again automatically do the datum conversion for you.

For a definitive account of how to determine an actual change in position, review my October 2002 column.

If you missed the column, you can access it via the Louisiana Sportsman web page by clicking on the Ask Capt. Paul GPS Info posting page.

The position in WGS datum that you asked about would convert to about 28º 10.150’ N. latitude/89º13.303’ W. longitude. It is in the Mississippi Canyon Leased Block #807, about 51.2 miles from the light at Southwest Pass.

Have a nice trip.

About Captain Paul Titus 192 Articles
Capt. Paul Titus has been responding to G.P.S questions on since 2000. He has been fishing and hunting in Louisiana since 1957. Titus holds a USCG license and conducts instruction courses in the use of GPS for private individuals and government agencies.