[gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

Casper Børgesen

Hi list!

 

I have just performed a test using gdal_translate to translate a GeoTIFF (referenced using a VRT file) to ASCII Grid, and I have stumpled upon an issue with precision.

 

This is my syntax:

 

gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=2 my_source.vrt my_target.asc

 

I have tried the above syntax using gdal 1.9.2 and 1.10 and these are examples from my results:

 

1.9.2:

3.77 3.83 3.89 3.87 3.79 3.49 3.03 2.50 1.96 1.56 1.29 0.95 0.77 0.54 0.53 0.41

 

1.10:

3.8 3.8 3.9 3.9 3.8 3.5 3 2.5 2 1.6 1.3 0.95 0.77 0.54 0.53 0.41

 

Has the meaning of DECIMAL_PRECISION changed from 1.9.2 to 1.10?

 

It seems like in 1.9.2 the meaning is X characters to the right of the comma and in 1.10 it means X describing characters?

 

I haven’t tried the 1.10.1 release because I can’t find the binaries on Tamas gisinternals.

 

 

Kind regards, Casper

 


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: [gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

peifer
Casper Børgesen wrote
This is my syntax:

gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=2 my_source.vrt my_target.asc

I have tried the above syntax using gdal 1.9.2 and 1.10 and these are examples from my results:

1.9.2:
...
3.77 3.83 3.89 3.87 3.79 3.49 3.03 2.50 1.96 1.56 1.29 0.95 0.77 0.54 0.53 0.41
...

1.10:
...
3.8 3.8 3.9 3.9 3.8 3.5 3 2.5 2 1.6 1.3 0.95 0.77 0.54 0.53 0.41
...

Has the meaning of DECIMAL_PRECISION changed from 1.9.2 to 1.10?

It seems like in 1.9.2 the meaning is X characters to the right of the comma and in 1.10 it means X describing characters?

I haven't tried the 1.10.1 release because I can't find the binaries on Tamas gisinternals.
Hmm. After a look into the source code [0] and some testing [1], it looks to me that DECIMAL_PRECISION is used as precision of a "%.<precision>g" format.

Format letter g means: "Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters."

For format letter g, the precision value defines: "Maximum number of *significant* digits."

I see there has been this change recently: http://trac.osgeo.org/gdal/changeset/25032/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp 

Hermann

[0] http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp#L1051

[1]

# Input file

$ cat in.asc
ncols 1
nrows 4
xllcorner 1
yllcorner 1
cellsize  1
0.00012345
1.12345678
22.1234567
333.123456

# Test run

$ for i in 1 2 3 4 5 ; do
  echo DECIMAL_PRECISION=$i &&
  gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=$i in.asc out.asc &&
  cat out.asc
done

# Output

DECIMAL_PRECISION=1

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001
 1
 2e+01
 3e+02

DECIMAL_PRECISION=2

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012
 1.1
 22
 3.3e+02

DECIMAL_PRECISION=3

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.000123
 1.12
 22.1
 333

DECIMAL_PRECISION=4

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001234
 1.123
 22.12
 333.1

DECIMAL_PRECISION=5

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012345
 1.1235
 22.123
 333.12
Reply | Threaded
Open this post in threaded view
|

Re: Change of DECIMAL_PRECISION in AAIGrid

Casper Børgesen
Hi Peifer.

Thank you for your comments. They do explain more in depth what is happening. But as seen in your first example, the value 333.123456 is reduced to 300 while the value 0.00012345 is reduced to 0.0001. I think this is a big change in the relative values of each number.

I am curious to know if the "new" implementation in GDAL 1.10 is actually what is wanted/expected?

I use the DECIMAL_PRECISION for rounding purposes to control how many digits I want to the right of the comma, not how many digits I want to use for the entire number.
I know I get a lot of zeros if I use DECIMAL_PRECISION=10 in GDAL 1.9.2 but I retain my precision.

In GDAL 1.10 I have to determine the number of digits to the left of the comma used by the largest number in my data set, add the number of digits I want to have to the right of the comma and use that value as DECIMAL_PRECISION. The result must be that I get a lot of (waste) digits to the right of the comma for small numbers and the expected result for large numbers. But isn't this similar to what the patch (mentioned ealier) wanted to avoid? :

> The AAIGrid float format string is "%6.20g"
> Firstly, the 6 doesn't really add anything, and when writing small integers it causes them to be space-padded, which is a waste of bytes.
> At present AAIGrid supports "-co DECIMAL_PRECISION=10" which changes the formatstring to "%.10f".
> This is a little inconsistent, and the %f causes integers to be written as "9.000000000" which is a horrible waste of bytes.

I could write my own GeoTIFF to AAIGrid exporter, but I don't believe that should be necessary :)


Kind regards, Casper


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of peifer
Sent: 2. september 2013 12:21
To: [hidden email]
Subject: Re: [gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

Casper Børgesen wrote

> This is my syntax:
>
> gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=2
> my_source.vrt my_target.asc
>
> I have tried the above syntax using gdal 1.9.2 and 1.10 and these are
> examples from my results:
>
> 1.9.2:
> ...
> 3.77 3.83 3.89 3.87 3.79 3.49 3.03 2.50 1.96 1.56 1.29 0.95 0.77 0.54
> 0.53
> 0.41
> ...
>
> 1.10:
> ...
> 3.8 3.8 3.9 3.9 3.8 3.5 3 2.5 2 1.6 1.3 0.95 0.77 0.54 0.53 0.41 ...
>
> Has the meaning of DECIMAL_PRECISION changed from 1.9.2 to 1.10?
>
> It seems like in 1.9.2 the meaning is X characters to the right of the
> comma and in 1.10 it means X describing characters?
>
> I haven't tried the 1.10.1 release because I can't find the binaries
> on Tamas gisinternals.

Hmm. After a look into the source code [0] and some testing [1], it looks to me that DECIMAL_PRECISION is used as precision of a "%.<precision>g" format.

Format letter g means: "Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters."

For format letter g, the precision value defines: "Maximum number of
*significant* digits."

I see there has been this change recently:
http://trac.osgeo.org/gdal/changeset/25032/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp 

Hermann

[0]
http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp#L1051

[1]

# Input file

$ cat in.asc
ncols 1
nrows 4
xllcorner 1
yllcorner 1
cellsize  1
0.00012345
1.12345678
22.1234567
333.123456

# Test run

$ for i in 1 2 3 4 5 ; do
  echo DECIMAL_PRECISION=$i &&
  gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=$i in.asc out.asc &&
  cat out.asc
done

# Output

DECIMAL_PRECISION=1

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001
 1
 2e+01
 3e+02

DECIMAL_PRECISION=2

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012
 1.1
 22
 3.3e+02

DECIMAL_PRECISION=3

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.000123
 1.12
 22.1
 333

DECIMAL_PRECISION=4

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001234
 1.123
 22.12
 333.1

DECIMAL_PRECISION=5

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012345
 1.1235
 22.123
 333.12




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-Change-of-DECIMAL-PRECISION-in-AAIGrid-tp5075524p5075585.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Change of DECIMAL_PRECISION in AAIGrid

peifer
 
I can't tell you how much thought there was behind the change from "%.<DECIMAL_PRECISION>f"  to "%.<DECIMAL_PRECISION>g". At least noone bothered to change the documentation which continues to tell us:  "You can alternatively specify the number of decimal places with the DECIMAL_PRECISION creation option." [http://www.gdal.org/frmt_various.html#AAIGrid]
 
"DECIMAL_PRECISION"  actually became a misnomer now and should perhaps be called "SIGNIFICANT_DIGITS" -- or the format should be changed back to "%<precision>f". Maybe some of the GDAL devs has an opinion about the issue.
 
Hermann
 
Gesendet: Montag, 02. September 2013 um 13:30 Uhr
Von: "Casper Børgesen (CABO)" <[hidden email]>
An: peifer <[hidden email]>, "[hidden email]" <[hidden email]>
Betreff: RE: [gdal-dev] Change of DECIMAL_PRECISION in AAIGrid
Hi Peifer.

Thank you for your comments. They do explain more in depth what is happening. But as seen in your first example, the value 333.123456 is reduced to 300 while the value 0.00012345 is reduced to 0.0001. I think this is a big change in the relative values of each number.

I am curious to know if the "new" implementation in GDAL 1.10 is actually what is wanted/expected?

I use the DECIMAL_PRECISION for rounding purposes to control how many digits I want to the right of the comma, not how many digits I want to use for the entire number.
I know I get a lot of zeros if I use DECIMAL_PRECISION=10 in GDAL 1.9.2 but I retain my precision.

In GDAL 1.10 I have to determine the number of digits to the left of the comma used by the largest number in my data set, add the number of digits I want to have to the right of the comma and use that value as DECIMAL_PRECISION. The result must be that I get a lot of (waste) digits to the right of the comma for small numbers and the expected result for large numbers. But isn't this similar to what the patch (mentioned ealier) wanted to avoid? :

> The AAIGrid float format string is "%6.20g"
> Firstly, the 6 doesn't really add anything, and when writing small integers it causes them to be space-padded, which is a waste of bytes.
> At present AAIGrid supports "-co DECIMAL_PRECISION=10" which changes the formatstring to "%.10f".
> This is a little inconsistent, and the %f causes integers to be written as "9.000000000" which is a horrible waste of bytes.

I could write my own GeoTIFF to AAIGrid exporter, but I don't believe that should be necessary :)


Kind regards, Casper


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of peifer
Sent: 2. september 2013 12:21
To: [hidden email]
Subject: Re: [gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

Casper Børgesen wrote
> This is my syntax:
>
> gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=2
> my_source.vrt my_target.asc
>
> I have tried the above syntax using gdal 1.9.2 and 1.10 and these are
> examples from my results:
>
> 1.9.2:
> ...
> 3.77 3.83 3.89 3.87 3.79 3.49 3.03 2.50 1.96 1.56 1.29 0.95 0.77 0.54
> 0.53
> 0.41
> ...
>
> 1.10:
> ...
> 3.8 3.8 3.9 3.9 3.8 3.5 3 2.5 2 1.6 1.3 0.95 0.77 0.54 0.53 0.41 ...
>
> Has the meaning of DECIMAL_PRECISION changed from 1.9.2 to 1.10?
>
> It seems like in 1.9.2 the meaning is X characters to the right of the
> comma and in 1.10 it means X describing characters?
>
> I haven't tried the 1.10.1 release because I can't find the binaries
> on Tamas gisinternals.

Hmm. After a look into the source code [0] and some testing [1], it looks to me that DECIMAL_PRECISION is used as precision of a "%.<precision>g" format.

Format letter g means: "Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters."

For format letter g, the precision value defines: "Maximum number of
*significant* digits."

I see there has been this change recently:
http://trac.osgeo.org/gdal/changeset/25032/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp

Hermann

[0]
http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp#L1051

[1]

# Input file

$ cat in.asc
ncols 1
nrows 4
xllcorner 1
yllcorner 1
cellsize 1
0.00012345
1.12345678
22.1234567
333.123456

# Test run

$ for i in 1 2 3 4 5 ; do
echo DECIMAL_PRECISION=$i &&
gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=$i in.asc out.asc &&
cat out.asc
done

# Output

DECIMAL_PRECISION=1

ncols 1
nrows 4
xllcorner 1.000000000000
yllcorner 1.000000000000
cellsize 1.000000000000
0.0001
1
2e+01
3e+02

DECIMAL_PRECISION=2

ncols 1
nrows 4
xllcorner 1.000000000000
yllcorner 1.000000000000
cellsize 1.000000000000
0.00012
1.1
22
3.3e+02

DECIMAL_PRECISION=3

ncols 1
nrows 4
xllcorner 1.000000000000
yllcorner 1.000000000000
cellsize 1.000000000000
0.000123
1.12
22.1
333

DECIMAL_PRECISION=4

ncols 1
nrows 4
xllcorner 1.000000000000
yllcorner 1.000000000000
cellsize 1.000000000000
0.0001234
1.123
22.12
333.1

DECIMAL_PRECISION=5

ncols 1
nrows 4
xllcorner 1.000000000000
yllcorner 1.000000000000
cellsize 1.000000000000
0.00012345
1.1235
22.123
333.12




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-Change-of-DECIMAL-PRECISION-in-AAIGrid-tp5075524p5075585.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev