[gdal-dev] gdal_merge.py

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] gdal_merge.py

Ian Reese
Hi Gdal,

I am having some issues with gdal_merge.py. 

When I merge two tifs together, the original nodata value is converted to 0, no matter what that original no data was.  Here is my test case:

Data:

https://data.linz.govt.nz/x/9iitN4 DEM_BA32_3501_2013

https://data.linz.govt.nz/x/WLurgo DEM_BA31_3550_2013

Running gdalinfo on both files reveals a 'nodata' value of -9999.  When I view the data as individual files in QGIS, the 'nodata'  regions are confirmed as 'nodata' or '-9999' and are rendered transparent.

Next I run gdal_merge.py:

gdal_merge.py -o test_merge.tif *.tif

I also try:

gdal_merge.py -n -9999 -a_nodata -9999 -o test_merge_wnodata.tif *.tif

No matter which variation of merge I run, those regions that once contained 'nodata' or '-9999' are converted to '0' and the nodata value is not retained after the merge.  When I load the test_merge_wnodata.tif into QGIS, those regions once containing 'nodata', are now filled with 0 and rendered as such.

Maybe this is a bug or am I using gdal_merge.py improperly.

Cheers,

Ian





This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or [hidden email]) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] FW: gdal_merge.py

Ian Reese

HI Gdal,

 

Just wanted to follow this up.

 

Cheers,

 

Ian

 

From: Ian Reese
Sent: Monday, 12 March 2018 6:55 p.m.
To: [hidden email]
Subject: gdal_merge.py

 

Hi Gdal,

I am having some issues with gdal_merge.py. 

When I merge two tifs together, the original nodata value is converted to 0, no matter what that original no data.  Here is my test case:

Data:

https://data.linz.govt.nz/x/9iitN4 DEM_BA32_3501_2013

https://data.linz.govt.nz/x/WLurgo DEM_BA31_3550_2013

Running gdalinfo on both files reveals a 'nodata' value of -9999.  When I view the data as individual files in QGIS, the 'nodata'  regions are confirmed as 'nodata' or '-9999' and are rendered transparent.

Next I run gdal_merge.py:

gdal_merge.py -o test_merge.tif *.tif

I also try:

gdal_merge.py -n -9999 -a_nodata -9999 -o test_merge_wnodata.tif *.tif

No matter which variation of merge I run, those regions that once contained 'nodata' or '-9999' are converted to '0' and the nodata value is not retained after the merge.  When I load the test_merge_wnodata.tif into QGIS, those regions once containing 'nodata', are now filled with 0 and rendered as such.

Maybe this is a bug or am I using gdal_merge.py improperly.

Cheers,

Ian




This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or [hidden email]) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: FW: gdal_merge.py

Raphael Das Gupta

Hi Ian, heya GDALers

On 27.03.2018 22:56, Ian Reese wrote:
Maybe this is a bug or am I using gdal_merge.py improperly.
I've recently become unsure whether (and when) gdal_merge.py should be used at all, or whether gdalbuildvrt followed by gdaltranslate or gdalwarp is always preferable. (Also asked here: https://gis.stackexchange.com/q/272344/51574 )

Regards,
Raphael

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: gdal_merge.py

Even Rouault-2
In reply to this post by Ian Reese
On lundi 12 mars 2018 05:55:10 CEST Ian Reese wrote:
> No matter which variation of merge I run, those regions that once contained
> 'nodata' or '-9999' are converted to '0' and the nodata value is not
> retained after the merge.  When I load the test_merge_wnodata.tif into
> QGIS, those regions once containing 'nodata', are now filled with 0 and
> rendered as such.

gdal_merge.py -n -9999 -a_nodata -9999 -o test_merge_wnodata.tif *.tif
or even
gdal_merge.py -a_nodata -9999 -o test_merge_wnodata.tif *.tif
work fine for me with recent enough GDAL (2.2)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: FW: gdal_merge.py

Marcos Dione
In reply to this post by Raphael Das Gupta
On Wed, Mar 28, 2018 at 02:28:57PM +0200, Raphael Das Gupta wrote:
> On 27.03.2018 22:56, Ian Reese wrote:
> > Maybe this is a bug or am I using gdal_merge.py improperly.
> I've recently become unsure whether (and when) gdal_merge.py should be
> used at all, or whether gdalbuildvrt followed by gdaltranslate or
> gdalwarp is always preferable. (Also asked here:
> https://gis.stackexchange.com/q/272344/51574 )

    in my opinion, gdal_merge should be used only if the merged version
will be used for more things or if you're going to use it with something
that doesn't know how to read the .vrt files (i.e., is not using gdal).

    .vrt files have the advantage that are fats to generate, are easy to
update if one of the sources is changed, and it could be faster if the
sources are compressed and the target is thought to be compressed too (I
have no numbers to back this up, it's just a hunch).

    Cheers,

        -- Marcos.

--
(Not so) Random fortune:
The technology industry sees itself as in rebellion against corporate
America: not corrupt, not buttoned-up, not empty. In fact, a tech company
can be as corrupt, soulless, and empty as any corporation, but being
unprofessional helps us maintain the belief that we are somehow different
from Wall Street.
            -- Shanley Kane
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: FW: gdal_merge.py

Raphael Das Gupta
On 28.03.2018 16:31, Marcos Dione wrote:
>     in my opinion, gdal_merge should be used only if the merged version
> will be used for more things or if you're going to use it with something
> that doesn't know how to read the .vrt files (i.e., is not using gdal).
Even then it seems to be faster to first create a (temporary) .vrt file
and then convert that to something else (thereby materializing the
content) with e.g. gdaltranslate. (And then throw away the temporary
.vrt file.)

In my (and others', see e.g.
https://stackoverflow.com/a/22602542/674064) experience, the combined
runtime of gdalbuildvrt and gdaltranslate will be shorter than that of
gdal_merge.py would be. (And the quality of the result might also be
better.)

Though I don't know whether that was just the case for the examples I
encountered so far or whether that's always the case. Also, I don't know
whether there are use-cases of gdal_merge.py that gdalbuildvrt +
gdaltranslate (or + gdalwarp) can't handle.

If not, maybe gdal_merge.py should be reimplemented as a wrapper of
gdalbuildvrt + gdaltranslate, so that the obvious (and most convenient)
choice is also the best one?

Cheers,
Raphael
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: FW: gdal_merge.py

Even Rouault-2
On mercredi 28 mars 2018 17:58:37 CEST Raphael Das Gupta wrote:

> On 28.03.2018 16:31, Marcos Dione wrote:
> >     in my opinion, gdal_merge should be used only if the merged version
> >
> > will be used for more things or if you're going to use it with something
> > that doesn't know how to read the .vrt files (i.e., is not using gdal).
>
> Even then it seems to be faster to first create a (temporary) .vrt file
> and then convert that to something else (thereby materializing the
> content) with e.g. gdaltranslate. (And then throw away the temporary
> .vrt file.)
>
> In my (and others', see e.g.
> https://stackoverflow.com/a/22602542/674064) experience, the combined
> runtime of gdalbuildvrt and gdaltranslate will be shorter than that of
> gdal_merge.py would be. (And the quality of the result might also be
> better.)
>
> Though I don't know whether that was just the case for the examples I
> encountered so far or whether that's always the case. Also, I don't know
> whether there are use-cases of gdal_merge.py that gdalbuildvrt +
> gdaltranslate (or + gdalwarp) can't handle.
>
> If not, maybe gdal_merge.py should be reimplemented as a wrapper of
> gdalbuildvrt + gdaltranslate, so that the obvious (and most convenient)
> choice is also the best one?

My summary of different solutions:

* gdalbuildvrt + gdaltranslate:
   - pros:
      + efficient regarding compression of output dataset
      + work with arbitrarily large input  datasets.
      + several resampling methods
   - cons:
      + does not handle as one would expect overlapping input datasets regarding masking / nodata / alpha.
      + cannot update an output dataset
      + all input datasets must have same projection


* gdalwarp:
   - pros:
        + handles properly nodata / mask / alpha in input datasets (with alpha blending for alpha)
        + several resampling methods.
        + can deal with input datasets in different projections
        + can update an existing output dataset
   - cons:
        + can lead to datasets larger than needed with compression methods (unless -wo OPTIMIZE_SIZE=TRUE).
        + May be slower than other methods.


* gdal_merge.py:
     - pros:
         + handles nodata / mask in input dataset (alpha seen as binary mask)
         + can update an existing output dataset
    - cons:
         + only nearest resampling
         + cannot deal with arbitrarily large input datasets (ingests each input dataset fully in memory)
         + not friendly for compressed output
         + all input datasets must have same projection


So indeed gdal_merge.py could potentially be emulated with gdalbuildvrt+gdal_translate if there's no
overlap between input datasets or if they don't have nodata/mask, and otherwise on gdalwarp.

Exercice left to readers :-)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev