[gdal-dev] gdal_translate with -projwin possible bug

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] gdal_translate with -projwin possible bug

Lauren O'Brien
Hello all,

I've noticed what I think may be a bug with gdal_translate, just looking for further information before deciding whether to report it. 

In version 1.11.4 and earlier, using the -projwin tag to subset a raster produces an output whose cell alignment and content matches the source dataset - it just grabs any cells that intersect the box and writes them to the output. 

In v2.1.0 (at least in a Windows environment), this no longer occurs. The output raster is distorted to match the exact extent of the -projwin coordinates. No resampling is involved, but the cell origins shift, and the cell sizes are either shrunk or stretched slightly to conform to the projwin box. This is a substantial change in behaviour which doesn't appear to have been documented.

From my point of view, its an unwelcome change. I've been using gdal_translate with -projwin and vsicurl to clip out sections of large national datasets without having to download the whole file first, and without needing to know things like the source dataset origin, cellsize etc in order to ensure output aligns with input. I need each dataset I clip out to 'stack' properly, and to represent an unaltered subset of the source data.

Please let me know if this is a true bug or just an undocumented change, and/or if there's a better alternative command to use.

Regards,
Lauren O'Brien

_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: [gdal-dev] gdal_translate with -projwin possible bug

Rutger
Hey Lauren,

I can replicate this with my Win64 installation of 2.1 (the conda-forge build).

Thanks for the heads up, i also do this all the time. Not having to calculate the exact extent was one of the benefits of using projwin. The current behavior could probably also be achieved with gdal_warp and nn-resampling for example, so at least for me its also an unwelcome change.  

The srcwin option probably still leaves the pixel placement untouched, but that would often require an additional conversion to pixel coordinates.

I think this also impact's the 'clipper' in QGIS, since the latest (2.16) also ships with GDAL 2.1.


Regards,
Rutger
Reply | Threaded
Open this post in threaded view
|

Re: gdal_translate with -projwin possible bug

jratike80
In reply to this post by Lauren O'Brien
Lauren O'Brien <leobrien <at> gmail.com> writes:

>
>
> Hello all,
>
> I've noticed what I think may be a bug with gdal_translate, just looking
for further information before deciding whether to report it. 
>
> In version 1.11.4 and earlier, using the -projwin tag to subset a raster
produces an output whose cell alignment and content matches the source
dataset - it just grabs any cells that intersect the box and writes them to
the output. 

I am sure that there are also users who consider that it is a bug if the
output does not exactly match -projwin.

I would have a try with gdalwarp by using -te and -tap together. I believe
that you would like to see something like -tap (target aligned pixels) also
in gdal_translate.

-Jukka Rahkonen-
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: gdal_translate with -projwin possible bug

Even Rouault-2
In reply to this post by Lauren O'Brien
Hi Lauren,

This change was semi intended as far as I can remember (the change that leads
to that was intended, the effect in this use case not) . It dates back to GDAL
2.1.0 where the VRT driver can now support floating point coordinates for
SrcWin / DestWin, and gdal_translate was updated accordingly to not round to
integer pixel coordinates deduced from world coordinates specified through -
projwin.  Sub pixel accuracy makes sense when using with a resampling that is
not nearest neighbour with the -r switch. But indeed for nearest neighbour, it
might result in a non intended shift.
Perhaps a good compromise would be to round to integer for nearest neighbour
only ?

Regarding Jukka's comment,
> I am sure that there are also users who consider that it is a bug if the
> output does not exactly match -projwin.

Using -a_ullr is a way of definining the output bounds for sure.

Even

> Hello all,
>
> I've noticed what I think may be a bug with gdal_translate, just looking
> for further information before deciding whether to report it.
>
> In version 1.11.4 and earlier, using the -projwin tag to subset a raster
> produces an output whose cell alignment and content matches the source
> dataset - it just grabs any cells that intersect the box and writes them to
> the output.
>
> In v2.1.0 (at least in a Windows environment), this no longer occurs. The
> output raster is distorted to match the exact extent of the -projwin
> coordinates. No resampling is involved, but the cell origins shift, and the
> cell sizes are either shrunk or stretched slightly to conform to the
> projwin box. This is a substantial change in behaviour which doesn't appear
> to have been documented.
>
> From my point of view, its an unwelcome change. I've been using
> gdal_translate with -projwin and vsicurl to clip out sections of large
> national datasets without having to download the whole file first, and
> without needing to know things like the source dataset origin, cellsize etc
> in order to ensure output aligns with input. I need each dataset I clip out
> to 'stack' properly, and to represent an unaltered subset of the source
> data.
>
> Please let me know if this is a true bug or just an undocumented change,
> and/or if there's a better alternative command to use.
>
> Regards,
> Lauren O'Brien

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: gdal_translate with -projwin possible bug

Lauren O'Brien

Hi Evan,

 

Ok I thought it might have been at least partially intended. Unfortunately, I think the use case the change is addressing is not what the majority of users are thinking of when they choose projwin.

 

As Rutger pointed out, the Clipper tool in QGIS 2.14 onwards is now incapable of producing an output that lines up with input, unless the user spends some time doing coordinate math to calculate a precise projwin. That’s a pretty major efficiency loss. Questions about how to clip a raster without resampling or alignment changes appear regularly on stackexchange (see http://gis.stackexchange.com/questions/201204/qgis-clipper-tool-misaliagnment , and http://gis.stackexchange.com/questions/203664/clip-a-tiff-raster-image-using-a-bounding-box-with-python) and gdal_translate with projwin is regularly recommended as the most viable solution.

 

Jukka, I tried gdal_warp with -te and -tap (one must also input -tr). It was better in that the cell sizes didn’t change, but the output cells moved slightly northwest, such that the center of the input cell became the lower right corner of the output cell. I guess that could be fixed by using gdal_edit with -a_ullr to nudge the output back into place, but that’s still extra work that shouldn’t be necessary.

 

I think it would be better if projwin’s functionality reverted to its previous behaviour as a default, and that an additional flag be made available for people who do want strict conformance with their specified bounding box as part of a resampling operation.

 

Either way, the way projwin works in different versions needs to be made explicit in the documentation.

 

Regards

Lauren

 

 

From: [hidden email]
Sent: Thursday, 28 July 2016 2:10 AM
To: [hidden email]; [hidden email]
Cc: [hidden email]
Subject: Re: [gdal-dev] gdal_translate with -projwin possible bug

 

Hi Lauren,

 

This change was semi intended as far as I can remember (the change that leads

to that was intended, the effect in this use case not) . It dates back to GDAL

2.1.0 where the VRT driver can now support floating point coordinates for

SrcWin / DestWin, and gdal_translate was updated accordingly to not round to

integer pixel coordinates deduced from world coordinates specified through -

projwin.  Sub pixel accuracy makes sense when using with a resampling that is

not nearest neighbour with the -r switch. But indeed for nearest neighbour, it

might result in a non intended shift.

Perhaps a good compromise would be to round to integer for nearest neighbour

only ?

 

Regarding Jukka's comment,

> I am sure that there are also users who consider that it is a bug if the

> output does not exactly match -projwin.

 

Using -a_ullr is a way of definining the output bounds for sure.

 

Even

 

> Hello all,

>

> I've noticed what I think may be a bug with gdal_translate, just looking

> for further information before deciding whether to report it.

>

> In version 1.11.4 and earlier, using the -projwin tag to subset a raster

> produces an output whose cell alignment and content matches the source

> dataset - it just grabs any cells that intersect the box and writes them to

> the output.

>

> In v2.1.0 (at least in a Windows environment), this no longer occurs. The

> output raster is distorted to match the exact extent of the -projwin

> coordinates. No resampling is involved, but the cell origins shift, and the

> cell sizes are either shrunk or stretched slightly to conform to the

> projwin box. This is a substantial change in behaviour which doesn't appear

> to have been documented.

>

> From my point of view, its an unwelcome change. I've been using

> gdal_translate with -projwin and vsicurl to clip out sections of large

> national datasets without having to download the whole file first, and

> without needing to know things like the source dataset origin, cellsize etc

> in order to ensure output aligns with input. I need each dataset I clip out

> to 'stack' properly, and to represent an unaltered subset of the source

> data.

>

> Please let me know if this is a true bug or just an undocumented change,

> and/or if there's a better alternative command to use.

>

> Regards,

> Lauren O'Brien

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com

 


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: gdal_translate with -projwin possible bug

Even Rouault-2
On Thursday 28 July 2016 09:18:36 Lauren O'Brien wrote:
> Hi Evan,
>
> Ok I thought it might have been at least partially intended. Unfortunately,
> I think the use case the change is addressing is not what the majority of
> users are thinking of when they choose projwin.

I've pushed a fix in line with my previous email:
https://trac.osgeo.org/gdal/ticket/6610

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: gdal_translate with -projwin possible bug

Rutger
Thanks Even,

There is probably, like Jukka mentioned, a use case for both ways, but i personally prefer your fix. If i want warp/shift functionality i use gdalwarp anyway, and then you know you have to be careful about pixel alignment etc.  

Even Rouault-2 wrote
I've pushed a fix in line with my previous email:
https://trac.osgeo.org/gdal/ticket/6610

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev