[gdal-dev] Potential issue with gdaladdo overviews setting alpha channel to nodata value instead of zero

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Potential issue with gdaladdo overviews setting alpha channel to nodata value instead of zero

Tim Waters (chippy)
Hello,

I'm seeing an issue which is present in 2.2.2 (tags/2.2.2 from github) and seen in 1.11.3 (Ubuntu 16.04) but not present in 1.10.1 (Ubuntu 14.04).

If a raster with an alpha channel has nodata set, and overviews via gdaladdo are created, the overviews use the nodata value for the value of the alpha channel.  However, using the nearest resampling method correctly sets the alpha to zero. All other resampling methods except nearest seem to give the same wrong result.

I've created a repository with all the files and readme containing the workflow.
https://github.com/timwaters/gdal_gdaladdo_issue with screenshots of the expected and observed files opened in Gimp to view the pixel values.

The use case is mapwarper.net where users upload ungeoreferenced images, they get converted to tifs and have overviews added, then they can mask them (gdal_rasterize) and then warp the masked files, with overviews added finally. I'm updating the server to run on Ubuntu 16.04 and will have to compile gdal.

Flow is:

#Apply overviews to ungeoreferenced raster
gdaladdo -r average treasure_island.tif 2 4 8 16

#Rasterise ungeoreferenced raster with the cropping mask
gdal_rasterize -i  -b 1 -b 2 -b 3 -burn 17 -burn 17 -burn 17  20.gml -l features treasure_island.tif

#Georeference & warp the raster
gdal_translate -a_srs '+init=epsg:4326' -of VRT treasure_island.tif temp.vrt  -gcp 100.38, 83.096, -122.377, 37.830 -gcp 430.1966, 370.08, -122.363, 37.8207 -gcp 313.69, 61.68, -122.368, 37.831
gdalwarp -rn  -dstalpha -srcnodata '17 17 17' -s_srs 'EPSG:4326' temp.vrt treasure_island_warped.tif -co TILED=YES -co COMPRESS=LZW

#Apply overviews to georeferenced raster
gdaladdo -r average treasure_island_warped.tif 2 4 8 16

Expected - transparent areas in overviews should be 17,17,17 with alpha channel set to 0
Observed - transparent areas in overviews are 17,17,17 with alpha also set to 17

Potential workarounds are 1) to set the nodata value to 0 instead of 17, however a few maps uploaded are black and white scans and so this might not be good for those maps which have pure black in them (that's why I'm using the 17 value), and 2) to use nearest neighbour for resampling, but it does not look good at all online.

many thanks in advance and your help is appreciated!

Regards,

Tim

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Potential issue with gdaladdo overviews setting alpha channel to nodata value instead of zero

Even Rouault-2

On jeudi 5 octobre 2017 18:46:32 CEST Tim Waters wrote:

> Hello,

>

> I'm seeing an issue which is present in 2.2.2 (tags/2.2.2 from github) and

> seen in 1.11.3 (Ubuntu 16.04) but not present in 1.10.1 (Ubuntu 14.04).

>

> If a raster with an alpha channel has nodata set, and overviews via

> gdaladdo are created, the overviews use the nodata value for the value of

> the alpha channel. However, using the nearest resampling method correctly

> sets the alpha to zero. All other resampling methods except nearest seem to

> give the same wrong result.

>

> I've created a repository with all the files and readme containing the

> workflow.

> https://github.com/timwaters/gdal_gdaladdo_issue with screenshots of the

> expected and observed files opened in Gimp to view the pixel values.

>

> The use case is mapwarper.net where users upload ungeoreferenced images,

> they get converted to tifs and have overviews added, then they can mask

> them (gdal_rasterize) and then warp the masked files, with overviews added

> finally. I'm updating the server to run on Ubuntu 16.04 and will have to

> compile gdal.

>

> Flow is:

>

> #Apply overviews to ungeoreferenced raster

> gdaladdo -r average treasure_island.tif 2 4 8 16

>

> #Rasterise ungeoreferenced raster with the cropping mask

> gdal_rasterize -i -b 1 -b 2 -b 3 -burn 17 -burn 17 -burn 17 20.gml -l

> features treasure_island.tif

>

> #Georeference & warp the raster

> gdal_translate -a_srs '+init=epsg:4326' -of VRT treasure_island.tif

> temp.vrt -gcp 100.38, 83.096, -122.377, 37.830 -gcp 430.1966, 370.08,

> -122.363, 37.8207 -gcp 313.69, 61.68, -122.368, 37.831

 

 

> gdalwarp -rn -dstalpha -srcnodata '17 17 17' -s_srs 'EPSG:4326' temp.vrt

> treasure_island_warped.tif -co TILED=YES -co COMPRESS=LZW

 

Tim,

 

The issue come from the above step. Basically this will add an alpha channel and also set implicitly a dest nodata value to 17. In GDAL extended TIFF, the nodata value applies to all channels. But when you have an alpha channel, having nodata values in addition doesn't really make sense and just confuse everything afterwards.

If you add -dstnodata none, this will fix the issue.

I think we should probably default to this behaviour when -dstalpha is specified.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Potential issue with gdaladdo overviews setting alpha channel to nodata value instead of zero

Tim Waters (chippy)
Even,

Many thanks! This did fix the issue.
Agreed, having both nodata and transparency would confuse things, and I see that this way also ensures that it works well with cropped maps with pure black pixels.

Best regards,

Tim



On 5 October 2017 at 22:21, Even Rouault <[hidden email]> wrote:

On jeudi 5 octobre 2017 18:46:32 CEST Tim Waters wrote:

> Hello,

>

> I'm seeing an issue which is present in 2.2.2 (tags/2.2.2 from github) and

> seen in 1.11.3 (Ubuntu 16.04) but not present in 1.10.1 (Ubuntu 14.04).

>

> If a raster with an alpha channel has nodata set, and overviews via

> gdaladdo are created, the overviews use the nodata value for the value of

> the alpha channel. However, using the nearest resampling method correctly

> sets the alpha to zero. All other resampling methods except nearest seem to

> give the same wrong result.

>

> I've created a repository with all the files and readme containing the

> workflow.

> https://github.com/timwaters/gdal_gdaladdo_issue with screenshots of the

> expected and observed files opened in Gimp to view the pixel values.

>

> The use case is mapwarper.net where users upload ungeoreferenced images,

> they get converted to tifs and have overviews added, then they can mask

> them (gdal_rasterize) and then warp the masked files, with overviews added

> finally. I'm updating the server to run on Ubuntu 16.04 and will have to

> compile gdal.

>

> Flow is:

>

> #Apply overviews to ungeoreferenced raster

> gdaladdo -r average treasure_island.tif 2 4 8 16

>

> #Rasterise ungeoreferenced raster with the cropping mask

> gdal_rasterize -i -b 1 -b 2 -b 3 -burn 17 -burn 17 -burn 17 20.gml -l

> features treasure_island.tif

>

> #Georeference & warp the raster

> gdal_translate -a_srs '+init=epsg:4326' -of VRT treasure_island.tif

> temp.vrt -gcp 100.38, 83.096, -122.377, 37.830 -gcp 430.1966, 370.08,

> -122.363, 37.8207 -gcp 313.69, 61.68, -122.368, 37.831

 

 

> gdalwarp -rn -dstalpha -srcnodata '17 17 17' -s_srs 'EPSG:4326' temp.vrt

> treasure_island_warped.tif -co TILED=YES -co COMPRESS=LZW

 

Tim,

 

The issue come from the above step. Basically this will add an alpha channel and also set implicitly a dest nodata value to 17. In GDAL extended TIFF, the nodata value applies to all channels. But when you have an alpha channel, having nodata values in addition doesn't really make sense and just confuse everything afterwards.

If you add -dstnodata none, this will fix the issue.

I think we should probably default to this behaviour when -dstalpha is specified.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com



_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Potential issue with gdaladdo overviews setting alpha channel to nodata value instead of zero

Even Rouault-2

On vendredi 6 octobre 2017 13:01:29 CEST Tim Waters wrote:

> Even,

>

> Many thanks! This did fix the issue.

> Agreed, having both nodata and transparency would confuse things, and I see

> that this way also ensures that it works well with cropped maps with pure

> black pixels.

 

I've changed in trunk the behaviour of gdalwarp as I suggested:

https://trac.osgeo.org/gdal/ticket/7075

 

The current behaviour is very likely a unintended side-effect of a GDAL 1.11.0 change:

* gdalwarp: copy nodata values from source to dest if -dstnodata is not given ; add option to not set dest nodata with -dstnodata None (#5087)

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev