Generating Cloud Optimized GeoTIFFs

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Generating Cloud Optimized GeoTIFFs

daunnc
This post was updated on .
Hi everybody! We're implementing COGs support in GeoTrellis, and there is appeared a question related to cloud optimized geotiff definition.

Following this page:

// in.tif is some random tiff.

1. gdaladdo in.tif 2 4 8 16 32
2. gdal_translate in.tif out.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATE

After that using this python script decided to validate the result and was confused by missing IFD_OFFSET tag.

So the questions are: Should it be present? Is it possible point me to the GDAL code base, to the place where it can or should be set? Should it be defined manually? Can BigTiffs be cloud optimized and can / should BigTiff have an offset == 8 as well (16 is a usual offset for them)? I discovered this link, but still confused about IFD_OFFSET tag necessity / requirement.

GDAL version: GDAL 2.2.1, released 2017/06/23, mac os sierra.

Thanks everybody in advance!
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

Even Rouault-2

On mardi 29 août 2017 09:27:33 CEST daunnc wrote:

> Hi everybody! We're implementing COGs support in GeoTrellis, and there is

> appeared a question related to cloud optimized geotiff definition.

>

> Following this <https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF>

> page:

>

> // in.tif is some random tiff.

>

> 1. gdaladdo in.tif 2 4 8 16 32

> 2. gdal_translate in.tif out.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES

> -co COMPRESS=DEFLATE

>

> After that using this

> <https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF#HowtocheckifaGeoTIFF

> hasacloudoptimizationinternalorganization> python script decided to validate

> the result and was confused by missing IFD_OFFSET tag.

 

confused ? Can you report the exact error ?

 

IFD_OFFSET should always be reported for any (valid) TIFF, be it COG or not. This is the offset in the file of the image directory, and we must check that it is right at the beginning of the file for efficient reading.

 

I suspect some issue with your setup, although I'm not clear which one. I'd have said perhaps you'd run the script against an older GDAL version, but the script does version checking..

To debug this you could try setting a breakpoint in GTiffRasterBand::GetMetadataItem and check what happens.

 

> Can BigTiffs be cloud optimized and can / should BigTiff

> have an offset == 8 as well (16 is a usual offset for them)?

 

You're absolutely right. I've just fixed the script to accept those.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

daunnc
This post was updated on .
Cool, thanks!

The error is missing IFD_OFFSET tag. Should it be specified manually or it is fine to have it missing? Is it a COG requirement?
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

Even Rouault-2

On mardi 29 août 2017 09:57:21 CEST daunnc wrote:

> Cool, thanks! Can you write here when it'll be updated?

 

It was already committed in GDAL trunk when I sent the previous email.

 

>

> The error is missing IFD_OFFSET tag. Should it be specified manually or it

> is fine to have it missing? Is it a COG requirement?

 

I'm not sure how to rephrase better what I wrote in my previous email. IFD_OFFSET is not a tag, this is a structural element of any TIFF file. There is compulsory an IFD for any TIFF file. The fact that you don't get this metadata item is a software issue, and should be solved (if you comment out that check, I'm pretty sure you'll have other issues then, so you need to find what's wrong in your setup). Your file has necessarily an IFD and thus an IFD offset.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

daunnc
This post was updated on .
In reply to this post by daunnc
Ah sorry reread your message. Yes, my tiff has no IFD_OFFSET tag (basically i mean that line int(main_band.GetMetadataItem('IFD_OFFSET', 'TIFF')) fails in this python validation script), but gdalinfo reads tiff info well and moreover reading tiff manually by bytes I can see that offset field / element has a correct value.
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

daunnc
Well I'll post the exact message here, as probably I could use not really correct terms in my messages:

python validate_cloud_optimized_geotiff.py in.tiff
Traceback (most recent call last):
  File "validate_cloud_optimized_geotiff.py", line 218, in <module>
    sys.exit(main())
  File "validate_cloud_optimized_geotiff.py", line 196, in main
    errors, _ = validate(filename)
  File "validate_cloud_optimized_geotiff.py", line 88, in validate
    ifd_offset = int(main_band.GetMetadataItem('IFD_OFFSET', 'TIFF'))

But however I got it, thanks a lot!
Reply | Threaded
Open this post in threaded view
|

Re: Generating Cloud Optimized GeoTIFFs

daunnc
In reply to this post by Even Rouault-2
> I suspect some issue with your setup

Thanks a lot, I agree that probably it's a software issue (breakpoint helped), and my GDAL version is 2010100 (I commented out this version check before as thought that it's not that important). Thanks again, and thanks for your reply, I'll upgrade GDAL and hope problems would go, as commenting this line out solves the problem and allows to validate tiff. Was really confused by an API and happy that it was just my fault.