After that using this python script decided to validate the result and was confused by missing IFD_OFFSET tag.
So the questions are: Should it be present? Is it possible point me to the GDAL code base, to the place where it can or should be set? Should it be defined manually? Can BigTiffs be cloud optimized and can / should BigTiff have an offset == 8 as well (16 is a usual offset for them)? I discovered this link, but still confused about IFD_OFFSET tag necessity / requirement.
GDAL version: GDAL 2.2.1, released 2017/06/23, mac os sierra.
> hasacloudoptimizationinternalorganization> python script decided to validate
> the result and was confused by missing IFD_OFFSET tag.
confused ? Can you report the exact error ?
IFD_OFFSET should always be reported for any (valid) TIFF, be it COG or not. This is the offset in the file of the image directory, and we must check that it is right at the beginning of the file for efficient reading.
I suspect some issue with your setup, although I'm not clear which one. I'd have said perhaps you'd run the script against an older GDAL version, but the script does version checking..
To debug this you could try setting a breakpoint in GTiffRasterBand::GetMetadataItem and check what happens.
> Can BigTiffs be cloud optimized and can / should BigTiff
> have an offset == 8 as well (16 is a usual offset for them)?
You're absolutely right. I've just fixed the script to accept those.
> Cool, thanks! Can you write here when it'll be updated?
It was already committed in GDAL trunk when I sent the previous email.
> The error is missing IFD_OFFSET tag. Should it be specified manually or it
> is fine to have it missing? Is it a COG requirement?
I'm not sure how to rephrase better what I wrote in my previous email. IFD_OFFSET is not a tag, this is a structural element of any TIFF file. There is compulsory an IFD for any TIFF file. The fact that you don't get this metadata item is a software issue, and should be solved (if you comment out that check, I'm pretty sure you'll have other issues then, so you need to find what's wrong in your setup). Your file has necessarily an IFD and thus an IFD offset.
Ah sorry reread your message. Yes, my tiff has no IFD_OFFSET tag (basically i mean that line int(main_band.GetMetadataItem('IFD_OFFSET', 'TIFF')) fails in this python validation script), but gdalinfo reads tiff info well and moreover reading tiff manually by bytes I can see that offset field / element has a correct value.
Well I'll post the exact message here, as probably I could use not really correct terms in my messages:
python validate_cloud_optimized_geotiff.py in.tiff
Traceback (most recent call last):
File "validate_cloud_optimized_geotiff.py", line 218, in <module> sys.exit(main())
File "validate_cloud_optimized_geotiff.py", line 196, in main
errors, _ = validate(filename)
File "validate_cloud_optimized_geotiff.py", line 88, in validate
ifd_offset = int(main_band.GetMetadataItem('IFD_OFFSET', 'TIFF'))
Thanks a lot, I agree that probably it's a software issue (breakpoint helped), and my GDAL version is 2010100 (I commented out this version check before as thought that it's not that important). Thanks again, and thanks for your reply, I'll upgrade GDAL and hope problems would go, as commenting this line out solves the problem and allows to validate tiff. Was really confused by an API and happy that it was just my fault.