[gdal-dev] Is there a reference documentation for GDALMetadata structure?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Is there a reference documentation for GDALMetadata structure?

geowolf
Hi,
I was wondering if there is any documentation on what a GDALMetadata XML structure might contain, both in terms
of element/attributes, and key names?

Online I've found this example:

But checking some files I have laying around there is more variety, for example:

<GDALMetadata>.  
    <Item name="Band_1">Max Band_1</Item>.  
    <Item name="Band_2">Min Band_1</Item>.  
    <Item name="Band_3">Mean Band_1</Item>.  
    <Item name="Band_4">SD Band_1</Item>.  
    <Item name="Band_5">MASD Band_1</Item>.  
    <Item name="Band_6">Valid Inputs</Item>.  
    <Item name="OFFSET" sample="0" role="offset">0</Item>.  
    <Item name="SCALE" sample="0" role="scale">0.000100000000000000005</Item>.  
    <Item name="OFFSET" sample="1" role="offset">0</Item>.  
    <Item name="SCALE" sample="1" role="scale">0.000100000000000000005</Item>.  
    <Item name="OFFSET" sample="2" role="offset">0</Item>.  
    <Item name="SCALE" sample="2" role="scale">0.000100000000000000005</Item>.  
    <Item name="OFFSET" sample="3" role="offset">0</Item>.  
    <Item name="SCALE" sample="3" role="scale">0.000100000000000000005</Item>.  
    <Item name="OFFSET" sample="4" role="offset">0</Item>.  
    <Item name="SCALE" sample="4" role="scale">0.000100000000000000005</Item>.  
    <Item name="OFFSET" sample="5" role="offset">0</Item>.  
    <Item name="SCALE" sample="5" role="scale">0.000100000000000000005</Item>.
</GDALMetadata>.

but also:

<GDALMetadata>.  
    <Item name="lat#units">degrees_north</Item>.  
    <Item name="LatLon_Projection#earth_radius">6371229</Item>.  
    <Item name="LatLon_Projection#grid_mapping_name">latitude_longitude</Item>.  
    <Item name="lon#units">degrees_east</Item>.  
    <Item name="NC_GLOBAL#Analysis_or_forecast_generating_process_identifier_defined_by_originating_centre">Coastal Ocean Circulation Model</Item>.  
    <Item name="NC_GLOBAL#Conventions">CF-1.6</Item>.  
    <Item name="NC_GLOBAL#featureType">GRID</Item>.  
    <Item name="NC_GLOBAL#file_format">GRIB-2</Item>.  
    <Item name="NC_GLOBAL#GRIB_table_version">2,1</Item>.  
    <Item name="NC_GLOBAL#history">Read using CDM IOSP GribCollection v3</Item>.  
    <Item name="NC_GLOBAL#Originating_or_generating_Center">US National Weather Service, National Centres for Environmental Prediction (NCEP)</Item>.  
    <Item name="NC_GLOBAL#Originating_or_generating_Subcenter">0</Item>.  
    <Item name="NC_GLOBAL#Type_of_generating_process">Forecast</Item>.  
    <Item name="NETCDF_DIM_EXTRA">{time}</Item>.  
    <Item name="NETCDF_DIM_time_DEF">{1,6}</Item>.  
    <Item name="NETCDF_DIM_time_VALUES">3</Item>.  
    <Item name="time#calendar">proleptic_gregorian</Item>.  
    <Item name="time#long_name">GRIB forecast or observation time</Item>.  
    <Item name="time#standard_name">time</Item>.  
    <Item name="time#units">Hour since 2014-01-23T00:00:00Z</Item>.  
    <Item name="u-component_of_current_surface#abbreviation">UOGRD</Item>.  
    <Item name="u-component_of_current_surface#coordinates">reftime time lat lon </Item>.  
    <Item name="u-component_of_current_surface#Grib2_Generating_Process_Type">Forecast</Item>.  
    <Item name="u-component_of_current_surface#Grib2_Level_Type">Ground or water surface</Item>.  
    <Item name="u-component_of_current_surface#Grib2_Parameter">{10,1,2}</Item>.  
    <Item name="u-component_of_current_surface#Grib2_Parameter_Category">Currents</Item>.  
    <Item name="u-component_of_current_surface#Grib2_Parameter_Discipline">Oceanographic products</Item>.  
    <Item name="u-component_of_current_surface#Grib2_Parameter_Name">u-component of current</Item>.  
    <Item name="u-component_of_current_surface#Grib_Variable_Id">VAR_10-1-2_L1</Item>.  
    <Item name="u-component_of_current_surface#grid_mapping">LatLon_Projection</Item>.  
    <Item name="u-component_of_current_surface#long_name">u-component of current @ Ground or water surface</Item>.  
    <Item name="u-component_of_current_surface#missing_value">nan</Item>.  
    <Item name="u-component_of_current_surface#units">m/s</Item>.  
    <Item name="abbreviation" sample="0">UOGRD</Item>.  
    <Item name="coordinates" sample="0">reftime time lat lon </Item>.  
    <Item name="Grib2_Generating_Process_Type" sample="0">Forecast</Item>.  
    <Item name="Grib2_Level_Type" sample="0">Ground or water surface</Item>.  
    <Item name="Grib2_Parameter" sample="0">{10,1,2}</Item>.  
    <Item name="Grib2_Parameter_Category" sample="0">Currents</Item>.  
    <Item name="Grib2_Parameter_Discipline" sample="0">Oceanographic products</Item>.  
    <Item name="Grib2_Parameter_Name" sample="0">u-component of current</Item>.  
    <Item name="Grib_Variable_Id" sample="0">VAR_10-1-2_L1</Item>.  
    <Item name="grid_mapping" sample="0">LatLon_Projection</Item>.  
    <Item name="long_name" sample="0">u-component of current @ Ground or water surface</Item>.  
    <Item name="missing_value" sample="0">nan</Item>.  
    <Item name="NETCDF_DIM_time" sample="0">3</Item>.  
    <Item name="NETCDF_VARNAME" sample="0">u-component_of_current_surface</Item>.  
    <Item name="units" sample="0">m/s</Item>.  
    <Item name="UNITTYPE" sample="0" role="unittype">m/s</Item>.
</GDALMetadata>

The periods at the end of the line are not valid, not sure where they are coming from, exiftool reports them out
of one tiff file I have handy, which was converted out of a grib file a few years ago

Cheers
Andrea

== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is there a reference documentation for GDALMetadata structure?

jratike80
Hi,

Perhaps here
https://github.com/OSGeo/gdal/blob/master/gdal/gcore/gdalpamdataset.cpp

-Jukka Rahkonen-


geowolf wrote
> Hi,
> I was wondering if there is any documentation on what a GDALMetadata XML
> structure might contain, both in terms
> of element/attributes, and key names?
> ...





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is there a reference documentation for GDALMetadata structure?

geowolf
Hi Jukka,
thank you for following up. I believe that provides information for the PAM dataset, which
is normally a sidecar containing statistical information such as min/max/histograms.
The GDAL metadata TIFF tag instead normally contains other information, such as scale/offset,
band names, units, and uses a XML that has a different structure.

Looking into the code there is something around here:

I see explicit treatment for scale/offset and a few others, not sure if there is anything else?
Like, if a band had a custom name, would it show up in the GDALMetadata structure in a significant way?
From my previous example, things like:

 <Item name="Band_1">Max Band_1</Item>.  
    <Item name="Band_2">Min Band_1</Item>.  
    <Item name="Band_3">Mean Band_1</Item>.  
    <Item name="Band_4">SD Band_1</Item>.  
    <Item name="Band_5">MASD Band_1</Item>.  
    <Item name="Band_6">Valid Inputs</Item>.  

are they actually used or just treated as "random strings"?
Looking at a GDALInfo output it looks like of the second, but maybe I'm missing something:

Metadata:
  AREA_OR_POINT=Area
  Band_1=Max Band_1
  Band_2=Min Band_1
  Band_3=Mean Band_1
  Band_4=SD Band_1
  Band_5=MASD Band_1
  Band_6=Valid Inputs
  UNITS=m
...
Band 1 Block=10x10 Type=Int32, ColorInterp=Gray
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 2 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 3 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 4 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 5 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 6 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001



Cheers
Andrea

On Mon, May 27, 2019 at 1:11 PM jratike80 <[hidden email]> wrote:
Hi,

Perhaps here
https://github.com/OSGeo/gdal/blob/master/gdal/gcore/gdalpamdataset.cpp

-Jukka Rahkonen-


geowolf wrote
> Hi,
> I was wondering if there is any documentation on what a GDALMetadata XML
> structure might contain, both in terms
> of element/attributes, and key names?
> ...





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is there a reference documentation for GDALMetadata structure?

jratike80
In reply to this post by jratike80

Hi Andrea,

 

I have understood that PAM may contain any metadata that cannot be stored into the file format itself.

 

I made a test but I honestly do not know what it proves, if anything.  However, if I tell GDAL to handle GeoTIFF as read-only and make some edits with

 

gdal_edit -ro -mo DATUM=WGS84 test.tif

gdal_edit -ro -scale 299 -offset 10 test.tif

 

then I get a tif.aux.xml file that contains

 

<PAMDataset>

  <Metadata>

    <MDI key="DATUM">WGS84</MDI>

  </Metadata>

  <PAMRasterBand band="1">

    <Offset>10</Offset>

    <Scale>299</Scale>

  </PAMRasterBand>

  <PAMRasterBand band="2">

    <Offset>10</Offset>

    <Scale>299</Scale>

  </PAMRasterBand>

  <PAMRasterBand band="3">

    <Offset>10</Offset>

    <Scale>299</Scale>

  </PAMRasterBand>

</PAMDataset>

 

If I run the commands without -ro then the tags of the GeoTIFF file are edited and no sidecar file appears.

 

-Jukka-

 

 

 

Lähettäjä: Andrea Aime <[hidden email]>
Lähetetty: maanantai 27. toukokuuta 2019 18.59
Vastaanottaja: Rahkonen Jukka (MML) <[hidden email]>
Kopio: [hidden email]
Aihe: Re: [gdal-dev] Is there a reference documentation for GDALMetadata structure?

 

Hi Jukka,

thank you for following up. I believe that provides information for the PAM dataset, which

is normally a sidecar containing statistical information such as min/max/histograms.

The GDAL metadata TIFF tag instead normally contains other information, such as scale/offset,

band names, units, and uses a XML that has a different structure.

 

Looking into the code there is something around here:

 

I see explicit treatment for scale/offset and a few others, not sure if there is anything else?

Like, if a band had a custom name, would it show up in the GDALMetadata structure in a significant way?

From my previous example, things like:

 

 <Item name="Band_1">Max Band_1</Item>.  
    <Item name="Band_2">Min Band_1</Item>.  
    <Item name="Band_3">Mean Band_1</Item>.  
    <Item name="Band_4">SD Band_1</Item>.  
    <Item name="Band_5">MASD Band_1</Item>.  
    <Item name="Band_6">Valid Inputs</Item>.  

 

are they actually used or just treated as "random strings"?

Looking at a GDALInfo output it looks like of the second, but maybe I'm missing something:

 

Metadata:
  AREA_OR_POINT=Area
  Band_1=Max Band_1
  Band_2=Min Band_1
  Band_3=Mean Band_1
  Band_4=SD Band_1
  Band_5=MASD Band_1
  Band_6=Valid Inputs
  UNITS=m
...
Band 1 Block=10x10 Type=Int32, ColorInterp=Gray
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 2 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 3 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 4 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 5 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001
Band 6 Block=10x10 Type=Int32, ColorInterp=Undefined
  NoData Value=-9999
  Offset: 0,   Scale:0.0001

 

 

Cheers

Andrea

 

On Mon, May 27, 2019 at 1:11 PM jratike80 <[hidden email]> wrote:

Hi,

Perhaps here
https://github.com/OSGeo/gdal/blob/master/gdal/gcore/gdalpamdataset.cpp

-Jukka Rahkonen-


geowolf wrote
> Hi,
> I was wondering if there is any documentation on what a GDALMetadata XML
> structure might contain, both in terms
> of element/attributes, and key names?
> ...





--
Sent from:
http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev


 

--

Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is there a reference documentation for GDALMetadata structure?

Even Rouault-2
In reply to this post by geowolf
On lundi 27 mai 2019 17:58:34 CEST Andrea Aime wrote:

> Hi Jukka,
> thank you for following up. I believe that provides information for the PAM
> dataset, which
> is normally a sidecar containing statistical information such as
> min/max/histograms.
> The GDAL metadata TIFF tag instead normally contains other information,
> such as scale/offset,
> band names, units, and uses a XML that has a different structure.
>
> Looking into the code there is something around here:
> https://github.com/OSGeo/gdal/blob/master/gdal/frmts/gtiff/geotiff.cpp#L1372
> 6

Andrea,

If you're looking for the specification of the GDAL metadata TIFF tag (which
is different from PAM .aux.xml), then the code will be your best friend as I
don't think there's a document describing it, together with experimentation

Here's how to set the names of a 2-band TIFF

<GDALMetadata>
  <Item name="DESCRIPTION" sample="0" role="description">my_band_1_name</Item>
  <Item name="DESCRIPTION" sample="1" role="description">my_band_2_name</Item>
</GDALMetadata>

The output of gdalinfo will be:

Band 1 Block=20x20 Type=Byte, ColorInterp=Gray
  Description = my_band_1_name
Band 2 Block=20x20 Type=Byte, ColorInterp=Gray
  Description = my_band_2_name

The samples with '.' you mention are corrupted. GDAL shouldn't produce such
files.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev