[gdal-dev] How to know which formats allow editing with ogrinfo?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] How to know which formats allow editing with ogrinfo?

jratike80

Hi,

 

Shapefile can be edited with ogrinfo

ogrinfo -dialect SQLite -sql "update foo set bar='14'" foo.shp

 

GML opens always as read-only with -sql option

ogrinfo -sql "select * from foo" foo.gml

Had to open data source read-only.

INFO: Open of `foo.gml'

      using driver `GML' successful.

 

How can I know which formats support editing with -sql and which not? I could not find any obvious metadata by checking the capabilities of a few different drivers.

 

-Jukka Rahkonen-


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to know which formats allow editing with ogrinfo?

Even Rouault-2
On vendredi 15 novembre 2019 08:52:48 CET Rahkonen Jukka (MML) wrote:

> Hi,
>
> Shapefile can be edited with ogrinfo
> ogrinfo -dialect SQLite -sql "update foo set bar='14'" foo.shp
>
> GML opens always as read-only with -sql option
> ogrinfo -sql "select * from foo" foo.gml
> Had to open data source read-only.
> INFO: Open of `foo.gml'
>       using driver `GML' successful.
>
> How can I know which formats support editing with -sql and which not? I
> could not find any obvious metadata by checking the capabilities of a few
> different drivers.

There's indeed no driver-level metadata indicating support for updates (update
through SQL is just a particular case). With a dataset opened, on a layer,
with the API you can query for fine grain capabilities with
TestCapability(OLCRandomWrite/OLCDeleteFeature/etc...)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to know which formats allow editing with ogrinfo?

Eli Adam
On Fri, Nov 15, 2019 at 3:58 AM Even Rouault <[hidden email]> wrote:

>
> On vendredi 15 novembre 2019 08:52:48 CET Rahkonen Jukka (MML) wrote:
> > Hi,
> >
> > Shapefile can be edited with ogrinfo
> > ogrinfo -dialect SQLite -sql "update foo set bar='14'" foo.shp
> >
> > GML opens always as read-only with -sql option
> > ogrinfo -sql "select * from foo" foo.gml
> > Had to open data source read-only.
> > INFO: Open of `foo.gml'
> >       using driver `GML' successful.
> >
> > How can I know which formats support editing with -sql and which not? I
> > could not find any obvious metadata by checking the capabilities of a few
> > different drivers.
>
> There's indeed no driver-level metadata indicating support for updates (update
> through SQL is just a particular case). With a dataset opened, on a layer,
> with the API you can query for fine grain capabilities with
> TestCapability(OLCRandomWrite/OLCDeleteFeature/etc...)

Can this be partly inferred from driver pages with the presence (or
absence) of items?  For instance,
https://gdal.org/drivers/vector/gml.html#driver-capabilities

The old driver pages
(https://web.archive.org/web/20190211030817/https://www.gdal.org/ogr_formats.html)
were tables that very effectively conveyed the capabilities (or
limitations) of the drivers and allowed for scanning for drivers that
had the capabilities that you wanted.  Would a similar table be
possible or desirable?

This got me thinking about the Sphinx migration a bit,
https://lists.osgeo.org/pipermail/gdal-dev/2019-May/050273.html.  It
looks great and the website is easy to use.  Thanks to everyone who
updated that!

There are a few things that might be missing.  The new page has
https://gdal.org/programs/raster_common_options.html#raster-common-options
and https://gdal.org/programs/vector_common_options.html#vector-common-options
I wonder if a list of uncommon options like this would be helpful,
https://trac.osgeo.org/gdal/wiki/ConfigOptions?  Same with some
overview table of driver capabilities?  If it is just a matter of
recreating a new page copied from the old, I would be up trying to
contribute to that.  I lack the detailed knowledge of the drivers to
be able to create it from scratch.

I was also curious to see if the OSGeo technical writing project would
impact any of the documentation.

Best regards, Eli






>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to know which formats allow editing with ogrinfo?

Even Rouault-2
> Can this be partly inferred from driver pages with the presence (or
> absence) of items?  For instance,
> https://gdal.org/drivers/vector/gml.html#driver-capabilities

Not really. They are only high level capabilities. Vector editing capabilities
are not reported in driver pages

> The old driver pages
> (https://web.archive.org/web/20190211030817/https://www.gdal.org/ogr_formats
> .html) were tables that very effectively conveyed the capabilities (or
> limitations) of the drivers and allowed for scanning for drivers that
> had the capabilities that you wanted.

The information in individual pages is roughly the same as in the global
table.

> Would a similar table be possible or desirable?

A few people have indeed noted the absence of the table. Not sure if this is
usual "resistance to change" phenomenon or if there is there really a strong
use case for "I don't want which format to use: let's look at the summary page
to select one".

If so, then yes if someone contributes it, that'd be welcome. However I think
we'd want it to be generated from a Python script from the individual .rst
pages, by parsing the directives like ".. supports_create::" that are in those
.rst files. That way that would make maintainance of the summary easier (and
initial version less tidious that doing that manually)

> There are a few things that might be missing.  The new page has
> https://gdal.org/programs/raster_common_options.html#raster-common-options
> and
> https://gdal.org/programs/vector_common_options.html#vector-common-options
> I wonder if a list of uncommon options like this would be helpful,
> https://trac.osgeo.org/gdal/wiki/ConfigOptions?

Migrating the wiki page to the doc could be useful. My own opinion is that
driver specific configuration options should be documented in the driver
specific page, rather than in a global config option page. Same with /vsi
specific options that should go to
https://gdal.org/user/virtual_file_systems.html if not already there.

Or if a global list is needed, I'd say a bit the same as for driver summaries,
have a way to compile the list of options from .rst files where they would be
mention and link from the summary to the actual page. But we don't have
currently any explicit directive for configuration options.

> I was also curious to see if the OSGeo technical writing project would
> impact any of the documentation.

There was initial discussion but no follow-up as far as GDAL is concerned.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to know which formats allow editing with ogrinfo?

jratike80
In reply to this post by jratike80
Hi,

Two most often needed items from the old format list for me as a community helper were "Creation" to answer "Why gdalwarp to format x fails" and "Compiled by default" to answer "Why everything with my GDAL to format x fails". The list was useful.

-Jukka-

-----Alkuperäinen viesti-----
Lähettäjä: Even Rouault <[hidden email]>
Lähetetty: perjantai 15. marraskuuta 2019 19.55
Vastaanottaja: [hidden email]
Kopio: [hidden email]; Rahkonen Jukka (MML) <[hidden email]>
Aihe: Re: [gdal-dev] How to know which formats allow editing with ogrinfo?

> Can this be partly inferred from driver pages with the presence (or
> absence) of items?  For instance,
> https://gdal.org/drivers/vector/gml.html#driver-capabilities

Not really. They are only high level capabilities. Vector editing capabilities are not reported in driver pages

> The old driver pages
> (https://web.archive.org/web/20190211030817/https://www.gdal.org/ogr_f
> ormats
> .html) were tables that very effectively conveyed the capabilities (or
> limitations) of the drivers and allowed for scanning for drivers that
> had the capabilities that you wanted.

The information in individual pages is roughly the same as in the global table.

> Would a similar table be possible or desirable?

A few people have indeed noted the absence of the table. Not sure if this is usual "resistance to change" phenomenon or if there is there really a strong use case for "I don't want which format to use: let's look at the summary page to select one".

If so, then yes if someone contributes it, that'd be welcome. However I think we'd want it to be generated from a Python script from the individual .rst pages, by parsing the directives like ".. supports_create::" that are in those .rst files. That way that would make maintainance of the summary easier (and initial version less tidious that doing that manually)

> There are a few things that might be missing.  The new page has
> https://gdal.org/programs/raster_common_options.html#raster-common-opt
> ions
> and
> https://gdal.org/programs/vector_common_options.html#vector-common-opt
> ions I wonder if a list of uncommon options like this would be
> helpful, https://trac.osgeo.org/gdal/wiki/ConfigOptions?

Migrating the wiki page to the doc could be useful. My own opinion is that driver specific configuration options should be documented in the driver specific page, rather than in a global config option page. Same with /vsi specific options that should go to https://gdal.org/user/virtual_file_systems.html if not already there.

Or if a global list is needed, I'd say a bit the same as for driver summaries, have a way to compile the list of options from .rst files where they would be mention and link from the summary to the actual page. But we don't have currently any explicit directive for configuration options.

> I was also curious to see if the OSGeo technical writing project would
> impact any of the documentation.

There was initial discussion but no follow-up as far as GDAL is concerned.

Even

--
Spatialys - Geospatial professional services http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: How to know which formats allow editing with ogrinfo?

Eli Adam
In reply to this post by Even Rouault-2
On Fri, Nov 15, 2019 at 9:55 AM Even Rouault <[hidden email]> wrote:

>
> > The old driver pages
> > (https://web.archive.org/web/20190211030817/https://www.gdal.org/ogr_formats
> > .html) were tables that very effectively conveyed the capabilities (or
> > limitations) of the drivers and allowed for scanning for drivers that
> > had the capabilities that you wanted.
>
> The information in individual pages is roughly the same as in the global
> table.
>
> > Would a similar table be possible or desirable?
>
> A few people have indeed noted the absence of the table. Not sure if this is
> usual "resistance to change" phenomenon or if there is there really a strong
> use case for "I don't want which format to use: let's look at the summary page
> to select one".

I've certainly found myself shopping for a format with that table.
Usually some middle format in an ETL toolchain, sometimes complicated
by cross platform considerations for driver availability.

>
> If so, then yes if someone contributes it, that'd be welcome. However I think
> we'd want it to be generated from a Python script from the individual .rst
> pages, by parsing the directives like ".. supports_create::" that are in those
> .rst files. That way that would make maintainance of the summary easier (and
> initial version less tidious that doing that manually)

Contributing it as a python script is an opportunity for someone other
than me.

>
> > There are a few things that might be missing.  The new page has
> > https://gdal.org/programs/raster_common_options.html#raster-common-options
> > and
> > https://gdal.org/programs/vector_common_options.html#vector-common-options
> > I wonder if a list of uncommon options like this would be helpful,
> > https://trac.osgeo.org/gdal/wiki/ConfigOptions?
>
> Migrating the wiki page to the doc could be useful. My own opinion is that
> driver specific configuration options should be documented in the driver
> specific page, rather than in a global config option page. Same with /vsi

I'll try moving some of these over and see how it goes.

> specific options that should go to
> https://gdal.org/user/virtual_file_systems.html if not already there.
>
> Or if a global list is needed, I'd say a bit the same as for driver summaries,
> have a way to compile the list of options from .rst files where they would be
> mention and link from the summary to the actual page. But we don't have
> currently any explicit directive for configuration options.

I'll start with driver specific.

Thanks, Eli

>
> > I was also curious to see if the OSGeo technical writing project would
> > impact any of the documentation.
>
> There was initial discussion but no follow-up as far as GDAL is concerned.
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Even Rouault-2
> > If so, then yes if someone contributes it, that'd be welcome. However I
> > think we'd want it to be generated from a Python script from the
> > individual .rst pages, by parsing the directives like "..
> > supports_create::" that are in those .rst files. That way that would make
> > maintainance of the summary easier (and initial version less tidious that
> > doing that manually)

OK, so I've added two new directives

.. built_in_by_default::

or

.. build_dependencies:: put here dependencies name

and just used them on a couple drivers for demo:
https://github.com/OSGeo/gdal/commit/0b3b1ed25bdde6f49a6515e0431d78bc9729ff83

A script that fetches all the information from individual pages
is run automatically at each documentation build and generates a summary table
at
https://gdal.org/drivers/raster/index.html
https://gdal.org/drivers/vector/index.html

==> What remains to be done is to go through all driver .rst pages and add
.. built_in_by_default:: or .. build_dependencies:: where appropriate so that
no "???" remains

Volunteers ?

> > Migrating the wiki page to the doc could be useful. My own opinion is that
> > driver specific configuration options should be documented in the driver
> > specific page, rather than in a global config option page. Same with /vsi

I've also added a specific role when you use a configuration option

:decl_configoption:`FOO`

Example of use:

https://github.com/OSGeo/gdal/commit/c8fcb6d941c0251219dbd57f701e5b161ce5085e

Similarly to the driver list, a script is run at each build to generate an
index of declared configuration options (if several pages declare the same
config option, several links will be created). The result is there:

https://gdal.org/user/configoptions_index.html

==> What remains to be done is tag appropriately configuration options
wherever they are already mentionned and add missing ones.

Volunteers ?

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Andrew C Aitchison-2
On Sat, 16 Nov 2019, Even Rouault wrote:

>>> If so, then yes if someone contributes it, that'd be welcome. However I
>>> think we'd want it to be generated from a Python script from the
>>> individual .rst pages, by parsing the directives like "..
>>> supports_create::" that are in those .rst files. That way that would make
>>> maintainance of the summary easier (and initial version less tidious that
>>> doing that manually)
>
> OK, so I've added two new directives
>
> .. built_in_by_default::
>
> or
>
> .. build_dependencies:: put here dependencies name
>
> and just used them on a couple drivers for demo:
> https://github.com/OSGeo/gdal/commit/0b3b1ed25bdde6f49a6515e0431d78bc9729ff83
>
> A script that fetches all the information from individual pages
> is run automatically at each documentation build and generates a summary table
> at
> https://gdal.org/drivers/raster/index.html
> https://gdal.org/drivers/vector/index.html

Thanks.

The summary table will need some styling.
At the moment I only see the short and and long name columns.
*If* I scroll down to the bottom, then I get a horizontal scroll bar
and can scroll to see that there are more columns.

I get this with Firefox and Opera on Linux and Firefox and IE11 on Win7
(yes I need to get a newer windows virtual machine).

--
Andrew C. Aitchison Kendal, UK
  [hidden email]
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Even Rouault-2
> The summary table will need some styling.
> At the moment I only see the short and and long name columns.
> *If* I scroll down to the bottom, then I get a horizontal scroll bar
> and can scroll to see that there are more columns.
>
> I get this with Firefox and Opera on Linux and Firefox and IE11 on Win7
> (yes I need to get a newer windows virtual machine).

Hum, displays correctly for me on
- Linux, Firefox 70.0.1
- Linux, Chromium 78.0.3904.97
- W10, Edge 44.18362.449.0
with browser in full width.

If I reduce the width, I also have the scroll bar.

The table is created there:
https://github.com/OSGeo/gdal/blob/master/gdal/doc/source/build_driver_summary.py#L67

and that generates the following HTML:
<colgroup>
<col style="width: 10%" />
<col style="width: 35%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 25%" />
</colgroup>

We'd probably need help from a Sphinx and/or CSS expert to improve that...
But if the width is too small, reducing the column width could make things unreadable,
so having at least the short and long name in reduced page width seems a
reasonable fallback behaviour.

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Eli Adam
In reply to this post by Even Rouault-2
On Sat, Nov 16, 2019 at 9:12 AM Even Rouault <[hidden email]> wrote:

>
> > > If so, then yes if someone contributes it, that'd be welcome. However I
> > > think we'd want it to be generated from a Python script from the
> > > individual .rst pages, by parsing the directives like "..
> > > supports_create::" that are in those .rst files. That way that would make
> > > maintainance of the summary easier (and initial version less tidious that
> > > doing that manually)
>
> OK, so I've added two new directives
>
> .. built_in_by_default::
>
> or
>
> .. build_dependencies:: put here dependencies name
>
> and just used them on a couple drivers for demo:
> https://github.com/OSGeo/gdal/commit/0b3b1ed25bdde6f49a6515e0431d78bc9729ff83
>
> A script that fetches all the information from individual pages
> is run automatically at each documentation build and generates a summary table
> at
> https://gdal.org/drivers/raster/index.html
> https://gdal.org/drivers/vector/index.html
>
> ==> What remains to be done is to go through all driver .rst pages and add
> .. built_in_by_default:: or .. build_dependencies:: where appropriate so that
> no "???" remains
>
> Volunteers ?

If the old table is a reliable source, I can do a good portion of them.

>
> > > Migrating the wiki page to the doc could be useful. My own opinion is that
> > > driver specific configuration options should be documented in the driver
> > > specific page, rather than in a global config option page. Same with /vsi
>
> I've also added a specific role when you use a configuration option
>
> :decl_configoption:`FOO`
>
> Example of use:
>
> https://github.com/OSGeo/gdal/commit/c8fcb6d941c0251219dbd57f701e5b161ce5085e
>
> Similarly to the driver list, a script is run at each build to generate an
> index of declared configuration options (if several pages declare the same
> config option, several links will be created). The result is there:
>
> https://gdal.org/user/configoptions_index.html
>
> ==> What remains to be done is tag appropriately configuration options
> wherever they are already mentionned and add missing ones.
>
> Volunteers ?

I can work on some already mentioned ones.

Thanks for adding this.

Best regards, Eli

>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Eli Adam
On Sun, Nov 17, 2019 at 5:42 PM Eli Adam <[hidden email]> wrote:
> > https://gdal.org/drivers/raster/index.html
> > https://gdal.org/drivers/vector/index.html

These two tables have been updated.  Please check them over.  Let me
know if anything needs to be changed/updated.

Best regards, Eli
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Joe Lee
Hi, Eli!

  I can see two rows for both HDF4 and HDF5 in [1] which points to the same document.
  For example,  HDF4 and HDF4Image points to [2].
  Can you consolidate them into one row?

[1] https://gdal.org/drivers/raster/index.html
[2] https://gdal.org/drivers/raster/hdf4.html#raster-hdf4

On 12/4/19, 12:00 PM, "gdal-dev on behalf of Eli Adam" <[hidden email] on behalf of [hidden email]> wrote:

    On Sun, Nov 17, 2019 at 5:42 PM Eli Adam <[hidden email]> wrote:
    > > https://gdal.org/drivers/raster/index.html
    > > https://gdal.org/drivers/vector/index.html
   
    These two tables have been updated.  Please check them over.  Let me
    know if anything needs to be changed/updated.
   
    Best regards, Eli
    _______________________________________________
    gdal-dev mailing list
    [hidden email]
    https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages [was Re: How to know which formats allow editing with ogrinfo?]

Even Rouault-2
> For example,  HDF4 and HDF4Image points to [2].
>   Can you consolidate them into one row?

Fixed. Website will be refreshed in a few mins

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Doc: driver + config options summary pages

Andrew C Aitchison-2
In reply to this post by Eli Adam
On Wed, 4 Dec 2019, Eli Adam wrote:

> On Sun, Nov 17, 2019 at 5:42 PM Eli Adam <[hidden email]> wrote:
>>> https://gdal.org/drivers/raster/index.html
>>> https://gdal.org/drivers/vector/index.html
>
> These two tables have been updated.  Please check them over.  Let me
> know if anything needs to be changed/updated.

Screen shot from Firefox on Linux, but I have the same issue with
Opera and Windows:
  https://www.aitchison.me.uk/rasterdrivers.png
If the window is *only* 1080 pixels wide I see no hint that there are
more than two columns and that if I scroll to the bottom I will be able
to scroll to the right and see more info about each driver.

--
Andrew C. Aitchison Kendal, UK
  [hidden email]
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev