[QGIS-Developer] OGC API Features provider

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[QGIS-Developer] OGC API Features provider

Even Rouault-2
Hi,

Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.

My initial thought would tend towards to try to add it into the existing WFS
provider. From a UI perspective, I've got some feedback that it would probably
be best to access it through the existing WFS entry to avoid cluttering the UI
with a new provider.
On the code level, the choice between having a dedicated provider or adding
functionnality in the existing WFS one is not so obvious. Technologically,
there is little in common between traditional WFS ( XML & GML based, KVP based
requests ), and OAPI-F (my own acronym for OGC API - Features) (JSON & GeoJSON
based, with a linking approach). But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.

Opinions about above directions welcome.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

ginetto

Luigi Pirelli

**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Book: Mastering QGIS3 - 3rd Edition
* Hire a team: http://www.qcooperative.net
**************************************************************************************************


On Mon, 23 Sep 2019 at 20:52, Even Rouault <[hidden email]> wrote:
Hi,

Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.

My initial thought would tend towards to try to add it into the existing WFS
provider. From a UI perspective, I've got some feedback that it would probably
be best to access it through the existing WFS entry to avoid cluttering the UI
with a new provider.
On the code level, the choice between having a dedicated provider or adding
functionnality in the existing WFS one is not so obvious. Technologically,
there is little in common between traditional WFS ( XML & GML based, KVP based
requests ), and OAPI-F (my own acronym for OGC API - Features) (JSON & GeoJSON
based, with a linking approach). But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.

Opinions about above directions welcome.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

ginetto
and this https://github.com/qgis/QGIS/pull/10016

Luigi Pirelli

**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Book: Mastering QGIS3 - 3rd Edition
* Hire a team: http://www.qcooperative.net
**************************************************************************************************


On Mon, 23 Sep 2019 at 22:22, Luigi Pirelli <[hidden email]> wrote:

Luigi Pirelli

**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Book: Mastering QGIS3 - 3rd Edition
* Hire a team: http://www.qcooperative.net
**************************************************************************************************


On Mon, 23 Sep 2019 at 20:52, Even Rouault <[hidden email]> wrote:
Hi,

Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.

My initial thought would tend towards to try to add it into the existing WFS
provider. From a UI perspective, I've got some feedback that it would probably
be best to access it through the existing WFS entry to avoid cluttering the UI
with a new provider.
On the code level, the choice between having a dedicated provider or adding
functionnality in the existing WFS one is not so obvious. Technologically,
there is little in common between traditional WFS ( XML & GML based, KVP based
requests ), and OAPI-F (my own acronym for OGC API - Features) (JSON & GeoJSON
based, with a linking approach). But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.

Opinions about above directions welcome.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Even Rouault-2
On lundi 23 septembre 2019 22:24:58 CEST Luigi Pirelli wrote:
> and this https://github.com/qgis/QGIS/pull/10016

Yes, we're talking about the same API. Alessandro's work was on the server
side, and here I plan to address the client side (potentially working with any
server conformant implementation)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Jeremy Palmer-3
In reply to this post by Even Rouault-2
Hi Even,

On Tue, Sep 24, 2019 at 4:52 AM Even Rouault <[hidden email]> wrote:
Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.


Great news - exciting!
 
But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.


Whichever path you take, it would be fantastic to use the same codebase for the WFS Spatialite based cache as it is well used and tested.

Cheers
Jeremy
 

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Régis Haubourg
In reply to this post by Even Rouault-2
Great news Even! 

Regards 
Régis

Le lun. 23 sept. 2019 à 20:52, Even Rouault <[hidden email]> a écrit :
Hi,

Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.

My initial thought would tend towards to try to add it into the existing WFS
provider. From a UI perspective, I've got some feedback that it would probably
be best to access it through the existing WFS entry to avoid cluttering the UI
with a new provider.
On the code level, the choice between having a dedicated provider or adding
functionnality in the existing WFS one is not so obvious. Technologically,
there is little in common between traditional WFS ( XML & GML based, KVP based
requests ), and OAPI-F (my own acronym for OGC API - Features) (JSON & GeoJSON
based, with a linking approach). But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.

Opinions about above directions welcome.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Alessandro Pasotti-2
In reply to this post by Even Rouault-2

On Mon, Sep 23, 2019 at 8:51 PM Even Rouault <[hidden email]> wrote:
Hi,

Just a quick note to mention, to avoid any potential duplication of work if
someone else is about to do that, that in the coming weeks I'll work on adding
support for "OGC API - Features" (previously known as WFS3) on the client
side, now that it is about to be finalized to its 1.0 version.


Hi Even,

I'm happy to see you taking this up!

 
My initial thought would tend towards to try to add it into the existing WFS
provider. From a UI perspective, I've got some feedback that it would probably
be best to access it through the existing WFS entry to avoid cluttering the UI
with a new provider.

I would probably advise against this approach, from the architectural point of view, I think that it would be better to address the new OGC JSON-based API with an abstract base class for providers that consume that kind of API and make WFS3 the first concrete implementation of that base.
 
On the code level, the choice between having a dedicated provider or adding
functionnality in the existing WFS one is not so obvious. Technologically,
there is little in common between traditional WFS ( XML & GML based, KVP based
requests ), and OAPI-F (my own acronym for OGC API - Features) (JSON & GeoJSON
based, with a linking approach). But the WFS provider has something which is
quite useful for the OAPI-F context, which is the local Spatialite-based cache
& the background download capability. Extracting that from the WFS provider
and be generic enough for multiple providers could be quite involved.


Re-using the WFS-sqlite feature cache looks a good idea, you might want to refactor it out to core to make it re-usable from a family of providers.
 
Opinions about above directions welcome.

Even

Looking forward to test it against QGIS Server, we will be providing a complete client+server solution!

Thanks!

--
Alessandro Pasotti
w3:   www.itopen.it

_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Even Rouault-2
> I would probably advise against this approach, from the architectural point
> of view, I think that it would be better to address the new OGC JSON-based
> API with an abstract base class for providers that consume that kind of API
> and make WFS3 the first concrete implementation of that base.

While I understand this idea, I see several difficulties:
- the candidate providers are the feature one, a coverage and a tile / map
one, but vector and raster providers have little in common
- at the time of writing, to the best of my knowledge, only the OAPI-F has
reached a sufficient degree of maturity in its spec and initial
implementations. In particular while I think there is an intention to have a
OAPI-Common at some stage, I don't think this has emerged yet (the current
github repo for it is a copy&paste of OAPI-F), so making an abstraction with
just a single implementation is not going to work well at that stage.
- if looking at the existing providers in QGIS, the arcgisrest ones seem to be
the closest to what you mention above. From what I see, the code between the
AMS and AFS is quite separate, likely for the reasons I mentionned in my first
point. They do have some utility functions qgsarcgisrestutils.h in common, so
that would be more the kind of communality I can imagine if a OAPI-Tile/Map
provider is later added. Perhaps things like parsing service metadata.

Regarding the UI, I'm not completely sure of the best option. I can understand
the opinion I got that reusing the WFS UI could be better because people are
familiar with it, and that would limit the number of provider entries (people
just have to figure out if the URL they are provided with is a WFS, OAPI-F or
ArcGIS REST Feature one...). It can also makes sense if the OAPI-F stuff is
added to the existing WFS provider.
The other point of view, having a dedicated UI & provider, has also its
advantages in term of code clarity (but provided the point below can be
solved)

> Re-using the WFS-sqlite feature cache looks a good idea, you might want to
> refactor it out to core to make it re-usable from a family of providers.

I haven't completely given up on trying that idea. Just scares me :-)

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Stefan Keller
Hi,

As I already talked about elsewhere I'm implementing a "minimal OGC-API Features
Server".

Now I'd like to test this service with QGIS as client. And I also
tested QGIS with the "kataster" service [1] the mentioned in the docs
[2].
Unfortunately I count not get both services to work (display) in QGIS.

* Our own service is being asked by QGIS with bbox and "limit=10&" any
time a zoom or pan occurs. But QGIS displays just 10 which are not
replaced by others whatever I do with QGIS.
* The kataster service asks about the CRS, then reports a large number
of features (which means there must be a different "limit="), and
finally does not answer requests from QGIS anymore.

Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
add limit=10?

Question 2: Does anybody know about any another OGC-API Features
service we could use to test?

:Stefan


[1] https://www.ldproxy.nrw.de/rest/services/kataster
[2] https://gdal.org/drivers/vector/oapif.html




Am Di., 24. Sept. 2019 um 11:26 Uhr schrieb Even Rouault
<[hidden email]>:

>
> > I would probably advise against this approach, from the architectural point
> > of view, I think that it would be better to address the new OGC JSON-based
> > API with an abstract base class for providers that consume that kind of API
> > and make WFS3 the first concrete implementation of that base.
>
> While I understand this idea, I see several difficulties:
> - the candidate providers are the feature one, a coverage and a tile / map
> one, but vector and raster providers have little in common
> - at the time of writing, to the best of my knowledge, only the OAPI-F has
> reached a sufficient degree of maturity in its spec and initial
> implementations. In particular while I think there is an intention to have a
> OAPI-Common at some stage, I don't think this has emerged yet (the current
> github repo for it is a copy&paste of OAPI-F), so making an abstraction with
> just a single implementation is not going to work well at that stage.
> - if looking at the existing providers in QGIS, the arcgisrest ones seem to be
> the closest to what you mention above. From what I see, the code between the
> AMS and AFS is quite separate, likely for the reasons I mentionned in my first
> point. They do have some utility functions qgsarcgisrestutils.h in common, so
> that would be more the kind of communality I can imagine if a OAPI-Tile/Map
> provider is later added. Perhaps things like parsing service metadata.
>
> Regarding the UI, I'm not completely sure of the best option. I can understand
> the opinion I got that reusing the WFS UI could be better because people are
> familiar with it, and that would limit the number of provider entries (people
> just have to figure out if the URL they are provided with is a WFS, OAPI-F or
> ArcGIS REST Feature one...). It can also makes sense if the OAPI-F stuff is
> added to the existing WFS provider.
> The other point of view, having a dedicated UI & provider, has also its
> advantages in term of code clarity (but provided the point below can be
> solved)
>
> > Re-using the WFS-sqlite feature cache looks a good idea, you might want to
> > refactor it out to core to make it re-usable from a family of providers.
>
> I haven't completely given up on trying that idea. Just scares me :-)
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> QGIS-Developer mailing list
> [hidden email]
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Even Rouault-2
On mardi 10 décembre 2019 20:36:22 CET Stefan Keller wrote:

> Hi,
>
> As I already talked about elsewhere I'm implementing a "minimal OGC-API
> Features Server".
>
> Now I'd like to test this service with QGIS as client. And I also
> tested QGIS with the "kataster" service [1] the mentioned in the docs
> [2].
> Unfortunately I count not get both services to work (display) in QGIS.
>
> * Our own service is being asked by QGIS with bbox and "limit=10&" any
> time a zoom or pan occurs. But QGIS displays just 10 which are not
> replaced by others whatever I do with QGIS.

Perhaps your service doesn't really return true unique ids ?

> * The kataster service asks about the CRS, then reports a large number
> of features (which means there must be a different "limit="), and
> finally does not answer requests from QGIS anymore.

Works for me.

>
> Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
> add limit=10?

Because that's the page size it detects from your service, or more presumably
the default value if your service doesn't implement /api properly. You can
tune it in the "WFS option" group of the "Modify WFS connection" dialog

>
> Question 2: Does anybody know about any another OGC-API Features
> service we could use to test?

See list in comment of https://github.com/qgis/QGIS/pull/32262

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Stefan Keller
Dear Even (dear Richard, dear all)

Many thanks for your replies. Thanks to your instant feedback we got
our minimal server implementation to work (also with QGIS).

Our issue was with the "/collections/castles/items" endpoint, where we
missed to include a link to the next page so that the (QGIS) reader
client can iterate through all the features pageuntil they’re all
read.

Finally, did I understood you correctly regarding the QGIS client options:

At 10. Dec. 2019 20:48 Even Rouault wrote:
> > Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
> > add limit=10?
>
> Because that's the page size it detects from your service, or more presumably
> the default value if your service doesn't implement /api properly. You can
> tune it in the "WFS option" group of the "Modify WFS connection" dialog

I've added the WFS3 (alias OGC API-F) service using QGIS menu item
"Add Vector Layer..." in menu "Layer".
So you are referring to entry field "Request step size" in the "Data
Soure Manager WMS/WMTS dialog?

:Stefan

[1] https://pro.arcgis.com/en/pro-app/help/data/services/use-wfs-services.htm


Am Di., 10. Dez. 2019 um 20:48 Uhr schrieb Even Rouault
<[hidden email]>:
> > Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
> > add limit=10?
>
> Because that's the page size it detects from your service, or more presumably
> the default value if your service doesn't implement /api properly. You can
> tune it in the "WFS option" group of the "Modify WFS connection" dialog


Am Di., 10. Dez. 2019 um 20:48 Uhr schrieb Even Rouault
<[hidden email]>:

>
> On mardi 10 décembre 2019 20:36:22 CET Stefan Keller wrote:
> > Hi,
> >
> > As I already talked about elsewhere I'm implementing a "minimal OGC-API
> > Features Server".
> >
> > Now I'd like to test this service with QGIS as client. And I also
> > tested QGIS with the "kataster" service [1] the mentioned in the docs
> > [2].
> > Unfortunately I count not get both services to work (display) in QGIS.
> >
> > * Our own service is being asked by QGIS with bbox and "limit=10&" any
> > time a zoom or pan occurs. But QGIS displays just 10 which are not
> > replaced by others whatever I do with QGIS.
>
> Perhaps your service doesn't really return true unique ids ?
>
> > * The kataster service asks about the CRS, then reports a large number
> > of features (which means there must be a different "limit="), and
> > finally does not answer requests from QGIS anymore.
>
> Works for me.
>
> >
> > Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
> > add limit=10?
>
> Because that's the page size it detects from your service, or more presumably
> the default value if your service doesn't implement /api properly. You can
> tune it in the "WFS option" group of the "Modify WFS connection" dialog
>
> >
> > Question 2: Does anybody know about any another OGC-API Features
> > service we could use to test?
>
> See list in comment of https://github.com/qgis/QGIS/pull/32262
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Reply | Threaded
Open this post in threaded view
|

Re: OGC API Features provider

Even Rouault-2
> Finally, did I understood you correctly regarding the QGIS client options:
>
> At 10. Dec. 2019 20:48 Even Rouault wrote:
> > > Question 1: Any idea what QGIS makes it's OGC-API-Features reader to
> > > add limit=10?
> >
> > Because that's the page size it detects from your service, or more
> > presumably the default value if your service doesn't implement /api
> > properly. You can tune it in the "WFS option" group of the "Modify WFS
> > connection" dialog
> I've added the WFS3 (alias OGC API-F) service using QGIS menu item
> "Add Vector Layer..." in menu "Layer".

No, this way you'll use the OGR WFS3/OAPIF driver

I mean the "Add WFS Layer" entry (which should be renamed by the way to "Add
WFS / OGC API - Features Layer") which will make you go to the "WFS / OGC API
- Features" tab of Data Source Manager. There you will use the native QGIS
provider (if using QGIS master)

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
QGIS-Developer mailing list
[hidden email]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer