Dropping support for GDAL < 2 ?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Dropping support for GDAL < 2 ?

Even Rouault-2
Hi,

I'd like to propose to drop support for GDAL < 2 in master.

I'm currently adding a new mapfile keyword LAYER.OPENOPTION to be able to
specify open options to GDAL and OGR drivers, which is a new feature of GDAL
2. While I can conditionnalize that by testing the GDAL version, I see this
more as cluttering of the codebase than something really needed.

GDAL 2.0 was released in June 2015 and is available in the latest stable
versions of Debian, Ubuntu, Fedora, etc.

Travis-CI uses GDAL 2.2 (from UbuntuGIS)
gisinternals builds used by AppVeyor even ship with GDAL 3.1dev

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: Dropping support for GDAL < 2 ?

Tamas Szekeres
Even,

I agree with dropping the support if needed.
The open option seems to be a driver specific attribute, wouldn't it be sufficient to use a processing directive for that?

Best regards,

Tamas



Even Rouault <[hidden email]> ezt írta (időpont: 2019. aug. 22., Cs, 12:44):
Hi,

I'd like to propose to drop support for GDAL < 2 in master.

I'm currently adding a new mapfile keyword LAYER.OPENOPTION to be able to
specify open options to GDAL and OGR drivers, which is a new feature of GDAL
2. While I can conditionnalize that by testing the GDAL version, I see this
more as cluttering of the codebase than something really needed.

GDAL 2.0 was released in June 2015 and is available in the latest stable
versions of Debian, Ubuntu, Fedora, etc.

Travis-CI uses GDAL 2.2 (from UbuntuGIS)
gisinternals builds used by AppVeyor even ship with GDAL 3.1dev

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev

_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: Dropping support for GDAL < 2 ?

Even Rouault-2
On jeudi 22 août 2019 15:29:29 CEST Tamas Szekeres wrote:
> Even,
>
> I agree with dropping the support if needed.
> The open option seems to be a driver specific attribute, wouldn't it be
> sufficient to use a processing directive for that?

I see PROCESSING as more for MapServer own special treatments.
And you would need to somehow indicate that the processing option is an open
option, so something like

PROCESSING "OO:KEY=VALUE"

I find
OPENOPTION "key" "value"
cleaner

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: Dropping support for GDAL < 2 ?

Seth G-2
+1 on dropping old GDAL versions for future MapServer releases. We've done this with SWIG and CMake recently.

With regards to the Mapfile syntax, there is precedent with the PROCESSING "OO:KEY=VALUE" approach in FORMATOPTION:

> FORMATOPTION "METADATA_ITEM:PRODUCER=MapServer, with GDAL trunk"

A new Mapfile keyword will require changes for SWIG support I believe?

My preference from a syntax point of view (although blissfully unaware of the implementation issues it will raise!) would be to turn PROCESSING into a block with the same syntax as METADATA e.g.

PROCESSING
   "CONTOUR_INTERVAL" "5"
   "CLUSTER_GET_ALL_SHAPES" ON"
   "OPENOPTION_KEY1" "VALUE1"
   "OPENOPTION_KEY2" "VALUE2"
END

SWIG could then use a dict type approach in the same way as METADATA to update and add, modify keys and values in the PROCESSING block.

Seth


--
web:http://geographika.co.uk
twitter: @geographika

On Thu, Aug 22, 2019, at 3:46 PM, Even Rouault wrote:

> On jeudi 22 août 2019 15:29:29 CEST Tamas Szekeres wrote:
> > Even,
> >
> > I agree with dropping the support if needed.
> > The open option seems to be a driver specific attribute, wouldn't it be
> > sufficient to use a processing directive for that?
>
> I see PROCESSING as more for MapServer own special treatments.
> And you would need to somehow indicate that the processing option is an open
> option, so something like
>
> PROCESSING "OO:KEY=VALUE"
>
> I find
> OPENOPTION "key" "value"
> cleaner
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> mapserver-dev mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

OPENOPTION (was Re: Dropping support for GDAL < 2 ?)

Even Rouault-2
> A new Mapfile keyword will require changes for SWIG support I believe?

In my initial implementation, this adds a new member (a hashTableObj) in the
struct layerObj. I don't think this would require specific SWIG changes.

>
> My preference from a syntax point of view (although blissfully unaware of
> the implementation issues it will raise!) would be to turn PROCESSING into
> a block with the same syntax as METADATA e.g.
>
> PROCESSING
>    "CONTOUR_INTERVAL" "5"
>    "CLUSTER_GET_ALL_SHAPES" ON"
>    "OPENOPTION_KEY1" "VALUE1"
>    "OPENOPTION_KEY2" "VALUE2"
> END

On the parser side there would be some complication to distinguish the old
PROCESSING "key=value" syntax with the new block approach. I'm not sure we
have a precedent of that.

And the processing member of layerObj is not exposed to SWIG currently. It
uses a char** processing + int numprocessing that are inconvenient to bind to
SWIG. They should likely be converted to a hashTableObj to be exposed to SWIG.

>
> SWIG could then use a dict type approach in the same way as METADATA to
> update and add, modify keys and values in the PROCESSING block.

If OPENOPTION uses a hashTableObj as METADATA does, then this would be similar
to interacting with metadata.

I'm also open to a block based approach for OPENOPTION if that seems better
OPENOPTION
        "key1" "value1"
        "key2" "value2
END


Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: OPENOPTION (was Re: Dropping support for GDAL < 2 ?)

Lime, Steve D (MNIT)

Would CONNECTIONOPTIONS be more generic? I could see that being useful for other drivers where key:value pairs are used as part of the configuration.


For 8.0 a broader conversion of the syntax for layer->processing and outputformat->formatoptions could be implemented. For the latter you'd get:


OUTPUTFORMAT
  NAME "kayml"
  DRIVER "TEMPLATE"
  MIMETYPE "application/vnd.google-earth.kml+xml"
  OPTIONS 
    "FILE" "myTemplate.kml"
    "ATTACHMENT" "queryResults.kml"
  END
END

In this case you could ensure backwards compatibility by adding recognizing FORMATOPTION as a shortcut.

Perhaps a new PROCESSINGOPTIONS block should be added and PROCESSING is handled as a shortcut for backwards compatability.

--Steve



From: mapserver-dev <[hidden email]> on behalf of Even Rouault <[hidden email]>
Sent: Friday, August 23, 2019 7:01 AM
To: [hidden email] <[hidden email]>
Subject: [mapserver-dev] OPENOPTION (was Re: Dropping support for GDAL < 2 ?)
 

> A new Mapfile keyword will require changes for SWIG support I believe?

In my initial implementation, this adds a new member (a hashTableObj) in the
struct layerObj. I don't think this would require specific SWIG changes.

>
> My preference from a syntax point of view (although blissfully unaware of
> the implementation issues it will raise!) would be to turn PROCESSING into
> a block with the same syntax as METADATA e.g.
>
> PROCESSING
>    "CONTOUR_INTERVAL" "5"
>    "CLUSTER_GET_ALL_SHAPES" ON"
>    "OPENOPTION_KEY1" "VALUE1"
>    "OPENOPTION_KEY2" "VALUE2"
> END

On the parser side there would be some complication to distinguish the old
PROCESSING "key=value" syntax with the new block approach. I'm not sure we
have a precedent of that.

And the processing member of layerObj is not exposed to SWIG currently. It
uses a char** processing + int numprocessing that are inconvenient to bind to
SWIG. They should likely be converted to a hashTableObj to be exposed to SWIG.

>
> SWIG could then use a dict type approach in the same way as METADATA to
> update and add, modify keys and values in the PROCESSING block.

If OPENOPTION uses a hashTableObj as METADATA does, then this would be similar
to interacting with metadata.

I'm also open to a block based approach for OPENOPTION if that seems better
OPENOPTION
        "key1" "value1"
        "key2" "value2
END


Even

--
Spatialys - Geospatial professional services
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com&amp;data=02%7C01%7Csteve.lime%40state.mn.us%7C2d231ecf76024bb5306308d727c1956c%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637021584697677675&amp;sdata=1M4fgEWOxeT91edSqgLw1%2B%2FeilqNaurlBqU0MxnEEPk%3D&amp;reserved=0
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&amp;data=02%7C01%7Csteve.lime%40state.mn.us%7C2d231ecf76024bb5306308d727c1956c%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C637021584697677675&amp;sdata=w4nqoDyBBfy2WvF8l0aFfeX95AfMeagf7JdnR8SCMPQ%3D&amp;reserved=0

_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: OPENOPTION (was Re: Dropping support for GDAL < 2 ?)

Even Rouault-2
On vendredi 23 août 2019 13:30:56 CEST Lime, Steve D (MNIT) wrote:
> Would CONNECTIONOPTIONS be more generic? I could see that being useful for
> other drivers where key:value pairs are used as part of the configuration.

Yes, I agree that the mechanism could potentially be used for other connection
types.

For completness, you'd probably also want a DATAOPTIONS to pass options
specific to the DATA keyword. I've always found the syntax for PostGIS data to
be a bit awkward.

So instead of

DATA "the_geom from (select * from polygon3d order by id) as foo using
srid=27700 using unique id"

you could have

DATA "select * from polygon3d order by id"
DATAOPTIONS
        "geometry_column" "the_geom"
        "srid" "27700"
        "unique_column" "id"
END

But this is just food for thought, and we probably don't want to break
existing PostGIS DATA syntax.
My scope for now would be just CONNECTIONOPTIONS and implementation for GDAL
and OGR connections.

>
>
> For 8.0 a broader conversion of the syntax for layer->processing and
> outputformat->formatoptions could be implemented. For the latter you'd get:
>
>
> OUTPUTFORMAT
>   NAME "kayml"
>   DRIVER "TEMPLATE"
>   MIMETYPE "application/vnd.google-earth.kml+xml"
>   OPTIONS
>     "FILE" "myTemplate.kml"
>     "ATTACHMENT" "queryResults.kml"
>   END
> END
>
> In this case you could ensure backwards compatibility by adding recognizing
> FORMATOPTION as a shortcut.
>
> Perhaps a new PROCESSINGOPTIONS block should be added and PROCESSING is
> handled as a shortcut for backwards compatability.

Makes sense

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: OPENOPTION (was Re: Dropping support for GDAL < 2 ?)

Lime, Steve D (MNIT)

Yes, I would leave it to driver owners to capitalize on CONNECTIONOPTIONS (and DATAOPTIONS - good idea!).


--Steve


From: Even Rouault <[hidden email]>
Sent: Friday, August 23, 2019 8:55 AM
To: Lime, Steve D (MNIT) <[hidden email]>
Cc: [hidden email] <[hidden email]>
Subject: Re: [mapserver-dev] OPENOPTION (was Re: Dropping support for GDAL < 2 ?)
 
This message may be from an external email source.
Do not select links or open attachments unless verified. Report all suspicious emails to Minnesota IT Services Security Operations Center.



On vendredi 23 août 2019 13:30:56 CEST Lime, Steve D (MNIT) wrote:
> Would CONNECTIONOPTIONS be more generic? I could see that being useful for
> other drivers where key:value pairs are used as part of the configuration.

Yes, I agree that the mechanism could potentially be used for other connection
types.

For completness, you'd probably also want a DATAOPTIONS to pass options
specific to the DATA keyword. I've always found the syntax for PostGIS data to
be a bit awkward.

So instead of

DATA "the_geom from (select * from polygon3d order by id) as foo using
srid=27700 using unique id"

you could have

DATA "select * from polygon3d order by id"
DATAOPTIONS
        "geometry_column"       "the_geom"
        "srid"                                  "27700"
        "unique_column"         "id"
END

But this is just food for thought, and we probably don't want to break
existing PostGIS DATA syntax.
My scope for now would be just CONNECTIONOPTIONS and implementation for GDAL
and OGR connections.

>
>
> For 8.0 a broader conversion of the syntax for layer->processing and
> outputformat->formatoptions could be implemented. For the latter you'd get:
>
>
> OUTPUTFORMAT
>   NAME "kayml"
>   DRIVER "TEMPLATE"
>   MIMETYPE "application/vnd.google-earth.kml+xml"
>   OPTIONS
>     "FILE" "myTemplate.kml"
>     "ATTACHMENT" "queryResults.kml"
>   END
> END
>
> In this case you could ensure backwards compatibility by adding recognizing
> FORMATOPTION as a shortcut.
>
> Perhaps a new PROCESSINGOPTIONS block should be added and PROCESSING is
> handled as a shortcut for backwards compatability.

Makes sense

Even

--
Spatialys - Geospatial professional services
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com&amp;data=02%7C01%7Csteve.lime%40state.mn.us%7C020baaa6275a47f47b2c08d727d184a8%7Ceb14b04624c445198f26b89c2159828c%7C0%7C1%7C637021653136627039&amp;sdata=odW%2FufcJzUm5CzN8KXSsMs6i1uax8W5cstQGGFuT31c%3D&amp;reserved=0

_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev