Corrupt attributes when trying to expose features through WFS

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Corrupt attributes when trying to expose features through WFS

Brecht Van Maldergem

Hi;

 

We’re using for quite some time MapServer already and are very pleased with it.
Recently though, we noticed a strange behavior.

We’re trying to load a ShapeFile, and expose it through a WFS, we get -on specific locations- a very strange output:

Attribute-values get shifted into other fields (eg text-values appear partially in number-fields) when fetching the output (as geojson in this case).
When loading the original .shp file directly into QGIS, all is fine.

When using the OGR ConnectionType, all is fine too.
So the problem only occurs with the mapserver-build-in shape-reader.

 

Someone who has a clue here?

We assume it could be related to the number of features; 16’948’045 features

Total FileSize is around 2.5GB

 

Thanks in advance!

 

 

This configuration for the WFS does not work;

LAYER

    NAME "segments"

    TYPE LINE

    DATA  "segments"

    METADATA

      "wfs_title"         "segments"

      "gml_include_items" "all"

      "gml_featureid"     "segmentID"

    END # METADATA

END # LAYER

 

This one does work though, but is too slow (response times around 1 second instead of 150ms)

LAYER

    NAME "segments"

    TYPE LINE

    CONNECTIONTYPE OGR

    CONNECTION "C:/ms4w/Apache/htdocs/Be-Mobile/flowCheckIssue/input/original/segments.shp"

      "wfs_title"         "segments"

      "gml_include_items" "all"

      "gml_featureid"     "segmentID"

    END # METADATA

END # LAYER

 

The shapefile itself contains a lot of features though: 16’948’045 features

 

ogrinfo ./segments.shp -al -so

INFO: Open of `./segments.shp'

      using driver `ESRI Shapefile' successful.

 

Layer name: segments

Geometry: Line String

Feature Count: 16948045

Extent: (2.432331, 49.369946) - (7.280179, 53.505310)

Layer SRS WKT:

GEOGCS["WGS 84",

    DATUM["WGS_1984",

        SPHEROID["WGS 84",6378137,298.257223563,

            AUTHORITY["EPSG","7030"]],

        AUTHORITY["EPSG","6326"]],

    PRIMEM["Greenwich",0,

        AUTHORITY["EPSG","8901"]],

    UNIT["degree",0.0174532925199433,

        AUTHORITY["EPSG","9122"]],

    AUTHORITY["EPSG","4326"]]

 

Setup:

» ogr2ogr --version

GDAL 1.11.3, released 2015/09/16

 

» ./mapserv -v                                                       

MapServer version 7.0.7 (MS4W 3.2.4) OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO SUPPORTS=SVG_SYMBOLS SUPPORTS=SVGCAIRO SUPPORTS=ICONV SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS SUPPORTS=POINT_Z_M INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE

 

Best Regards,

Brecht Van Maldergem

 


_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev
Reply | Threaded
Open this post in threaded view
|

Re: Corrupt attributes when trying to expose features through WFS

jmckenna
Administrator
Hi Brecht,

It will be hard to debug this without access to that shapefile.

I do notice that your ogrogr version does not match the MS4W version
(meaning that you have multiple GDAL installations on that same machine,
which is going to cause trouble).  The GDAL version should be "GDAL
2.2.3, released 2017/11/20"

But that could be unrelated.

I am very glad to hear though that you are so happy with MS4W in your
organization.

-jeff



--
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/


On 2018-03-02 10:51 AM, Brecht Van Maldergem wrote:

> Hi;
>
> We’re using for quite some time MapServer already and are very pleased
> with it.
> Recently though, we noticed a strange behavior.
>
> We’re trying to load a ShapeFile, and expose it through a WFS, we get
> -on specific locations- a very strange output:
>
> Attribute-values get shifted into other fields (eg text-values appear
> partially in number-fields) when fetching the output (as geojson in this
> case).
> When loading the original .shp file directly into QGIS, all is fine.
>
> When using the OGR ConnectionType, all is fine too.
> So the problem only occurs with the mapserver-build-in shape-reader.
>
> Someone who has a clue here?
>
> We assume it could be related to the number of features; 16’948’045 features
>
> Total FileSize is around 2.5GB
>
> Thanks in advance!
>
> This configuration for the WFS does *not* work;
>
> LAYER
>
>      NAME "segments"
>
>      TYPE LINE
>
>      DATA  "segments"
>
>      METADATA
>
>        "wfs_title"         "segments"
>
>        "gml_include_items" "all"
>
>        "gml_featureid"     "segmentID"
>
>      END # METADATA
>
> END # LAYER
>
> This one does work though, but is too slow (response times around 1
> second instead of 150ms)
>
> LAYER
>
>      NAME "segments"
>
>      TYPE LINE
>
>      CONNECTIONTYPE OGR
>
>      CONNECTION
> "C:/ms4w/Apache/htdocs/Be-Mobile/flowCheckIssue/input/original/segments.shp"
>
>        "wfs_title"         "segments"
>
>        "gml_include_items" "all"
>
>        "gml_featureid"     "segmentID"
>
>      END # METADATA
>
> END # LAYER
>
> The shapefile itself contains a lot of features though: 16’948’045 features
>
> ogrinfo ./segments.shp -al -so
>
> INFO: Open of `./segments.shp'
>
>        using driver `ESRI Shapefile' successful.
>
> Layer name: segments
>
> Geometry: Line String
>
> Feature Count: 16948045
>
> Extent: (2.432331, 49.369946) - (7.280179, 53.505310)
>
> Layer SRS WKT:
>
> GEOGCS["WGS 84",
>
>      DATUM["WGS_1984",
>
>          SPHEROID["WGS 84",6378137,298.257223563,
>
>              AUTHORITY["EPSG","7030"]],
>
>          AUTHORITY["EPSG","6326"]],
>
>      PRIMEM["Greenwich",0,
>
>          AUTHORITY["EPSG","8901"]],
>
>      UNIT["degree",0.0174532925199433,
>
>          AUTHORITY["EPSG","9122"]],
>
>      AUTHORITY["EPSG","4326"]]
>
> *_Setup:_*
>
> » ogr2ogr --version
>
> GDAL 1.11.3, released 2015/09/16
>
> » ./mapserv -v
>
> MapServer version 7.0.7 (MS4W 3.2.4) OUTPUT=PNG OUTPUT=JPEG OUTPUT=KML
> SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=CAIRO
> SUPPORTS=SVG_SYMBOLS SUPPORTS=SVGCAIRO SUPPORTS=ICONV SUPPORTS=FRIBIDI
> SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER
> SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER
> SUPPORTS=FASTCGI SUPPORTS=THREADS SUPPORTS=GEOS SUPPORTS=POINT_Z_M
> INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE
>
> Best Regards,
>
> *Brecht Van Maldergem
>
> *
>
_______________________________________________
mapserver-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev