PyCSW Harvesting.

Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

PyCSW Harvesting.

bslunab
Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.

I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.

Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?

Q: I'm currently using Mapserver 7.7.0-dev in windows.

Regards.

Thanks a lot.

_______________________________________________
pycsw-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pycsw-devel
Reply | Threaded
Open this post in threaded view
|

Re: PyCSW Harvesting.

Tom Kralidis
Hi Brian: thanks for the info.  Comments interleaved:

On Tue, Oct 13, 2020 at 4:13 PM BRIAN STIFFENN LUNA BOLIVAR
<[hidden email]> wrote:
>
> Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.
>

What kind of installation problems are you having?

> I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.
>
> Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?
>

You can run a CSW-T Harvest XML POST request with the following XML request:

https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xml

Here, you would replace the <Source> element content with your WMS
base URL.  To invoke XML POST requests, see the following
FAQ item: https://pycsw.org/faq/#how-can-i-make-csw-post-xml-requests

Hope this helps.

..Tom

> Q: I'm currently using Mapserver 7.7.0-dev in windows.
>
> Regards.
>
_______________________________________________
pycsw-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pycsw-devel
Reply | Threaded
Open this post in threaded view
|

Re: PyCSW Harvesting.

bslunab
Thanks a lot for your response Tom, so I have a question, should I follow this steps to harvest a wms service:

1. Modify the following XML request from my WMS service: https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xmlv (I named: post.xml and save it in c:\geiper\post\post.xml)
2. Set in my default.cfg transactions=true
3. When it's done, I should execute this command line:

python bin/pycsw-admin.py -c post_xml -u http://localhost/pycsw/csw.py -x c:\geiper\post\post.xml

4. Refresh the harvested records with the following command line:

python bin/pycsw-admin.py  -c refresh_harvested_records -f default.cfg
Those were all the steps I must follow to harvest records from WMS?

If yes, I get the following error:

b'<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<!-- pycsw 2.4.2 -->\n<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" language="en-US" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"><ows:Exception exceptionCode="NoApplicableCode" locator="service"><ows:ExceptionText>Exception: document not well-formed.\nError: EntityRef: expecting \';\', line 3, column 95 (&lt;string&gt;, line 3).</ows:ExceptionText></ows:Exception></ows:ExceptionReport>'

And My post.xml is formed like this:



  <ResourceType>http://www.opengis.net/wms</ResourceType>
  <ResourceFormat>application/xml</ResourceFormat>
</Harvest>

Thank you so much for your help.




El mar., 13 oct. 2020 a las 19:28, Tom Kralidis (<[hidden email]>) escribió:
Hi Brian: thanks for the info.  Comments interleaved:

On Tue, Oct 13, 2020 at 4:13 PM BRIAN STIFFENN LUNA BOLIVAR
<[hidden email]> wrote:
>
> Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.
>

What kind of installation problems are you having?

> I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.
>
> Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?
>

You can run a CSW-T Harvest XML POST request with the following XML request:

https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xml

Here, you would replace the <Source> element content with your WMS
base URL.  To invoke XML POST requests, see the following
FAQ item: https://pycsw.org/faq/#how-can-i-make-csw-post-xml-requests

Hope this helps.

..Tom

> Q: I'm currently using Mapserver 7.7.0-dev in windows.
>
> Regards.
>

_______________________________________________
pycsw-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pycsw-devel
Reply | Threaded
Open this post in threaded view
|

Re: PyCSW Harvesting.

Tom Kralidis
Hi Brian: it looks like the "&" characters in the <Source> element are
causing XML parsing issues.  They are also
not required in the context of a CSW Harvest request.  Can you update
your XML request with just the base URL
in <Source>?  The request parameters are autogenerated by pycsw accordingly:

<?xml version="1.0" encoding="UTF-8"?>
<Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:gmd="http://www.isotc211.org/2005/gmd"
xmlns:ows="http://www.opengis.net/ows"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:dct="http://purl.org/dc/terms/"
xmlns:gml="http://www.opengis.net/gml"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd"
service="CSW" version="2.0.2">
  <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map</Source>
  <ResourceType>http://www.opengis.net/wms</ResourceType>
  <ResourceFormat>application/xml</ResourceFormat>
</Harvest>

Are you able to try again and report back?

Thanks

..Tom

On Tue, Oct 13, 2020 at 10:04 PM BRIAN STIFFENN LUNA BOLIVAR
<[hidden email]> wrote:

>
> Thanks a lot for your response Tom, so I have a question, should I follow this steps to harvest a wms service:
>
> 1. Modify the following XML request from my WMS service: https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xmlv (I named: post.xml and save it in c:\geiper\post\post.xml)
> 2. Set in my default.cfg transactions=true
> 3. When it's done, I should execute this command line:
>
> python bin/pycsw-admin.py -c post_xml -u http://localhost/pycsw/csw.py -x c:\geiper\post\post.xml
>
> 4. Refresh the harvested records with the following command line:
>
> python bin/pycsw-admin.py  -c refresh_harvested_records -f default.cfg
>
> Those were all the steps I must follow to harvest records from WMS?
>
> If yes, I get the following error:
>
> b'<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<!-- pycsw 2.4.2 -->\n<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" language="en-US" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"><ows:Exception exceptionCode="NoApplicableCode" locator="service"><ows:ExceptionText>Exception: document not well-formed.\nError: EntityRef: expecting \';\', line 3, column 95 (&lt;string&gt;, line 3).</ows:ExceptionText></ows:Exception></ows:ExceptionReport>'
>
> And My post.xml is formed like this:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:ows="http://www.opengis.net/ows" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW" version="2.0.2">
>
>   <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map&SERVICE=WMS&VERSION=1.3.0</Source>
>
>   <ResourceType>http://www.opengis.net/wms</ResourceType>
>   <ResourceFormat>application/xml</ResourceFormat>
> </Harvest>
>
> Thank you so much for your help.
>
>
>
>
>
>
>
> El mar., 13 oct. 2020 a las 19:28, Tom Kralidis (<[hidden email]>) escribió:
>>
>> Hi Brian: thanks for the info.  Comments interleaved:
>>
>> On Tue, Oct 13, 2020 at 4:13 PM BRIAN STIFFENN LUNA BOLIVAR
>> <[hidden email]> wrote:
>> >
>> > Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.
>> >
>>
>> What kind of installation problems are you having?
>>
>> > I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.
>> >
>> > Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?
>> >
>>
>> You can run a CSW-T Harvest XML POST request with the following XML request:
>>
>> https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xml
>>
>> Here, you would replace the <Source> element content with your WMS
>> base URL.  To invoke XML POST requests, see the following
>> FAQ item: https://pycsw.org/faq/#how-can-i-make-csw-post-xml-requests
>>
>> Hope this helps.
>>
>> ..Tom
>>
>> > Q: I'm currently using Mapserver 7.7.0-dev in windows.
>> >
>> > Regards.
>> >
_______________________________________________
pycsw-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pycsw-devel