A process with JSON inputs and outputs?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

A process with JSON inputs and outputs?

dhohls
Hi all

I am just starting out with pyWPS, and am struggling to get a basic Process working, that takes JSON data as input and produces JSON data as output.

Any feedback on this from the community would be appreciated!

Thanks
Derek

********
PART ONE: My Example

I have posted an example of what I am trying to achieve on github:

https://gist.github.com/gamesbook/3c3ff83f9348a01fff3750f53346db08

I first tried to call the Process with this:

wget --content-on-error -O - 'http://localhost:5000/wps?service=WPS&request=Execute&version=1.0.0&identifier=say_goodbye&datainputs=person={"firstname": "Derek","surname":"Richards","title": "Mr"}'

I get this response:

!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>

The server log from the demo server shows this error trace:

[2017-01-10] ERROR in app: Exception on /wps [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1615, in full_dispatch_request
    return self.finalize_request(rv)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1630, in finalize_request
    response = self.make_response(rv)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1740, in make_response
    rv = self.response_class.force_type(rv, request.environ)
  File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.11.15-py2.7.egg/werkzeug/wrappers.py", line 847, in force_type
    response = BaseResponse(*_run_wsgi_app(response, environ))
  File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.11.15-py2.7.egg/werkzeug/wrappers.py", line 57, in _run_wsgi_app
    return _run_wsgi_app(*args)
  File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.11.15-py2.7.egg/werkzeug/test.py", line 871, in run_wsgi_app
    app_rv = app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/Werkzeug-0.11.15-py2.7.egg/werkzeug/wrappers.py", line 291, in application
    return f(*args[:-2] + (request,))(*args[-2:])
  File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 584, in __call__
    request_uuid
  File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 306, in execute
    response = self._parse_and_execute(process, wps_request, uuid)
  File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 339, in _parse_and_execute
    inpt, wps_request.inputs[inpt.identifier])
  File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 452, in create_complex_inputs
    frmt = data_input.supported_formats[0]
TypeError: 'NoneType' object has no attribute '__getitem__'

As there is no reference point in the above trace to any line in the code I wrote, its not obvious what is triggering this error?

I then created a simple JSON file (`test.json`):

{
    "firstname": "Derek",
    "surname":"Richards",
    "title": "Mr"
}

When I try and call the WPS with this command:

curl -H "Content-Type: application/json" -X POST -F 'datainputs=person=@test.json'  http://localhost:5000/wps?service=WPS&request=Execute&version=1.0.0&identifier=say_goodbye

I get this response:


<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd" version="1.0.0">
  <ows:Exception exceptionCode="NoApplicableCode" locator="" >
      <ows:ExceptionText>Start tag expected, '<' not found, line 1, column 1</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

I assume the POST syntax is wrong, but cannot see what to change?

********
PART TWO: General Comments

I have struggled a bit with the documentation, as it seems to go to a certain point and then be incomplete.  For example, the page at https://github.com/geopython/pywps-demo/blob/master/docs/flask.rst ends by saying:

"NB it is recommended to use HTTP POST requests for invoking WPS Execute operations as normally DataInputs will be more complex."

But then it does not give any example of such an HTTP POST request for a Process with a ComplexInput. It would really help to have examples that work via:
 * command line
 * JavaScript
 * Python

Similarly, there is a fairly detailed example of creating a Process in http://pywps.readthedocs.io/en/latest/process.html , but it also stops short of showing how this process is actually invoked.

The test code in https://github.com/geopython/pywps-demo/blob/master/tests/test_execute.py runs tests for some of the demo processes; but not all of them; it would be helpful to see how the more complex ones are invoked.

The other issue I have is how to debug a new Process; the XML responses from the Server seem quite minimal and I cannot find any guidelines for interpreting them.  Is there a way to ensure that a new Process is constructed properly; are there obvious mistakes and pitfalls to avoid; etc.?



--

This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.

Please consider the environment before printing this email.
_______________________________________________
pywps-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/pywps-dev
Reply | Threaded
Open this post in threaded view
|

Re: A process with JSON inputs and outputs?

Jachym Cepicky
Hi Derek,

another long-time-waiting mail in my stack - I'm very sorry for letting
you wait so long and even it might be too late, I try to answer hoping,
it will be still valid:

Dne 10.1.2017 v 13:32 Derek Hohls napsal(a):

> Hi all
>
> I am just starting out with pyWPS, and am struggling to get a basic Process working, that takes JSON data as input and produces JSON data as output.
>
> Any feedback on this from the community would be appreciated!
>
> Thanks
> Derek
>
> ********
> PART ONE: My Example
>
> I have posted an example of what I am trying to achieve on github:
>
> https://gist.github.com/gamesbook/3c3ff83f9348a01fff3750f53346db08
>
> I first tried to call the Process with this:
>
> wget --content-on-error -O - 'http://localhost:5000/wps?service=WPS&request=Execute&version=1.0.0&identifier=say_goodbye&datainputs=person={"firstname": "Derek","surname":"Richards","title": "Mr"}'
>
> I get this response:
>
> !DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
> <title>500 Internal Server Error</title>
> <h1>Internal Server Error</h1>
> <p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>
>
> The server log from the demo server shows this error trace:

[...]

>   File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 452, in create_complex_inputs
>     frmt = data_input.supported_formats[0]
> TypeError: 'NoneType' object has no attribute '__getitem__'

In your process, the input has default_format=[] -> change it to
supported_formats

>
> As there is no reference point in the above trace to any line in the code I wrote, its not obvious what is triggering this error?
>
> I then created a simple JSON file (`test.json`):
>
> {
>     "firstname": "Derek",
>     "surname":"Richards",
>     "title": "Mr"
> }
>
> When I try and call the WPS with this command:
>
> curl -H "Content-Type: application/json" -X POST -F 'datainputs=person=@test.json'  http://localhost:5000/wps?service=WPS&request=Execute&version=1.0.0&identifier=say_goodbye
>
> I get this response:
>
>
> <ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd" version="1.0.0">
>   <ows:Exception exceptionCode="NoApplicableCode" locator="" >
>       <ows:ExceptionText>Start tag expected, '<' not found, line 1, column 1</ows:ExceptionText>
>   </ows:Exception>
> </ows:ExceptionReport>

WPS assumes, it must be XML input (see some examples in
pywps-demo/static/requests/


>
> I assume the POST syntax is wrong, but cannot see what to change?
>
> ********
> PART TWO: General Comments
>
> I have struggled a bit with the documentation, as it seems to go to a certain point and then be incomplete.  For example, the page at https://github.com/geopython/pywps-demo/blob/master/docs/flask.rst ends by saying:
>
> "NB it is recommended to use HTTP POST requests for invoking WPS Execute operations as normally DataInputs will be more complex."
>
> But then it does not give any example of such an HTTP POST request for a Process with a ComplexInput. It would really help to have examples that work via:
>  * command line
>  * JavaScript
>  * Python

Well, examples of the request are stored in

pywps-demo/static/requests

and

PyWPS/tests/requests/

You might be right in this point, ti is assumed, that you are familiar
with the OGC WPS spec (http://opengeospatial.org/standards/wps),
therefore the documentation is a bit spare at this point.

>
> Similarly, there is a fairly detailed example of creating a Process in http://pywps.readthedocs.io/en/latest/process.html , but it also stops short of showing how this process is actually invoked.

Hm, you might be right at this point too - the documentation continues
in pywps-demo project, but it would be good to have it linked there at least

>
> The test code in https://github.com/geopython/pywps-demo/blob/master/tests/test_execute.py runs tests for some of the demo processes; but not all of them; it would be helpful to see how the more complex ones are invoked.

As I already said, more complex processes should use POST request
encoded to as XML files - you might have link to OGC WPS and request
examples already.

>
> The other issue I have is how to debug a new Process; the XML responses from the Server seem quite minimal and I cannot find any guidelines for interpreting them.  Is there a way to ensure that a new Process is constructed properly; are there obvious mistakes and pitfalls to avoid; etc.?


The server response is not very verbose, that is true, we try to put as
much as possible into server logs.

I'm not aware of many pitfalls or mistakes, if you follow the docs a
bit, it should be working (but as you can see on your github, some bugs
are still there)

Sorry one more time for such late response

your contribution to documentation would be appreciated, if you go any
further

Jachym

--
Jachym Cepicky
e-mail: [hidden email]
twitter: @jachymc
_______________________________________________
pywps-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pywps-dev