[pdal] execute a pipeline from python

Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[pdal] execute a pipeline from python

Dan Lipsa
Hi all,
I can enable streaming in a pipeline using

./pdal pipeline --stream dsm.json

Can I do the same thing in python?
What is the option that I need to set?

pipeline = pdal.Pipeline(json)
pipeline.validate()  # check if our JSON and options were good
count = pipeline.execute()

Thanks,
Dan





_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] execute a pipeline from python

Howard Butler-3


On 1/31/18 1:37 PM, Dan Lipsa wrote:

> Hi all,
> I can enable streaming in a pipeline using
>
> ./pdal pipeline --stream dsm.json
>
> Can I do the same thing in python?
> What is the option that I need to set?
>
> pipeline = pdal.Pipeline(json)
> pipeline.validate()  # check if our JSON and options were good
> count = pipeline.execute()
>
Dan,

No, that bit of Python API isn't done yet. There's a bit of a question
of what we should do there too? Presumably the API would have the user
set a chunk size and then return an iterator. The API to follow would be
the PipelineKernel.cpp implementation, but apply it to the Python/Cython
extension stuff.

I'd be happy to merge a patch that did this.

Howard


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal

signature.asc (560 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] execute a pipeline from python

Dan Lipsa
Hi Howard,
Thanks for your answer.

I was thinking more along the lines of what the command line tool does: just set on option "stream" to the pipeline.
Is this choosing a default chunk (buffer) size?

For instance the following pipeline takes several point cloud "bricks"
and generates the dsm for the whole surface covered.


{
  "pipeline": [
    "a.bpf",
    "b.bpf",
    {
      "resolution": 0.25,
      "filename":"test.tif",
      "output_type": "max",
      "window_size": "20",
      "bounds": "([740698.8745, 741144.3546], [4404897.242, 4405465.48])"
    }
  ]
}




On Wed, Jan 31, 2018 at 2:43 PM, Howard Butler <[hidden email]> wrote:


On 1/31/18 1:37 PM, Dan Lipsa wrote:
> Hi all,
> I can enable streaming in a pipeline using
>
> ./pdal pipeline --stream dsm.json
>
> Can I do the same thing in python?
> What is the option that I need to set?
>
> pipeline = pdal.Pipeline(json)
> pipeline.validate()  # check if our JSON and options were good
> count = pipeline.execute()
>

Dan,

No, that bit of Python API isn't done yet. There's a bit of a question
of what we should do there too? Presumably the API would have the user
set a chunk size and then return an iterator. The API to follow would be
the PipelineKernel.cpp implementation, but apply it to the Python/Cython
extension stuff.

I'd be happy to merge a patch that did this.

Howard


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] execute a pipeline from python

Howard Butler-3
When you .execute() a pipeline in Python, you get back the data in the pipeline.arrays variable. In a stream scenario, that behavior would be disrupted, or we would need to have it return an iterator or something. 

If you want to just execute a pipeline in stream mode without caring for the numpy arrays of the data, is the a reason to not simply use subprocess and invoke the shell?

Howard



On Jan 31, 2018, at 1:54 PM, Dan Lipsa <[hidden email]> wrote:

Hi Howard,
Thanks for your answer.

I was thinking more along the lines of what the command line tool does: just set on option "stream" to the pipeline.
Is this choosing a default chunk (buffer) size?

For instance the following pipeline takes several point cloud "bricks"
and generates the dsm for the whole surface covered.


{
  "pipeline": [
    "a.bpf",
    "b.bpf",
    {
      "resolution": 0.25,
      "filename":"test.tif",
      "output_type": "max",
      "window_size": "20",
      "bounds": "([740698.8745, 741144.3546], [4404897.242, 4405465.48])"
    }
  ]
}




On Wed, Jan 31, 2018 at 2:43 PM, Howard Butler <[hidden email]> wrote:


On 1/31/18 1:37 PM, Dan Lipsa wrote:
> Hi all,
> I can enable streaming in a pipeline using
>
> ./pdal pipeline --stream dsm.json
>
> Can I do the same thing in python?
> What is the option that I need to set?
>
> pipeline = pdal.Pipeline(json)
> pipeline.validate()  # check if our JSON and options were good
> count = pipeline.execute()
>

Dan,

No, that bit of Python API isn't done yet. There's a bit of a question
of what we should do there too? Presumably the API would have the user
set a chunk size and then return an iterator. The API to follow would be
the PipelineKernel.cpp implementation, but apply it to the Python/Cython
extension stuff.

I'd be happy to merge a patch that did this.

Howard


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal



_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal

signature.asc (540 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] execute a pipeline from python

Dan Lipsa


On Wed, Jan 31, 2018 at 3:11 PM, Howard Butler <[hidden email]> wrote:
When you .execute() a pipeline in Python, you get back the data in the pipeline.arrays variable. In a stream scenario, that behavior would be disrupted, or we would need to have it return an iterator or something. 

I see.
 

If you want to just execute a pipeline in stream mode without caring for the numpy arrays of the data, is the a reason to not simply use subprocess and invoke the shell?
I was just thinking about not going through the shell, but probably the effect of that compared with the total time would be negligible.

Thanks again!
 

 
Howard




On Jan 31, 2018, at 1:54 PM, Dan Lipsa <[hidden email]> wrote:

Hi Howard,
Thanks for your answer.

I was thinking more along the lines of what the command line tool does: just set on option "stream" to the pipeline.
Is this choosing a default chunk (buffer) size?

For instance the following pipeline takes several point cloud "bricks"
and generates the dsm for the whole surface covered.


{
  "pipeline": [
    "a.bpf",
    "b.bpf",
    {
      "resolution": 0.25,
      "filename":"test.tif",
      "output_type": "max",
      "window_size": "20",
      "bounds": "([740698.8745, 741144.3546], [4404897.242, 4405465.48])"
    }
  ]
}




On Wed, Jan 31, 2018 at 2:43 PM, Howard Butler <[hidden email]> wrote:


On 1/31/18 1:37 PM, Dan Lipsa wrote:
> Hi all,
> I can enable streaming in a pipeline using
>
> ./pdal pipeline --stream dsm.json
>
> Can I do the same thing in python?
> What is the option that I need to set?
>
> pipeline = pdal.Pipeline(json)
> pipeline.validate()  # check if our JSON and options were good
> count = pipeline.execute()
>

Dan,

No, that bit of Python API isn't done yet. There's a bit of a question
of what we should do there too? Presumably the API would have the user
set a chunk size and then return an iterator. The API to follow would be
the PipelineKernel.cpp implementation, but apply it to the Python/Cython
extension stuff.

I'd be happy to merge a patch that did this.

Howard


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal




_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal