[pdal] tweaking parameters in filters.smrf

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[pdal] tweaking parameters in filters.smrf

Karl North

Greetings PDAL’ers.

 

I’m new to PDAL, and have recently begun the testing of a PDAL pipeline in hopes of improving an existing automated workflow which uses a TerraScan macro to classify ground points in raw Lidar point clouds.  My pipeline is based on the SMRF filter.

 

By including the ELM and OUTLIER filters, and then juggling the cell size, scalar, slope, threshold, widow size and the cut grid parameters in the SMRF filter, I can achieve a final point cloud that is mostly equivalent to the TSCAN result in many terrain/vegetation/feature scenarios.  Of course, I would like to use PDAL to surpass the TSCAN results.  The main issue that I currently have with my PDAL result is the presence of too much high frequency “noise” visible in the surface defined by the ground classified points.  A lesser issue is the delicate balance, when using the cut grid parameter to help eliminate building points, between the removal of building features and preserving sufficient terrain detail.

 

I should point out that we’ve invested several years (off and on) in tweaking our TSCAN macro with the help of paid support staff to achieve the current excellent result, and that the macro includes 40+ individual steps to achieve this result.  In comparison, to date we have only devoted a couple of weeks to testing/tweaking the PDAL filter parameters.

 

I’ve searched on-line for tutorials/suggestions/experiences that users have had with tweaking the SMRF parameters in various terrain types, vegetation density/types, and cultural feature density/types but have really not found a definitive guide.

 

Is there anyone out there on the mailing list that can suggest such write-ups, or would be willing to share their insights?

 

Perhaps there are other (as yet undiscovered) PDAL filters that I might include in my pipeline either before or after SMRF that would be better suited to the remaining cleanup than tweaking the SMRF parameters themselves.

 

I can share sample LAZ data illustrating a SMRF-vs-TSCAN comparison of current my state-of-the-art, if this will help clarify the current issue(s).  To do so, I could use some guidance on reasonable limits on file size for sharing such a sample.

 

Thanks in advance for any/all suggestions.

 

Karl


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] tweaking parameters in filters.smrf

Bradley Chambers
Karl,

On Fri, Jun 7, 2019 at 12:09 PM Karl North <[hidden email]> wrote:

The main issue that I currently have with my PDAL result is the presence of too much high frequency “noise” visible in the surface defined by the ground classified points.  A lesser issue is the delicate balance, when using the cut grid parameter to help eliminate building points, between the removal of building features and preserving sufficient terrain detail.


The issue you've mentioned with and without cut is well known. It would be interesting to hear more about what you are seeing with the high frequency noise though.
 

Is there anyone out there on the mailing list that can suggest such write-ups, or would be willing to share their insights?


I don't know of any write-ups other than the original paper. The original author also has his MATLAB code on GitHub (https://github.com/thomaspingel/neilpy), though I don't recall that there are any hints on what parameters to tweak for specific environments.
 

Perhaps there are other (as yet undiscovered) PDAL filters that I might include in my pipeline either before or after SMRF that would be better suited to the remaining cleanup than tweaking the SMRF parameters themselves.


One thing that comes to mind, though it would take some experimentation, would be to run a cluster filter (https://pdal.io/stages/filters.cluster.html#filters-cluster) after SMRF and then to filter out clusters that were small. I'd maybe use this without cut to find those roof segments that got left behind and may end up as their own clusters.
 

I can share sample LAZ data illustrating a SMRF-vs-TSCAN comparison of current my state-of-the-art, if this will help clarify the current issue(s).  To do so, I could use some guidance on reasonable limits on file size for sharing such a sample.


It would certainly be interesting to see the data. Other users in the past have posted to Google Drive, Dropbox, or similar.

Brad

_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] tweaking parameters in filters.smrf

Karl North

Brad:

 

Thanks for the suggestions.

 

I’ve done some reading and begun playing around with the cluster filter.  One problem is that I’m new enough to PDAL that I cannot find a way to add/define a dimension and store the cluster ID into either LAS or LAZ format on output.  I think it should be possible to just hijack an existing field in the point data records for current testing purposes.  Does this sound possible?  I’m floundering a bit trying to find an example of something similar.  Can you point me in the right direction?

 

I can get a BPF output, but currently have no way to review the result.  I’m now downloading 64-bit QT Reader, after finding a thread by HoBu saying that this viewer would work for a quick review.

 

I’ve also tried using pdal translate to convert the BPF to LAS, but cannot find any trace of the ClusterID in the resulting LAS file.

 

Thanks for any and all assistance,

 

Karl

 

 

From: Bradley Chambers <[hidden email]>
Sent: Friday, June 7, 2019 11:29 AM
To: Karl North <[hidden email]>
Cc: [hidden email]
Subject: Re: [pdal] tweaking parameters in filters.smrf

 

Karl,

 

On Fri, Jun 7, 2019 at 12:09 PM Karl North <[hidden email]> wrote:

The main issue that I currently have with my PDAL result is the presence of too much high frequency “noise” visible in the surface defined by the ground classified points.  A lesser issue is the delicate balance, when using the cut grid parameter to help eliminate building points, between the removal of building features and preserving sufficient terrain detail.

 

The issue you've mentioned with and without cut is well known. It would be interesting to hear more about what you are seeing with the high frequency noise though.

 

Is there anyone out there on the mailing list that can suggest such write-ups, or would be willing to share their insights?

 

I don't know of any write-ups other than the original paper. The original author also has his MATLAB code on GitHub (https://github.com/thomaspingel/neilpy), though I don't recall that there are any hints on what parameters to tweak for specific environments.

 

Perhaps there are other (as yet undiscovered) PDAL filters that I might include in my pipeline either before or after SMRF that would be better suited to the remaining cleanup than tweaking the SMRF parameters themselves.

 

One thing that comes to mind, though it would take some experimentation, would be to run a cluster filter (https://pdal.io/stages/filters.cluster.html#filters-cluster) after SMRF and then to filter out clusters that were small. I'd maybe use this without cut to find those roof segments that got left behind and may end up as their own clusters.

 

I can share sample LAZ data illustrating a SMRF-vs-TSCAN comparison of current my state-of-the-art, if this will help clarify the current issue(s).  To do so, I could use some guidance on reasonable limits on file size for sharing such a sample.

 

It would certainly be interesting to see the data. Other users in the past have posted to Google Drive, Dropbox, or similar.

 

Brad


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] tweaking parameters in filters.smrf

Howard Butler-3


On Jun 12, 2019, at 9:36 AM, Karl North <[hidden email]> wrote:

Brad:
 
Thanks for the suggestions.
 
I’ve done some reading and begun playing around with the cluster filter.  One problem is that I’m new enough to PDAL that I cannot find a way to add/define a dimension and store the cluster ID into either LAS or LAZ format on output.  I think it should be possible to just hijack an existing field in the point data records for current testing purposes.  Does this sound possible?  I’m floundering a bit trying to find an example of something similar.  Can you point me in the right direction?

Use the filters.ferry to copy ClusterID => PointSourceId downstream of your filters.cluster invocation.



I can get a BPF output, but currently have no way to review the result.  I’m now downloading 64-bit QT Reader, after finding a thread by HoBu saying that this viewer would work for a quick review.

I don't know if that is still viable.

You can also write to LAS with extra bytes

--writers.las.extra_dims=ClusterID

but you'll need something that can view extra LAS dimensions. 


Howard


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] tweaking parameters in filters.smrf

Karl North

Howard:

 

Excellent input.  Thanks.  I’ll first check into using “ferry” to copy ClusterID into the LAS, and then look into adding/viewing extra dimensions in LAS/LAZ in my current software options.

 

QT Reader, by the way, was able to load the BPF file but didn’t make it immediately obvious how I might view the cluster ID.  This is, of course, and about 5 minutes of experience…  😊

 

Karl

 

 

From: Howard Butler <[hidden email]>
Sent: Wednesday, June 12, 2019 9:41 AM
To: Karl North <[hidden email]>
Cc: Bradley Chambers <[hidden email]>; [hidden email]
Subject: Re: [pdal] tweaking parameters in filters.smrf

 

 



On Jun 12, 2019, at 9:36 AM, Karl North <[hidden email]> wrote:

 

Brad:

 

Thanks for the suggestions.

 

I’ve done some reading and begun playing around with the cluster filter.  One problem is that I’m new enough to PDAL that I cannot find a way to add/define a dimension and store the cluster ID into either LAS or LAZ format on output.  I think it should be possible to just hijack an existing field in the point data records for current testing purposes.  Does this sound possible?  I’m floundering a bit trying to find an example of something similar.  Can you point me in the right direction?

 

Use the filters.ferry to copy ClusterID => PointSourceId downstream of your filters.cluster invocation.

 

 



I can get a BPF output, but currently have no way to review the result.  I’m now downloading 64-bit QT Reader, after finding a thread by HoBu saying that this viewer would work for a quick review.

 

I don't know if that is still viable.

 

You can also write to LAS with extra bytes

 

--writers.las.extra_dims=ClusterID

 

but you'll need something that can view extra LAS dimensions. 

 

 

Howard

 


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] tweaking parameters in filters.smrf

Karl North
In reply to this post by Howard Butler-3

Moving along here, and have encountered a new question…

 

I’m getting some promising results from filters.cluster but would certainly like to be able to apply this filter ( and others ) multiple times with different settings while using writers.las after each filter so that I can review the progress of the overall pipeline processing after each step completes.

 

Is there a way to get my pipeline to write a LAS after each of the steps/filters?  I’d like to save interim results for review, but cannot find a good example in the docs or in the list archive.

 

 

 

From: Howard Butler <[hidden email]>
Sent: Wednesday, June 12, 2019 9:41 AM
To: Karl North <[hidden email]>
Cc: Bradley Chambers <[hidden email]>; [hidden email]
Subject: Re: [pdal] tweaking parameters in filters.smrf

 

 



On Jun 12, 2019, at 9:36 AM, Karl North <[hidden email]> wrote:

 

Brad:

 

Thanks for the suggestions.

 

I’ve done some reading and begun playing around with the cluster filter.  One problem is that I’m new enough to PDAL that I cannot find a way to add/define a dimension and store the cluster ID into either LAS or LAZ format on output.  I think it should be possible to just hijack an existing field in the point data records for current testing purposes.  Does this sound possible?  I’m floundering a bit trying to find an example of something similar.  Can you point me in the right direction?

 

Use the filters.ferry to copy ClusterID => PointSourceId downstream of your filters.cluster invocation.

 

 



I can get a BPF output, but currently have no way to review the result.  I’m now downloading 64-bit QT Reader, after finding a thread by HoBu saying that this viewer would work for a quick review.

 

I don't know if that is still viable.

 

You can also write to LAS with extra bytes

 

--writers.las.extra_dims=ClusterID

 

but you'll need something that can view extra LAS dimensions. 

 

 

Howard

 


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal