[pdal] filtering duplicate points

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[pdal] filtering duplicate points

Andrew Cunliffe
Dear all,

I am hoping someone might be able to help me to use PDAL to filter for duplicate points in a point cloud.

By duplicate, I mean points that are less than a threshold distance (e.g. 0.001 m) from other points in the cloud. I'm working with photogrammetrically-generated point clouds, in some cases ca. 20-30% of the points are 'duplicates' so removing these has some big benefits later in the pipeline.

I'm looking for a filter that functions like 'Remove duplicate points' in CloudCompare. I wasn't able to see anything suitable under the filters > cull documentation, unless filters.voxelcenternearestneighbor would achieve this with cell set to 0.001?

If anyone else thinks that a filter.duplicate function might be helpful, I'm happy to submit a feature request via Github.

Thanks
Andy

--
Dr Andrew Cunliffe

Research Fellow

School of Geography
University of Exeter


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] filtering duplicate points

Michael Smith

Andrew,

 

You can use filters.sample and just set a small radius.

 

Mike

 

 

-- 

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

 

 

From: pdal <[hidden email]> on behalf of Andrew Cunliffe <[hidden email]>
Date: Monday, May 13, 2019 at 8:21 AM
To: PDAL Mailing List <[hidden email]>
Subject: [pdal] filtering duplicate points

 

Dear all,

 

I am hoping someone might be able to help me to use PDAL to filter for duplicate points in a point cloud.

 

By duplicate, I mean points that are less than a threshold distance (e.g. 0.001 m) from other points in the cloud. I'm working with photogrammetrically-generated point clouds, in some cases ca. 20-30% of the points are 'duplicates' so removing these has some big benefits later in the pipeline.

 

I'm looking for a filter that functions like 'Remove duplicate points' in CloudCompare. I wasn't able to see anything suitable under the filters > cull documentation, unless filters.voxelcenternearestneighbor would achieve this with cell set to 0.001?

 

If anyone else thinks that a filter.duplicate function might be helpful, I'm happy to submit a feature request via Github.

 

Thanks

Andy


--

Dr Andrew Cunliffe

Research Fellow
School of Geography
University of Exeter

 

_______________________________________________ pdal mailing list [hidden email] https://lists.osgeo.org/mailman/listinfo/pdal


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] filtering duplicate points

Andrew Cunliffe
Thanks for the suggestion Mike,

I just tried running
                        {
                            "type": "filters.sample",
                            "radius": 0.001
                        },
on a test cloud with n = 6650 points. With a 0.001 m threshold CloudCompare removes 1782 as duplicates (ca. 27%), and makes no visible difference to the displayed cloud, so the points removed appear genuine. However, with the above parameters, this PDAL pipeline only removes 6 duplicates.

I wondered whether filters.sample perhaps only considers proximity in XY space? (this isn't currently specified in the documentation). Apologies if I have missed something simple, I'm still learning PDAL, but I like it.

Thanks,
Andy



On Mon, 13 May 2019 at 13:26, Michael Smith <[hidden email]> wrote:

Andrew,

 

You can use filters.sample and just set a small radius.

 

Mike

 

 

-- 

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

 

 

From: pdal <[hidden email]> on behalf of Andrew Cunliffe <[hidden email]>
Date: Monday, May 13, 2019 at 8:21 AM
To: PDAL Mailing List <[hidden email]>
Subject: [pdal] filtering duplicate points

 

Dear all,

 

I am hoping someone might be able to help me to use PDAL to filter for duplicate points in a point cloud.

 

By duplicate, I mean points that are less than a threshold distance (e.g. 0.001 m) from other points in the cloud. I'm working with photogrammetrically-generated point clouds, in some cases ca. 20-30% of the points are 'duplicates' so removing these has some big benefits later in the pipeline.

 

I'm looking for a filter that functions like 'Remove duplicate points' in CloudCompare. I wasn't able to see anything suitable under the filters > cull documentation, unless filters.voxelcenternearestneighbor would achieve this with cell set to 0.001?

 

If anyone else thinks that a filter.duplicate function might be helpful, I'm happy to submit a feature request via Github.

 

Thanks

Andy


--

Dr Andrew Cunliffe

Research Fellow
School of Geography
University of Exeter

 

_______________________________________________ pdal mailing list [hidden email] https://lists.osgeo.org/mailman/listinfo/pdal



--
Dr Andrew Cunliffe

Research Fellow in Dryland Carbon Dynamics

School of Geography, College of Life and Environmental Science,
University of Exeter



_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] filtering duplicate points

andrew.bell.ia@gmail.com

filters.sample works in 3D.  I'm not sure about the difference you're seeing.  I don't know what cloud compare is doing.  You're welcome to submit a bug report if you have some data that you think isn't being handled properly.

On Mon, May 13, 2019 at 8:56 AM Andrew Cunliffe <[hidden email]> wrote:
Thanks for the suggestion Mike,

I just tried running
                        {
                            "type": "filters.sample",
                            "radius": 0.001
                        },
on a test cloud with n = 6650 points. With a 0.001 m threshold CloudCompare removes 1782 as duplicates (ca. 27%), and makes no visible difference to the displayed cloud, so the points removed appear genuine. However, with the above parameters, this PDAL pipeline only removes 6 duplicates.

I wondered whether filters.sample perhaps only considers proximity in XY space? (this isn't currently specified in the documentation). Apologies if I have missed something simple, I'm still learning PDAL, but I like it.

Thanks,
Andy



On Mon, 13 May 2019 at 13:26, Michael Smith <[hidden email]> wrote:

Andrew,

 

You can use filters.sample and just set a small radius.

 

Mike

 

 

-- 

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

 

 

From: pdal <[hidden email]> on behalf of Andrew Cunliffe <[hidden email]>
Date: Monday, May 13, 2019 at 8:21 AM
To: PDAL Mailing List <[hidden email]>
Subject: [pdal] filtering duplicate points

 

Dear all,

 

I am hoping someone might be able to help me to use PDAL to filter for duplicate points in a point cloud.

 

By duplicate, I mean points that are less than a threshold distance (e.g. 0.001 m) from other points in the cloud. I'm working with photogrammetrically-generated point clouds, in some cases ca. 20-30% of the points are 'duplicates' so removing these has some big benefits later in the pipeline.

 

I'm looking for a filter that functions like 'Remove duplicate points' in CloudCompare. I wasn't able to see anything suitable under the filters > cull documentation, unless filters.voxelcenternearestneighbor would achieve this with cell set to 0.001?

 

If anyone else thinks that a filter.duplicate function might be helpful, I'm happy to submit a feature request via Github.

 

Thanks

Andy


--

Dr Andrew Cunliffe

Research Fellow
School of Geography
University of Exeter

 

_______________________________________________ pdal mailing list [hidden email] https://lists.osgeo.org/mailman/listinfo/pdal



--
Dr Andrew Cunliffe

Research Fellow in Dryland Carbon Dynamics

School of Geography, College of Life and Environmental Science,
University of Exeter


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal


--
Andrew Bell
[hidden email]

_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal
Reply | Threaded
Open this post in threaded view
|

Re: [pdal] filtering duplicate points

andrew.bell.ia@gmail.com
Andrew,

A look at the cloud compare code suggests it works very differently from PDAL filters.sample and depends on the way that points are distributed in an octree.  Rather than checking actual distance, CC tries to remove points in octree cells, that may be closer than the specified distance, depending on the octree organization.  You might try setting the distance smaller in PDAL to see if you get results more to your liking.  (This is based on a very cursory look at the CC code, so take it with a grain of salt.)

On Mon, May 13, 2019 at 8:59 AM Andrew Bell <[hidden email]> wrote:

filters.sample works in 3D.  I'm not sure about the difference you're seeing.  I don't know what cloud compare is doing.  You're welcome to submit a bug report if you have some data that you think isn't being handled properly.

On Mon, May 13, 2019 at 8:56 AM Andrew Cunliffe <[hidden email]> wrote:
Thanks for the suggestion Mike,

I just tried running
                        {
                            "type": "filters.sample",
                            "radius": 0.001
                        },
on a test cloud with n = 6650 points. With a 0.001 m threshold CloudCompare removes 1782 as duplicates (ca. 27%), and makes no visible difference to the displayed cloud, so the points removed appear genuine. However, with the above parameters, this PDAL pipeline only removes 6 duplicates.

I wondered whether filters.sample perhaps only considers proximity in XY space? (this isn't currently specified in the documentation). Apologies if I have missed something simple, I'm still learning PDAL, but I like it.

Thanks,
Andy



On Mon, 13 May 2019 at 13:26, Michael Smith <[hidden email]> wrote:

Andrew,

 

You can use filters.sample and just set a small radius.

 

Mike

 

 

-- 

Michael Smith

Remote Sensing/GIS Center

US Army Corps of Engineers

 

 

From: pdal <[hidden email]> on behalf of Andrew Cunliffe <[hidden email]>
Date: Monday, May 13, 2019 at 8:21 AM
To: PDAL Mailing List <[hidden email]>
Subject: [pdal] filtering duplicate points

 

Dear all,

 

I am hoping someone might be able to help me to use PDAL to filter for duplicate points in a point cloud.

 

By duplicate, I mean points that are less than a threshold distance (e.g. 0.001 m) from other points in the cloud. I'm working with photogrammetrically-generated point clouds, in some cases ca. 20-30% of the points are 'duplicates' so removing these has some big benefits later in the pipeline.

 

I'm looking for a filter that functions like 'Remove duplicate points' in CloudCompare. I wasn't able to see anything suitable under the filters > cull documentation, unless filters.voxelcenternearestneighbor would achieve this with cell set to 0.001?

 

If anyone else thinks that a filter.duplicate function might be helpful, I'm happy to submit a feature request via Github.

 

Thanks

Andy


--

Dr Andrew Cunliffe

Research Fellow
School of Geography
University of Exeter

 

_______________________________________________ pdal mailing list [hidden email] https://lists.osgeo.org/mailman/listinfo/pdal



--
Dr Andrew Cunliffe

Research Fellow in Dryland Carbon Dynamics

School of Geography, College of Life and Environmental Science,
University of Exeter


_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal


--
Andrew Bell
[hidden email]


--
Andrew Bell
[hidden email]

_______________________________________________
pdal mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/pdal