Is Overlay-NG active?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Is Overlay-NG active?

Roger Bivand
Can I ask whether there is, or could be, a function exposed in the C_API,
or a header variable say in geoc_c.h, showing whether the running GEOS is
using Overlay-NG or not?

After help from the list, I've now run checks on R packages either
themselves linking to GEOS, or using functions from packages which do link
to GEOS. A half-dozen or so fail on unit tests, typically because the
ordering  of coordinates varies (say same polygon, but starting at  a
different place), or the ordering of sub-geometries (say slivers from a
Union operation) varies from before Overlay-NG. The objects are the same
(for given precision), but unit tests in packages compare the WKT of the
output geometry with the expected WKT (often generated from output before
Overlay-NG).

So if we could provide a way for the unit tests to compare correctly
branching on Overlay-NG or not, the package maintainers could avoid
having to scramble when platforms and R packages linking to GEOS begin to
appear.

Another question raised by package maintainers - do we know when 3.9.0 may
be expected, and will it have Overlay-NG? However, providing them with an
easy way to ask the GEOS runtime if it is Overlay-NG or not will relieve
the pressure.

Anyway, the failure count is very low, a half-dozen from hundreds is fine
(and if they didn't write tests, that isn't our problem...).

Roger

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Paul Ramsey
The "plan" as I see it is to, for 3.9, actually make NG the default for the standard intersection(), union(), unaryunion(), difference(), symdifference() calls. So the "test" will be "is this GEOS 3.9". I don't want to go forward into the future where a released GEOS has two potential behaviour modes, that way lies madness. We needed to keep a switch in place to allow other development to continue and to compare before/after easily on the working branch (and to allow people like you to test the new without suddenly have everything break on ordinary testing).

Does that make sense?

P

> On Oct 15, 2020, at 8:59 AM, Roger Bivand <[hidden email]> wrote:
>
> Can I ask whether there is, or could be, a function exposed in the C_API, or a header variable say in geoc_c.h, showing whether the running GEOS is using Overlay-NG or not?
>
> After help from the list, I've now run checks on R packages either themselves linking to GEOS, or using functions from packages which do link to GEOS. A half-dozen or so fail on unit tests, typically because the ordering  of coordinates varies (say same polygon, but starting at  a different place), or the ordering of sub-geometries (say slivers from a Union operation) varies from before Overlay-NG. The objects are the same (for given precision), but unit tests in packages compare the WKT of the output geometry with the expected WKT (often generated from output before Overlay-NG).
>
> So if we could provide a way for the unit tests to compare correctly branching on Overlay-NG or not, the package maintainers could avoid having to scramble when platforms and R packages linking to GEOS begin to appear.
>
> Another question raised by package maintainers - do we know when 3.9.0 may be expected, and will it have Overlay-NG? However, providing them with an easy way to ask the GEOS runtime if it is Overlay-NG or not will relieve the pressure.
>
> Anyway, the failure count is very low, a half-dozen from hundreds is fine (and if they didn't write tests, that isn't our problem...).
>
> Roger
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> _______________________________________________
> geos-devel mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/geos-devel

_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Regina Obe
Roger,

I'll let Paul answer the question of when we can expect a 3.9 as he probably has a better idea than anyone else.
All I know is the plan is we'll be releasing GEOS 3.9 and PostGIS 3.1 around the same time since there is some functionality in PostGIS 3.1 that leverages GEOS 3.9 if it is compiled with GEOS 3.9.

As far as testing, on the PostGIS side to make the old Geos and new Geos agree, I've been applying ST_Normalize

Which looks to come from Geos CAPI - GEOSNormalize

Can you use that to deal with the  failures you are running into?  That should make the wkt of both GEOS old and new agree.

Thanks,
Regina

> -----Original Message-----
> From: geos-devel [mailto:[hidden email]] On Behalf Of
> Paul Ramsey
> Sent: Thursday, October 15, 2020 12:39 PM
> To: [hidden email]; GEOS Development List <geos-
> [hidden email]>
> Subject: Re: [geos-devel] Is Overlay-NG active?
>
> The "plan" as I see it is to, for 3.9, actually make NG the default for the
> standard intersection(), union(), unaryunion(), difference(), symdifference()
> calls. So the "test" will be "is this GEOS 3.9". I don't want to go forward into
> the future where a released GEOS has two potential behaviour modes, that
> way lies madness. We needed to keep a switch in place to allow other
> development to continue and to compare before/after easily on the working
> branch (and to allow people like you to test the new without suddenly have
> everything break on ordinary testing).
>
> Does that make sense?
>
> P
>
> > On Oct 15, 2020, at 8:59 AM, Roger Bivand <[hidden email]> wrote:
> >
> > Can I ask whether there is, or could be, a function exposed in the C_API, or
> a header variable say in geoc_c.h, showing whether the running GEOS is
> using Overlay-NG or not?
> >
> > After help from the list, I've now run checks on R packages either
> themselves linking to GEOS, or using functions from packages which do link
> to GEOS. A half-dozen or so fail on unit tests, typically because the ordering
> of coordinates varies (say same polygon, but starting at  a different place), or
> the ordering of sub-geometries (say slivers from a Union operation) varies
> from before Overlay-NG. The objects are the same (for given precision), but
> unit tests in packages compare the WKT of the output geometry with the
> expected WKT (often generated from output before Overlay-NG).
> >
> > So if we could provide a way for the unit tests to compare correctly
> branching on Overlay-NG or not, the package maintainers could avoid having
> to scramble when platforms and R packages linking to GEOS begin to appear.
> >
> > Another question raised by package maintainers - do we know when 3.9.0
> may be expected, and will it have Overlay-NG? However, providing them
> with an easy way to ask the GEOS runtime if it is Overlay-NG or not will
> relieve the pressure.
> >
> > Anyway, the failure count is very low, a half-dozen from hundreds is fine
> (and if they didn't write tests, that isn't our problem...).
> >
> > Roger
> >
> > --
> > Roger Bivand
> > Department of Economics, Norwegian School of Economics, Helleveien 30,
> > N-5045 Bergen, Norway.
> > voice: +47 55 95 93 55; e-mail: [hidden email]
> > https://orcid.org/0000-0003-2392-6140
> > https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> > _______________________________________________
> > geos-devel mailing list
> > [hidden email]
> > https://lists.osgeo.org/mailman/listinfo/geos-devel
>
> _______________________________________________
> geos-devel mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/geos-devel

_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Sandro Santilli-4
In reply to this post by Roger Bivand
On Thu, Oct 15, 2020 at 05:59:39PM +0200, Roger Bivand wrote:
> Can I ask whether there is, or could be, a function exposed in the C_API, or
> a header variable say in geoc_c.h, showing whether the running GEOS is using
> Overlay-NG or not?

There are functions with Prec suffix, in current GEOS master branch,
that can be used to explicitly request OverlayNG.

The new signatures will keep making sense even when OverlayNG becomes
the default because they allow you to specify a grid size which would
be used for the operation.

--strk;
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Roger Bivand
In reply to this post by Regina Obe
On Thu, 15 Oct 2020, Regina Obe wrote:

> Roger,
>
> I'll let Paul answer the question of when we can expect a 3.9 as he
> probably has a better idea than anyone else. All I know is the plan is
> we'll be releasing GEOS 3.9 and PostGIS 3.1 around the same time since
> there is some functionality in PostGIS 3.1 that leverages GEOS 3.9 if it
> is compiled with GEOS 3.9.
>
> As far as testing, on the PostGIS side to make the old Geos and new Geos
> agree, I've been applying ST_Normalize
>
> Which looks to come from Geos CAPI - GEOSNormalize
>
> Can you use that to deal with the failures you are running into?  That
> should make the wkt of both GEOS old and new agree.

Thank you, yes, probably. However, both legacy overlay and Overlay-NG
differ from GEOSNormalize() (as far as I can see from a simple test), so
the half-dozen package maintainers will need to go from legacy to
normalized and then avoid having to condition on the GEOS version.

Best wishes,

Roger

>
> Thanks,
> Regina
>
>> -----Original Message-----
>> From: geos-devel [mailto:[hidden email]] On Behalf Of
>> Paul Ramsey
>> Sent: Thursday, October 15, 2020 12:39 PM
>> To: [hidden email]; GEOS Development List <geos-
>> [hidden email]>
>> Subject: Re: [geos-devel] Is Overlay-NG active?
>>
>> The "plan" as I see it is to, for 3.9, actually make NG the default for the
>> standard intersection(), union(), unaryunion(), difference(), symdifference()
>> calls. So the "test" will be "is this GEOS 3.9". I don't want to go forward into
>> the future where a released GEOS has two potential behaviour modes, that
>> way lies madness. We needed to keep a switch in place to allow other
>> development to continue and to compare before/after easily on the working
>> branch (and to allow people like you to test the new without suddenly have
>> everything break on ordinary testing).
>>
>> Does that make sense?
>>
>> P
>>
>>> On Oct 15, 2020, at 8:59 AM, Roger Bivand <[hidden email]> wrote:
>>>
>>> Can I ask whether there is, or could be, a function exposed in the C_API, or
>> a header variable say in geoc_c.h, showing whether the running GEOS is
>> using Overlay-NG or not?
>>>
>>> After help from the list, I've now run checks on R packages either
>> themselves linking to GEOS, or using functions from packages which do link
>> to GEOS. A half-dozen or so fail on unit tests, typically because the ordering
>> of coordinates varies (say same polygon, but starting at  a different place), or
>> the ordering of sub-geometries (say slivers from a Union operation) varies
>> from before Overlay-NG. The objects are the same (for given precision), but
>> unit tests in packages compare the WKT of the output geometry with the
>> expected WKT (often generated from output before Overlay-NG).
>>>
>>> So if we could provide a way for the unit tests to compare correctly
>> branching on Overlay-NG or not, the package maintainers could avoid having
>> to scramble when platforms and R packages linking to GEOS begin to appear.
>>>
>>> Another question raised by package maintainers - do we know when 3.9.0
>> may be expected, and will it have Overlay-NG? However, providing them
>> with an easy way to ask the GEOS runtime if it is Overlay-NG or not will
>> relieve the pressure.
>>>
>>> Anyway, the failure count is very low, a half-dozen from hundreds is fine
>> (and if they didn't write tests, that isn't our problem...).
>>>
>>> Roger
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics, Helleveien 30,
>>> N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>> _______________________________________________
>>> geos-devel mailing list
>>> [hidden email]
>>> https://lists.osgeo.org/mailman/listinfo/geos-devel
>>
>> _______________________________________________
>> geos-devel mailing list
>> [hidden email]
>> https://lists.osgeo.org/mailman/listinfo/geos-devel
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Roger Bivand
In reply to this post by Paul Ramsey
On Thu, 15 Oct 2020, Paul Ramsey wrote:

> The "plan" as I see it is to, for 3.9, actually make NG the default for
> the standard intersection(), union(), unaryunion(), difference(),
> symdifference() calls. So the "test" will be "is this GEOS 3.9". I don't
> want to go forward into the future where a released GEOS has two
> potential behaviour modes, that way lies madness. We needed to keep a
> switch in place to allow other development to continue and to compare
> before/after easily on the working branch (and to allow people like you
> to test the new without suddenly have everything break on ordinary
> testing).
>
> Does that make sense?

Yes, it does, thank you.

Best wishes,

Roger

>
> P
>
>> On Oct 15, 2020, at 8:59 AM, Roger Bivand <[hidden email]> wrote:
>>
>> Can I ask whether there is, or could be, a function exposed in the C_API, or a header variable say in geoc_c.h, showing whether the running GEOS is using Overlay-NG or not?
>>
>> After help from the list, I've now run checks on R packages either themselves linking to GEOS, or using functions from packages which do link to GEOS. A half-dozen or so fail on unit tests, typically because the ordering  of coordinates varies (say same polygon, but starting at  a different place), or the ordering of sub-geometries (say slivers from a Union operation) varies from before Overlay-NG. The objects are the same (for given precision), but unit tests in packages compare the WKT of the output geometry with the expected WKT (often generated from output before Overlay-NG).
>>
>> So if we could provide a way for the unit tests to compare correctly branching on Overlay-NG or not, the package maintainers could avoid having to scramble when platforms and R packages linking to GEOS begin to appear.
>>
>> Another question raised by package maintainers - do we know when 3.9.0 may be expected, and will it have Overlay-NG? However, providing them with an easy way to ask the GEOS runtime if it is Overlay-NG or not will relieve the pressure.
>>
>> Anyway, the failure count is very low, a half-dozen from hundreds is fine (and if they didn't write tests, that isn't our problem...).
>>
>> Roger
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>> _______________________________________________
>> geos-devel mailing list
>> [hidden email]
>> https://lists.osgeo.org/mailman/listinfo/geos-devel
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Roger Bivand
In reply to this post by Sandro Santilli-4
On Thu, 15 Oct 2020, Sandro Santilli wrote:

> On Thu, Oct 15, 2020 at 05:59:39PM +0200, Roger Bivand wrote:
>> Can I ask whether there is, or could be, a function exposed in the
>> C_API, or a header variable say in geoc_c.h, showing whether the
>> running GEOS is using Overlay-NG or not?
>
> There are functions with Prec suffix, in current GEOS master branch,
> that can be used to explicitly request OverlayNG.
>
> The new signatures will keep making sense even when OverlayNG becomes
> the default because they allow you to specify a grid size which would
> be used for the operation.

Thanks - this will be useful anyway, for example see the discussion in
(with apologies for github):

https://github.com/r-spatial/sf/issues/1510

where the resolution for a regression in intersection and/or STRtrees from
3.6 to 3.7/3.8 was associated with an input problem challenging the use of
0 precision in the R sf package by default.

Best wishes,

Roger

OT: We've been worried by not being able to build GEOS on Solaris 10 on
ix64/x64 with GEOS after 3.6 - we had had a Sparc Solaris to run checks on
a big-endian system, and have continued on Intel Solaris, and because the
build train is divergent, it finds lots of assumptions made in coding, and
since 3.6, we haven't been able to install GEOS. With Apple Silicon
coming, it would be helpful to solve the Solaris build problem to keep our
check systems up to date:

https://cran.r-project.org/web/checks/check_summary.html
https://cran.r-project.org/web/checks/check_flavors.html

>
> --strk;
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Paul Ramsey
I share your concern about niche architectures. We are running ARM64 in CI now. We have no access to an easy bigendian platform to test on. I tried Travis S390 for a while, but the CI itself was very unreliable on that platform, frequent timeouts made it hard to keep in the test matrix, so I dropped it. Access to a big-endian machine with a modern build chain would be handy for occasional manual test runs, and of course debugging. "It don't work" is so nonactionable, and with the only access to alternative architectures being CI sometimes, it's hard to fix problems that do arise (see the litany of weird MIPS failures that Debian reports)

P.

> On Oct 15, 2020, at 12:39 PM, Roger Bivand <[hidden email]> wrote:
>
> OT: We've been worried by not being able to build GEOS on Solaris 10 on ix64/x64 with GEOS after 3.6 - we had had a Sparc Solaris to run checks on a big-endian system, and have continued on Intel Solaris, and because the build train is divergent, it finds lots of assumptions made in coding, and since 3.6, we haven't been able to install GEOS. With Apple Silicon coming, it would be helpful to solve the Solaris build problem to keep our check systems up to date:

_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Roger Bivand
In reply to this post by Roger Bivand
I apologise for brevity, this blog shows some current thinking: 


Roger

---
Roger Bivand
Department of Economics
Norwegian School of Economics
Bergen, Norway
[hidden email]


Fra: Paul Ramsey <[hidden email]>
Sendt: torsdag 15. oktober 2020, 21:45
Til: [hidden email]; GEOS Development List
Emne: Re: [geos-devel] Is Overlay-NG active?

I share your concern about niche architectures. We are running ARM64 in CI now. We have no access to an easy bigendian platform to test on. I tried Travis S390 for a while, but the CI itself was very unreliable on that platform, frequent timeouts made it hard to keep in the test matrix, so I dropped it. Access to a big-endian machine with a modern build chain would be handy for occasional manual test runs, and of course debugging. "It don't work" is so nonactionable, and with the only access to alternative architectures being CI sometimes, it's hard to fix problems that do arise (see the litany of weird MIPS failures that Debian reports) P. > On Oct 15, 2020, at 12:39 PM, Roger Bivand wrote: > > OT: We've been worried by not being able to build GEOS on Solaris 10 on ix64/x64 with GEOS after 3.6 - we had had a Sparc Solaris to run checks on a big-endian system, and have continued on Intel Solaris, and because the build train is divergent, it finds lots of assumptions made in coding, and since 3.6, we haven't been able to install GEOS. With Apple Silicon coming, it would be helpful to solve the Solaris build problem to keep our check systems up to date:

_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Testing OverlayNG robustness in R-sf

Martin Davis-3
In reply to this post by Roger Bivand

On Thu, Oct 15, 2020 at 9:00 AM Roger Bivand <[hidden email]> wrote:

Anyway, the failure count is very low, a half-dozen from hundreds is fine
(and if they didn't write tests, that isn't our problem...).

What would be especially interesting is to see whether OverlayNG resolves the various overlay robustness issues reported in R-sf issues.  

E.g.
 

_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Regina Obe
In reply to this post by Roger Bivand
> > Roger,
> >
> > I'll let Paul answer the question of when we can expect a 3.9 as he
> > probably has a better idea than anyone else. All I know is the plan is
> > we'll be releasing GEOS 3.9 and PostGIS 3.1 around the same time since
> > there is some functionality in PostGIS 3.1 that leverages GEOS 3.9 if
> > it is compiled with GEOS 3.9.
> >
> > As far as testing, on the PostGIS side to make the old Geos and new
> > Geos agree, I've been applying ST_Normalize
> >
> > Which looks to come from Geos CAPI - GEOSNormalize
> >
> > Can you use that to deal with the failures you are running into?  That
> > should make the wkt of both GEOS old and new agree.
>
> Thank you, yes, probably. However, both legacy overlay and Overlay-NG
> differ from GEOSNormalize() (as far as I can see from a simple test), so
the
> half-dozen package maintainers will need to go from legacy to normalized
> and then avoid having to condition on the GEOS version.
>
> Best wishes,
>
> Roger
>

Yah both legacy and NG both defer from GEOSNormalize, but the GEOSNormalize
of both is equivalent so what we do is

Essentially

WKT(GeosNormalize(outputfromfunc)) = WKT(GeosNormalize(oldexpectedresult))

Not ideal I know and maybe much harder to do in your setup than it was on
our tests as it does require changing your wkt expected output to the
normalized version.



_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is Overlay-NG active?

Roger Bivand
On Thu, 15 Oct 2020, Regina Obe wrote:

>>> Roger,
>>>
>>> I'll let Paul answer the question of when we can expect a 3.9 as he
>>> probably has a better idea than anyone else. All I know is the plan is
>>> we'll be releasing GEOS 3.9 and PostGIS 3.1 around the same time since
>>> there is some functionality in PostGIS 3.1 that leverages GEOS 3.9 if
>>> it is compiled with GEOS 3.9.
>>>
>>> As far as testing, on the PostGIS side to make the old Geos and new
>>> Geos agree, I've been applying ST_Normalize
>>>
>>> Which looks to come from Geos CAPI - GEOSNormalize
>>>
>>> Can you use that to deal with the failures you are running into?  That
>>> should make the wkt of both GEOS old and new agree.
>>
>> Thank you, yes, probably. However, both legacy overlay and Overlay-NG
>> differ from GEOSNormalize() (as far as I can see from a simple test), so
> the
>> half-dozen package maintainers will need to go from legacy to normalized
>> and then avoid having to condition on the GEOS version.
>>
>> Best wishes,
>>
>> Roger
>>
>
> Yah both legacy and NG both defer from GEOSNormalize, but the GEOSNormalize
> of both is equivalent so what we do is
>
> Essentially
>
> WKT(GeosNormalize(outputfromfunc)) = WKT(GeosNormalize(oldexpectedresult))
>
> Not ideal I know and maybe much harder to do in your setup than it was on
> our tests as it does require changing your wkt expected output to the
> normalized version.

Regina,

Thanks again. Since we accept that we can rely on the existing WKT
representations remaining fixed < "3.9.0", and that Overlay-NG will be
used >= "3.9.0", we are choosing to advise package maintainers to add a
condition to their tests to add the new >= "3.9.0" WKT when GEOS >=
"3.9.0". Not ideal, but we can return to normalization later if this isn't
enough.

Best wishes,

Roger

>
>
>
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Testing OverlayNG robustness in R-sf

Roger Bivand
In reply to this post by Martin Davis-3
On Thu, 15 Oct 2020, Martin Davis wrote:

> On Thu, Oct 15, 2020 at 9:00 AM Roger Bivand <[hidden email]> wrote:
>
>>
>> Anyway, the failure count is very low, a half-dozen from hundreds is fine
>> (and if they didn't write tests, that isn't our problem...).
>>
>
> What would be especially interesting is to see whether OverlayNG resolves
> the various overlay robustness issues reported in R-sf issues.
>
> E.g.
> https://github.com/r-spatial/sf/issues/794

(I had a local copy of the all.rds file)

> all %>% st_intersection -> o
Error in CPL_nary_intersection(x) :
   Evaluation error: TopologyException: found non-noded intersection
between LINESTRING (28.2168 45.4666, 28.2153 45.4669) and LINESTRING
(28.2153 45.4669, 28.2168 45.4666) at 28.215309983502991
45.466850983501708.
> all %>% st_set_precision(1e8) %>% st_intersection -> o
Error in CPL_nary_intersection(x) :
   Evaluation error: TopologyException: found non-noded intersection
between LINESTRING (40.1413 4.01209, 40.1296 4.00625) and LINESTRING
(40.1413 4.01209, 40.1296 4.00625) at 40.141307598308956
4.0120866067334404.



> https://github.com/r-spatial/sf/issues/603

> library(sf)
Linking to GEOS 3.9.0dev, GDAL 3.2.0dev-b51dc28ab7, PROJ 7.2.0
> p1 <- readRDS("p1.rds")
> p2 <- readRDS("p2.rds")
> p3 <- sf::st_difference(p1, p2)

so now runs with Overlay-NG and sf's default 0 precision, but sf has also
changed since late 2017.

> https://github.com/r-spatial/sf/issues/936
>

Error in CPL_nary_intersection(x) :
   Evaluation error: TopologyException: Input geom 0 is invalid:
Self-intersection at 314406.13208707399 -5762352.8343122564.

> o <- st_make_valid(test_set)
> overlap <- st_intersection(o)

Error in CPL_nary_intersection(x) :
   Evaluation error: TopologyException: found non-noded intersection
between LINESTRING (316483 -5.76278e+06, 316483 -5.76278e+06) and
LINESTRING (316483 -5.76278e+06, 316411 -5.76283e+06) at
316482.77665721485 -5762778.376178137.

Cases 794 and 936 are all-by-all intersections, and as noted in the latter
issue, the problems occurring differ by input order.

Please ask off-list if someone needs all.rds from 794.

Roger

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: Testing OverlayNG robustness in R-sf

Martin Davis-3


On Fri, Oct 16, 2020 at 11:59 AM Roger Bivand <[hidden email]> wrote:
Cases 794 and 936 are all-by-all intersections, and as noted in the latter
issue, the problems occurring differ by input order.

Well, that's disappointing that 794 and 936 still have failures.  Especially because 794 was also reported as JTS 300:


and the test data there is incoporated in the robustness test suilte in both JTS [1] and GEOS [2] , where it passes using OverlayNG.

Is there any way to get a dump of the actual input geometries to GEOS which are causing these errors? 


_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Reply | Threaded
Open this post in threaded view
|

Re: Testing OverlayNG robustness in R-sf

Roger Bivand
On Fri, 16 Oct 2020, Martin Davis wrote:

> On Fri, Oct 16, 2020 at 11:59 AM Roger Bivand <[hidden email]> wrote:
>
>> Cases 794 and 936 are all-by-all intersections, and as noted in the latter
>> issue, the problems occurring differ by input order.
>>
>
> Well, that's disappointing that 794 and 936 still have failures.
> Especially because 794 was also reported as JTS 300:
>
> https://github.com/locationtech/jts/issues/300
>
> and the test data there is incoporated in the robustness test suilte in
> both JTS [1] and GEOS [2] , where it passes using OverlayNG.
>
> Is there any way to get a dump of the actual input geometries to GEOS which
> are causing these errors?

I've prepared a 13.7GB WKT file (I think - trying to tidy it up), but this
looks like overkill. My present take after stepping back from the original
question - which language multipolygons intersect each other and what are
the combined geometries - to run N=7510 intersections between all and each
in turn. These did not fail with Overlay-NG, but maybe didn't fail before
either. All with all was probably an unhelpful understanding of how
intersection works initially. Further, the list of all-with-each contains
many non-polygon artefacts, which probably need to be discarded anyway.

So far for now,

Roger

>
> [1]
> https://github.com/locationtech/jts/blob/master/modules/tests/src/test/resources/testxml/robust/overlay/TestOverlay-jts-300.xml
> [2]
> https://git.osgeo.org/gitea/geos/geos/src/branch/master/tests/xmltester/tests/robust/overlay/TestOverlay-jts-300.xml
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
geos-devel mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/geos-devel
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway