Infinite coordinates in geometries

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Infinite coordinates in geometries

Sandro Santilli-2

I've found the bug in the postgis picksplit function.
This function is the exact copy of rtree_picksplit from rtree_gist,
so this might be of interest to the rtree_gist guys as well.

To determine destination of the node, this kind of checks are performed:

 cur->low.y - pageunion.low.y  < pageunion.high.y - cur->high
 cur->low.y - pageunion.low.y  > pageunion.high.y - cur->high
 cur->low.y - pageunion.low.y == pageunion.high.y - cur->high

Substracting Inf from Inf gives Nan, which does not match any
of the checks.

I've currently (CVS) made the == check (equality) the default,
so to have Infinite valued keys be cached by an ADD-TO-LESS-POPULATED
rule.

I dunno the consequences of this in terms of overall index functionality,
but you don't get the error message anymore ;)
I'm afraid Infinite geometry presence in a GiST index makes the index
unusable because it eventualy ends up with an infinite plane as root
node and with an arbitrary tree built under it.

Maybe the picksplit function or whatever other GiST
construction function invoked at every new inserted geometry or at
while table scan should warn the user, or skip infinite geometries
(making them invisible by the index).

Any comment from GiST friends ?

--strk;

On Tue, Jun 01, 2004 at 10:14:55AM -0700, Paul Ramsey wrote:

> Quoting strk <[hidden email]>:
>
> > On Tue, Jun 01, 2004 at 10:00:02AM -0700, Paul Ramsey wrote:
> > > No, they shouldn't create Inf geometries.
> > > It's 'POINT(Inf Inf)'::geometry, no comma, BTW.
> > > P.
> >
> > Oops... it works, sorry :)
> > Now, what should I do ?
> > Drop Inf geometry support or use it to find other bugs ?
>
> I think we should drop it, what are your thoughts? If we leave it in, there is
> an implied rational behavior to apply in the presence of these things which
> could be very hard to figure out. I suppose from an indexing point of view they
> are no worse than NULL geometries in semantic terms.
>
> >
> > The operation our guy was trying to execute is not specified,
> > I tried filling a table with Inf points, building a gist index,
> > querying it... no memory alloc problems so far.
> >
> >
> > --strk;
> >
> > >
> > > Quoting strk <[hidden email]>:
> > >
> > > > On Tue, Jun 01, 2004 at 09:53:04AM -0700, Paul Ramsey wrote:
> > > > > Intriguing... doing POINT(Inf,Inf) does not do the trick? I would have
> > > > thought
> > > > > that a2d() would interpret the Inf token properly.
> > > > > P.
> > > >
> > > > test=# select 'POINT(Inf, Inf)'::geometry;
> > > > ERROR:  couldnt parse object in GEOMETRY
> > > >
> > > > If John Hagstrand could reproduce the Inf geoms that would help...
> > > > I think the bug is actually there.. postgis functions should not
> > > > produce such geoms, should 'hem ?
> > > >
> > > > --strk;
> > > >
> > > > >
> > > > > Quoting strk <[hidden email]>:
> > > > >
> > > > > > On Tue, Jun 01, 2004 at 09:13:33AM -0700, Paul Ramsey wrote:
> > > > > > > Sandro,
> > > > > > > Could you check for a reasonable place to handle this special case?
> > I
> > > > do
> > > > > > not
> > > > > > > think Inf geometries are something we want to handle, necessarily
> > :)
> > > > > > > Paul
> > > > > >
> > > > > > The question is: how did this Inf geometry get created ?
> > > > > > I dunno how to make one, for debugging purposes...
> > > > > > --strk;
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > ----- Forwarded message from [hidden email]
> > -----
> > > > > > >     Date: Tue, 1 Jun 2004 10:59:32 -0500
> > > > > > >     From: John Hagstrand <[hidden email]>
> > > > > > > Reply-To: John Hagstrand <[hidden email]>
> > > > > > >  Subject: RE: [GENERAL] ERROR: invalid memory alloc request size 0
> > > > > > >       To: Paul Ramsey <[hidden email]>
> > > > > > >
> > > > > > > Hello Paul,
> > > > > > >
> > > > > > > I solved this problem.
> > > > > > >
> > > > > > > I had 20 or 30 records in that table with a geometry column like
> > this:
> > > > > > > POINT(-inf -inf).  I deleted those records, and then dropped and
> > > > recreated
> > > > > > > the index.  That solved the problem.  I can now do inserts and
> > updates
> > > > > > > without getting the malloc error.
> > > > > > >
> > > > > > > Thanks for your help.
> > > > > > > I look forward to seeing you in Ottawa next week.
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > Sent: Thursday, April 15, 2004 10:49 PM
> > > > > > > > To: John Hagstrand
> > > > > > > > Cc: Tom Lane; [hidden email]
> > > > > > > > Subject: Re: [GENERAL] ERROR: invalid memory alloc request size 0
> > > > > > > >
> > > > > > > >
> > > > > > > > John,
> > > > > > > > A pg_dump would be sufficient, or a shape file, and a SQL log of
> > the
> > > > > > > > commands you ran.
> > > > > > > > Thanks!
> > > > > > > > Paul
> > > > > > > >
> > > > > > > > On Thursday, April 15, 2004, at 12:27 AM, John Hagstrand wrote:
> > > > > > > >
> > > > > > > > > Hello Paul,
> > > > > > > > >
> > > > > > > > > Thanks for helping.  I'd like to send you the test data set.
> > > > > > > > > What format would you like?
> > > > > > > > >
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >> -----Original Message-----
> > > > > > > > >> From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > >> Sent: Wednesday, April 14, 2004 11:28 PM
> > > > > > > > >> To: Tom Lane
> > > > > > > > >> Cc: John Hagstrand; [hidden email]
> > > > > > > > >> Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > size 0
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> We'll need the test data set that does this, of course, and
> > some
> > > > > > > > >> version info. The indexing routines seemed pretty solid
> > (tested on
> > > > > > > > >> many
> > > > > > > > >> many millions of geometries), I am surprised this has come up.
> > > > > > > > >>
> > > > > > > > >> P.
> > > > > > > > >>
> > > > > > > > >> On Wednesday, April 14, 2004, at 08:53 PM, Tom Lane wrote:
> > > > > > > > >>
> > > > > > > > >>> "John Hagstrand" <[hidden email]>
> > writes:
> > > > > > > > >>>> Here is a more detailed backtrace.
> > > > > > > > >>>
> > > > > > > > >>>> (gdb) bt
> > > > > > > > >>>> #0  errfinish (dummy=0) at elog.c:319
> > > > > > > > >>>> #1  0x081a85de in elog_finish (elevel=20,
> > > > > > > > >>>>     fmt=0x8231d80 "invalid memory alloc request size %lu")
> > at
> > > > > > > > >>>> elog.c:853
> > > > > > > > >>>> #2  0x081b3dd6 in MemoryContextAlloc (context=0x9f10598,
> > size=0)
> > > > at
> > > > > > > > >>>> mcxt.c:482
> > > > > > > > >>>> #3  0x0807411d in gistSplit (r=0xbe9a1938, buffer=805,
> > > > > > > > >>>> itup=0x9f25128,
> > > > > > > > >>>>     len=0xbfec3eac, giststate=0xbfec4070, res=0xbfec402c) at
> > > > > > > > >>>> gist.c:1348
> > > > > > > > >>>
> > > > > > > > >>> Okay, this makes it pretty clear that the problem is the
> > > > picksplit
> > > > > > > > >>> function is setting v.spl_nleft to zero.  In other words it's
> > > > failed
> > > > > > > > >>> to "split" the page at all (everything in the original index
> > page
> > > > has
> > > > > > > > >>> gone into the righthand child).  This seems to me to be
> > > > necessarily
> > > > > > > > >>> a bug in the index opclass's picksplit function.
> > > > > > > > >>>
> > > > > > > > >>> You previously said that the column in question is a PostGIS
> > > > Geometry
> > > > > > > > >>> column, so I'm of the opinion this is a PostGIS bug.  (Paul,
> > any
> > > > > > > > >>> suggestions about how to probe deeper?)
> > > > > > > > >>>
> > > > > > > > >>> regards, tom lane
> > > > > > > > >>>
> > > > > > > > >>       Paul Ramsey
> > > > > > > > >>       Refractions Research
> > > > > > > > >>       Email: [hidden email]
> > > > > > > > >>       Phone: (250) 885-0632
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >       Paul Ramsey
> > > > > > > >       Refractions Research
> > > > > > > >       Email: [hidden email]
> > > > > > > >       Phone: (250) 885-0632
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > ----- End forwarded message -----
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Infinite coordinates in geometries

David Blasby-3
Good points, strk.

I dont think infinite geometries are valid (I'll check the spec tomorrow).  
POINT(Inf Inf) isnt really defining where the point is in space.

Inf coordinates are likely to screw up more than just the GiST index.

My gut feeling is to make them invisible to the index (treat them like NULL),
and any query of the form:

SELECT * FROM <table> && 'BOX3D(-Inf -Inf, ...)';

to throw an error.  

Either that or throw an error whenever your geometry has an Inf or NaN in it.

What does JTS do with Inf coordinates?  Is a geometry with Inf coordinates
valid (ie. geometry.isvalid() = true)?

Anyone else have suggestions?

Quoting strk <[hidden email]>:

>
> I've found the bug in the postgis picksplit function.
> This function is the exact copy of rtree_picksplit from rtree_gist,
> so this might be of interest to the rtree_gist guys as well.
>
> To determine destination of the node, this kind of checks are performed:
>
>  cur->low.y - pageunion.low.y  < pageunion.high.y - cur->high
>  cur->low.y - pageunion.low.y  > pageunion.high.y - cur->high
>  cur->low.y - pageunion.low.y == pageunion.high.y - cur->high
>
> Substracting Inf from Inf gives Nan, which does not match any
> of the checks.
>
> I've currently (CVS) made the == check (equality) the default,
> so to have Infinite valued keys be cached by an ADD-TO-LESS-POPULATED
> rule.
>
> I dunno the consequences of this in terms of overall index functionality,
> but you don't get the error message anymore ;)
> I'm afraid Infinite geometry presence in a GiST index makes the index
> unusable because it eventualy ends up with an infinite plane as root
> node and with an arbitrary tree built under it.
>
> Maybe the picksplit function or whatever other GiST
> construction function invoked at every new inserted geometry or at
> while table scan should warn the user, or skip infinite geometries
> (making them invisible by the index).
>
> Any comment from GiST friends ?
>
> --strk;
>
> On Tue, Jun 01, 2004 at 10:14:55AM -0700, Paul Ramsey wrote:
> > Quoting strk <[hidden email]>:
> >
> > > On Tue, Jun 01, 2004 at 10:00:02AM -0700, Paul Ramsey wrote:
> > > > No, they shouldn't create Inf geometries.
> > > > It's 'POINT(Inf Inf)'::geometry, no comma, BTW.
> > > > P.
> > >
> > > Oops... it works, sorry :)
> > > Now, what should I do ?
> > > Drop Inf geometry support or use it to find other bugs ?
> >
> > I think we should drop it, what are your thoughts? If we leave it in, there
> is
> > an implied rational behavior to apply in the presence of these things which
> > could be very hard to figure out. I suppose from an indexing point of view
> they
> > are no worse than NULL geometries in semantic terms.
> >
> > >
> > > The operation our guy was trying to execute is not specified,
> > > I tried filling a table with Inf points, building a gist index,
> > > querying it... no memory alloc problems so far.
> > >
> > >
> > > --strk;
> > >
> > > >
> > > > Quoting strk <[hidden email]>:
> > > >
> > > > > On Tue, Jun 01, 2004 at 09:53:04AM -0700, Paul Ramsey wrote:
> > > > > > Intriguing... doing POINT(Inf,Inf) does not do the trick? I would
> have
> > > > > thought
> > > > > > that a2d() would interpret the Inf token properly.
> > > > > > P.
> > > > >
> > > > > test=# select 'POINT(Inf, Inf)'::geometry;
> > > > > ERROR:  couldnt parse object in GEOMETRY
> > > > >
> > > > > If John Hagstrand could reproduce the Inf geoms that would help...
> > > > > I think the bug is actually there.. postgis functions should not
> > > > > produce such geoms, should 'hem ?
> > > > >
> > > > > --strk;
> > > > >
> > > > > >
> > > > > > Quoting strk <[hidden email]>:
> > > > > >
> > > > > > > On Tue, Jun 01, 2004 at 09:13:33AM -0700, Paul Ramsey wrote:
> > > > > > > > Sandro,
> > > > > > > > Could you check for a reasonable place to handle this special
> case?
> > > I
> > > > > do
> > > > > > > not
> > > > > > > > think Inf geometries are something we want to handle,
> necessarily
> > > :)
> > > > > > > > Paul
> > > > > > >
> > > > > > > The question is: how did this Inf geometry get created ?
> > > > > > > I dunno how to make one, for debugging purposes...
> > > > > > > --strk;
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > ----- Forwarded message from
> [hidden email]
> > > -----
> > > > > > > >     Date: Tue, 1 Jun 2004 10:59:32 -0500
> > > > > > > >     From: John Hagstrand <[hidden email]>
> > > > > > > > Reply-To: John Hagstrand <[hidden email]>
> > > > > > > >  Subject: RE: [GENERAL] ERROR: invalid memory alloc request
> size 0
> > > > > > > >       To: Paul Ramsey <[hidden email]>
> > > > > > > >
> > > > > > > > Hello Paul,
> > > > > > > >
> > > > > > > > I solved this problem.
> > > > > > > >
> > > > > > > > I had 20 or 30 records in that table with a geometry column
> like
> > > this:
> > > > > > > > POINT(-inf -inf).  I deleted those records, and then dropped
> and
> > > > > recreated
> > > > > > > > the index.  That solved the problem.  I can now do inserts and
> > > updates
> > > > > > > > without getting the malloc error.
> > > > > > > >
> > > > > > > > Thanks for your help.
> > > > > > > > I look forward to seeing you in Ottawa next week.
> > > > > > > > John
> > > > > > > >
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > Sent: Thursday, April 15, 2004 10:49 PM
> > > > > > > > > To: John Hagstrand
> > > > > > > > > Cc: Tom Lane; [hidden email]
> > > > > > > > > Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> size 0
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > John,
> > > > > > > > > A pg_dump would be sufficient, or a shape file, and a SQL log
> of
> > > the
> > > > > > > > > commands you ran.
> > > > > > > > > Thanks!
> > > > > > > > > Paul
> > > > > > > > >
> > > > > > > > > On Thursday, April 15, 2004, at 12:27 AM, John Hagstrand
> wrote:
> > > > > > > > >
> > > > > > > > > > Hello Paul,
> > > > > > > > > >
> > > > > > > > > > Thanks for helping.  I'd like to send you the test data
> set.
> > > > > > > > > > What format would you like?
> > > > > > > > > >
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >> -----Original Message-----
> > > > > > > > > >> From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > >> Sent: Wednesday, April 14, 2004 11:28 PM
> > > > > > > > > >> To: Tom Lane
> > > > > > > > > >> Cc: John Hagstrand; [hidden email]
> > > > > > > > > >> Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > size 0
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> We'll need the test data set that does this, of course,
> and
> > > some
> > > > > > > > > >> version info. The indexing routines seemed pretty solid
> > > (tested on
> > > > > > > > > >> many
> > > > > > > > > >> many millions of geometries), I am surprised this has come
> up.
> > > > > > > > > >>
> > > > > > > > > >> P.
> > > > > > > > > >>
> > > > > > > > > >> On Wednesday, April 14, 2004, at 08:53 PM, Tom Lane wrote:
> > > > > > > > > >>
> > > > > > > > > >>> "John Hagstrand" <[hidden email]>
> > > writes:
> > > > > > > > > >>>> Here is a more detailed backtrace.
> > > > > > > > > >>>
> > > > > > > > > >>>> (gdb) bt
> > > > > > > > > >>>> #0  errfinish (dummy=0) at elog.c:319
> > > > > > > > > >>>> #1  0x081a85de in elog_finish (elevel=20,
> > > > > > > > > >>>>     fmt=0x8231d80 "invalid memory alloc request size
> %lu")
> > > at
> > > > > > > > > >>>> elog.c:853
> > > > > > > > > >>>> #2  0x081b3dd6 in MemoryContextAlloc (context=0x9f10598,
> > > size=0)
> > > > > at
> > > > > > > > > >>>> mcxt.c:482
> > > > > > > > > >>>> #3  0x0807411d in gistSplit (r=0xbe9a1938, buffer=805,
> > > > > > > > > >>>> itup=0x9f25128,
> > > > > > > > > >>>>     len=0xbfec3eac, giststate=0xbfec4070,
> res=0xbfec402c) at
> > > > > > > > > >>>> gist.c:1348
> > > > > > > > > >>>
> > > > > > > > > >>> Okay, this makes it pretty clear that the problem is the
> > > > > picksplit
> > > > > > > > > >>> function is setting v.spl_nleft to zero.  In other words
> it's
> > > > > failed
> > > > > > > > > >>> to "split" the page at all (everything in the original
> index
> > > page
> > > > > has
> > > > > > > > > >>> gone into the righthand child).  This seems to me to be
> > > > > necessarily
> > > > > > > > > >>> a bug in the index opclass's picksplit function.
> > > > > > > > > >>>
> > > > > > > > > >>> You previously said that the column in question is a
> PostGIS
> > > > > Geometry
> > > > > > > > > >>> column, so I'm of the opinion this is a PostGIS bug.
> (Paul,
> > > any
> > > > > > > > > >>> suggestions about how to probe deeper?)
> > > > > > > > > >>>
> > > > > > > > > >>> regards, tom lane
> > > > > > > > > >>>
> > > > > > > > > >>       Paul Ramsey
> > > > > > > > > >>       Refractions Research
> > > > > > > > > >>       Email: [hidden email]
> > > > > > > > > >>       Phone: (250) 885-0632
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >       Paul Ramsey
> > > > > > > > >       Refractions Research
> > > > > > > > >       Email: [hidden email]
> > > > > > > > >       Phone: (250) 885-0632
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > ----- End forwarded message -----
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> _______________________________________________
> postgis-devel mailing list
> [hidden email]
> http://postgis.refractions.net/mailman/listinfo/postgis-devel
>





Reply | Threaded
Open this post in threaded view
|

Re: Infinite coordinates in geometries

Sandro Santilli-2
On Wed, Jun 02, 2004 at 05:51:06PM -0700, Dave Blasby wrote:

> Good points, strk.
>
> I dont think infinite geometries are valid (I'll check the spec tomorrow).  
> POINT(Inf Inf) isnt really defining where the point is in space.
>
> Inf coordinates are likely to screw up more than just the GiST index.
>
> My gut feeling is to make them invisible to the index (treat them like NULL),
> and any query of the form:
>
> SELECT * FROM <table> && 'BOX3D(-Inf -Inf, ...)';
>
> to throw an error.  
>
> Either that or throw an error whenever your geometry has an Inf or NaN in it.

I've made WKT parser handle Inf coordinates in geom throwing an error
and GiST ggeometry_compress skip Inf geoms as it does with EMPTY ones.

WKB parser would still accept Inf I guess...

>
> What does JTS do with Inf coordinates?  Is a geometry with Inf coordinates
> valid (ie. geometry.isvalid() = true)?

Dunno... can not make JTS parse an Inf geom from .xml ...

--strk;

>
> Anyone else have suggestions?
>
> Quoting strk <[hidden email]>:
>
> >
> > I've found the bug in the postgis picksplit function.
> > This function is the exact copy of rtree_picksplit from rtree_gist,
> > so this might be of interest to the rtree_gist guys as well.
> >
> > To determine destination of the node, this kind of checks are performed:
> >
> >  cur->low.y - pageunion.low.y  < pageunion.high.y - cur->high
> >  cur->low.y - pageunion.low.y  > pageunion.high.y - cur->high
> >  cur->low.y - pageunion.low.y == pageunion.high.y - cur->high
> >
> > Substracting Inf from Inf gives Nan, which does not match any
> > of the checks.
> >
> > I've currently (CVS) made the == check (equality) the default,
> > so to have Infinite valued keys be cached by an ADD-TO-LESS-POPULATED
> > rule.
> >
> > I dunno the consequences of this in terms of overall index functionality,
> > but you don't get the error message anymore ;)
> > I'm afraid Infinite geometry presence in a GiST index makes the index
> > unusable because it eventualy ends up with an infinite plane as root
> > node and with an arbitrary tree built under it.
> >
> > Maybe the picksplit function or whatever other GiST
> > construction function invoked at every new inserted geometry or at
> > while table scan should warn the user, or skip infinite geometries
> > (making them invisible by the index).
> >
> > Any comment from GiST friends ?
> >
> > --strk;
> >
> > On Tue, Jun 01, 2004 at 10:14:55AM -0700, Paul Ramsey wrote:
> > > Quoting strk <[hidden email]>:
> > >
> > > > On Tue, Jun 01, 2004 at 10:00:02AM -0700, Paul Ramsey wrote:
> > > > > No, they shouldn't create Inf geometries.
> > > > > It's 'POINT(Inf Inf)'::geometry, no comma, BTW.
> > > > > P.
> > > >
> > > > Oops... it works, sorry :)
> > > > Now, what should I do ?
> > > > Drop Inf geometry support or use it to find other bugs ?
> > >
> > > I think we should drop it, what are your thoughts? If we leave it in, there
> > is
> > > an implied rational behavior to apply in the presence of these things which
> > > could be very hard to figure out. I suppose from an indexing point of view
> > they
> > > are no worse than NULL geometries in semantic terms.
> > >
> > > >
> > > > The operation our guy was trying to execute is not specified,
> > > > I tried filling a table with Inf points, building a gist index,
> > > > querying it... no memory alloc problems so far.
> > > >
> > > >
> > > > --strk;
> > > >
> > > > >
> > > > > Quoting strk <[hidden email]>:
> > > > >
> > > > > > On Tue, Jun 01, 2004 at 09:53:04AM -0700, Paul Ramsey wrote:
> > > > > > > Intriguing... doing POINT(Inf,Inf) does not do the trick? I would
> > have
> > > > > > thought
> > > > > > > that a2d() would interpret the Inf token properly.
> > > > > > > P.
> > > > > >
> > > > > > test=# select 'POINT(Inf, Inf)'::geometry;
> > > > > > ERROR:  couldnt parse object in GEOMETRY
> > > > > >
> > > > > > If John Hagstrand could reproduce the Inf geoms that would help...
> > > > > > I think the bug is actually there.. postgis functions should not
> > > > > > produce such geoms, should 'hem ?
> > > > > >
> > > > > > --strk;
> > > > > >
> > > > > > >
> > > > > > > Quoting strk <[hidden email]>:
> > > > > > >
> > > > > > > > On Tue, Jun 01, 2004 at 09:13:33AM -0700, Paul Ramsey wrote:
> > > > > > > > > Sandro,
> > > > > > > > > Could you check for a reasonable place to handle this special
> > case?
> > > > I
> > > > > > do
> > > > > > > > not
> > > > > > > > > think Inf geometries are something we want to handle,
> > necessarily
> > > > :)
> > > > > > > > > Paul
> > > > > > > >
> > > > > > > > The question is: how did this Inf geometry get created ?
> > > > > > > > I dunno how to make one, for debugging purposes...
> > > > > > > > --strk;
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > ----- Forwarded message from
> > [hidden email]
> > > > -----
> > > > > > > > >     Date: Tue, 1 Jun 2004 10:59:32 -0500
> > > > > > > > >     From: John Hagstrand <[hidden email]>
> > > > > > > > > Reply-To: John Hagstrand <[hidden email]>
> > > > > > > > >  Subject: RE: [GENERAL] ERROR: invalid memory alloc request
> > size 0
> > > > > > > > >       To: Paul Ramsey <[hidden email]>
> > > > > > > > >
> > > > > > > > > Hello Paul,
> > > > > > > > >
> > > > > > > > > I solved this problem.
> > > > > > > > >
> > > > > > > > > I had 20 or 30 records in that table with a geometry column
> > like
> > > > this:
> > > > > > > > > POINT(-inf -inf).  I deleted those records, and then dropped
> > and
> > > > > > recreated
> > > > > > > > > the index.  That solved the problem.  I can now do inserts and
> > > > updates
> > > > > > > > > without getting the malloc error.
> > > > > > > > >
> > > > > > > > > Thanks for your help.
> > > > > > > > > I look forward to seeing you in Ottawa next week.
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > > Sent: Thursday, April 15, 2004 10:49 PM
> > > > > > > > > > To: John Hagstrand
> > > > > > > > > > Cc: Tom Lane; [hidden email]
> > > > > > > > > > Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > size 0
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > John,
> > > > > > > > > > A pg_dump would be sufficient, or a shape file, and a SQL log
> > of
> > > > the
> > > > > > > > > > commands you ran.
> > > > > > > > > > Thanks!
> > > > > > > > > > Paul
> > > > > > > > > >
> > > > > > > > > > On Thursday, April 15, 2004, at 12:27 AM, John Hagstrand
> > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hello Paul,
> > > > > > > > > > >
> > > > > > > > > > > Thanks for helping.  I'd like to send you the test data
> > set.
> > > > > > > > > > > What format would you like?
> > > > > > > > > > >
> > > > > > > > > > > John
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >> -----Original Message-----
> > > > > > > > > > >> From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > > >> Sent: Wednesday, April 14, 2004 11:28 PM
> > > > > > > > > > >> To: Tom Lane
> > > > > > > > > > >> Cc: John Hagstrand; [hidden email]
> > > > > > > > > > >> Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > > size 0
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> We'll need the test data set that does this, of course,
> > and
> > > > some
> > > > > > > > > > >> version info. The indexing routines seemed pretty solid
> > > > (tested on
> > > > > > > > > > >> many
> > > > > > > > > > >> many millions of geometries), I am surprised this has come
> > up.
> > > > > > > > > > >>
> > > > > > > > > > >> P.
> > > > > > > > > > >>
> > > > > > > > > > >> On Wednesday, April 14, 2004, at 08:53 PM, Tom Lane wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>> "John Hagstrand" <[hidden email]>
> > > > writes:
> > > > > > > > > > >>>> Here is a more detailed backtrace.
> > > > > > > > > > >>>
> > > > > > > > > > >>>> (gdb) bt
> > > > > > > > > > >>>> #0  errfinish (dummy=0) at elog.c:319
> > > > > > > > > > >>>> #1  0x081a85de in elog_finish (elevel=20,
> > > > > > > > > > >>>>     fmt=0x8231d80 "invalid memory alloc request size
> > %lu")
> > > > at
> > > > > > > > > > >>>> elog.c:853
> > > > > > > > > > >>>> #2  0x081b3dd6 in MemoryContextAlloc (context=0x9f10598,
> > > > size=0)
> > > > > > at
> > > > > > > > > > >>>> mcxt.c:482
> > > > > > > > > > >>>> #3  0x0807411d in gistSplit (r=0xbe9a1938, buffer=805,
> > > > > > > > > > >>>> itup=0x9f25128,
> > > > > > > > > > >>>>     len=0xbfec3eac, giststate=0xbfec4070,
> > res=0xbfec402c) at
> > > > > > > > > > >>>> gist.c:1348
> > > > > > > > > > >>>
> > > > > > > > > > >>> Okay, this makes it pretty clear that the problem is the
> > > > > > picksplit
> > > > > > > > > > >>> function is setting v.spl_nleft to zero.  In other words
> > it's
> > > > > > failed
> > > > > > > > > > >>> to "split" the page at all (everything in the original
> > index
> > > > page
> > > > > > has
> > > > > > > > > > >>> gone into the righthand child).  This seems to me to be
> > > > > > necessarily
> > > > > > > > > > >>> a bug in the index opclass's picksplit function.
> > > > > > > > > > >>>
> > > > > > > > > > >>> You previously said that the column in question is a
> > PostGIS
> > > > > > Geometry
> > > > > > > > > > >>> column, so I'm of the opinion this is a PostGIS bug.
> > (Paul,
> > > > any
> > > > > > > > > > >>> suggestions about how to probe deeper?)
> > > > > > > > > > >>>
> > > > > > > > > > >>> regards, tom lane
> > > > > > > > > > >>>
> > > > > > > > > > >>       Paul Ramsey
> > > > > > > > > > >>       Refractions Research
> > > > > > > > > > >>       Email: [hidden email]
> > > > > > > > > > >>       Phone: (250) 885-0632
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >       Paul Ramsey
> > > > > > > > > >       Refractions Research
> > > > > > > > > >       Email: [hidden email]
> > > > > > > > > >       Phone: (250) 885-0632
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ----- End forwarded message -----
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > _______________________________________________
> > postgis-devel mailing list
> > [hidden email]
> > http://postgis.refractions.net/mailman/listinfo/postgis-devel
> >
>
>
>
>
> _______________________________________________
> postgis-devel mailing list
> [hidden email]
> http://postgis.refractions.net/mailman/listinfo/postgis-devel

Reply | Threaded
Open this post in threaded view
|

Re: Infinite coordinates in geometries

Sandro Santilli-2
I've added ! finite coordinates check in WKB parser as well..
Now it is *really* hard to detect other ! finite() related bugs.
A way could be making the GiST index routines warn in the presence
of non finite coordinates to try tracking where have them been
introduced. What do you think ? Is it worth it ?

As an alternative, we could have a check_for_infinite_geoms(schema,
table, column) func, checking for bounding box.
A simple check is actually checking extent(<column>) as any infinite
or NaN coordinate would make its extent infinite or NaN and this would
be the case for any union as well..

--strk;



On Thu, Jun 03, 2004 at 10:07:04AM +0200, strk wrote:

> On Wed, Jun 02, 2004 at 05:51:06PM -0700, Dave Blasby wrote:
> > Good points, strk.
> >
> > I dont think infinite geometries are valid (I'll check the spec tomorrow).  
> > POINT(Inf Inf) isnt really defining where the point is in space.
> >
> > Inf coordinates are likely to screw up more than just the GiST index.
> >
> > My gut feeling is to make them invisible to the index (treat them like NULL),
> > and any query of the form:
> >
> > SELECT * FROM <table> && 'BOX3D(-Inf -Inf, ...)';
> >
> > to throw an error.  
> >
> > Either that or throw an error whenever your geometry has an Inf or NaN in it.
>
> I've made WKT parser handle Inf coordinates in geom throwing an error
> and GiST ggeometry_compress skip Inf geoms as it does with EMPTY ones.
>
> WKB parser would still accept Inf I guess...
>
> >
> > What does JTS do with Inf coordinates?  Is a geometry with Inf coordinates
> > valid (ie. geometry.isvalid() = true)?
>
> Dunno... can not make JTS parse an Inf geom from .xml ...
>
> --strk;
>
> >
> > Anyone else have suggestions?
> >
> > Quoting strk <[hidden email]>:
> >
> > >
> > > I've found the bug in the postgis picksplit function.
> > > This function is the exact copy of rtree_picksplit from rtree_gist,
> > > so this might be of interest to the rtree_gist guys as well.
> > >
> > > To determine destination of the node, this kind of checks are performed:
> > >
> > >  cur->low.y - pageunion.low.y  < pageunion.high.y - cur->high
> > >  cur->low.y - pageunion.low.y  > pageunion.high.y - cur->high
> > >  cur->low.y - pageunion.low.y == pageunion.high.y - cur->high
> > >
> > > Substracting Inf from Inf gives Nan, which does not match any
> > > of the checks.
> > >
> > > I've currently (CVS) made the == check (equality) the default,
> > > so to have Infinite valued keys be cached by an ADD-TO-LESS-POPULATED
> > > rule.
> > >
> > > I dunno the consequences of this in terms of overall index functionality,
> > > but you don't get the error message anymore ;)
> > > I'm afraid Infinite geometry presence in a GiST index makes the index
> > > unusable because it eventualy ends up with an infinite plane as root
> > > node and with an arbitrary tree built under it.
> > >
> > > Maybe the picksplit function or whatever other GiST
> > > construction function invoked at every new inserted geometry or at
> > > while table scan should warn the user, or skip infinite geometries
> > > (making them invisible by the index).
> > >
> > > Any comment from GiST friends ?
> > >
> > > --strk;
> > >
> > > On Tue, Jun 01, 2004 at 10:14:55AM -0700, Paul Ramsey wrote:
> > > > Quoting strk <[hidden email]>:
> > > >
> > > > > On Tue, Jun 01, 2004 at 10:00:02AM -0700, Paul Ramsey wrote:
> > > > > > No, they shouldn't create Inf geometries.
> > > > > > It's 'POINT(Inf Inf)'::geometry, no comma, BTW.
> > > > > > P.
> > > > >
> > > > > Oops... it works, sorry :)
> > > > > Now, what should I do ?
> > > > > Drop Inf geometry support or use it to find other bugs ?
> > > >
> > > > I think we should drop it, what are your thoughts? If we leave it in, there
> > > is
> > > > an implied rational behavior to apply in the presence of these things which
> > > > could be very hard to figure out. I suppose from an indexing point of view
> > > they
> > > > are no worse than NULL geometries in semantic terms.
> > > >
> > > > >
> > > > > The operation our guy was trying to execute is not specified,
> > > > > I tried filling a table with Inf points, building a gist index,
> > > > > querying it... no memory alloc problems so far.
> > > > >
> > > > >
> > > > > --strk;
> > > > >
> > > > > >
> > > > > > Quoting strk <[hidden email]>:
> > > > > >
> > > > > > > On Tue, Jun 01, 2004 at 09:53:04AM -0700, Paul Ramsey wrote:
> > > > > > > > Intriguing... doing POINT(Inf,Inf) does not do the trick? I would
> > > have
> > > > > > > thought
> > > > > > > > that a2d() would interpret the Inf token properly.
> > > > > > > > P.
> > > > > > >
> > > > > > > test=# select 'POINT(Inf, Inf)'::geometry;
> > > > > > > ERROR:  couldnt parse object in GEOMETRY
> > > > > > >
> > > > > > > If John Hagstrand could reproduce the Inf geoms that would help...
> > > > > > > I think the bug is actually there.. postgis functions should not
> > > > > > > produce such geoms, should 'hem ?
> > > > > > >
> > > > > > > --strk;
> > > > > > >
> > > > > > > >
> > > > > > > > Quoting strk <[hidden email]>:
> > > > > > > >
> > > > > > > > > On Tue, Jun 01, 2004 at 09:13:33AM -0700, Paul Ramsey wrote:
> > > > > > > > > > Sandro,
> > > > > > > > > > Could you check for a reasonable place to handle this special
> > > case?
> > > > > I
> > > > > > > do
> > > > > > > > > not
> > > > > > > > > > think Inf geometries are something we want to handle,
> > > necessarily
> > > > > :)
> > > > > > > > > > Paul
> > > > > > > > >
> > > > > > > > > The question is: how did this Inf geometry get created ?
> > > > > > > > > I dunno how to make one, for debugging purposes...
> > > > > > > > > --strk;
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ----- Forwarded message from
> > > [hidden email]
> > > > > -----
> > > > > > > > > >     Date: Tue, 1 Jun 2004 10:59:32 -0500
> > > > > > > > > >     From: John Hagstrand <[hidden email]>
> > > > > > > > > > Reply-To: John Hagstrand <[hidden email]>
> > > > > > > > > >  Subject: RE: [GENERAL] ERROR: invalid memory alloc request
> > > size 0
> > > > > > > > > >       To: Paul Ramsey <[hidden email]>
> > > > > > > > > >
> > > > > > > > > > Hello Paul,
> > > > > > > > > >
> > > > > > > > > > I solved this problem.
> > > > > > > > > >
> > > > > > > > > > I had 20 or 30 records in that table with a geometry column
> > > like
> > > > > this:
> > > > > > > > > > POINT(-inf -inf).  I deleted those records, and then dropped
> > > and
> > > > > > > recreated
> > > > > > > > > > the index.  That solved the problem.  I can now do inserts and
> > > > > updates
> > > > > > > > > > without getting the malloc error.
> > > > > > > > > >
> > > > > > > > > > Thanks for your help.
> > > > > > > > > > I look forward to seeing you in Ottawa next week.
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > > > Sent: Thursday, April 15, 2004 10:49 PM
> > > > > > > > > > > To: John Hagstrand
> > > > > > > > > > > Cc: Tom Lane; [hidden email]
> > > > > > > > > > > Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > size 0
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > John,
> > > > > > > > > > > A pg_dump would be sufficient, or a shape file, and a SQL log
> > > of
> > > > > the
> > > > > > > > > > > commands you ran.
> > > > > > > > > > > Thanks!
> > > > > > > > > > > Paul
> > > > > > > > > > >
> > > > > > > > > > > On Thursday, April 15, 2004, at 12:27 AM, John Hagstrand
> > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hello Paul,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for helping.  I'd like to send you the test data
> > > set.
> > > > > > > > > > > > What format would you like?
> > > > > > > > > > > >
> > > > > > > > > > > > John
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >> -----Original Message-----
> > > > > > > > > > > >> From: Paul Ramsey [mailto:[hidden email]]
> > > > > > > > > > > >> Sent: Wednesday, April 14, 2004 11:28 PM
> > > > > > > > > > > >> To: Tom Lane
> > > > > > > > > > > >> Cc: John Hagstrand; [hidden email]
> > > > > > > > > > > >> Subject: Re: [GENERAL] ERROR: invalid memory alloc request
> > > > > size 0
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> We'll need the test data set that does this, of course,
> > > and
> > > > > some
> > > > > > > > > > > >> version info. The indexing routines seemed pretty solid
> > > > > (tested on
> > > > > > > > > > > >> many
> > > > > > > > > > > >> many millions of geometries), I am surprised this has come
> > > up.
> > > > > > > > > > > >>
> > > > > > > > > > > >> P.
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Wednesday, April 14, 2004, at 08:53 PM, Tom Lane wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> "John Hagstrand" <[hidden email]>
> > > > > writes:
> > > > > > > > > > > >>>> Here is a more detailed backtrace.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>> (gdb) bt
> > > > > > > > > > > >>>> #0  errfinish (dummy=0) at elog.c:319
> > > > > > > > > > > >>>> #1  0x081a85de in elog_finish (elevel=20,
> > > > > > > > > > > >>>>     fmt=0x8231d80 "invalid memory alloc request size
> > > %lu")
> > > > > at
> > > > > > > > > > > >>>> elog.c:853
> > > > > > > > > > > >>>> #2  0x081b3dd6 in MemoryContextAlloc (context=0x9f10598,
> > > > > size=0)
> > > > > > > at
> > > > > > > > > > > >>>> mcxt.c:482
> > > > > > > > > > > >>>> #3  0x0807411d in gistSplit (r=0xbe9a1938, buffer=805,
> > > > > > > > > > > >>>> itup=0x9f25128,
> > > > > > > > > > > >>>>     len=0xbfec3eac, giststate=0xbfec4070,
> > > res=0xbfec402c) at
> > > > > > > > > > > >>>> gist.c:1348
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Okay, this makes it pretty clear that the problem is the
> > > > > > > picksplit
> > > > > > > > > > > >>> function is setting v.spl_nleft to zero.  In other words
> > > it's
> > > > > > > failed
> > > > > > > > > > > >>> to "split" the page at all (everything in the original
> > > index
> > > > > page
> > > > > > > has
> > > > > > > > > > > >>> gone into the righthand child).  This seems to me to be
> > > > > > > necessarily
> > > > > > > > > > > >>> a bug in the index opclass's picksplit function.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> You previously said that the column in question is a
> > > PostGIS
> > > > > > > Geometry
> > > > > > > > > > > >>> column, so I'm of the opinion this is a PostGIS bug.
> > > (Paul,
> > > > > any
> > > > > > > > > > > >>> suggestions about how to probe deeper?)
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> regards, tom lane
> > > > > > > > > > > >>>
> > > > > > > > > > > >>       Paul Ramsey
> > > > > > > > > > > >>       Refractions Research
> > > > > > > > > > > >>       Email: [hidden email]
> > > > > > > > > > > >>       Phone: (250) 885-0632
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >       Paul Ramsey
> > > > > > > > > > >       Refractions Research
> > > > > > > > > > >       Email: [hidden email]
> > > > > > > > > > >       Phone: (250) 885-0632
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ----- End forwarded message -----
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > _______________________________________________
> > > postgis-devel mailing list
> > > [hidden email]
> > > http://postgis.refractions.net/mailman/listinfo/postgis-devel
> > >
> >
> >
> >
> >
> > _______________________________________________
> > postgis-devel mailing list
> > [hidden email]
> > http://postgis.refractions.net/mailman/listinfo/postgis-devel
> _______________________________________________
> postgis-devel mailing list
> [hidden email]
> http://postgis.refractions.net/mailman/listinfo/postgis-devel