[postgis] OGR and PostGIS

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[postgis] OGR and PostGIS

Frank Warmerdam
Folks,

I have a C++ library called 'OGR Simple Features' which provides a
moderately abstracted way of interacting with a variety of GIS vector
formats, including DGN, MapInfo binary, MapInfo .mid/.mif, Shapefiles,
Tiger, NTF, S-57, SDTS and some others.

I am now working on integrating OGR with PostGIS.  I have successfully
built and installed PostGIS 0.1 on PostgreSQL 7.1.2 on Linux (Debian).

I have run the provided tests, and got a few variations.  Are these of
any concern?   Note that the "make test" script doesn't seem to preclean
the output files, so I wasted a few minutes with that.

diff.log:
376,378c376,378
<        geom        
< -------------------
<  POINT(Infinity 0)
---
>      geom    
> --------------
>  POINT(inf 0)
382,384c382,384
<         geom        
< --------------------
<  POINT(-Infinity 0)
---
>      geom      
> ---------------
>  POINT(-inf 0)
672c672
<    884
---
>    880
740c740
< INSERT 18857599 1
---
> INSERT 20649 1
793c793
<     72080 |    72080
---
>     72076 |    72076

Also, I tried the example inserts from the README.postgis, and had some
problems with the first polygon geometry:

This insert:

INSERT INTO geom_test ( gid, geom, name )
  VALUES ( 1, 'POLYGON((0 0 0,0 5 0,5 5 0,5 0 0,0 0 0))', '3D Square');

Resulted in this from a select *:

   1 | POLYGON((0 0 0,0 5.31146529464635e-315 0,5.31146529464635e-315 5.31146529464635e-315 0,5.31146529464635e-315 0 0,0 0 0)) | 3D Square

Note that the other two geometries worked just fine.

Finally, I have integrated the ability into OGR to read PostGIS tables.
They are identied as any table with a field that has the type "Geometry".
This is now working fine for read access, and I will start work on creating
new geometric tables from OGR.

What is the best way for my OGR library to determine if a PostgeSQL
instance has the PostGIS extensions loaded?  Should I access the types
table, and look for a Geometry type?

Anyone interested in looking at OGR can find it at:

  http://gdal.velocet.ca/projects/opengis/

Best regards,
 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

Re: [postgis] OGR and PostGIS

Paul Ramsey-2
Frank Warmerdam wrote:

> I have run the provided tests, and got a few variations.  Are these of
> any concern?   Note that the "make test" script doesn't seem to preclean
> the output files, so I wasted a few minutes with that.

Frankly, our regression test could use alot of work, it's hardly better
than nothing at all at this point.

> This insert:
>
> INSERT INTO geom_test ( gid, geom, name )
>   VALUES ( 1, 'POLYGON((0 0 0,0 5 0,5 5 0,5 0 0,0 0 0))', '3D Square');
>
> Resulted in this from a select *:
>
>    1 | POLYGON((0 0 0,0 5.31146529464635e-315 0,5.31146529464635e-315 5.31146529464635e-315 0,5.31146529464635e-315 0 0,0 0 0)) | 3D Square


This is very disturbing... suffice to say, the same thing doesn't happen
on our installation... I will try and get PostgreSQL/PostGIS running on
a Linux box here and see if we can duplicate it. Does the problem
*always* happen, or did it happen just the first time you tried that
particular geometry?


--
      __
     /
     | Paul Ramsey
     | Refractions Research
     | Email: [hidden email]
     | Phone: (250) 885-0632
     \_

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

Re: [postgis] OGR and PostGIS

Frank Warmerdam
Paul Ramsey wrote:

>
> Frank Warmerdam wrote:
>
> > I have run the provided tests, and got a few variations.  Are these of
> > any concern?   Note that the "make test" script doesn't seem to preclean
> > the output files, so I wasted a few minutes with that.
>
> Frankly, our regression test could use alot of work, it's hardly better
> than nothing at all at this point.
>
> > This insert:
> >
> > INSERT INTO geom_test ( gid, geom, name )
> >   VALUES ( 1, 'POLYGON((0 0 0,0 5 0,5 5 0,5 0 0,0 0 0))', '3D Square');
> >
> > Resulted in this from a select *:
> >
> >    1 | POLYGON((0 0 0,0 5.31146529464635e-315 0,5.31146529464635e-315 5.31146529464635e-315 0,5.31146529464635e-315 0 0,0 0 0)) | 3D Square
>
> This is very disturbing... suffice to say, the same thing doesn't happen
> on our installation... I will try and get PostgreSQL/PostGIS running on
> a Linux box here and see if we can duplicate it. Does the problem
> *always* happen, or did it happen just the first time you tried that
> particular geometry?

Paul,

It happened twice on the same insert statement, while the rest were fine
so it is reproducable.  I will try doing a little debugging on this myself
and see if I can figure out what is going on.  A good chance to get my hands
wet on the code.  I can also provide a login account on my system for debugging
if I can't figure it out myself.

Best regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

Re: [postgis] OGR and PostGIS

David Blasby-3
In reply to this post by Frank Warmerdam
Frank Warmerdam wrote:
>
> Folks,


Frank, I'm really happy to see that you've dived into PostGIS and given
a bunch of really helpfull comments.


> <        geom
> < -------------------
> <  POINT(Infinity 0)
> ---
> >      geom
> > --------------
> >  POINT(inf 0)
> 382,384c382,384
> <         geom
> < --------------------
> <  POINT(-Infinity 0)
> ---
> >      geom
> > ---------------
> >  POINT(-inf 0)


These are fine.  It caused by how printf("%g", (double) 1e999 ) is
interpreted.  On our Solaris machine, it prints "Infinity"; on an i386
it prints "inf".
Our solaris machine correctly parses "Infinity", "infinity", "inf", and
"Inf" as valid instances of the IEEE 'infinity' value.  I expect that
this is the case on your i386 machine.  We're creating a linux i386
installation as I type, so I can test this there.


The next two problems are more interesting;


> 672c672
> <    884
> ---
> >    880
> 740c740
> < INSERT 18857599 1
> ---
> > INSERT 20649 1
> 793c793
> <     72080 |    72080
> ---
> >     72076 |    72076


These values are the size of the underlying geometry structure (using
the mem_size() function).  Its very strange that our solaris machine
would give a different number of bytes than your i386.  Most likely this
is being causes by (1) different size of base types (ie. sizeof(bool) on
i386 != sizeof(bool) on sparc) or (2) memory alignment differences.  


> INSERT INTO geom_test ( gid, geom, name )
>   VALUES ( 1, 'POLYGON((0 0 0,0 5 0,5 5 0,5 0 0,0 0 0))', '3D Square');
>
> Resulted in this from a select *:
>
>    1 | POLYGON((0 0 0,0 5.31146529464635e-315 0,5.31146529464635e-315 5.31146529464635e-315 0,5.31146529464635e-315 0 0,0 0 0)) | 3D Square


This is probably caused by the same problem as above.  It works
correctly on our solaris install.  

I'll take a good look at the structures and see why you are consistently
4 bytes shorter than I am.

> Finally, I have integrated the ability into OGR to read PostGIS tables.
> They are identied as any table with a field that has the type "Geometry".
> This is now working fine for read access, and I will start work on creating
> new geometric tables from OGR.

Nice!
 
> What is the best way for my OGR library to determine if a PostgeSQL
> instance has the PostGIS extensions loaded?  Should I access the types
> table, and look for a Geometry type?

There are two types you will be interested in;

select OID from pg_type where typname = 'geometry';
select OID from pg_type where typname = 'wkb';


if you do a 'select * from geom_table', you will get your geometry back
as a block of text (wkt format).

if you do a 'select gid,name, wkb_xdr(geom) from geom_table', you'll get
your geometry back as type wkb (in WKB).  (only in release 0.2 or above,
and you have to use binary cursors)

The wkb type is just like the internal type 'text'.  The first 4 bytes
represent the size of the structure, then the actual wkb stream.

Paul has installed PostGIS on a i386 linux box, so I should have no
problem tracking down the problem.  When I'm finished I'll send you out
a pre-release 0.2 (with the wkb stuff).  Included is a simple program
that uses binary cursors to read binary data out of the database.

dave

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

Re: [postgis] Odd Insert Problem

Paul Ramsey-2
In reply to this post by Frank Warmerdam
Frank,
The problem occurs identically on our linux installation, so it is
definitately an architecture issue. I suspect that some judicious
ifndefs using the information from postgis.h will clear it up. Dave has
it under the debugger right now.
Paul

Frank Warmerdam wrote:
> > > This insert:
> > >
> > > INSERT INTO geom_test ( gid, geom, name )
> > >   VALUES ( 1, 'POLYGON((0 0 0,0 5 0,5 5 0,5 0 0,0 0 0))', '3D Square');
> > >
> > > Resulted in this from a select *:
> > >
> > >    1 | POLYGON((0 0 0,0 5.31146529464635e-315 0,5.31146529464635e-315 5.31146529464635e-315 0,5.31146529464635e-315 0 0,0 0 0)) | 3D Square
> >

--
      __
     /
     | Paul Ramsey
     | Refractions Research
     | Email: [hidden email]
     | Phone: (250) 885-0632
     \_

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

Re: [postgis] Odd Insert Problem

Frank Warmerdam
Paul Ramsey wrote:
>
> Frank,
> The problem occurs identically on our linux installation, so it is
> definitately an architecture issue. I suspect that some judicious
> ifndefs using the information from postgis.h will clear it up. Dave has
> it under the debugger right now.
> Paul

Paul,

The error seems to be in the assumptions about 4 vs. 8 byte alignment.  It
seems that under linux stuff only needs to be four byte aligned.

Strangely I found the polygons did work under pgaccess, a tcl/tk frontend
to Postgresql.  Tracking this down the difference was that the geometry
objects in geometry_out() happened to be four byte aligned in one case (the
broken case), and happened to be eight byte aligned when accessed from
pgaccess.  

I believe that this logic to "eight byte align" the pts pointers is not
appropriate on the linux architecture.  It seems you will need some sort
of build time logic to determine if this operation is required on a
per-platform basis.

            if ((int32) pts != (((int32)pts>>3)<<3) )
                pts = (POINT3D *) ((char *) pts + 4);


I printed out the values of some stuff from psql (wrong):

First point in polygon:(0.000000000000,0.000000000000,0.000000000000)
First Point Offset: 84:0x8217db0:0x8217d5c  <-- geometry
                           ^
                          pts

>From pgaccess:
First point in polygon:(0.000000000000,4.000000000000,0.000000000000)
First Point Offset: 80:0x8218748:0x82186f8

In the pgaccess case the geometry pointer fell on an eight byte
boundary, so no adjustment was done.  In the psql case the offset
to the pts within the record was "adjusted" to achieve eight byte
alighnment but it really shouldn't have been.

Best regards,
                   
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



Reply | Threaded
Open this post in threaded view
|

[postgis] Fixed - odd insert problem

David Blasby-3


Frank Warmerdam wrote:
> The error seems to be in the assumptions about 4 vs. 8 byte alignment.  It
> seems that under linux stuff only needs to be four byte aligned.

This is exactly what was wrong.  Sparc aligns doubles on 8 bytes and
i386 aligns on 4 bytes.  This causes problems when the GEOMETRY
structure was originally created on a 4-byte boundary, then later
re-allocated on a 8-byte boundary (or vis-versa).  This would never
happen on a sparc.

The solution is easy, replace everything that looks like:

>             if ((int32) pts != (((int32)pts>>3)<<3) )
>                 pts = (POINT3D *) ((char *) pts + 4);

with
             pts = (POINT3D *) MAXALIGN(pts);

I was going to do this in a later release anyways since its more
readable.
{ On a sparc, the MAXALIGN macro will align to 8 byte boundaries, and on
i386 to 4 byte boundaries}

 
Release 0.2 will have this fix in it.

dave

To unsubscribe from this group, send an email to:
[hidden email]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/