rgrass7: readVECT() issue with drivers other than ESRI Shapefile

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Hi,

I am trying to get vector data from GRASS GIS 7 into R using rgrass7.
The default driver is SHAPE which is not able to maintain the longer
than 10 chars column names etc. So I tried GeoJSON which would do the
job, yet I get:

> mydata <- readVECT("table_2010_2014", driver="GeoJSON")
Error in .read_vect_non_plugin(vname = vname, layer = layer, type = type,  :
  Requested driver GeoJSON not supported

Where does the error message come from? I guess R?

So, reading ?readVECT I see:
...
  driver: default ‘"ESRI Shapefile"’; a valid OGR writable driver name
          to define the file format for intermediate files, one of
          ‘c("GML", "SQLite"), c("ESRI_Shapefile", "MapInfo_File")’

Is there any reason that GeoJSON is excluded?

Trying with GML (since I really need to preserve the column names) it fails:

> mydata <- readVECT("table_2010_2014", driver="GML")
Error in paste(gtmpfl1, shname, sep = .Platform$file.sep) :
  object 'shname' not found

Likewise also the SQLite driver fails...

Guess I'm missing something...

best,
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Roger Bivand
Hi,

This has been lagging and needs attention - please provide a reproducible
example. A work-around may also be to copy across the column names, but
driver= is never regularly tested, and probably should be. rgrass7 uses:

nc_basic_spm_grass7

Roger

On Wed, 17 Jun 2015, Markus Neteler wrote:

> Hi,
>
> I am trying to get vector data from GRASS GIS 7 into R using rgrass7.
> The default driver is SHAPE which is not able to maintain the longer
> than 10 chars column names etc. So I tried GeoJSON which would do the
> job, yet I get:
>
>> mydata <- readVECT("table_2010_2014", driver="GeoJSON")
> Error in .read_vect_non_plugin(vname = vname, layer = layer, type = type,  :
>  Requested driver GeoJSON not supported
>
> Where does the error message come from? I guess R?
>
> So, reading ?readVECT I see:
> ...
>  driver: default ‘"ESRI Shapefile"’; a valid OGR writable driver name
>          to define the file format for intermediate files, one of
>          ‘c("GML", "SQLite"), c("ESRI_Shapefile", "MapInfo_File")’
>
> Is there any reason that GeoJSON is excluded?
>
> Trying with GML (since I really need to preserve the column names) it fails:
>
>> mydata <- readVECT("table_2010_2014", driver="GML")
> Error in paste(gtmpfl1, shname, sep = .Platform$file.sep) :
>  object 'shname' not found
>
> Likewise also the SQLite driver fails...
>
> Guess I'm missing something...
>
> best,
> Markus
> _______________________________________________
> grass-stats mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/grass-stats
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: [hidden email]

_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Dear Roger,

On Wed, Jun 17, 2015 at 12:41 PM, Roger Bivand <[hidden email]> wrote:
> Hi,
>
> This has been lagging and needs attention - please provide a reproducible
> example. A work-around may also be to copy across the column names, but
> driver= is never regularly tested, and probably should be. rgrass7 uses:
>
> nc_basic_spm_grass7

Here an example for that location

g.copy vector=geonames,mygeonames
v.db.renamecolumn mygeonames column=ALTERNATEN,ALTERNATE_NAME

GRASS 7.0.1RC1 (nc_basic_spm_grass7):~ > R
# R version 3.2.0 (2015-04-16) -- "Full of Ingredients"

> library(rgrass7)
> geonames <- readVECT("mygeonames")
Warning 1: Field NAME of width 1000 truncated to 254.
Warning 1: Field ASCIINAME of width 1000 truncated to 254.
Warning 1: Field FEATURECLA of width 1000 truncated to 254.
Warning 1: Field FEATURECOD of width 1000 truncated to 254.
Warning 1: Field COUNTRYCOD of width 1000 truncated to 254.
Warning 1: Field CC2 of width 1000 truncated to 254.
Warning 1: Field ADMIN1 of width 1000 truncated to 254.
Warning 1: Field TIMEZONE of width 1000 truncated to 254.
Warning 1: Field MODIFICATI of width 1000 truncated to 254.
ERROR 6: Failed to add field named 'ALTERNATE_NAME'
Exporting 1088 features...
 100%
v.out.ogr complete. 1088 features (Point type) written to <mygeonam>
(ESRI_Shapefile format).
OGR data source with driver: ESRI Shapefile
Source: "/home/neteler/grassdata/nc_basic_spm_grass7/user1/.tmp/pgis_north",
layer: "mygeonam"
with 1088 features
It has 17 fields

--> to avoid unneeded trunkation, I would switch to GeoJSON as driver:

> geonames <- readVECT("mygeonames", driver="GeoJSON")
Error in .read_vect_non_plugin(vname = vname, layer = layer, type = type,  :
  Requested driver GeoJSON not supported

--> given that it is not yet added to the available driver list, I try GML:

> geonames <- readVECT("mygeonames", driver="GML")
Error in paste(gtmpfl1, shname, sep = .Platform$file.sep) :
  object 'shname' not found

If it matters:
> .Platform$file.sep
[1] "/"

I am on Fedora 21:
uname -a
Linux pgis_north 4.0.4-202.fc21.x86_64 #1 SMP Wed May 27 22:28:42 UTC
2015 x86_64 x86_64 x86_64 GNU/Linux

thanks,
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Dear Roger,

On Wed, Jun 17, 2015 at 2:36 PM, Markus Neteler <[hidden email]> wrote:
>> geonames <- readVECT("mygeonames", driver="GML")
> Error in paste(gtmpfl1, shname, sep = .Platform$file.sep) :
>   object 'shname' not found

I fixed it locally. Do you want a diff or me committing it?

Best
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Roger Bivand
Dear Markus,

You have commit rights, so please consider that route. Before committing,
I increment Version: in rgrass7/DESCRIPTION, and run:

R CMD build rgrass7

in spgrass/pkg, creating rgrass7_0.1-*.tar.gz, then start GRASS 7 in the
example location (NC basic), and run:

R CMD check --run-dontrun --run-donttest rgrass7_0.1-*.tar.gz

If it passes without obvious problems caused by the changes made, I
commit.

It may be worth adding a suitable example to the help page in man/,
guarded in case some platform doesn't have the required driver on the
rgdal side. I think that GML and GeoJSON are problematic because of the
binary-string-binary conversions for coordinates, but we need working
cases to start with.

Of course, if it is more convenient for you, a diff for me to patch in
would also be fine.

Thanks for pushing (not a git pun) this!

Best wishes,

Roger

On Thu, 18 Jun 2015, Markus Neteler wrote:

> Dear Roger,
>
> On Wed, Jun 17, 2015 at 2:36 PM, Markus Neteler <[hidden email]> wrote:
>>> geonames <- readVECT("mygeonames", driver="GML")
>> Error in paste(gtmpfl1, shname, sep = .Platform$file.sep) :
>>   object 'shname' not found
>
> I fixed it locally. Do you want a diff or me committing it?
>
> Best
> Markus
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: [hidden email]

_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Dear Roger,

On Thu, Jun 18, 2015 at 10:28 AM, Roger Bivand <[hidden email]> wrote:
> Dear Markus,
>
> You have commit rights, so please consider that route.

ok will do!

> Before committing, I increment Version: in rgrass7/DESCRIPTION, and run:

Should I put 0.2-0 or 0.1-1 ?

> R CMD build rgrass7
>
> in spgrass/pkg, creating rgrass7_0.1-*.tar.gz, then start GRASS 7 in the
> example location (NC basic), and run:
>
> R CMD check --run-dontrun --run-donttest rgrass7_0.1-*.tar.gz
>
> If it passes without obvious problems caused by the changes made, I commit.

My (small) changes do not cause any troubles but there is this unrelated issue:

GRASS 7.0.1RC1 (nc_basic_spm_grass7):~/software/rgrass7_svn/pkg > R
CMD check --run-dontrun --run-donttest rgrass7_0.2-*.tar.gz
* using log directory ‘/home/neteler/software/rgrass7_svn/pkg/rgrass7.Rcheck’
* using R version 3.2.0 (2015-04-16)
* using platform: x86_64-redhat-linux-gnu (64-bit)
...
* checking for unstated dependencies in examples ... OK
* checking examples ... ERROR
Running examples in ‘rgrass7-Ex.R’ failed
The error most likely occurred in:

> ### Name: initGRASS
> ### Title: Initiate GRASS session
> ### Aliases: initGRASS get.GIS_LOCK set.GIS_LOCK unset.GIS_LOCK
> ###   unlink_.gislock
> ### Keywords: spatial
>
> ### ** Examples
>
>
> initGRASS("/usr/bin/grass-7.0.0", home=tempdir())
Error in initGRASS("/usr/bin/grass-7.0.0", home = tempdir()) :
  A GRASS location is already in use; to override, set override=TRUE
Execution halted
* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR
See
  ‘/home/neteler/software/rgrass7_svn/pkg/rgrass7.Rcheck/00check.log’
for details.

I don't know where the "override=TRUE" needs to be used.

> It may be worth adding a suitable example to the help page in man/, guarded
> in case some platform doesn't have the required driver on the rgdal side. I
> think that GML and GeoJSON are problematic because of the
> binary-string-binary conversions for coordinates, but we need working cases
> to start with.

Ok, my first commit will just be the small bugfix.

Regarding the idea to avoid SHAPE as internal data transfer:
The GeoJSON implementation failed because it does not accept a layer name.
But I have a (for me) working GML solution implemented which I would
prefer to send for inspection: a few lines but they look like a hack
to me. Not knowing well what's behind a rgdal call conditionalized
upon the rgdal package version, I likely implemented a workaround. At
least I can start with my analysis now :-)

> Of course, if it is more convenient for you, a diff for me to patch in would
> also be fine.
>
> Thanks for pushing (not a git pun) this!

I'm happy to "commit".

Best wishes,
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Roger Bivand
On Thu, 18 Jun 2015, Markus Neteler wrote:

> Dear Roger,
>
> On Thu, Jun 18, 2015 at 10:28 AM, Roger Bivand <[hidden email]> wrote:
>> Dear Markus,
>>
>> You have commit rights, so please consider that route.
>
> ok will do!
>
>> Before committing, I increment Version: in rgrass7/DESCRIPTION, and run:
>
> Should I put 0.2-0 or 0.1-1 ?
0.1-1, as this would be a small correction of an existing functionality.

>
>> R CMD build rgrass7
>>
>> in spgrass/pkg, creating rgrass7_0.1-*.tar.gz, then start GRASS 7 in the
>> example location (NC basic), and run:
>>
>> R CMD check --run-dontrun --run-donttest rgrass7_0.1-*.tar.gz
>>
>> If it passes without obvious problems caused by the changes made, I commit.
>
> My (small) changes do not cause any troubles but there is this unrelated
> issue:
I'll fix this once you commit; it is protected by \dontrun{} in the help
file examples, but --run-dontrun overrides that. CRAN often uses
--run-dontrun --run-donttest to be sure that the code included in these
exceptions is valid. I need to find a way to bracket this properly.

Sorry, I should have tried myself first!

Roger

>
> GRASS 7.0.1RC1 (nc_basic_spm_grass7):~/software/rgrass7_svn/pkg > R
> CMD check --run-dontrun --run-donttest rgrass7_0.2-*.tar.gz
> * using log directory ‘/home/neteler/software/rgrass7_svn/pkg/rgrass7.Rcheck’
> * using R version 3.2.0 (2015-04-16)
> * using platform: x86_64-redhat-linux-gnu (64-bit)
> ...
> * checking for unstated dependencies in examples ... OK
> * checking examples ... ERROR
> Running examples in ‘rgrass7-Ex.R’ failed
> The error most likely occurred in:
>
>> ### Name: initGRASS
>> ### Title: Initiate GRASS session
>> ### Aliases: initGRASS get.GIS_LOCK set.GIS_LOCK unset.GIS_LOCK
>> ###   unlink_.gislock
>> ### Keywords: spatial
>>
>> ### ** Examples
>>
>>
>> initGRASS("/usr/bin/grass-7.0.0", home=tempdir())
> Error in initGRASS("/usr/bin/grass-7.0.0", home = tempdir()) :
>  A GRASS location is already in use; to override, set override=TRUE
> Execution halted
> * checking PDF version of manual ... OK
> * DONE
>
> Status: 1 ERROR
> See
>  ‘/home/neteler/software/rgrass7_svn/pkg/rgrass7.Rcheck/00check.log’
> for details.
>
> I don't know where the "override=TRUE" needs to be used.
>
>> It may be worth adding a suitable example to the help page in man/, guarded
>> in case some platform doesn't have the required driver on the rgdal side. I
>> think that GML and GeoJSON are problematic because of the
>> binary-string-binary conversions for coordinates, but we need working cases
>> to start with.
>
> Ok, my first commit will just be the small bugfix.
>
> Regarding the idea to avoid SHAPE as internal data transfer:
> The GeoJSON implementation failed because it does not accept a layer name.
> But I have a (for me) working GML solution implemented which I would
> prefer to send for inspection: a few lines but they look like a hack
> to me. Not knowing well what's behind a rgdal call conditionalized
> upon the rgdal package version, I likely implemented a workaround. At
> least I can start with my analysis now :-)
>
>> Of course, if it is more convenient for you, a diff for me to patch in would
>> also be fine.
>>
>> Thanks for pushing (not a git pun) this!
>
> I'm happy to "commit".
>
> Best wishes,
> Markus
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: [hidden email]

_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Dear Roger,

On Thu, Jun 18, 2015 at 11:51 AM, Roger Bivand <[hidden email]> wrote:
> On Thu, 18 Jun 2015, Markus Neteler wrote:
...
>> Should I put 0.2-0 or 0.1-1 ?
>
> 0.1-1, as this would be a small correction of an existing functionality.

Fine, commited as revision 32.

Concerning the GRASS GIS -R data exchange:
GML also not perfect - the yet uncommited local change is to use
"SQLite" rather than SHAPE since it also preserves the projection
metadata (which GML doesn't). Works smoothly here so far.

...
> I'll fix this once you commit; it is protected by \dontrun{} in the help
> file examples, but --run-dontrun overrides that. CRAN often uses
> --run-dontrun --run-donttest to be sure that the code included in these
> exceptions is valid. I need to find a way to bracket this properly.
>
> Sorry, I should have tried myself first!

No problem at all!

best,
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
All,

attached a simple (potentially ugly?) change to enable "SQLite" as a
new driver for reading GRASS GIS 7 vector data into R
(readVECT("vectmap, driver="SQLite")).
I would really appreciate to see "SQLite" support added since it seems
to be the only lossless OGR driver (e.g. unlike the default "SHAPE"
file driver).

I'll not commit this myself to r-forge since I am not sure if it
should be implemented differently.
Please verify.

thanks,
Markus

_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats

rgrass7_svn.diff (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Markus Neteler
Roger, all,

greetings from the community sprint:
http://grasswiki.osgeo.org/wiki/Talk:GRASS_Community_Sprint_Como_2015


On Fri, Jun 19, 2015 at 9:15 AM, Markus Neteler <[hidden email]> wrote:

> All,
>
> attached a simple (potentially ugly?) change to enable "SQLite" as a
> new driver for reading GRASS GIS 7 vector data into R
> (readVECT("vectmap, driver="SQLite")).
> I would really appreciate to see "SQLite" support added since it seems
> to be the only lossless OGR driver (e.g. unlike the default "SHAPE"
> file driver).
>
> I'll not commit this myself to r-forge since I am not sure if it
> should be implemented differently.
> Please verify.


Do you have any suggestions how to add SQLite support for a better
vector interface?

thanks
Markus
_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Reply | Threaded
Open this post in threaded view
|

Re: rgrass7: readVECT() issue with drivers other than ESRI Shapefile

Roger Bivand
No possibility of helping now, mostly offline to 8 August.

SQLite depends on the GDAL driver, which depends on an external library
(or several). The Windows and OSX rgdal binaries are built in as minimal
way as possible, linking statically. Adding SQLite possibly with
Spatialite is not currently feasible, and binary builds typically include
GEOS, which we don't want as it is used through rgeos, not rgdal. We
certainly should try to do something, but not by introducing any external
dependencies, so first check drivers available in the CRAN Windoes and OSX
binaries (which are built by the CRAN team, not me).

Roger

On Sun, 19 Jul 2015, Markus Neteler wrote:

> Roger, all,
>
> greetings from the community sprint:
> http://grasswiki.osgeo.org/wiki/Talk:GRASS_Community_Sprint_Como_2015
>
>
> On Fri, Jun 19, 2015 at 9:15 AM, Markus Neteler <[hidden email]> wrote:
>> All,
>>
>> attached a simple (potentially ugly?) change to enable "SQLite" as a
>> new driver for reading GRASS GIS 7 vector data into R
>> (readVECT("vectmap, driver="SQLite")).
>> I would really appreciate to see "SQLite" support added since it seems
>> to be the only lossless OGR driver (e.g. unlike the default "SHAPE"
>> file driver).
>>
>> I'll not commit this myself to r-forge since I am not sure if it
>> should be implemented differently.
>> Please verify.
>
>
> Do you have any suggestions how to add SQLite support for a better
> vector interface?
>
> thanks
> Markus
>

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; fax +47 55 95 91 00
e-mail: [hidden email]

_______________________________________________
grass-stats mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-stats
Roger Bivand
NHH Norwegian School of Economics, Bergen, Norway