[gdal-dev] Exporting large geojsons to postgis using ogr2ogr

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Exporting large geojsons to postgis using ogr2ogr

Hemant Bist-2
Hi,
ogr2ogr is repeatedly failing for me with the following error when I try to import a large geojson (5gb) to postgres. The geojson itself was created by ogr2ogr from a ESRI gdb file.
I would appreciate any pointers for workarounds. I only found 1] which indicates that I may have to manually(using custom script) split the geojson file.

 ogr2ogr  -f "PostgreSQL" -append -update -nlt GEOMETRY -t_srs EPSG:4326 PG:"dbname=mydb  tables=dummy" vol2/myuser/delme/a.json'  -nln tmpx -lco FID=gid -lco PRECISION=NO

ERROR 1: GeoJSON parsing error: buffer size overflow (at offset 0)
ERROR 4: Failed to read GeoJSON data
FAILURE:
Unable to open datasource `/vol2/myuser/delme/a.json' with the following drivers.
  -> PCIDSK
  -> JPEG2000
  -> PDF
  -> ESRI Shapefile


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Exporting large geojsons to postgis using ogr2ogr

Even Rouault-2

On jeudi 2 novembre 2017 13:39:56 CET Hemant Bist wrote:

> Hi,

> ogr2ogr is repeatedly failing for me with the following error when I try to

> import a large geojson (5gb) to postgres. The geojson itself was created by

> ogr2ogr from a ESRI gdb file.

> I would appreciate any pointers for workarounds. I only found 1] which

> indicates that I may have to manually(using custom script) split the

> geojson file.

 

I've fixed this a couple days ago in GDAL trunk to be able to read arbitrarily large GeoJSON files with little RAM.

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev