Converting multilayer KML to single layer output

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Converting multilayer KML to single layer output

Rutger
Hey,

I am trying to convert a kml-file containing multiple layers to a new output (GeoJSON for example), which only supports a single layer. All geometry types in the kml are identical, all polygon.

The only way i can make it work (by using gdal.VectorTranslate) is to first write a single layer to a file in 'overwrite' mode. And then run almost the identical command for the remaining layers in 'append' mode. Writing all layers in append mode sort of works, it creates the file and writes all layers, but it doesn't close the file properly. In the case of GeoJSON, it doesnt write ']}' at the end of the file.

What i currently do is:

res = gdal.VectorTranslate(outfile, infile, format='GeoJSON', accessMode='overwrite', layers=layers[0], layerName='outlayer')
res = None

res = gdal.VectorTranslate(outfile, infile, format='GeoJSON', accessMode='append', layers=layers[1:], layerName='outlayer')
res = None


Where 'layers' is a Python list populated by iterating over the layers and calling 'lyr.GetName()'.

I dont like needing two commands to do this, and also iterating over all layers manually, just to get the names, seems a bit redundant. So i'm wondering if anyone knows ways to improve upon this.

Regards,
Rutger

Reply | Threaded
Open this post in threaded view
|

Re: Converting multilayer KML to single layer output

Even Rouault-2

On mercredi 30 novembre 2016 05:44:54 CET Rutger wrote:

> Hey,

>

> I am trying to convert a kml-file containing multiple layers to a new output

> (GeoJSON for example), which only supports a single layer. All geometry

> types in the kml are identical, all polygon.

>

> The only way i can make it work (by using gdal.VectorTranslate) is to first

> write a single layer to a file in 'overwrite' mode. And then run almost the

> identical command for the remaining layers in 'append' mode. Writing all

> layers in append mode sort of works, it creates the file and writes all

> layers, but it doesn't close the file properly. In the case of GeoJSON, it

> doesnt write ']}' at the end of the file.

 

I can't reproduce your issue with the file not being terminated with ]}

 

Hum maybe this is due to trying to append several layers with layers[1:] syntax. You should probably

 

>

> What i currently do is:

> /

> res = gdal.VectorTranslate(outfile, infile, format='GeoJSON',

> accessMode='overwrite', layers=layers[0], layerName='outlayer')

> res = None

 

Note that currently layers must be provided with an array, and not without a string (otherwise it iterates over each character of the string...) This is a defect I'm going to fix.

 

Note also that the 'overwrite' mode is without effect here (and will actually not do what you want). overwrite is for multi-layer datasets where you want to overwrite an existing layer. In the GeoJSON driver, this is not implemented. You should just remove it and in that case the whole file will be overwritten.

 

 

>

> res = gdal.VectorTranslate(outfile, infile, format='GeoJSON',

> accessMode='append', layers=layers[1:], layerName='outlayer')

> res = None/

>

> Where 'layers' is a Python list populated by iterating over the layers and

> calling 'lyr.GetName()'.

>

> I dont like needing two commands to do this, and also iterating over all

> layers manually, just to get the names, seems a bit redundant. So i'm

> wondering if anyone knows ways to improve upon this.

 

Actually I've just tested the following successfully (with trunk)

 

from osgeo import gdal

 

gdal.VectorTranslate('out.json', 'src', layerName = 'outlayer', format = 'GeoJSON', accessMode='append', layers = ['poly', 'poly2', 'poly3'])

 

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Converting multilayer KML to single layer output

Rutger
Hey,

Thank you for your reponse.

Even Rouault-2 wrote
I can't reproduce your issue with the file not being terminated with ]}

Hum maybe this is due to trying to append several layers with layers[1:] syntax. You should
probably

Note that currently layers must be provided with an array, and not without a string
(otherwise it iterates over each character of the string...) This is a defect I'm going to fix.

Note also that the 'overwrite' mode is without effect here (and will actually not do what you
want). overwrite is for multi-layer datasets where you want to overwrite an existing layer. In
the GeoJSON driver, this is not implemented. You should just remove it and in that case the
whole file will be overwritten.
You're right of course about 'layers' not accepting a string if you try to specify a single layer, i noticed that mistake in my code later on as well. Its not affecting the behavior i'm experiencing. I work in the Jupyter Notebook and that moves you away from the console where the GDAL warnings are printed, which is why i didn't noticed it right away. GDAL prints a nice 'ERROR 1: Couldn't fetch requested layer 'a'!' warning which makes the sliced string obvious.

Even Rouault-2 wrote
Actually I've just tested the following successfully (with trunk)

from osgeo import gdal

gdal.VectorTranslate('out.json', 'src', layerName = 'outlayer', format = 'GeoJSON',
accessMode='append', layers = ['poly', 'poly2', 'poly3'])
Good to know that it should work like that, i started off with something similar.

After some more trial and error i think I've found whats wrong. I was first really focused on the keyword options, hence trying all access modes etc.

It turns out that its more a flushing/releasing issue, perhaps in combination with the Jupyter Notebook (holding a reference to the object?). If i display the returned dataset, Jupyter displays the string 'repr'. I do this all the time with GDAL datasets just to confirm i actually have a dataset, whereas a failed command would result in "None".

But somehow, in this case, closing the dataset with "ds = None" fails to release the file handle (and doesnt flush?), but only when let Jupyter display the dataset (repr).  If i close the dataset right away, it works as expected.

Here is a notebook showing what i did:
http://nbviewer.jupyter.org/gist/RutgerK/7da186e23b3ec20526c9a1c16c5cf98a

Is the method which doesn't work something that should work, or is it simply bad practice? I'm pretty sure i do this with the raster utilities all the time, without any problems.


Regards,
Rutger