ogr and utf-8 encoding

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

ogr and utf-8 encoding

Roger André
Hi all,

I'm converting a CSV file that contains names which are utf-8 encoded
into a shapefile with ogr in Python.  It appears from my research that
there is currently no way to specify an encoding type with ogr for the
fields in the target shapefile.  I see that there is an RFC to
"enhance" gdal/ogr so that it can respect input encoding, but am not
sure if I've missed another obvious way to preserve the diacritics in
my input data.

Could someone please let me know if, A) there is a way to preserve
unicode encoding from source and pass it into a new shapefile, B) what
the status is on RFC-5, "Unicode support in GDAL", and when this
functionality might be available.

Thanks very much,

Roger
--
_______________________________________________
Gdal-dev mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: ogr and utf-8 encoding

Frank Warmerdam
Roger André wrote:

> Hi all,
>
> I'm converting a CSV file that contains names which are utf-8 encoded
> into a shapefile with ogr in Python.  It appears from my research that
> there is currently no way to specify an encoding type with ogr for the
> fields in the target shapefile.  I see that there is an RFC to
> "enhance" gdal/ogr so that it can respect input encoding, but am not
> sure if I've missed another obvious way to preserve the diacritics in
> my input data.
>
> Could someone please let me know if, A) there is a way to preserve
> unicode encoding from source and pass it into a new shapefile, B) what
> the status is on RFC-5, "Unicode support in GDAL", and when this
> functionality might be available.

Roger,

My understanding is that no work on utf-8 or RFC-5 has been done yet.
Currently there is no mechanism to set the code page for shapefiles nor for
identifying the code page from csv files.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org

_______________________________________________
Gdal-dev mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

ogr and utf-8 encoding

Uwe Schmitz-2
In reply to this post by Roger André
Roger,
Frank,

>
> My understanding is that no work on utf-8 or RFC-5 has been done yet.
> Currently there is no mechanism to set the code page for
> shapefiles nor for
> identifying the code page from csv files.
>
at least for shape-files it seems to be possible to
specify a code page. Please take a look at:

http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=27345

and especially
http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=21106
http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=26015

But I fear this is not widely used.

best wishes
uwe

---------------------------------------------------------
... Uwe Schmitz  Landesvermessungsamt Nordrhein-Westfalen
... Muffendorfer Str. 19 - 21              D - 53177 Bonn
... E-mail:       [hidden email]
... Internet:     http://www.lverma.nrw.de


_______________________________________________
Gdal-dev mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: ogr and utf-8 encoding

Stephen Woodbridge
[hidden email] wrote:

> Roger,
> Frank,
>
>> My understanding is that no work on utf-8 or RFC-5 has been done yet.
>> Currently there is no mechanism to set the code page for
>> shapefiles nor for
>> identifying the code page from csv files.
>>
> at least for shape-files it seems to be possible to
> specify a code page. Please take a look at:
>
> http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=27345
>
> and especially
> http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=21106
> http://support.esri.com/index.cfm?fa=knowledgebase.techarticles.articleShow&d=26015
>
> But I fear this is not widely used.

It would be nice if shapelib supported this with an interface to iconv.
This could also be useful in shp2pgsql and pgsql2shp.

-Steve
_______________________________________________
Gdal-dev mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: ogr and utf-8 encoding

Didrik Pinte-2-3

On Tue, 2007-10-09 at 09:28 -0400, Stephen Woodbridge wrote:
>
> It would be nice if shapelib supported this with an interface to iconv.
> This could also be useful in shp2pgsql and pgsql2shp.
>
> -Steve

I think it could be interesting to inform you that Bram de Greve has
done some work on the utf-8 support in shapelib.

This was done in the version used with Thuban. For more information,
look here :

http://www.intevation.de/pipermail/thuban-devel/2007-April/002034.html 

Didrik

_______________________________________________
Gdal-dev mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/gdal-dev

signature.asc (196 bytes) Download Attachment