Integration of Japanese GIS formats into mainline GDAL ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
Hi,

Sorry for writing in English, but I don't unfortunately speak Japanese.

I'm one of the GDAL developers.

I've noticed that there are "un-official" versions of GDAL floating around, with
a few drivers for new Japanese-specific formats. I was wondering if there was a
reason to keep them outside of the official GDAL tree ? Integrating them into
mainline GDAL would probably make life of Japanese users easier. GDAL has
already support for a lof of national specific formats, mainly in Europe and
North America. Would be cool for Asia to join in ;-)

In a recent discussion thread, the following link is mentionned:
 https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip

I can see that it contains 2 new GDAL drivers ( FGD: Japanese Fundamental
Geographic Data (.xml) & FG-GML :  Japanese Fundamental Geographic Data
GML(.xml) ) and one OGR driver (FGD).
Unfortunatelly there's no author or copyright information in the header of the
corresponding source files. Would their author(s) be willing to release them
under the X/MIT license and contribute them to the mainline GDAL project ?
Perhaps some test files to write minimalistic regressions tests would be
useful.

Let me know if I can be useful for anything. I can also help for writing new
drivers if needed.

Best regards,

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Hiroo Imaki
Hi Even,

I am not the developer but he and I working at a same company. If he can agree on the license and release his efforts, that would be wonderful for all Japanese data uses.

He hesitates writing his message in English. So, I can be a translator between you guys. I hope no lost in translation!

I will get back to you once I hear from him.

Hiroo

On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault <[hidden email]> wrote:
Hi,

Sorry for writing in English, but I don't unfortunately speak Japanese.

I'm one of the GDAL developers.

I've noticed that there are "un-official" versions of GDAL floating around, with
a few drivers for new Japanese-specific formats. I was wondering if there was a
reason to keep them outside of the official GDAL tree ? Integrating them into
mainline GDAL would probably make life of Japanese users easier. GDAL has
already support for a lof of national specific formats, mainly in Europe and
North America. Would be cool for Asia to join in ;-)

In a recent discussion thread, the following link is mentionned:
 https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip

I can see that it contains 2 new GDAL drivers ( FGD: Japanese Fundamental
Geographic Data (.xml) & FG-GML :  Japanese Fundamental Geographic Data
GML(.xml) ) and one OGR driver (FGD).
Unfortunatelly there's no author or copyright information in the header of the
corresponding source files. Would their author(s) be willing to release them
under the X/MIT license and contribute them to the mainline GDAL project ?
Perhaps some test files to write minimalistic regressions tests would be
useful.

Let me know if I can be useful for anything. I can also help for writing new
drivers if needed.

Best regards,

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss



--
Hiroo Imaki
Pacific Spatial Solutions, LLC
1523 Chatham Colony Ct.
Reston, VA  20190
http://www.geopacific.org (GIS info site)
[hidden email] (private)


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Hiroo Imaki
Hi Even,

I talk to Yamate-san who has been working on those three drivers. He explained that each driver has little bit different copy right background he need to resolve before he can merge them to the main.

The first one, FGD driver, which he wrote for a project and he does not have control on its copy right. So, he would like to leave it at this moment.

For the second one, FG-GML driver, he wrote it and he owns its copy right at this moment. But he said he can release with the licence you suggested.

The last one, FGD ogr driver, which he also wrote and it is highly possible he can release it under the MIT licence. But he need to make sure his client can agree on this. So, he said he will contact to his client.

So, the last two drivers can be merge into the main GDAL driver. It is little bit confusing background but I am sure we can sort these out.

I will let you know when he is ready to work with you.

Thanks!

Hiroo



On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
Hi Even,

I am not the developer but he and I working at a same company. If he can agree on the license and release his efforts, that would be wonderful for all Japanese data uses.

He hesitates writing his message in English. So, I can be a translator between you guys. I hope no lost in translation!

I will get back to you once I hear from him.

Hiroo

On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault <[hidden email]> wrote:
Hi,

Sorry for writing in English, but I don't unfortunately speak Japanese.

I'm one of the GDAL developers.

I've noticed that there are "un-official" versions of GDAL floating around, with
a few drivers for new Japanese-specific formats. I was wondering if there was a
reason to keep them outside of the official GDAL tree ? Integrating them into
mainline GDAL would probably make life of Japanese users easier. GDAL has
already support for a lof of national specific formats, mainly in Europe and
North America. Would be cool for Asia to join in ;-)

In a recent discussion thread, the following link is mentionned:
 https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip

I can see that it contains 2 new GDAL drivers ( FGD: Japanese Fundamental
Geographic Data (.xml) & FG-GML :  Japanese Fundamental Geographic Data
GML(.xml) ) and one OGR driver (FGD).
Unfortunatelly there's no author or copyright information in the header of the
corresponding source files. Would their author(s) be willing to release them
under the X/MIT license and contribute them to the mainline GDAL project ?
Perhaps some test files to write minimalistic regressions tests would be
useful.

Let me know if I can be useful for anything. I can also help for writing new
drivers if needed.

Best regards,

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss



--
Hiroo Imaki
Pacific Spatial Solutions, LLC
1523 Chatham Colony Ct.
Reston, VA  20190
http://www.geopacific.org (GIS info site)
[hidden email] (private)




--
Hiroo Imaki
Pacific Spatial Solutions, LLC
1523 Chatham Colony Ct.
Reston, VA  20190
http://www.geopacific.org (GIS info site)
[hidden email] (private)


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Yoichi Kayama

これは本家にコントリビュートするべきだとずっと言っておるのですが無視されてます。
2015/05/01 10:17 "Hiroo Imaki" <[hidden email]>:
>
> Hi Even,
>
> I talk to Yamate-san who has been working on those three drivers. He explained that each driver has little bit different copy right background he need to resolve before he can merge them to the main.
>
> The first one, FGD driver, which he wrote for a project and he does not have control on its copy right. So, he would like to leave it at this moment.
>
> For the second one, FG-GML driver, he wrote it and he owns its copy right at this moment. But he said he can release with the licence you suggested.
>
> The last one, FGD ogr driver, which he also wrote and it is highly possible he can release it under the MIT licence. But he need to make sure his client can agree on this. So, he said he will contact to his client.
>
> So, the last two drivers can be merge into the main GDAL driver. It is little bit confusing background but I am sure we can sort these out.
>
> I will let you know when he is ready to work with you.
>
> Thanks!
>
> Hiroo
>
>
>
> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
>>
>> Hi Even,
>>
>> I am not the developer but he and I working at a same company. If he can agree on the license and release his efforts, that would be wonderful for all Japanese data uses.
>>
>> He hesitates writing his message in English. So, I can be a translator between you guys. I hope no lost in translation!
>>
>> I will get back to you once I hear from him.
>>
>> Hiroo
>>
>> On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> Sorry for writing in English, but I don't unfortunately speak Japanese.
>>>
>>> I'm one of the GDAL developers.
>>>
>>> I've noticed that there are "un-official" versions of GDAL floating around, with
>>> a few drivers for new Japanese-specific formats. I was wondering if there was a
>>> reason to keep them outside of the official GDAL tree ? Integrating them into
>>> mainline GDAL would probably make life of Japanese users easier. GDAL has
>>> already support for a lof of national specific formats, mainly in Europe and
>>> North America. Would be cool for Asia to join in ;-)
>>>
>>> In a recent discussion thread, the following link is mentionned:
>>>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>>>
>>> I can see that it contains 2 new GDAL drivers ( FGD: Japanese Fundamental
>>> Geographic Data (.xml) & FG-GML :  Japanese Fundamental Geographic Data
>>> GML(.xml) ) and one OGR driver (FGD).
>>> Unfortunatelly there's no author or copyright information in the header of the
>>> corresponding source files. Would their author(s) be willing to release them
>>> under the X/MIT license and contribute them to the mainline GDAL project ?
>>> Perhaps some test files to write minimalistic regressions tests would be
>>> useful.
>>>
>>> Let me know if I can be useful for anything. I can also help for writing new
>>> drivers if needed.
>>>
>>> Best regards,
>>>
>>> Even
>>>
>>> --
>>> Spatialys - Geospatial professional services
>>> http://www.spatialys.com
>>> _______________________________________________
>>> OSGeoJapan-discuss mailing list
>>> [hidden email]
>>> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>>
>>
>>
>>
>> --
>> Hiroo Imaki
>> Pacific Spatial Solutions, LLC
>> 1523 Chatham Colony Ct.
>> Reston, VA  20190
>> [hidden email]
>> http://www.pacificspatial.com
>> http://www.geopacific.org (GIS info site)
>> [hidden email] (private)
>>
>
>
>
> --
> Hiroo Imaki
> Pacific Spatial Solutions, LLC
> 1523 Chatham Colony Ct.
> Reston, VA  20190
> [hidden email]
> http://www.pacificspatial.com
> http://www.geopacific.org (GIS info site)
> [hidden email] (private)
>
>
> _______________________________________________
> OSGeoJapan-discuss mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
In reply to this post by Hiroo Imaki
Hi Hiroo,

OK I let you take the necessary time sort that out. Being completely an
outsider and not having the "big picture" of the existing formats and how
historic or up-to-date they are, it is also not clear to me which drivers
might be of general interest for the Japanese community, as opposed to the
ones which would really be of very restricted scope of interest.

(Just a word of caution: as those archives are public, and the new drivers are
included in a distribution of source code that is labelled as X/MIT in its
LICENSE.TXT file, people could in good faith assume that all the files,
including the extra drivers are also X/MIT)

I tried the FGD ogr driver on the file
https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/data/shift_jis/FG-
GML-13-RailCL25000-20080331-0001.xml
(found by googling FG-GML)

It is not recognized by the driver. Looking more closely, the driver seems to
expect a <GI> element that isn't found in that file. It appears that there are
several versions of FGD GML, right ? The above file is apparently v3 (since
there's a v3 schema next to it). Perhaps the driver is v4 (or a previous
version) ? I tried to remove the test for the presence of <GI> but it crashes
later during parsing of dates, apparently not liking the <gml:timePosition>
element within <lfSpanFr>

I found this project : http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
https://github.com/mizutuu/jpgisgmlv4togml-converter
It includes the schema for FG-GML v4 and I can't see any reference to the <GI>
element in it either.
With some trickery related to encoding conversion, I managed to use that
project to produce a standard GML file that the OGR GML driver handles well :

cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis --
to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py > out.gml

I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a few
datasets. Those I tried are not handled by the OGR FGD driver. Some of them
look like GML with a particular application schema. They are unfortunately not
standard enough for the OGR GML driver to understand them out of the box,
although I believe it could be tweaked to parse them. Their header look like:

<ksj:Dataset gml:id="A09Dataset"
xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:xlink="http://www.w3.org/1999/xlink" xm
lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app KsjAppSchema-
A09-v3_1.xsd">

There's a shapefile next to them, with apparently same content.

So it appears there's a variety of GML-like formats ;-)

Even

> Hi Even,
>
> I talk to Yamate-san who has been working on those three drivers. He
> explained that each driver has little bit different copy right background
> he need to resolve before he can merge them to the main.
>
> The first one, FGD driver, which he wrote for a project and he does not
> have control on its copy right. So, he would like to leave it at this
> moment.
>
> For the second one, FG-GML driver, he wrote it and he owns its copy right
> at this moment. But he said he can release with the licence you suggested.
>
> The last one, FGD ogr driver, which he also wrote and it is highly possible
> he can release it under the MIT licence. But he need to make sure his
> client can agree on this. So, he said he will contact to his client.
>
> So, the last two drivers can be merge into the main GDAL driver. It is
> little bit confusing background but I am sure we can sort these out.
>
> I will let you know when he is ready to work with you.
>
> Thanks!
>
> Hiroo
>
> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
> > Hi Even,
> >
> > I am not the developer but he and I working at a same company. If he can
> > agree on the license and release his efforts, that would be wonderful for
> > all Japanese data uses.
> >
> > He hesitates writing his message in English. So, I can be a translator
> > between you guys. I hope no lost in translation!
> >
> > I will get back to you once I hear from him.
> >
> > Hiroo
> >
> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> > <[hidden email]
> >
> > > wrote:
> >> Hi,
> >>
> >> Sorry for writing in English, but I don't unfortunately speak Japanese.
> >>
> >> I'm one of the GDAL developers.
> >>
> >> I've noticed that there are "un-official" versions of GDAL floating
> >> around, with
> >> a few drivers for new Japanese-specific formats. I was wondering if
> >> there was a
> >> reason to keep them outside of the official GDAL tree ? Integrating them
> >> into
> >> mainline GDAL would probably make life of Japanese users easier. GDAL
> >> has already support for a lof of national specific formats, mainly in
> >> Europe and
> >> North America. Would be cool for Asia to join in ;-)
> >>
> >> In a recent discussion thread, the following link is mentionned:
> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >>
> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> Unfortunatelly there's no author or copyright information in the header
> >> of the
> >> corresponding source files. Would their author(s) be willing to release
> >> them
> >> under the X/MIT license and contribute them to the mainline GDAL project
> >> ? Perhaps some test files to write minimalistic regressions tests would
> >> be useful.
> >>
> >> Let me know if I can be useful for anything. I can also help for writing
> >> new
> >> drivers if needed.
> >>
> >> Best regards,
> >>
> >> Even
> >>
> >> --
> >> Spatialys - Geospatial professional services
> >> http://www.spatialys.com
> >> _______________________________________________
> >> OSGeoJapan-discuss mailing list
> >> [hidden email]
> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >
> > --
> > Hiroo Imaki
> > Pacific Spatial Solutions, LLC
> > 1523 Chatham Colony Ct.
> > Reston, VA  20190
> > [hidden email]
> > http://www.pacificspatial.com
> > http://www.geopacific.org (GIS info site)
> > [hidden email] (private)

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Hiroo Imaki
Hi Even,

All players have started talking about this topic. Japanese people are in the middle of long vacation season and I will restart this conversation with all contributors the end of second week of May.

I will keep posting.

Thanks for your nice offer and contributions to make this happens!

Hiroo

On Fri, May 1, 2015 at 5:59 AM, Even Rouault <[hidden email]> wrote:
Hi Hiroo,

OK I let you take the necessary time sort that out. Being completely an
outsider and not having the "big picture" of the existing formats and how
historic or up-to-date they are, it is also not clear to me which drivers
might be of general interest for the Japanese community, as opposed to the
ones which would really be of very restricted scope of interest.

(Just a word of caution: as those archives are public, and the new drivers are
included in a distribution of source code that is labelled as X/MIT in its
LICENSE.TXT file, people could in good faith assume that all the files,
including the extra drivers are also X/MIT)

I tried the FGD ogr driver on the file
<a href="https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/data/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml" target="_blank">https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/data/shift_jis/FG-
GML-13-RailCL25000-20080331-0001.xml
(found by googling FG-GML)

It is not recognized by the driver. Looking more closely, the driver seems to
expect a <GI> element that isn't found in that file. It appears that there are
several versions of FGD GML, right ? The above file is apparently v3 (since
there's a v3 schema next to it). Perhaps the driver is v4 (or a previous
version) ? I tried to remove the test for the presence of <GI> but it crashes
later during parsing of dates, apparently not liking the <gml:timePosition>
element within <lfSpanFr>

I found this project : http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
https://github.com/mizutuu/jpgisgmlv4togml-converter
It includes the schema for FG-GML v4 and I can't see any reference to the <GI>
element in it either.
With some trickery related to encoding conversion, I managed to use that
project to produce a standard GML file that the OGR GML driver handles well :

cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis --
to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py > out.gml

I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a few
datasets. Those I tried are not handled by the OGR FGD driver. Some of them
look like GML with a particular application schema. They are unfortunately not
standard enough for the OGR GML driver to understand them out of the box,
although I believe it could be tweaked to parse them. Their header look like:

<ksj:Dataset gml:id="A09Dataset"
xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:xlink="http://www.w3.org/1999/xlink" xm
lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app KsjAppSchema-
A09-v3_1.xsd">

There's a shapefile next to them, with apparently same content.

So it appears there's a variety of GML-like formats ;-)

Even

> Hi Even,
>
> I talk to Yamate-san who has been working on those three drivers. He
> explained that each driver has little bit different copy right background
> he need to resolve before he can merge them to the main.
>
> The first one, FGD driver, which he wrote for a project and he does not
> have control on its copy right. So, he would like to leave it at this
> moment.
>
> For the second one, FG-GML driver, he wrote it and he owns its copy right
> at this moment. But he said he can release with the licence you suggested.
>
> The last one, FGD ogr driver, which he also wrote and it is highly possible
> he can release it under the MIT licence. But he need to make sure his
> client can agree on this. So, he said he will contact to his client.
>
> So, the last two drivers can be merge into the main GDAL driver. It is
> little bit confusing background but I am sure we can sort these out.
>
> I will let you know when he is ready to work with you.
>
> Thanks!
>
> Hiroo
>
> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
> > Hi Even,
> >
> > I am not the developer but he and I working at a same company. If he can
> > agree on the license and release his efforts, that would be wonderful for
> > all Japanese data uses.
> >
> > He hesitates writing his message in English. So, I can be a translator
> > between you guys. I hope no lost in translation!
> >
> > I will get back to you once I hear from him.
> >
> > Hiroo
> >
> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> > <[hidden email]
> >
> > > wrote:
> >> Hi,
> >>
> >> Sorry for writing in English, but I don't unfortunately speak Japanese.
> >>
> >> I'm one of the GDAL developers.
> >>
> >> I've noticed that there are "un-official" versions of GDAL floating
> >> around, with
> >> a few drivers for new Japanese-specific formats. I was wondering if
> >> there was a
> >> reason to keep them outside of the official GDAL tree ? Integrating them
> >> into
> >> mainline GDAL would probably make life of Japanese users easier. GDAL
> >> has already support for a lof of national specific formats, mainly in
> >> Europe and
> >> North America. Would be cool for Asia to join in ;-)
> >>
> >> In a recent discussion thread, the following link is mentionned:
> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >>
> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> Unfortunatelly there's no author or copyright information in the header
> >> of the
> >> corresponding source files. Would their author(s) be willing to release
> >> them
> >> under the X/MIT license and contribute them to the mainline GDAL project
> >> ? Perhaps some test files to write minimalistic regressions tests would
> >> be useful.
> >>
> >> Let me know if I can be useful for anything. I can also help for writing
> >> new
> >> drivers if needed.
> >>
> >> Best regards,
> >>
> >> Even
> >>
> >> --
> >> Spatialys - Geospatial professional services
> >> http://www.spatialys.com
> >> _______________________________________________
> >> OSGeoJapan-discuss mailing list
> >> [hidden email]
> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >
> > --
> > Hiroo Imaki
> > Pacific Spatial Solutions, LLC
> > 1523 Chatham Colony Ct.
> > Reston, VA  20190
> > [hidden email]
> > http://www.pacificspatial.com
> > http://www.geopacific.org (GIS info site)
> > [hidden email] (private)

--
Spatialys - Geospatial professional services
http://www.spatialys.com



--
Hiroo Imaki
Pacific Spatial Solutions, LLC
1523 Chatham Colony Ct.
Reston, VA  20190
http://www.geopacific.org (GIS info site)
[hidden email] (private)


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
In reply to this post by Even Rouault-2
Hi Even,
CC: Hiroo-san, Yamate-san, list

Excuse me for cutting in this topic.

I am neither the developer nor the player, but I tried mext_gdal
test version (gdal110_test.zip) at last weekend and probably I understood
some data related part, so I mention some of your question about datas.
(But, if someone find my mistake, please point it out here...)

2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:

> :
> I tried the FGD ogr driver on the file
> https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/data/shift_jis/FG-
> GML-13-RailCL25000-20080331-0001.xml
> (found by googling FG-GML)
>
> It is not recognized by the driver. Looking more closely, the driver seems to
> expect a <GI> element that isn't found in that file. It appears that there are
> several versions of FGD GML, right ? The above file is apparently v3 (since
> there's a v3 schema next to it). Perhaps the driver is v4 (or a previous
> version) ? I tried to remove the test for the presence of <GI> but it crashes
> later during parsing of dates, apparently not liking the <gml:timePosition>
> element within <lfSpanFr>

There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
in FGD data,
and current mext_gdal test version seems to support the following pair.

=====
1. FGD driver: DLD v3 (5/10m mesh DEM)
2. FG-GML driver: GML v3 (5/10m mesh DEM)
3. FGD driver: DLD v3 (General vector data)
=====

Above link which you found use GML v3, but it is not 5/10m mesh DEM,
so you couldn't convert it.

> I found this project : http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> https://github.com/mizutuu/jpgisgmlv4togml-converter
> It includes the schema for FG-GML v4 and I can't see any reference to the <GI>
> element in it either.
> With some trickery related to encoding conversion, I managed to use that
> project to produce a standard GML file that the OGR GML driver handles well :
>
> cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis --
> to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py > out.gml

And above GML v4 is not supported (both of 5/10m mesh DEM and
General vector data) by current mext_gdal for now.

> I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a few
> datasets. Those I tried are not handled by the OGR FGD driver. Some of them
> look like GML with a particular application schema. They are unfortunately not
> standard enough for the OGR GML driver to understand them out of the box,
> although I believe it could be tweaked to parse them. Their header look like:
>
> <ksj:Dataset gml:id="A09Dataset"
> xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> xmlns:gml="http://www.opengis.net/gml/3.2"
> xmlns:xlink="http://www.w3.org/1999/xlink" xm
> lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app KsjAppSchema-
> A09-v3_1.xsd">
>
> There's a shapefile next to them, with apparently same content.

And about above MLIT NLNI(National Land Numerical Information) data,
it is not FGD data...

> So it appears there's a variety of GML-like formats ;-)

Yes, sure...

If you want to use mext_gdal test data, I just uploaded example dummy data
to the following repository. So, please try to use those files.
https://github.com/sanak/dev-notes/tree/master/datas/fgd
(The dummy data location is near by Japan Wakayama prefecture,
but on the pacific ocean.)

Regards,


>> Hi Even,
>>
>> I talk to Yamate-san who has been working on those three drivers. He
>> explained that each driver has little bit different copy right background
>> he need to resolve before he can merge them to the main.
>>
>> The first one, FGD driver, which he wrote for a project and he does not
>> have control on its copy right. So, he would like to leave it at this
>> moment.
>>
>> For the second one, FG-GML driver, he wrote it and he owns its copy right
>> at this moment. But he said he can release with the licence you suggested.
>>
>> The last one, FGD ogr driver, which he also wrote and it is highly possible
>> he can release it under the MIT licence. But he need to make sure his
>> client can agree on this. So, he said he will contact to his client.
>>
>> So, the last two drivers can be merge into the main GDAL driver. It is
>> little bit confusing background but I am sure we can sort these out.
>>
>> I will let you know when he is ready to work with you.
>>
>> Thanks!
>>
>> Hiroo
>>
>> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
>> > Hi Even,
>> >
>> > I am not the developer but he and I working at a same company. If he can
>> > agree on the license and release his efforts, that would be wonderful for
>> > all Japanese data uses.
>> >
>> > He hesitates writing his message in English. So, I can be a translator
>> > between you guys. I hope no lost in translation!
>> >
>> > I will get back to you once I hear from him.
>> >
>> > Hiroo
>> >
>> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> > <[hidden email]
>> >
>> > > wrote:
>> >> Hi,
>> >>
>> >> Sorry for writing in English, but I don't unfortunately speak Japanese.
>> >>
>> >> I'm one of the GDAL developers.
>> >>
>> >> I've noticed that there are "un-official" versions of GDAL floating
>> >> around, with
>> >> a few drivers for new Japanese-specific formats. I was wondering if
>> >> there was a
>> >> reason to keep them outside of the official GDAL tree ? Integrating them
>> >> into
>> >> mainline GDAL would probably make life of Japanese users easier. GDAL
>> >> has already support for a lof of national specific formats, mainly in
>> >> Europe and
>> >> North America. Would be cool for Asia to join in ;-)
>> >>
>> >> In a recent discussion thread, the following link is mentionned:
>> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> >>
>> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
>> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> >> Unfortunatelly there's no author or copyright information in the header
>> >> of the
>> >> corresponding source files. Would their author(s) be willing to release
>> >> them
>> >> under the X/MIT license and contribute them to the mainline GDAL project
>> >> ? Perhaps some test files to write minimalistic regressions tests would
>> >> be useful.
>> >>
>> >> Let me know if I can be useful for anything. I can also help for writing
>> >> new
>> >> drivers if needed.
>> >>
>> >> Best regards,
>> >>
>> >> Even
>> >>
>> >> --
>> >> Spatialys - Geospatial professional services
>> >> http://www.spatialys.com
>> >> _______________________________________________
>> >> OSGeoJapan-discuss mailing list
>> >> [hidden email]
>> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >
>> > --
>> > Hiroo Imaki
>> > Pacific Spatial Solutions, LLC
>> > 1523 Chatham Colony Ct.
>> > Reston, VA  20190
>> > [hidden email]
>> > http://www.pacificspatial.com
>> > http://www.geopacific.org (GIS info site)
>> > [hidden email] (private)
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> OSGeoJapan-discuss mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
Ko-san,

Thanks a lot for bringing some light on this.

I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all findings up
to now and trace tasks that I've in mind that should be completed for
integretation into upstream GDAL.

Best regards,

Even

> Hi Even,
> CC: Hiroo-san, Yamate-san, list
>
> Excuse me for cutting in this topic.
>
> I am neither the developer nor the player, but I tried mext_gdal
> test version (gdal110_test.zip) at last weekend and probably I understood
> some data related part, so I mention some of your question about datas.
> (But, if someone find my mistake, please point it out here...)
>
> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
> > I tried the FGD ogr driver on the file
> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/dat
> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
> > (found by googling FG-GML)
> >
> > It is not recognized by the driver. Looking more closely, the driver
> > seems to expect a <GI> element that isn't found in that file. It appears
> > that there are several versions of FGD GML, right ? The above file is
> > apparently v3 (since there's a v3 schema next to it). Perhaps the driver
> > is v4 (or a previous version) ? I tried to remove the test for the
> > presence of <GI> but it crashes later during parsing of dates,
> > apparently not liking the <gml:timePosition> element within <lfSpanFr>
>
> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
> and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
> in FGD data,
> and current mext_gdal test version seems to support the following pair.
>
> =====
> 1. FGD driver: DLD v3 (5/10m mesh DEM)
> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
> 3. FGD driver: DLD v3 (General vector data)
> =====
>
> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
> so you couldn't convert it.
>
> > I found this project :
> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> > https://github.com/mizutuu/jpgisgmlv4togml-converter
> > It includes the schema for FG-GML v4 and I can't see any reference to the
> > <GI> element in it either.
> > With some trickery related to encoding conversion, I managed to use that
> > project to produce a standard GML file that the OGR GML driver handles
> > well :
> >
> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis
> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py >
> > out.gml
>
> And above GML v4 is not supported (both of 5/10m mesh DEM and
> General vector data) by current mext_gdal for now.
>
> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a
> > few datasets. Those I tried are not handled by the OGR FGD driver. Some
> > of them look like GML with a particular application schema. They are
> > unfortunately not standard enough for the OGR GML driver to understand
> > them out of the box, although I believe it could be tweaked to parse
> > them. Their header look like:
> >
> > <ksj:Dataset gml:id="A09Dataset"
> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> > xmlns:gml="http://www.opengis.net/gml/3.2"
> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
> > KsjAppSchema- A09-v3_1.xsd">
> >
> > There's a shapefile next to them, with apparently same content.
>
> And about above MLIT NLNI(National Land Numerical Information) data,
> it is not FGD data...
>
> > So it appears there's a variety of GML-like formats ;-)
>
> Yes, sure...
>
> If you want to use mext_gdal test data, I just uploaded example dummy data
> to the following repository. So, please try to use those files.
> https://github.com/sanak/dev-notes/tree/master/datas/fgd
> (The dummy data location is near by Japan Wakayama prefecture,
> but on the pacific ocean.)
>
> Regards,
>
> >> Hi Even,
> >>
> >> I talk to Yamate-san who has been working on those three drivers. He
> >> explained that each driver has little bit different copy right
> >> background he need to resolve before he can merge them to the main.
> >>
> >> The first one, FGD driver, which he wrote for a project and he does not
> >> have control on its copy right. So, he would like to leave it at this
> >> moment.
> >>
> >> For the second one, FG-GML driver, he wrote it and he owns its copy
> >> right at this moment. But he said he can release with the licence you
> >> suggested.
> >>
> >> The last one, FGD ogr driver, which he also wrote and it is highly
> >> possible he can release it under the MIT licence. But he need to make
> >> sure his client can agree on this. So, he said he will contact to his
> >> client.
> >>
> >> So, the last two drivers can be merge into the main GDAL driver. It is
> >> little bit confusing background but I am sure we can sort these out.
> >>
> >> I will let you know when he is ready to work with you.
> >>
> >> Thanks!
> >>
> >> Hiroo
> >>
> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
> >> > Hi Even,
> >> >
> >> > I am not the developer but he and I working at a same company. If he
> >> > can agree on the license and release his efforts, that would be
> >> > wonderful for all Japanese data uses.
> >> >
> >> > He hesitates writing his message in English. So, I can be a translator
> >> > between you guys. I hope no lost in translation!
> >> >
> >> > I will get back to you once I hear from him.
> >> >
> >> > Hiroo
> >> >
> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> >> > <[hidden email]
> >> >
> >> > > wrote:
> >> >> Hi,
> >> >>
> >> >> Sorry for writing in English, but I don't unfortunately speak
> >> >> Japanese.
> >> >>
> >> >> I'm one of the GDAL developers.
> >> >>
> >> >> I've noticed that there are "un-official" versions of GDAL floating
> >> >> around, with
> >> >> a few drivers for new Japanese-specific formats. I was wondering if
> >> >> there was a
> >> >> reason to keep them outside of the official GDAL tree ? Integrating
> >> >> them into
> >> >> mainline GDAL would probably make life of Japanese users easier. GDAL
> >> >> has already support for a lof of national specific formats, mainly in
> >> >> Europe and
> >> >> North America. Would be cool for Asia to join in ;-)
> >> >>
> >> >> In a recent discussion thread, the following link is mentionned:
> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >> >>
> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> >> Unfortunatelly there's no author or copyright information in the
> >> >> header of the
> >> >> corresponding source files. Would their author(s) be willing to
> >> >> release them
> >> >> under the X/MIT license and contribute them to the mainline GDAL
> >> >> project ? Perhaps some test files to write minimalistic regressions
> >> >> tests would be useful.
> >> >>
> >> >> Let me know if I can be useful for anything. I can also help for
> >> >> writing new
> >> >> drivers if needed.
> >> >>
> >> >> Best regards,
> >> >>
> >> >> Even
> >> >>
> >> >> --
> >> >> Spatialys - Geospatial professional services
> >> >> http://www.spatialys.com
> >> >> _______________________________________________
> >> >> OSGeoJapan-discuss mailing list
> >> >> [hidden email]
> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> >
> >> > --
> >> > Hiroo Imaki
> >> > Pacific Spatial Solutions, LLC
> >> > 1523 Chatham Colony Ct.
> >> > Reston, VA  20190
> >> > [hidden email]
> >> > http://www.pacificspatial.com
> >> > http://www.geopacific.org (GIS info site)
> >> > [hidden email] (private)
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > _______________________________________________
> > OSGeoJapan-discuss mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
Hi Even,

Thanks for quick response and creating the ticket.

I made a mistake in list No.3 item name (not "FGD driver", but "FGD
ogr driver"),
so I fixed it on the ticket.
https://trac.osgeo.org/gdal/ticket/6046#comment:2

Regards,

Ko


2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:

> Ko-san,
>
> Thanks a lot for bringing some light on this.
>
> I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all findings up
> to now and trace tasks that I've in mind that should be completed for
> integretation into upstream GDAL.
>
> Best regards,
>
> Even
>
>> Hi Even,
>> CC: Hiroo-san, Yamate-san, list
>>
>> Excuse me for cutting in this topic.
>>
>> I am neither the developer nor the player, but I tried mext_gdal
>> test version (gdal110_test.zip) at last weekend and probably I understood
>> some data related part, so I mention some of your question about datas.
>> (But, if someone find my mistake, please point it out here...)
>>
>> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
>> > I tried the FGD ogr driver on the file
>> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/dat
>> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
>> > (found by googling FG-GML)
>> >
>> > It is not recognized by the driver. Looking more closely, the driver
>> > seems to expect a <GI> element that isn't found in that file. It appears
>> > that there are several versions of FGD GML, right ? The above file is
>> > apparently v3 (since there's a v3 schema next to it). Perhaps the driver
>> > is v4 (or a previous version) ? I tried to remove the test for the
>> > presence of <GI> but it crashes later during parsing of dates,
>> > apparently not liking the <gml:timePosition> element within <lfSpanFr>
>>
>> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
>> and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
>> in FGD data,
>> and current mext_gdal test version seems to support the following pair.
>>
>> =====
>> 1. FGD driver: DLD v3 (5/10m mesh DEM)
>> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
>> 3. FGD driver: DLD v3 (General vector data)
>> =====
>>
>> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
>> so you couldn't convert it.
>>
>> > I found this project :
>> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
>> > https://github.com/mizutuu/jpgisgmlv4togml-converter
>> > It includes the schema for FG-GML v4 and I can't see any reference to the
>> > <GI> element in it either.
>> > With some trickery related to encoding conversion, I managed to use that
>> > project to produce a standard GML file that the OGR GML driver handles
>> > well :
>> >
>> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis
>> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py >
>> > out.gml
>>
>> And above GML v4 is not supported (both of 5/10m mesh DEM and
>> General vector data) by current mext_gdal for now.
>>
>> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a
>> > few datasets. Those I tried are not handled by the OGR FGD driver. Some
>> > of them look like GML with a particular application schema. They are
>> > unfortunately not standard enough for the OGR GML driver to understand
>> > them out of the box, although I believe it could be tweaked to parse
>> > them. Their header look like:
>> >
>> > <ksj:Dataset gml:id="A09Dataset"
>> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
>> > xmlns:gml="http://www.opengis.net/gml/3.2"
>> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
>> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
>> > KsjAppSchema- A09-v3_1.xsd">
>> >
>> > There's a shapefile next to them, with apparently same content.
>>
>> And about above MLIT NLNI(National Land Numerical Information) data,
>> it is not FGD data...
>>
>> > So it appears there's a variety of GML-like formats ;-)
>>
>> Yes, sure...
>>
>> If you want to use mext_gdal test data, I just uploaded example dummy data
>> to the following repository. So, please try to use those files.
>> https://github.com/sanak/dev-notes/tree/master/datas/fgd
>> (The dummy data location is near by Japan Wakayama prefecture,
>> but on the pacific ocean.)
>>
>> Regards,
>>
>> >> Hi Even,
>> >>
>> >> I talk to Yamate-san who has been working on those three drivers. He
>> >> explained that each driver has little bit different copy right
>> >> background he need to resolve before he can merge them to the main.
>> >>
>> >> The first one, FGD driver, which he wrote for a project and he does not
>> >> have control on its copy right. So, he would like to leave it at this
>> >> moment.
>> >>
>> >> For the second one, FG-GML driver, he wrote it and he owns its copy
>> >> right at this moment. But he said he can release with the licence you
>> >> suggested.
>> >>
>> >> The last one, FGD ogr driver, which he also wrote and it is highly
>> >> possible he can release it under the MIT licence. But he need to make
>> >> sure his client can agree on this. So, he said he will contact to his
>> >> client.
>> >>
>> >> So, the last two drivers can be merge into the main GDAL driver. It is
>> >> little bit confusing background but I am sure we can sort these out.
>> >>
>> >> I will let you know when he is ready to work with you.
>> >>
>> >> Thanks!
>> >>
>> >> Hiroo
>> >>
>> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
>> >> > Hi Even,
>> >> >
>> >> > I am not the developer but he and I working at a same company. If he
>> >> > can agree on the license and release his efforts, that would be
>> >> > wonderful for all Japanese data uses.
>> >> >
>> >> > He hesitates writing his message in English. So, I can be a translator
>> >> > between you guys. I hope no lost in translation!
>> >> >
>> >> > I will get back to you once I hear from him.
>> >> >
>> >> > Hiroo
>> >> >
>> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> >> > <[hidden email]
>> >> >
>> >> > > wrote:
>> >> >> Hi,
>> >> >>
>> >> >> Sorry for writing in English, but I don't unfortunately speak
>> >> >> Japanese.
>> >> >>
>> >> >> I'm one of the GDAL developers.
>> >> >>
>> >> >> I've noticed that there are "un-official" versions of GDAL floating
>> >> >> around, with
>> >> >> a few drivers for new Japanese-specific formats. I was wondering if
>> >> >> there was a
>> >> >> reason to keep them outside of the official GDAL tree ? Integrating
>> >> >> them into
>> >> >> mainline GDAL would probably make life of Japanese users easier. GDAL
>> >> >> has already support for a lof of national specific formats, mainly in
>> >> >> Europe and
>> >> >> North America. Would be cool for Asia to join in ;-)
>> >> >>
>> >> >> In a recent discussion thread, the following link is mentionned:
>> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> >> >>
>> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
>> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> >> >> Unfortunatelly there's no author or copyright information in the
>> >> >> header of the
>> >> >> corresponding source files. Would their author(s) be willing to
>> >> >> release them
>> >> >> under the X/MIT license and contribute them to the mainline GDAL
>> >> >> project ? Perhaps some test files to write minimalistic regressions
>> >> >> tests would be useful.
>> >> >>
>> >> >> Let me know if I can be useful for anything. I can also help for
>> >> >> writing new
>> >> >> drivers if needed.
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Even
>> >> >>
>> >> >> --
>> >> >> Spatialys - Geospatial professional services
>> >> >> http://www.spatialys.com
>> >> >> _______________________________________________
>> >> >> OSGeoJapan-discuss mailing list
>> >> >> [hidden email]
>> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >> >
>> >> > --
>> >> > Hiroo Imaki
>> >> > Pacific Spatial Solutions, LLC
>> >> > 1523 Chatham Colony Ct.
>> >> > Reston, VA  20190
>> >> > [hidden email]
>> >> > http://www.pacificspatial.com
>> >> > http://www.geopacific.org (GIS info site)
>> >> > [hidden email] (private)
>> >
>> > --
>> > Spatialys - Geospatial professional services
>> > http://www.spatialys.com
>> > _______________________________________________
>> > OSGeoJapan-discuss mailing list
>> > [hidden email]
>> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Minoru Akagi
In reply to this post by Even Rouault-2
Hi,

We had an online hackathon about this topic on the same day as QGIS
HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
vector data (v4) with the OGR GML driver.

I prepared .gfs schema files, then registered them to the GML registry
and made a few changes to the GML driver code. Finally, I managed to
read FGD (GML) vector data with the driver! See the work branch [1]
and the hackpad [2][3] for the details.

I think we should make efforts for the support of recent format rather
than old formats that are no longer distributed from GSI.

Any comments?

[1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
    The branch is based on 1.11 (the current GDAL version of OSGeo4W).
[2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In Japanese)
[3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English summary)

Minoru

2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:

> Ko-san,
>
> Thanks a lot for bringing some light on this.
>
> I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all findings up
> to now and trace tasks that I've in mind that should be completed for
> integretation into upstream GDAL.
>
> Best regards,
>
> Even
>
>> Hi Even,
>> CC: Hiroo-san, Yamate-san, list
>>
>> Excuse me for cutting in this topic.
>>
>> I am neither the developer nor the player, but I tried mext_gdal
>> test version (gdal110_test.zip) at last weekend and probably I understood
>> some data related part, so I mention some of your question about datas.
>> (But, if someone find my mistake, please point it out here...)
>>
>> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
>> > I tried the FGD ogr driver on the file
>> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/dat
>> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
>> > (found by googling FG-GML)
>> >
>> > It is not recognized by the driver. Looking more closely, the driver
>> > seems to expect a <GI> element that isn't found in that file. It appears
>> > that there are several versions of FGD GML, right ? The above file is
>> > apparently v3 (since there's a v3 schema next to it). Perhaps the driver
>> > is v4 (or a previous version) ? I tried to remove the test for the
>> > presence of <GI> but it crashes later during parsing of dates,
>> > apparently not liking the <gml:timePosition> element within <lfSpanFr>
>>
>> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
>> and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
>> in FGD data,
>> and current mext_gdal test version seems to support the following pair.
>>
>> =====
>> 1. FGD driver: DLD v3 (5/10m mesh DEM)
>> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
>> 3. FGD driver: DLD v3 (General vector data)
>> =====
>>
>> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
>> so you couldn't convert it.
>>
>> > I found this project :
>> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
>> > https://github.com/mizutuu/jpgisgmlv4togml-converter
>> > It includes the schema for FG-GML v4 and I can't see any reference to the
>> > <GI> element in it either.
>> > With some trickery related to encoding conversion, I managed to use that
>> > project to produce a standard GML file that the OGR GML driver handles
>> > well :
>> >
>> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv --from-code=shift-jis
>> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py >
>> > out.gml
>>
>> And above GML v4 is not supported (both of 5/10m mesh DEM and
>> General vector data) by current mext_gdal for now.
>>
>> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded a
>> > few datasets. Those I tried are not handled by the OGR FGD driver. Some
>> > of them look like GML with a particular application schema. They are
>> > unfortunately not standard enough for the OGR GML driver to understand
>> > them out of the box, although I believe it could be tweaked to parse
>> > them. Their header look like:
>> >
>> > <ksj:Dataset gml:id="A09Dataset"
>> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
>> > xmlns:gml="http://www.opengis.net/gml/3.2"
>> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
>> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
>> > KsjAppSchema- A09-v3_1.xsd">
>> >
>> > There's a shapefile next to them, with apparently same content.
>>
>> And about above MLIT NLNI(National Land Numerical Information) data,
>> it is not FGD data...
>>
>> > So it appears there's a variety of GML-like formats ;-)
>>
>> Yes, sure...
>>
>> If you want to use mext_gdal test data, I just uploaded example dummy data
>> to the following repository. So, please try to use those files.
>> https://github.com/sanak/dev-notes/tree/master/datas/fgd
>> (The dummy data location is near by Japan Wakayama prefecture,
>> but on the pacific ocean.)
>>
>> Regards,
>>
>> >> Hi Even,
>> >>
>> >> I talk to Yamate-san who has been working on those three drivers. He
>> >> explained that each driver has little bit different copy right
>> >> background he need to resolve before he can merge them to the main.
>> >>
>> >> The first one, FGD driver, which he wrote for a project and he does not
>> >> have control on its copy right. So, he would like to leave it at this
>> >> moment.
>> >>
>> >> For the second one, FG-GML driver, he wrote it and he owns its copy
>> >> right at this moment. But he said he can release with the licence you
>> >> suggested.
>> >>
>> >> The last one, FGD ogr driver, which he also wrote and it is highly
>> >> possible he can release it under the MIT licence. But he need to make
>> >> sure his client can agree on this. So, he said he will contact to his
>> >> client.
>> >>
>> >> So, the last two drivers can be merge into the main GDAL driver. It is
>> >> little bit confusing background but I am sure we can sort these out.
>> >>
>> >> I will let you know when he is ready to work with you.
>> >>
>> >> Thanks!
>> >>
>> >> Hiroo
>> >>
>> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
>> >> > Hi Even,
>> >> >
>> >> > I am not the developer but he and I working at a same company. If he
>> >> > can agree on the license and release his efforts, that would be
>> >> > wonderful for all Japanese data uses.
>> >> >
>> >> > He hesitates writing his message in English. So, I can be a translator
>> >> > between you guys. I hope no lost in translation!
>> >> >
>> >> > I will get back to you once I hear from him.
>> >> >
>> >> > Hiroo
>> >> >
>> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> >> > <[hidden email]
>> >> >
>> >> > > wrote:
>> >> >> Hi,
>> >> >>
>> >> >> Sorry for writing in English, but I don't unfortunately speak
>> >> >> Japanese.
>> >> >>
>> >> >> I'm one of the GDAL developers.
>> >> >>
>> >> >> I've noticed that there are "un-official" versions of GDAL floating
>> >> >> around, with
>> >> >> a few drivers for new Japanese-specific formats. I was wondering if
>> >> >> there was a
>> >> >> reason to keep them outside of the official GDAL tree ? Integrating
>> >> >> them into
>> >> >> mainline GDAL would probably make life of Japanese users easier. GDAL
>> >> >> has already support for a lof of national specific formats, mainly in
>> >> >> Europe and
>> >> >> North America. Would be cool for Asia to join in ;-)
>> >> >>
>> >> >> In a recent discussion thread, the following link is mentionned:
>> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> >> >>
>> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
>> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> >> >> Unfortunatelly there's no author or copyright information in the
>> >> >> header of the
>> >> >> corresponding source files. Would their author(s) be willing to
>> >> >> release them
>> >> >> under the X/MIT license and contribute them to the mainline GDAL
>> >> >> project ? Perhaps some test files to write minimalistic regressions
>> >> >> tests would be useful.
>> >> >>
>> >> >> Let me know if I can be useful for anything. I can also help for
>> >> >> writing new
>> >> >> drivers if needed.
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >> Even
>> >> >>
>> >> >> --
>> >> >> Spatialys - Geospatial professional services
>> >> >> http://www.spatialys.com
>> >> >> _______________________________________________
>> >> >> OSGeoJapan-discuss mailing list
>> >> >> [hidden email]
>> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >> >
>> >> > --
>> >> > Hiroo Imaki
>> >> > Pacific Spatial Solutions, LLC
>> >> > 1523 Chatham Colony Ct.
>> >> > Reston, VA  20190
>> >> > [hidden email]
>> >> > http://www.pacificspatial.com
>> >> > http://www.geopacific.org (GIS info site)
>> >> > [hidden email] (private)
>> >
>> > --
>> > Spatialys - Geospatial professional services
>> > http://www.spatialys.com
>> > _______________________________________________
>> > OSGeoJapan-discuss mailing list
>> > [hidden email]
>> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> _______________________________________________
> OSGeoJapan-discuss mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
Hi Minoru,
>
> We had an online hackathon about this topic on the same day as QGIS
> HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
> vector data (v4) with the OGR GML driver.
>
> I prepared .gfs schema files, then registered them to the GML registry
> and made a few changes to the GML driver code. Finally, I managed to
> read FGD (GML) vector data with the driver! See the work branch [1]
> and the hackpad [2][3] for the details.

This is cool you managed to use the GML registry mechanism.

I think however that the changes done in ogrgmldatasource.cpp in this branch
should be amended to restore the previous behaviour when the namespace prefix
is defined in the registry, and the behaviour of the branch when it is not
defined, so that full control is still available.

I'd add to the todo list to add a few words to drv_gml.html to mention that
FGD GML v4 is supported in the "Particular GML application schemas" section.

>
> I think we should make efforts for the support of recent format rather
> than old formats that are no longer distributed from GSI.

That makes sense. Sometimes supporting old formats can be of interest when
people are interested in historic data that isn't available to more modern
formats, but this is admitedly a particular use case.

Best regards,

Even

>
> Any comments?
>
> [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
>     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
> [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In Japanese)
> [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
> summary)
>
> Minoru
>
> 2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:
> > Ko-san,
> >
> > Thanks a lot for bringing some light on this.
> >
> > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
> > findings up to now and trace tasks that I've in mind that should be
> > completed for integretation into upstream GDAL.
> >
> > Best regards,
> >
> > Even
> >
> >> Hi Even,
> >> CC: Hiroo-san, Yamate-san, list
> >>
> >> Excuse me for cutting in this topic.
> >>
> >> I am neither the developer nor the player, but I tried mext_gdal
> >> test version (gdal110_test.zip) at last weekend and probably I understood
> >> some data related part, so I mention some of your question about datas.
> >> (But, if someone find my mistake, please point it out here...)
> >>
> >> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
> >> > I tried the FGD ogr driver on the file
> >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/d
> >> > at
> >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
> >> > (found by googling FG-GML)
> >> >
> >> > It is not recognized by the driver. Looking more closely, the driver
> >> > seems to expect a <GI> element that isn't found in that file. It
> >> > appears
> >> > that there are several versions of FGD GML, right ? The above file is
> >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
> >> > driver
> >> > is v4 (or a previous version) ? I tried to remove the test for the
> >> > presence of <GI> but it crashes later during parsing of dates,
> >> > apparently not liking the <gml:timePosition> element within <lfSpanFr>
> >>
> >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
> >> and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
> >> in FGD data,
> >> and current mext_gdal test version seems to support the following pair.
> >>
> >> =====
> >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
> >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
> >> 3. FGD driver: DLD v3 (General vector data)
> >> =====
> >>
> >> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
> >> so you couldn't convert it.
> >>
> >> > I found this project :
> >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
> >> > It includes the schema for FG-GML v4 and I can't see any reference to
> >> > the
> >> > <GI> element in it either.
> >> > With some trickery related to encoding conversion, I managed to use
> >> > that
> >> > project to produce a standard GML file that the OGR GML driver handles
> >> > well :
> >> >
> >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
> >> > --from-code=shift-jis
> >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py >
> >> > out.gml
> >>
> >> And above GML v4 is not supported (both of 5/10m mesh DEM and
> >> General vector data) by current mext_gdal for now.
> >>
> >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded
> >> > a
> >> > few datasets. Those I tried are not handled by the OGR FGD driver. Some
> >> > of them look like GML with a particular application schema. They are
> >> > unfortunately not standard enough for the OGR GML driver to understand
> >> > them out of the box, although I believe it could be tweaked to parse
> >> > them. Their header look like:
> >> >
> >> > <ksj:Dataset gml:id="A09Dataset"
> >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> >> > xmlns:gml="http://www.opengis.net/gml/3.2"
> >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
> >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
> >> > KsjAppSchema- A09-v3_1.xsd">
> >> >
> >> > There's a shapefile next to them, with apparently same content.
> >>
> >> And about above MLIT NLNI(National Land Numerical Information) data,
> >> it is not FGD data...
> >>
> >> > So it appears there's a variety of GML-like formats ;-)
> >>
> >> Yes, sure...
> >>
> >> If you want to use mext_gdal test data, I just uploaded example dummy
> >> data
> >> to the following repository. So, please try to use those files.
> >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
> >> (The dummy data location is near by Japan Wakayama prefecture,
> >> but on the pacific ocean.)
> >>
> >> Regards,
> >>
> >> >> Hi Even,
> >> >>
> >> >> I talk to Yamate-san who has been working on those three drivers. He
> >> >> explained that each driver has little bit different copy right
> >> >> background he need to resolve before he can merge them to the main.
> >> >>
> >> >> The first one, FGD driver, which he wrote for a project and he does
> >> >> not
> >> >> have control on its copy right. So, he would like to leave it at this
> >> >> moment.
> >> >>
> >> >> For the second one, FG-GML driver, he wrote it and he owns its copy
> >> >> right at this moment. But he said he can release with the licence you
> >> >> suggested.
> >> >>
> >> >> The last one, FGD ogr driver, which he also wrote and it is highly
> >> >> possible he can release it under the MIT licence. But he need to make
> >> >> sure his client can agree on this. So, he said he will contact to his
> >> >> client.
> >> >>
> >> >> So, the last two drivers can be merge into the main GDAL driver. It is
> >> >> little bit confusing background but I am sure we can sort these out.
> >> >>
> >> >> I will let you know when he is ready to work with you.
> >> >>
> >> >> Thanks!
> >> >>
> >> >> Hiroo
> >> >>
> >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
> >> >> > Hi Even,
> >> >> >
> >> >> > I am not the developer but he and I working at a same company. If he
> >> >> > can agree on the license and release his efforts, that would be
> >> >> > wonderful for all Japanese data uses.
> >> >> >
> >> >> > He hesitates writing his message in English. So, I can be a
> >> >> > translator
> >> >> > between you guys. I hope no lost in translation!
> >> >> >
> >> >> > I will get back to you once I hear from him.
> >> >> >
> >> >> > Hiroo
> >> >> >
> >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> >> >> > <[hidden email]
> >> >> >
> >> >> > > wrote:
> >> >> >> Hi,
> >> >> >>
> >> >> >> Sorry for writing in English, but I don't unfortunately speak
> >> >> >> Japanese.
> >> >> >>
> >> >> >> I'm one of the GDAL developers.
> >> >> >>
> >> >> >> I've noticed that there are "un-official" versions of GDAL floating
> >> >> >> around, with
> >> >> >> a few drivers for new Japanese-specific formats. I was wondering if
> >> >> >> there was a
> >> >> >> reason to keep them outside of the official GDAL tree ? Integrating
> >> >> >> them into
> >> >> >> mainline GDAL would probably make life of Japanese users easier.
> >> >> >> GDAL
> >> >> >> has already support for a lof of national specific formats, mainly
> >> >> >> in
> >> >> >> Europe and
> >> >> >> North America. Would be cool for Asia to join in ;-)
> >> >> >>
> >> >> >> In a recent discussion thread, the following link is mentionned:
> >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >> >> >>
> >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
> >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> >> >> Unfortunatelly there's no author or copyright information in the
> >> >> >> header of the
> >> >> >> corresponding source files. Would their author(s) be willing to
> >> >> >> release them
> >> >> >> under the X/MIT license and contribute them to the mainline GDAL
> >> >> >> project ? Perhaps some test files to write minimalistic regressions
> >> >> >> tests would be useful.
> >> >> >>
> >> >> >> Let me know if I can be useful for anything. I can also help for
> >> >> >> writing new
> >> >> >> drivers if needed.
> >> >> >>
> >> >> >> Best regards,
> >> >> >>
> >> >> >> Even
> >> >> >>
> >> >> >> --
> >> >> >> Spatialys - Geospatial professional services
> >> >> >> http://www.spatialys.com
> >> >> >> _______________________________________________
> >> >> >> OSGeoJapan-discuss mailing list
> >> >> >> [hidden email]
> >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> >> >
> >> >> > --
> >> >> > Hiroo Imaki
> >> >> > Pacific Spatial Solutions, LLC
> >> >> > 1523 Chatham Colony Ct.
> >> >> > Reston, VA  20190
> >> >> > [hidden email]
> >> >> > http://www.pacificspatial.com
> >> >> > http://www.geopacific.org (GIS info site)
> >> >> > [hidden email] (private)
> >> >
> >> > --
> >> > Spatialys - Geospatial professional services
> >> > http://www.spatialys.com
> >> > _______________________________________________
> >> > OSGeoJapan-discuss mailing list
> >> > [hidden email]
> >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > _______________________________________________
> > OSGeoJapan-discuss mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Hiroo Imaki
Hi Even, Akagi-san, and Ko-san 

Ko-san, I am sorry I missed your message you set to the list last month. I was on vacation and did not check email for a while.

I stepped in this conversation last May and has been talking to those people who worked on GDAL and OGR drivers for the Japanese data. But I realized that I am not that useful on this topic (sorry!). 

Here is what I found so far. For the old drivers (JPGIS driver for vector data and DEM raster data driver), owners of those agreed on releasing them with the MIT license. For the JPGIS (GML) vector driver, there is one more step to release it but it hasn't reached the point to release it. 

Anyway,I totally agree on Akagi-san's comment and people who worked on this should make efforts for the support of recent format rather than old formats that are no longer distributed from GSI.

I hope we can release what we already have soon too.

Akagi-san and Ko-san, great job!!

Hiroo

On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault <[hidden email]> wrote:
Hi Minoru,
>
> We had an online hackathon about this topic on the same day as QGIS
> HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
> vector data (v4) with the OGR GML driver.
>
> I prepared .gfs schema files, then registered them to the GML registry
> and made a few changes to the GML driver code. Finally, I managed to
> read FGD (GML) vector data with the driver! See the work branch [1]
> and the hackpad [2][3] for the details.

This is cool you managed to use the GML registry mechanism.

I think however that the changes done in ogrgmldatasource.cpp in this branch
should be amended to restore the previous behaviour when the namespace prefix
is defined in the registry, and the behaviour of the branch when it is not
defined, so that full control is still available.

I'd add to the todo list to add a few words to drv_gml.html to mention that
FGD GML v4 is supported in the "Particular GML application schemas" section.

>
> I think we should make efforts for the support of recent format rather
> than old formats that are no longer distributed from GSI.

That makes sense. Sometimes supporting old formats can be of interest when
people are interested in historic data that isn't available to more modern
formats, but this is admitedly a particular use case.

Best regards,

Even

>
> Any comments?
>
> [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
>     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
> [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In Japanese)
> [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
> summary)
>
> Minoru
>
> 2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:
> > Ko-san,
> >
> > Thanks a lot for bringing some light on this.
> >
> > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
> > findings up to now and trace tasks that I've in mind that should be
> > completed for integretation into upstream GDAL.
> >
> > Best regards,
> >
> > Even
> >
> >> Hi Even,
> >> CC: Hiroo-san, Yamate-san, list
> >>
> >> Excuse me for cutting in this topic.
> >>
> >> I am neither the developer nor the player, but I tried mext_gdal
> >> test version (gdal110_test.zip) at last weekend and probably I understood
> >> some data related part, so I mention some of your question about datas.
> >> (But, if someone find my mistake, please point it out here...)
> >>
> >> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
> >> > I tried the FGD ogr driver on the file
> >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/d
> >> > at
> >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
> >> > (found by googling FG-GML)
> >> >
> >> > It is not recognized by the driver. Looking more closely, the driver
> >> > seems to expect a <GI> element that isn't found in that file. It
> >> > appears
> >> > that there are several versions of FGD GML, right ? The above file is
> >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
> >> > driver
> >> > is v4 (or a previous version) ? I tried to remove the test for the
> >> > presence of <GI> but it crashes later during parsing of dates,
> >> > apparently not liking the <gml:timePosition> element within <lfSpanFr>
> >>
> >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
> >> and 2 (or more) data types (5/10m mesh DEM / General vector data .etc)
> >> in FGD data,
> >> and current mext_gdal test version seems to support the following pair.
> >>
> >> =====
> >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
> >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
> >> 3. FGD driver: DLD v3 (General vector data)
> >> =====
> >>
> >> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
> >> so you couldn't convert it.
> >>
> >> > I found this project :
> >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
> >> > It includes the schema for FG-GML v4 and I can't see any reference to
> >> > the
> >> > <GI> element in it either.
> >> > With some trickery related to encoding conversion, I managed to use
> >> > that
> >> > project to produce a standard GML file that the OGR GML driver handles
> >> > well :
> >> >
> >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
> >> > --from-code=shift-jis
> >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python jpgisgml2gml.py >
> >> > out.gml
> >>
> >> And above GML v4 is not supported (both of 5/10m mesh DEM and
> >> General vector data) by current mext_gdal for now.
> >>
> >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and downloaded
> >> > a
> >> > few datasets. Those I tried are not handled by the OGR FGD driver. Some
> >> > of them look like GML with a particular application schema. They are
> >> > unfortunately not standard enough for the OGR GML driver to understand
> >> > them out of the box, although I believe it could be tweaked to parse
> >> > them. Their header look like:
> >> >
> >> > <ksj:Dataset gml:id="A09Dataset"
> >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> >> > xmlns:gml="http://www.opengis.net/gml/3.2"
> >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
> >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
> >> > KsjAppSchema- A09-v3_1.xsd">
> >> >
> >> > There's a shapefile next to them, with apparently same content.
> >>
> >> And about above MLIT NLNI(National Land Numerical Information) data,
> >> it is not FGD data...
> >>
> >> > So it appears there's a variety of GML-like formats ;-)
> >>
> >> Yes, sure...
> >>
> >> If you want to use mext_gdal test data, I just uploaded example dummy
> >> data
> >> to the following repository. So, please try to use those files.
> >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
> >> (The dummy data location is near by Japan Wakayama prefecture,
> >> but on the pacific ocean.)
> >>
> >> Regards,
> >>
> >> >> Hi Even,
> >> >>
> >> >> I talk to Yamate-san who has been working on those three drivers. He
> >> >> explained that each driver has little bit different copy right
> >> >> background he need to resolve before he can merge them to the main.
> >> >>
> >> >> The first one, FGD driver, which he wrote for a project and he does
> >> >> not
> >> >> have control on its copy right. So, he would like to leave it at this
> >> >> moment.
> >> >>
> >> >> For the second one, FG-GML driver, he wrote it and he owns its copy
> >> >> right at this moment. But he said he can release with the licence you
> >> >> suggested.
> >> >>
> >> >> The last one, FGD ogr driver, which he also wrote and it is highly
> >> >> possible he can release it under the MIT licence. But he need to make
> >> >> sure his client can agree on this. So, he said he will contact to his
> >> >> client.
> >> >>
> >> >> So, the last two drivers can be merge into the main GDAL driver. It is
> >> >> little bit confusing background but I am sure we can sort these out.
> >> >>
> >> >> I will let you know when he is ready to work with you.
> >> >>
> >> >> Thanks!
> >> >>
> >> >> Hiroo
> >> >>
> >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]> wrote:
> >> >> > Hi Even,
> >> >> >
> >> >> > I am not the developer but he and I working at a same company. If he
> >> >> > can agree on the license and release his efforts, that would be
> >> >> > wonderful for all Japanese data uses.
> >> >> >
> >> >> > He hesitates writing his message in English. So, I can be a
> >> >> > translator
> >> >> > between you guys. I hope no lost in translation!
> >> >> >
> >> >> > I will get back to you once I hear from him.
> >> >> >
> >> >> > Hiroo
> >> >> >
> >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> >> >> > <[hidden email]
> >> >> >
> >> >> > > wrote:
> >> >> >> Hi,
> >> >> >>
> >> >> >> Sorry for writing in English, but I don't unfortunately speak
> >> >> >> Japanese.
> >> >> >>
> >> >> >> I'm one of the GDAL developers.
> >> >> >>
> >> >> >> I've noticed that there are "un-official" versions of GDAL floating
> >> >> >> around, with
> >> >> >> a few drivers for new Japanese-specific formats. I was wondering if
> >> >> >> there was a
> >> >> >> reason to keep them outside of the official GDAL tree ? Integrating
> >> >> >> them into
> >> >> >> mainline GDAL would probably make life of Japanese users easier.
> >> >> >> GDAL
> >> >> >> has already support for a lof of national specific formats, mainly
> >> >> >> in
> >> >> >> Europe and
> >> >> >> North America. Would be cool for Asia to join in ;-)
> >> >> >>
> >> >> >> In a recent discussion thread, the following link is mentionned:
> >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >> >> >>
> >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese Fundamental
> >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> >> >> Unfortunatelly there's no author or copyright information in the
> >> >> >> header of the
> >> >> >> corresponding source files. Would their author(s) be willing to
> >> >> >> release them
> >> >> >> under the X/MIT license and contribute them to the mainline GDAL
> >> >> >> project ? Perhaps some test files to write minimalistic regressions
> >> >> >> tests would be useful.
> >> >> >>
> >> >> >> Let me know if I can be useful for anything. I can also help for
> >> >> >> writing new
> >> >> >> drivers if needed.
> >> >> >>
> >> >> >> Best regards,
> >> >> >>
> >> >> >> Even
> >> >> >>
> >> >> >> --
> >> >> >> Spatialys - Geospatial professional services
> >> >> >> http://www.spatialys.com
> >> >> >> _______________________________________________
> >> >> >> OSGeoJapan-discuss mailing list
> >> >> >> [hidden email]
> >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> >> >
> >> >> > --
> >> >> > Hiroo Imaki
> >> >> > Pacific Spatial Solutions, LLC
> >> >> > 1523 Chatham Colony Ct.
> >> >> > Reston, VA  20190
> >> >> > [hidden email]
> >> >> > http://www.pacificspatial.com
> >> >> > http://www.geopacific.org (GIS info site)
> >> >> > [hidden email] (private)
> >> >
> >> > --
> >> > Spatialys - Geospatial professional services
> >> > http://www.spatialys.com
> >> > _______________________________________________
> >> > OSGeoJapan-discuss mailing list
> >> > [hidden email]
> >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > _______________________________________________
> > OSGeoJapan-discuss mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss



--
Hiroo Imaki
Pacific Spatial Solutions, LLC
1523 Chatham Colony Ct.
Reston, VA  20190
http://www.geopacific.org (GIS info site)
[hidden email] (private)


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
Hi Hiroo-san,
CC: Akagi-san, Even, list

Thanks for your comments.

2015-08-19 9:58 GMT+09:00 Hiroo Imaki <[hidden email]>:
> Hi Even, Akagi-san, and Ko-san
>
> Ko-san, I am sorry I missed your message you set to the list last month. I
> was on vacation and did not check email for a while.
>
> I stepped in this conversation last May and has been talking to those people
> who worked on GDAL and OGR drivers for the Japanese data. But I realized
> that I am not that useful on this topic (sorry!).

Okay, no problem.

> Here is what I found so far. For the old drivers (JPGIS driver for vector
> data and DEM raster data driver), owners of those agreed on releasing them
> with the MIT license. For the JPGIS (GML) vector driver, there is one more
> step to release it but it hasn't reached the point to release it.

Okay, thanks for information.
I guess that existence(under procedure) JPGIS (GML) v4 vector driver
implementation
is different from Akagi-san's GML driver improvement, and not conflict
with each other.
So, I think that both drivers will be available in near future.

> Anyway,I totally agree on Akagi-san's comment and people who worked on this
> should make efforts for the support of recent format rather than old formats
> that are no longer distributed from GSI.
>
> I hope we can release what we already have soon too.

Okay.
At QGIS Hackfest Tokyo/Hokkaido/Osaka (2015-08-01),
Kayama-san renamed Github "qgis-jp" group name to "OSGeo-jp".
So, we will be able to work Japanese GDAL adjustment/improvement together
after forking "OSGeo/gdal" to "OSGeo-jp/gdal".
(I am thinking to prepare JPGIS (GML) v4 dummy data, then put those to
the forked repository.)

> Akagi-san and Ko-san, great job!!

Thank you, too!

Regards,

Ko

> On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault <[hidden email]>
> wrote:
>>
>> Hi Minoru,
>> >
>> > We had an online hackathon about this topic on the same day as QGIS
>> > HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
>> > vector data (v4) with the OGR GML driver.
>> >
>> > I prepared .gfs schema files, then registered them to the GML registry
>> > and made a few changes to the GML driver code. Finally, I managed to
>> > read FGD (GML) vector data with the driver! See the work branch [1]
>> > and the hackpad [2][3] for the details.
>>
>> This is cool you managed to use the GML registry mechanism.
>>
>> I think however that the changes done in ogrgmldatasource.cpp in this
>> branch
>> should be amended to restore the previous behaviour when the namespace
>> prefix
>> is defined in the registry, and the behaviour of the branch when it is not
>> defined, so that full control is still available.
>>
>> I'd add to the todo list to add a few words to drv_gml.html to mention
>> that
>> FGD GML v4 is supported in the "Particular GML application schemas"
>> section.
>>
>> >
>> > I think we should make efforts for the support of recent format rather
>> > than old formats that are no longer distributed from GSI.
>>
>> That makes sense. Sometimes supporting old formats can be of interest when
>> people are interested in historic data that isn't available to more modern
>> formats, but this is admitedly a particular use case.
>>
>> Best regards,
>>
>> Even
>>
>> >
>> > Any comments?
>> >
>> > [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
>> >     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
>> > [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In
>> > Japanese)
>> > [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
>> > summary)
>> >
>> > Minoru
>> >
>> > 2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:
>> > > Ko-san,
>> > >
>> > > Thanks a lot for bringing some light on this.
>> > >
>> > > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
>> > > findings up to now and trace tasks that I've in mind that should be
>> > > completed for integretation into upstream GDAL.
>> > >
>> > > Best regards,
>> > >
>> > > Even
>> > >
>> > >> Hi Even,
>> > >> CC: Hiroo-san, Yamate-san, list
>> > >>
>> > >> Excuse me for cutting in this topic.
>> > >>
>> > >> I am neither the developer nor the player, but I tried mext_gdal
>> > >> test version (gdal110_test.zip) at last weekend and probably I
>> > >> understood
>> > >> some data related part, so I mention some of your question about
>> > >> datas.
>> > >> (But, if someone find my mistake, please point it out here...)
>> > >>
>> > >> 2015-05-01 18:59 GMT+09:00 Even Rouault <[hidden email]>:
>> > >> > I tried the FGD ogr driver on the file
>> > >> >
>> > >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/d
>> > >> > at
>> > >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
>> > >> > (found by googling FG-GML)
>> > >> >
>> > >> > It is not recognized by the driver. Looking more closely, the
>> > >> > driver
>> > >> > seems to expect a <GI> element that isn't found in that file. It
>> > >> > appears
>> > >> > that there are several versions of FGD GML, right ? The above file
>> > >> > is
>> > >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
>> > >> > driver
>> > >> > is v4 (or a previous version) ? I tried to remove the test for the
>> > >> > presence of <GI> but it crashes later during parsing of dates,
>> > >> > apparently not liking the <gml:timePosition> element within
>> > >> > <lfSpanFr>
>> > >>
>> > >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
>> > >> and 2 (or more) data types (5/10m mesh DEM / General vector data
>> > >> .etc)
>> > >> in FGD data,
>> > >> and current mext_gdal test version seems to support the following
>> > >> pair.
>> > >>
>> > >> =====
>> > >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
>> > >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
>> > >> 3. FGD driver: DLD v3 (General vector data)
>> > >> =====
>> > >>
>> > >> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
>> > >> so you couldn't convert it.
>> > >>
>> > >> > I found this project :
>> > >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
>> > >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
>> > >> > It includes the schema for FG-GML v4 and I can't see any reference
>> > >> > to
>> > >> > the
>> > >> > <GI> element in it either.
>> > >> > With some trickery related to encoding conversion, I managed to use
>> > >> > that
>> > >> > project to produce a standard GML file that the OGR GML driver
>> > >> > handles
>> > >> > well :
>> > >> >
>> > >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
>> > >> > --from-code=shift-jis
>> > >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python
>> > >> > jpgisgml2gml.py >
>> > >> > out.gml
>> > >>
>> > >> And above GML v4 is not supported (both of 5/10m mesh DEM and
>> > >> General vector data) by current mext_gdal for now.
>> > >>
>> > >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and
>> > >> > downloaded
>> > >> > a
>> > >> > few datasets. Those I tried are not handled by the OGR FGD driver.
>> > >> > Some
>> > >> > of them look like GML with a particular application schema. They
>> > >> > are
>> > >> > unfortunately not standard enough for the OGR GML driver to
>> > >> > understand
>> > >> > them out of the box, although I believe it could be tweaked to
>> > >> > parse
>> > >> > them. Their header look like:
>> > >> >
>> > >> > <ksj:Dataset gml:id="A09Dataset"
>> > >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
>> > >> > xmlns:gml="http://www.opengis.net/gml/3.2"
>> > >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
>> > >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> > >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
>> > >> > KsjAppSchema- A09-v3_1.xsd">
>> > >> >
>> > >> > There's a shapefile next to them, with apparently same content.
>> > >>
>> > >> And about above MLIT NLNI(National Land Numerical Information) data,
>> > >> it is not FGD data...
>> > >>
>> > >> > So it appears there's a variety of GML-like formats ;-)
>> > >>
>> > >> Yes, sure...
>> > >>
>> > >> If you want to use mext_gdal test data, I just uploaded example dummy
>> > >> data
>> > >> to the following repository. So, please try to use those files.
>> > >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
>> > >> (The dummy data location is near by Japan Wakayama prefecture,
>> > >> but on the pacific ocean.)
>> > >>
>> > >> Regards,
>> > >>
>> > >> >> Hi Even,
>> > >> >>
>> > >> >> I talk to Yamate-san who has been working on those three drivers.
>> > >> >> He
>> > >> >> explained that each driver has little bit different copy right
>> > >> >> background he need to resolve before he can merge them to the
>> > >> >> main.
>> > >> >>
>> > >> >> The first one, FGD driver, which he wrote for a project and he
>> > >> >> does
>> > >> >> not
>> > >> >> have control on its copy right. So, he would like to leave it at
>> > >> >> this
>> > >> >> moment.
>> > >> >>
>> > >> >> For the second one, FG-GML driver, he wrote it and he owns its
>> > >> >> copy
>> > >> >> right at this moment. But he said he can release with the licence
>> > >> >> you
>> > >> >> suggested.
>> > >> >>
>> > >> >> The last one, FGD ogr driver, which he also wrote and it is highly
>> > >> >> possible he can release it under the MIT licence. But he need to
>> > >> >> make
>> > >> >> sure his client can agree on this. So, he said he will contact to
>> > >> >> his
>> > >> >> client.
>> > >> >>
>> > >> >> So, the last two drivers can be merge into the main GDAL driver.
>> > >> >> It is
>> > >> >> little bit confusing background but I am sure we can sort these
>> > >> >> out.
>> > >> >>
>> > >> >> I will let you know when he is ready to work with you.
>> > >> >>
>> > >> >> Thanks!
>> > >> >>
>> > >> >> Hiroo
>> > >> >>
>> > >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]>
>> > >> >> wrote:
>> > >> >> > Hi Even,
>> > >> >> >
>> > >> >> > I am not the developer but he and I working at a same company.
>> > >> >> > If he
>> > >> >> > can agree on the license and release his efforts, that would be
>> > >> >> > wonderful for all Japanese data uses.
>> > >> >> >
>> > >> >> > He hesitates writing his message in English. So, I can be a
>> > >> >> > translator
>> > >> >> > between you guys. I hope no lost in translation!
>> > >> >> >
>> > >> >> > I will get back to you once I hear from him.
>> > >> >> >
>> > >> >> > Hiroo
>> > >> >> >
>> > >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> > >> >> > <[hidden email]
>> > >> >> >
>> > >> >> > > wrote:
>> > >> >> >> Hi,
>> > >> >> >>
>> > >> >> >> Sorry for writing in English, but I don't unfortunately speak
>> > >> >> >> Japanese.
>> > >> >> >>
>> > >> >> >> I'm one of the GDAL developers.
>> > >> >> >>
>> > >> >> >> I've noticed that there are "un-official" versions of GDAL
>> > >> >> >> floating
>> > >> >> >> around, with
>> > >> >> >> a few drivers for new Japanese-specific formats. I was
>> > >> >> >> wondering if
>> > >> >> >> there was a
>> > >> >> >> reason to keep them outside of the official GDAL tree ?
>> > >> >> >> Integrating
>> > >> >> >> them into
>> > >> >> >> mainline GDAL would probably make life of Japanese users
>> > >> >> >> easier.
>> > >> >> >> GDAL
>> > >> >> >> has already support for a lof of national specific formats,
>> > >> >> >> mainly
>> > >> >> >> in
>> > >> >> >> Europe and
>> > >> >> >> North America. Would be cool for Asia to join in ;-)
>> > >> >> >>
>> > >> >> >> In a recent discussion thread, the following link is
>> > >> >> >> mentionned:
>> > >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> > >> >> >>
>> > >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> > >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese
>> > >> >> >> Fundamental
>> > >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> > >> >> >> Unfortunatelly there's no author or copyright information in
>> > >> >> >> the
>> > >> >> >> header of the
>> > >> >> >> corresponding source files. Would their author(s) be willing to
>> > >> >> >> release them
>> > >> >> >> under the X/MIT license and contribute them to the mainline
>> > >> >> >> GDAL
>> > >> >> >> project ? Perhaps some test files to write minimalistic
>> > >> >> >> regressions
>> > >> >> >> tests would be useful.
>> > >> >> >>
>> > >> >> >> Let me know if I can be useful for anything. I can also help
>> > >> >> >> for
>> > >> >> >> writing new
>> > >> >> >> drivers if needed.
>> > >> >> >>
>> > >> >> >> Best regards,
>> > >> >> >>
>> > >> >> >> Even
>> > >> >> >>
>> > >> >> >> --
>> > >> >> >> Spatialys - Geospatial professional services
>> > >> >> >> http://www.spatialys.com
>> > >> >> >> _______________________________________________
>> > >> >> >> OSGeoJapan-discuss mailing list
>> > >> >> >> [hidden email]
>> > >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> > >> >> >
>> > >> >> > --
>> > >> >> > Hiroo Imaki
>> > >> >> > Pacific Spatial Solutions, LLC
>> > >> >> > 1523 Chatham Colony Ct.
>> > >> >> > Reston, VA  20190
>> > >> >> > [hidden email]
>> > >> >> > http://www.pacificspatial.com
>> > >> >> > http://www.geopacific.org (GIS info site)
>> > >> >> > [hidden email] (private)
>> > >> >
>> > >> > --
>> > >> > Spatialys - Geospatial professional services
>> > >> > http://www.spatialys.com
>> > >> > _______________________________________________
>> > >> > OSGeoJapan-discuss mailing list
>> > >> > [hidden email]
>> > >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> > >
>> > > --
>> > > Spatialys - Geospatial professional services
>> > > http://www.spatialys.com
>> > > _______________________________________________
>> > > OSGeoJapan-discuss mailing list
>> > > [hidden email]
>> > > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>> _______________________________________________
>> OSGeoJapan-discuss mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>
>
>
>
> --
> Hiroo Imaki
> Pacific Spatial Solutions, LLC
> 1523 Chatham Colony Ct.
> Reston, VA  20190
> [hidden email]
> http://www.pacificspatial.com
> http://www.geopacific.org (GIS info site)
> [hidden email] (private)
>
>
> _______________________________________________
> OSGeoJapan-discuss mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
Hi all,

I was wondering if there had been progress on this ? I can see a
https://github.com/OSGeo-jp/gdal repository, but it doesn't seem to contain
changes for now.

Best regards,

Even

> Hi Hiroo-san,
> CC: Akagi-san, Even, list
>
> Thanks for your comments.
>
> 2015-08-19 9:58 GMT+09:00 Hiroo Imaki <[hidden email]>:
> > Hi Even, Akagi-san, and Ko-san
> >
> > Ko-san, I am sorry I missed your message you set to the list last month.
> > I was on vacation and did not check email for a while.
> >
> > I stepped in this conversation last May and has been talking to those
> > people who worked on GDAL and OGR drivers for the Japanese data. But I
> > realized that I am not that useful on this topic (sorry!).
>
> Okay, no problem.
>
> > Here is what I found so far. For the old drivers (JPGIS driver for vector
> > data and DEM raster data driver), owners of those agreed on releasing
> > them with the MIT license. For the JPGIS (GML) vector driver, there is
> > one more step to release it but it hasn't reached the point to release
> > it.
>
> Okay, thanks for information.
> I guess that existence(under procedure) JPGIS (GML) v4 vector driver
> implementation
> is different from Akagi-san's GML driver improvement, and not conflict
> with each other.
> So, I think that both drivers will be available in near future.
>
> > Anyway,I totally agree on Akagi-san's comment and people who worked on
> > this should make efforts for the support of recent format rather than
> > old formats that are no longer distributed from GSI.
> >
> > I hope we can release what we already have soon too.
>
> Okay.
> At QGIS Hackfest Tokyo/Hokkaido/Osaka (2015-08-01),
> Kayama-san renamed Github "qgis-jp" group name to "OSGeo-jp".
> So, we will be able to work Japanese GDAL adjustment/improvement together
> after forking "OSGeo/gdal" to "OSGeo-jp/gdal".
> (I am thinking to prepare JPGIS (GML) v4 dummy data, then put those to
> the forked repository.)
>
> > Akagi-san and Ko-san, great job!!
>
> Thank you, too!
>
> Regards,
>
> Ko
>
> > On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault
> > <[hidden email]>
> >
> > wrote:
> >> Hi Minoru,
> >>
> >> > We had an online hackathon about this topic on the same day as QGIS
> >> > HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
> >> > vector data (v4) with the OGR GML driver.
> >> >
> >> > I prepared .gfs schema files, then registered them to the GML registry
> >> > and made a few changes to the GML driver code. Finally, I managed to
> >> > read FGD (GML) vector data with the driver! See the work branch [1]
> >> > and the hackpad [2][3] for the details.
> >>
> >> This is cool you managed to use the GML registry mechanism.
> >>
> >> I think however that the changes done in ogrgmldatasource.cpp in this
> >> branch
> >> should be amended to restore the previous behaviour when the namespace
> >> prefix
> >> is defined in the registry, and the behaviour of the branch when it is
> >> not defined, so that full control is still available.
> >>
> >> I'd add to the todo list to add a few words to drv_gml.html to mention
> >> that
> >> FGD GML v4 is supported in the "Particular GML application schemas"
> >> section.
> >>
> >> > I think we should make efforts for the support of recent format rather
> >> > than old formats that are no longer distributed from GSI.
> >>
> >> That makes sense. Sometimes supporting old formats can be of interest
> >> when people are interested in historic data that isn't available to
> >> more modern formats, but this is admitedly a particular use case.
> >>
> >> Best regards,
> >>
> >> Even
> >>
> >> > Any comments?
> >> >
> >> > [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
> >> >
> >> >     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
> >> >
> >> > [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In
> >> > Japanese)
> >> > [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
> >> > summary)
> >> >
> >> > Minoru
> >> >
> >> > 2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:
> >> > > Ko-san,
> >> > >
> >> > > Thanks a lot for bringing some light on this.
> >> > >
> >> > > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
> >> > > findings up to now and trace tasks that I've in mind that should be
> >> > > completed for integretation into upstream GDAL.
> >> > >
> >> > > Best regards,
> >> > >
> >> > > Even
> >> > >
> >> > >> Hi Even,
> >> > >> CC: Hiroo-san, Yamate-san, list
> >> > >>
> >> > >> Excuse me for cutting in this topic.
> >> > >>
> >> > >> I am neither the developer nor the player, but I tried mext_gdal
> >> > >> test version (gdal110_test.zip) at last weekend and probably I
> >> > >> understood
> >> > >> some data related part, so I mention some of your question about
> >> > >> datas.
> >> > >> (But, if someone find my mistake, please point it out here...)
> >> > >>
> >> > >> 2015-05-01 18:59 GMT+09:00 Even Rouault
<[hidden email]>:

> >> > >> > I tried the FGD ogr driver on the file
> >> > >> >
> >> > >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicod
> >> > >> > e_jp/d at
> >> > >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
> >> > >> > (found by googling FG-GML)
> >> > >> >
> >> > >> > It is not recognized by the driver. Looking more closely, the
> >> > >> > driver
> >> > >> > seems to expect a <GI> element that isn't found in that file. It
> >> > >> > appears
> >> > >> > that there are several versions of FGD GML, right ? The above
> >> > >> > file is
> >> > >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
> >> > >> > driver
> >> > >> > is v4 (or a previous version) ? I tried to remove the test for
> >> > >> > the presence of <GI> but it crashes later during parsing of
> >> > >> > dates, apparently not liking the <gml:timePosition> element
> >> > >> > within <lfSpanFr>
> >> > >>
> >> > >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
> >> > >> and 2 (or more) data types (5/10m mesh DEM / General vector data
> >> > >> .etc)
> >> > >> in FGD data,
> >> > >> and current mext_gdal test version seems to support the following
> >> > >> pair.
> >> > >>
> >> > >> =====
> >> > >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
> >> > >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
> >> > >> 3. FGD driver: DLD v3 (General vector data)
> >> > >> =====
> >> > >>
> >> > >> Above link which you found use GML v3, but it is not 5/10m mesh
> >> > >> DEM, so you couldn't convert it.
> >> > >>
> >> > >> > I found this project :
> >> > >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> >> > >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
> >> > >> > It includes the schema for FG-GML v4 and I can't see any
> >> > >> > reference to
> >> > >> > the
> >> > >> > <GI> element in it either.
> >> > >> > With some trickery related to encoding conversion, I managed to
> >> > >> > use that
> >> > >> > project to produce a standard GML file that the OGR GML driver
> >> > >> > handles
> >> > >> > well :
> >> > >> >
> >> > >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
> >> > >> > --from-code=shift-jis
> >> > >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python
> >> > >> > jpgisgml2gml.py >
> >> > >> > out.gml
> >> > >>
> >> > >> And above GML v4 is not supported (both of 5/10m mesh DEM and
> >> > >> General vector data) by current mext_gdal for now.
> >> > >>
> >> > >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and
> >> > >> > downloaded
> >> > >> > a
> >> > >> > few datasets. Those I tried are not handled by the OGR FGD
> >> > >> > driver. Some
> >> > >> > of them look like GML with a particular application schema. They
> >> > >> > are
> >> > >> > unfortunately not standard enough for the OGR GML driver to
> >> > >> > understand
> >> > >> > them out of the box, although I believe it could be tweaked to
> >> > >> > parse
> >> > >> > them. Their header look like:
> >> > >> >
> >> > >> > <ksj:Dataset gml:id="A09Dataset"
> >> > >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> >> > >> > xmlns:gml="http://www.opengis.net/gml/3.2"
> >> > >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
> >> > >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >> > >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
> >> > >> > KsjAppSchema- A09-v3_1.xsd">
> >> > >> >
> >> > >> > There's a shapefile next to them, with apparently same content.
> >> > >>
> >> > >> And about above MLIT NLNI(National Land Numerical Information)
> >> > >> data, it is not FGD data...
> >> > >>
> >> > >> > So it appears there's a variety of GML-like formats ;-)
> >> > >>
> >> > >> Yes, sure...
> >> > >>
> >> > >> If you want to use mext_gdal test data, I just uploaded example
> >> > >> dummy data
> >> > >> to the following repository. So, please try to use those files.
> >> > >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
> >> > >> (The dummy data location is near by Japan Wakayama prefecture,
> >> > >> but on the pacific ocean.)
> >> > >>
> >> > >> Regards,
> >> > >>
> >> > >> >> Hi Even,
> >> > >> >>
> >> > >> >> I talk to Yamate-san who has been working on those three
> >> > >> >> drivers. He
> >> > >> >> explained that each driver has little bit different copy right
> >> > >> >> background he need to resolve before he can merge them to the
> >> > >> >> main.
> >> > >> >>
> >> > >> >> The first one, FGD driver, which he wrote for a project and he
> >> > >> >> does
> >> > >> >> not
> >> > >> >> have control on its copy right. So, he would like to leave it at
> >> > >> >> this
> >> > >> >> moment.
> >> > >> >>
> >> > >> >> For the second one, FG-GML driver, he wrote it and he owns its
> >> > >> >> copy
> >> > >> >> right at this moment. But he said he can release with the
> >> > >> >> licence you
> >> > >> >> suggested.
> >> > >> >>
> >> > >> >> The last one, FGD ogr driver, which he also wrote and it is
> >> > >> >> highly possible he can release it under the MIT licence. But he
> >> > >> >> need to make
> >> > >> >> sure his client can agree on this. So, he said he will contact
> >> > >> >> to his
> >> > >> >> client.
> >> > >> >>
> >> > >> >> So, the last two drivers can be merge into the main GDAL driver.
> >> > >> >> It is
> >> > >> >> little bit confusing background but I am sure we can sort these
> >> > >> >> out.
> >> > >> >>
> >> > >> >> I will let you know when he is ready to work with you.
> >> > >> >>
> >> > >> >> Thanks!
> >> > >> >>
> >> > >> >> Hiroo
> >> > >> >>
> >> > >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]>
> >> > >> >>
> >> > >> >> wrote:
> >> > >> >> > Hi Even,
> >> > >> >> >
> >> > >> >> > I am not the developer but he and I working at a same company.
> >> > >> >> > If he
> >> > >> >> > can agree on the license and release his efforts, that would
> >> > >> >> > be wonderful for all Japanese data uses.
> >> > >> >> >
> >> > >> >> > He hesitates writing his message in English. So, I can be a
> >> > >> >> > translator
> >> > >> >> > between you guys. I hope no lost in translation!
> >> > >> >> >
> >> > >> >> > I will get back to you once I hear from him.
> >> > >> >> >
> >> > >> >> > Hiroo
> >> > >> >> >
> >> > >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> >> > >> >> > <[hidden email]
> >> > >> >> >
> >> > >> >> > > wrote:
> >> > >> >> >> Hi,
> >> > >> >> >>
> >> > >> >> >> Sorry for writing in English, but I don't unfortunately speak
> >> > >> >> >> Japanese.
> >> > >> >> >>
> >> > >> >> >> I'm one of the GDAL developers.
> >> > >> >> >>
> >> > >> >> >> I've noticed that there are "un-official" versions of GDAL
> >> > >> >> >> floating
> >> > >> >> >> around, with
> >> > >> >> >> a few drivers for new Japanese-specific formats. I was
> >> > >> >> >> wondering if
> >> > >> >> >> there was a
> >> > >> >> >> reason to keep them outside of the official GDAL tree ?
> >> > >> >> >> Integrating
> >> > >> >> >> them into
> >> > >> >> >> mainline GDAL would probably make life of Japanese users
> >> > >> >> >> easier.
> >> > >> >> >> GDAL
> >> > >> >> >> has already support for a lof of national specific formats,
> >> > >> >> >> mainly
> >> > >> >> >> in
> >> > >> >> >> Europe and
> >> > >> >> >> North America. Would be cool for Asia to join in ;-)
> >> > >> >> >>
> >> > >> >> >> In a recent discussion thread, the following link is
> >> > >> >> >>
> >> > >> >> >> mentionned:
> >> > >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >> > >> >> >>
> >> > >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> > >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese
> >> > >> >> >> Fundamental
> >> > >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> > >> >> >> Unfortunatelly there's no author or copyright information in
> >> > >> >> >> the
> >> > >> >> >> header of the
> >> > >> >> >> corresponding source files. Would their author(s) be willing
> >> > >> >> >> to release them
> >> > >> >> >> under the X/MIT license and contribute them to the mainline
> >> > >> >> >> GDAL
> >> > >> >> >> project ? Perhaps some test files to write minimalistic
> >> > >> >> >> regressions
> >> > >> >> >> tests would be useful.
> >> > >> >> >>
> >> > >> >> >> Let me know if I can be useful for anything. I can also help
> >> > >> >> >> for
> >> > >> >> >> writing new
> >> > >> >> >> drivers if needed.
> >> > >> >> >>
> >> > >> >> >> Best regards,
> >> > >> >> >>
> >> > >> >> >> Even
> >> > >> >> >>
> >> > >> >> >> --
> >> > >> >> >> Spatialys - Geospatial professional services
> >> > >> >> >> http://www.spatialys.com
> >> > >> >> >> _______________________________________________
> >> > >> >> >> OSGeoJapan-discuss mailing list
> >> > >> >> >> [hidden email]
> >> > >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> > >> >> >
> >> > >> >> > --
> >> > >> >> > Hiroo Imaki
> >> > >> >> > Pacific Spatial Solutions, LLC
> >> > >> >> > 1523 Chatham Colony Ct.
> >> > >> >> > Reston, VA  20190
> >> > >> >> > [hidden email]
> >> > >> >> > http://www.pacificspatial.com
> >> > >> >> > http://www.geopacific.org (GIS info site)
> >> > >> >> > [hidden email] (private)
> >> > >> >
> >> > >> > --
> >> > >> > Spatialys - Geospatial professional services
> >> > >> > http://www.spatialys.com
> >> > >> > _______________________________________________
> >> > >> > OSGeoJapan-discuss mailing list
> >> > >> > [hidden email]
> >> > >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> > >
> >> > > --
> >> > > Spatialys - Geospatial professional services
> >> > > http://www.spatialys.com
> >> > > _______________________________________________
> >> > > OSGeoJapan-discuss mailing list
> >> > > [hidden email]
> >> > > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >>
> >> --
> >> Spatialys - Geospatial professional services
> >> http://www.spatialys.com
> >> _______________________________________________
> >> OSGeoJapan-discuss mailing list
> >> [hidden email]
> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >
> > --
> > Hiroo Imaki
> > Pacific Spatial Solutions, LLC
> > 1523 Chatham Colony Ct.
> > Reston, VA  20190
> > [hidden email]
> > http://www.pacificspatial.com
> > http://www.geopacific.org (GIS info site)
> > [hidden email] (private)
> >
> >
> > _______________________________________________
> > OSGeoJapan-discuss mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
Hi Even,

Sorry for our low activity.

We are still discussing about this topic,
and it may take more time...
(Hopefully, we will take some action in this autumn...)

So, could you wait for a while ?

Regards,

Ko

2015-10-08 18:03 GMT+09:00 Even Rouault <[hidden email]>:

> Hi all,
>
> I was wondering if there had been progress on this ? I can see a
> https://github.com/OSGeo-jp/gdal repository, but it doesn't seem to contain
> changes for now.
>
> Best regards,
>
> Even
>
>> Hi Hiroo-san,
>> CC: Akagi-san, Even, list
>>
>> Thanks for your comments.
>>
>> 2015-08-19 9:58 GMT+09:00 Hiroo Imaki <[hidden email]>:
>> > Hi Even, Akagi-san, and Ko-san
>> >
>> > Ko-san, I am sorry I missed your message you set to the list last month.
>> > I was on vacation and did not check email for a while.
>> >
>> > I stepped in this conversation last May and has been talking to those
>> > people who worked on GDAL and OGR drivers for the Japanese data. But I
>> > realized that I am not that useful on this topic (sorry!).
>>
>> Okay, no problem.
>>
>> > Here is what I found so far. For the old drivers (JPGIS driver for vector
>> > data and DEM raster data driver), owners of those agreed on releasing
>> > them with the MIT license. For the JPGIS (GML) vector driver, there is
>> > one more step to release it but it hasn't reached the point to release
>> > it.
>>
>> Okay, thanks for information.
>> I guess that existence(under procedure) JPGIS (GML) v4 vector driver
>> implementation
>> is different from Akagi-san's GML driver improvement, and not conflict
>> with each other.
>> So, I think that both drivers will be available in near future.
>>
>> > Anyway,I totally agree on Akagi-san's comment and people who worked on
>> > this should make efforts for the support of recent format rather than
>> > old formats that are no longer distributed from GSI.
>> >
>> > I hope we can release what we already have soon too.
>>
>> Okay.
>> At QGIS Hackfest Tokyo/Hokkaido/Osaka (2015-08-01),
>> Kayama-san renamed Github "qgis-jp" group name to "OSGeo-jp".
>> So, we will be able to work Japanese GDAL adjustment/improvement together
>> after forking "OSGeo/gdal" to "OSGeo-jp/gdal".
>> (I am thinking to prepare JPGIS (GML) v4 dummy data, then put those to
>> the forked repository.)
>>
>> > Akagi-san and Ko-san, great job!!
>>
>> Thank you, too!
>>
>> Regards,
>>
>> Ko
>>
>> > On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault
>> > <[hidden email]>
>> >
>> > wrote:
>> >> Hi Minoru,
>> >>
>> >> > We had an online hackathon about this topic on the same day as QGIS
>> >> > HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
>> >> > vector data (v4) with the OGR GML driver.
>> >> >
>> >> > I prepared .gfs schema files, then registered them to the GML registry
>> >> > and made a few changes to the GML driver code. Finally, I managed to
>> >> > read FGD (GML) vector data with the driver! See the work branch [1]
>> >> > and the hackpad [2][3] for the details.
>> >>
>> >> This is cool you managed to use the GML registry mechanism.
>> >>
>> >> I think however that the changes done in ogrgmldatasource.cpp in this
>> >> branch
>> >> should be amended to restore the previous behaviour when the namespace
>> >> prefix
>> >> is defined in the registry, and the behaviour of the branch when it is
>> >> not defined, so that full control is still available.
>> >>
>> >> I'd add to the todo list to add a few words to drv_gml.html to mention
>> >> that
>> >> FGD GML v4 is supported in the "Particular GML application schemas"
>> >> section.
>> >>
>> >> > I think we should make efforts for the support of recent format rather
>> >> > than old formats that are no longer distributed from GSI.
>> >>
>> >> That makes sense. Sometimes supporting old formats can be of interest
>> >> when people are interested in historic data that isn't available to
>> >> more modern formats, but this is admitedly a particular use case.
>> >>
>> >> Best regards,
>> >>
>> >> Even
>> >>
>> >> > Any comments?
>> >> >
>> >> > [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
>> >> >
>> >> >     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
>> >> >
>> >> > [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In
>> >> > Japanese)
>> >> > [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
>> >> > summary)
>> >> >
>> >> > Minoru
>> >> >
>> >> > 2015-07-20 23:23 GMT+09:00 Even Rouault <[hidden email]>:
>> >> > > Ko-san,
>> >> > >
>> >> > > Thanks a lot for bringing some light on this.
>> >> > >
>> >> > > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
>> >> > > findings up to now and trace tasks that I've in mind that should be
>> >> > > completed for integretation into upstream GDAL.
>> >> > >
>> >> > > Best regards,
>> >> > >
>> >> > > Even
>> >> > >
>> >> > >> Hi Even,
>> >> > >> CC: Hiroo-san, Yamate-san, list
>> >> > >>
>> >> > >> Excuse me for cutting in this topic.
>> >> > >>
>> >> > >> I am neither the developer nor the player, but I tried mext_gdal
>> >> > >> test version (gdal110_test.zip) at last weekend and probably I
>> >> > >> understood
>> >> > >> some data related part, so I mention some of your question about
>> >> > >> datas.
>> >> > >> (But, if someone find my mistake, please point it out here...)
>> >> > >>
>> >> > >> 2015-05-01 18:59 GMT+09:00 Even Rouault
> <[hidden email]>:
>> >> > >> > I tried the FGD ogr driver on the file
>> >> > >> >
>> >> > >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicod
>> >> > >> > e_jp/d at
>> >> > >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
>> >> > >> > (found by googling FG-GML)
>> >> > >> >
>> >> > >> > It is not recognized by the driver. Looking more closely, the
>> >> > >> > driver
>> >> > >> > seems to expect a <GI> element that isn't found in that file. It
>> >> > >> > appears
>> >> > >> > that there are several versions of FGD GML, right ? The above
>> >> > >> > file is
>> >> > >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
>> >> > >> > driver
>> >> > >> > is v4 (or a previous version) ? I tried to remove the test for
>> >> > >> > the presence of <GI> but it crashes later during parsing of
>> >> > >> > dates, apparently not liking the <gml:timePosition> element
>> >> > >> > within <lfSpanFr>
>> >> > >>
>> >> > >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
>> >> > >> and 2 (or more) data types (5/10m mesh DEM / General vector data
>> >> > >> .etc)
>> >> > >> in FGD data,
>> >> > >> and current mext_gdal test version seems to support the following
>> >> > >> pair.
>> >> > >>
>> >> > >> =====
>> >> > >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
>> >> > >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
>> >> > >> 3. FGD driver: DLD v3 (General vector data)
>> >> > >> =====
>> >> > >>
>> >> > >> Above link which you found use GML v3, but it is not 5/10m mesh
>> >> > >> DEM, so you couldn't convert it.
>> >> > >>
>> >> > >> > I found this project :
>> >> > >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
>> >> > >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
>> >> > >> > It includes the schema for FG-GML v4 and I can't see any
>> >> > >> > reference to
>> >> > >> > the
>> >> > >> > <GI> element in it either.
>> >> > >> > With some trickery related to encoding conversion, I managed to
>> >> > >> > use that
>> >> > >> > project to produce a standard GML file that the OGR GML driver
>> >> > >> > handles
>> >> > >> > well :
>> >> > >> >
>> >> > >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
>> >> > >> > --from-code=shift-jis
>> >> > >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python
>> >> > >> > jpgisgml2gml.py >
>> >> > >> > out.gml
>> >> > >>
>> >> > >> And above GML v4 is not supported (both of 5/10m mesh DEM and
>> >> > >> General vector data) by current mext_gdal for now.
>> >> > >>
>> >> > >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and
>> >> > >> > downloaded
>> >> > >> > a
>> >> > >> > few datasets. Those I tried are not handled by the OGR FGD
>> >> > >> > driver. Some
>> >> > >> > of them look like GML with a particular application schema. They
>> >> > >> > are
>> >> > >> > unfortunately not standard enough for the OGR GML driver to
>> >> > >> > understand
>> >> > >> > them out of the box, although I believe it could be tweaked to
>> >> > >> > parse
>> >> > >> > them. Their header look like:
>> >> > >> >
>> >> > >> > <ksj:Dataset gml:id="A09Dataset"
>> >> > >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
>> >> > >> > xmlns:gml="http://www.opengis.net/gml/3.2"
>> >> > >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
>> >> > >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> >> > >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
>> >> > >> > KsjAppSchema- A09-v3_1.xsd">
>> >> > >> >
>> >> > >> > There's a shapefile next to them, with apparently same content.
>> >> > >>
>> >> > >> And about above MLIT NLNI(National Land Numerical Information)
>> >> > >> data, it is not FGD data...
>> >> > >>
>> >> > >> > So it appears there's a variety of GML-like formats ;-)
>> >> > >>
>> >> > >> Yes, sure...
>> >> > >>
>> >> > >> If you want to use mext_gdal test data, I just uploaded example
>> >> > >> dummy data
>> >> > >> to the following repository. So, please try to use those files.
>> >> > >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
>> >> > >> (The dummy data location is near by Japan Wakayama prefecture,
>> >> > >> but on the pacific ocean.)
>> >> > >>
>> >> > >> Regards,
>> >> > >>
>> >> > >> >> Hi Even,
>> >> > >> >>
>> >> > >> >> I talk to Yamate-san who has been working on those three
>> >> > >> >> drivers. He
>> >> > >> >> explained that each driver has little bit different copy right
>> >> > >> >> background he need to resolve before he can merge them to the
>> >> > >> >> main.
>> >> > >> >>
>> >> > >> >> The first one, FGD driver, which he wrote for a project and he
>> >> > >> >> does
>> >> > >> >> not
>> >> > >> >> have control on its copy right. So, he would like to leave it at
>> >> > >> >> this
>> >> > >> >> moment.
>> >> > >> >>
>> >> > >> >> For the second one, FG-GML driver, he wrote it and he owns its
>> >> > >> >> copy
>> >> > >> >> right at this moment. But he said he can release with the
>> >> > >> >> licence you
>> >> > >> >> suggested.
>> >> > >> >>
>> >> > >> >> The last one, FGD ogr driver, which he also wrote and it is
>> >> > >> >> highly possible he can release it under the MIT licence. But he
>> >> > >> >> need to make
>> >> > >> >> sure his client can agree on this. So, he said he will contact
>> >> > >> >> to his
>> >> > >> >> client.
>> >> > >> >>
>> >> > >> >> So, the last two drivers can be merge into the main GDAL driver.
>> >> > >> >> It is
>> >> > >> >> little bit confusing background but I am sure we can sort these
>> >> > >> >> out.
>> >> > >> >>
>> >> > >> >> I will let you know when he is ready to work with you.
>> >> > >> >>
>> >> > >> >> Thanks!
>> >> > >> >>
>> >> > >> >> Hiroo
>> >> > >> >>
>> >> > >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <[hidden email]>
>> >> > >> >>
>> >> > >> >> wrote:
>> >> > >> >> > Hi Even,
>> >> > >> >> >
>> >> > >> >> > I am not the developer but he and I working at a same company.
>> >> > >> >> > If he
>> >> > >> >> > can agree on the license and release his efforts, that would
>> >> > >> >> > be wonderful for all Japanese data uses.
>> >> > >> >> >
>> >> > >> >> > He hesitates writing his message in English. So, I can be a
>> >> > >> >> > translator
>> >> > >> >> > between you guys. I hope no lost in translation!
>> >> > >> >> >
>> >> > >> >> > I will get back to you once I hear from him.
>> >> > >> >> >
>> >> > >> >> > Hiroo
>> >> > >> >> >
>> >> > >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> >> > >> >> > <[hidden email]
>> >> > >> >> >
>> >> > >> >> > > wrote:
>> >> > >> >> >> Hi,
>> >> > >> >> >>
>> >> > >> >> >> Sorry for writing in English, but I don't unfortunately speak
>> >> > >> >> >> Japanese.
>> >> > >> >> >>
>> >> > >> >> >> I'm one of the GDAL developers.
>> >> > >> >> >>
>> >> > >> >> >> I've noticed that there are "un-official" versions of GDAL
>> >> > >> >> >> floating
>> >> > >> >> >> around, with
>> >> > >> >> >> a few drivers for new Japanese-specific formats. I was
>> >> > >> >> >> wondering if
>> >> > >> >> >> there was a
>> >> > >> >> >> reason to keep them outside of the official GDAL tree ?
>> >> > >> >> >> Integrating
>> >> > >> >> >> them into
>> >> > >> >> >> mainline GDAL would probably make life of Japanese users
>> >> > >> >> >> easier.
>> >> > >> >> >> GDAL
>> >> > >> >> >> has already support for a lof of national specific formats,
>> >> > >> >> >> mainly
>> >> > >> >> >> in
>> >> > >> >> >> Europe and
>> >> > >> >> >> North America. Would be cool for Asia to join in ;-)
>> >> > >> >> >>
>> >> > >> >> >> In a recent discussion thread, the following link is
>> >> > >> >> >>
>> >> > >> >> >> mentionned:
>> >> > >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> >> > >> >> >>
>> >> > >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> >> > >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese
>> >> > >> >> >> Fundamental
>> >> > >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> >> > >> >> >> Unfortunatelly there's no author or copyright information in
>> >> > >> >> >> the
>> >> > >> >> >> header of the
>> >> > >> >> >> corresponding source files. Would their author(s) be willing
>> >> > >> >> >> to release them
>> >> > >> >> >> under the X/MIT license and contribute them to the mainline
>> >> > >> >> >> GDAL
>> >> > >> >> >> project ? Perhaps some test files to write minimalistic
>> >> > >> >> >> regressions
>> >> > >> >> >> tests would be useful.
>> >> > >> >> >>
>> >> > >> >> >> Let me know if I can be useful for anything. I can also help
>> >> > >> >> >> for
>> >> > >> >> >> writing new
>> >> > >> >> >> drivers if needed.
>> >> > >> >> >>
>> >> > >> >> >> Best regards,
>> >> > >> >> >>
>> >> > >> >> >> Even
>> >> > >> >> >>
>> >> > >> >> >> --
>> >> > >> >> >> Spatialys - Geospatial professional services
>> >> > >> >> >> http://www.spatialys.com
>> >> > >> >> >> _______________________________________________
>> >> > >> >> >> OSGeoJapan-discuss mailing list
>> >> > >> >> >> [hidden email]
>> >> > >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >> > >> >> >
>> >> > >> >> > --
>> >> > >> >> > Hiroo Imaki
>> >> > >> >> > Pacific Spatial Solutions, LLC
>> >> > >> >> > 1523 Chatham Colony Ct.
>> >> > >> >> > Reston, VA  20190
>> >> > >> >> > [hidden email]
>> >> > >> >> > http://www.pacificspatial.com
>> >> > >> >> > http://www.geopacific.org (GIS info site)
>> >> > >> >> > [hidden email] (private)
>> >> > >> >
>> >> > >> > --
>> >> > >> > Spatialys - Geospatial professional services
>> >> > >> > http://www.spatialys.com
>> >> > >> > _______________________________________________
>> >> > >> > OSGeoJapan-discuss mailing list
>> >> > >> > [hidden email]
>> >> > >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >> > >
>> >> > > --
>> >> > > Spatialys - Geospatial professional services
>> >> > > http://www.spatialys.com
>> >> > > _______________________________________________
>> >> > > OSGeoJapan-discuss mailing list
>> >> > > [hidden email]
>> >> > > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >>
>> >> --
>> >> Spatialys - Geospatial professional services
>> >> http://www.spatialys.com
>> >> _______________________________________________
>> >> OSGeoJapan-discuss mailing list
>> >> [hidden email]
>> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> >
>> > --
>> > Hiroo Imaki
>> > Pacific Spatial Solutions, LLC
>> > 1523 Chatham Colony Ct.
>> > Reston, VA  20190
>> > [hidden email]
>> > http://www.pacificspatial.com
>> > http://www.geopacific.org (GIS info site)
>> > [hidden email] (private)
>> >
>> >
>> > _______________________________________________
>> > OSGeoJapan-discuss mailing list
>> > [hidden email]
>> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2
> So, could you wait for a while ?

Ko-san,

Sure, I was just seeking for news. GDAL 2.1 will probably not be released
before April or May next year (but as usual, the earlier it is integrated, the
better.)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
Hi Even,

> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
> before April or May next year (but as usual, the earlier it is integrated, the
> better.)

Okay.
Thanks for confirmation and useful information!

Regards,

Ko

2015-10-08 21:30 GMT+09:00 Even Rouault <[hidden email]>:

>> So, could you wait for a while ?
>
> Ko-san,
>
> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
> before April or May next year (but as usual, the earlier it is integrated, the
> better.)
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Hiroshi Miura(@osmf)
Hi Even-san,  Ko-san and Akagi-san,

Here was very old thread but I eventually pick it up.
That is because I've worked on python converter that
I posted recently and then got feedback about this thread.

I'd like to join this effort, and already does some works
target to release in GDAL v2.2. Mostly port the code on
current trunk HEAD.

https://github.com/miurahr/gdal/tree/ogr_frmt_gml_fgdgmlv4

There is an issue to prepare many tests and it should be done
before merge.

https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX#

best regards,

Hiroshi
RD of OpenStreetMap Foundation Japan

On 2015年10月08日 21:41, Ko Nagase wrote:

> Hi Even,
>
>> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
>> before April or May next year (but as usual, the earlier it is integrated, the
>> better.)
> Okay.
> Thanks for confirmation and useful information!
>
> Regards,
>
> Ko
>
> 2015-10-08 21:30 GMT+09:00 Even Rouault <[hidden email]>:
>>> So, could you wait for a while ?
>> Ko-san,
>>
>> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
>> before April or May next year (but as usual, the earlier it is integrated, the
>> better.)
>>
>> Even
>>

_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault-2

On samedi 18 mars 2017 08:45:17 CET Hiroshi Miura(@osmf) wrote:

> Hi Even-san, Ko-san and Akagi-san,

>

> Here was very old thread but I eventually pick it up.

> That is because I've worked on python converter that

> I posted recently and then got feedback about this thread.

>

> I'd like to join this effort, and already does some works

> target to release in GDAL v2.2. Mostly port the code on

> current trunk HEAD.

>

> https://github.com/miurahr/gdal/tree/ogr_frmt_gml_fgdgmlv4

>

> There is an issue to prepare many tests and it should be done

> before merge.

 

Cool, perhaps you could submit a pull request, even if it is not completely ready, so you can get early feedback ?

 

Best,

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Integration of Japanese GIS formats into mainline GDAL ?

Ko Nagase
In reply to this post by Hiroshi Miura(@osmf)
Hi Miura-san, Akagi-san, Even,

Sorry for my no response and activity,
and thanks for proceeding with this topic.

Regards,

Ko

2017-03-18 8:45 GMT+09:00 Hiroshi Miura(@osmf) <[hidden email]>:

> Hi Even-san,  Ko-san and Akagi-san,
>
> Here was very old thread but I eventually pick it up.
> That is because I've worked on python converter that
> I posted recently and then got feedback about this thread.
>
> I'd like to join this effort, and already does some works
> target to release in GDAL v2.2. Mostly port the code on
> current trunk HEAD.
>
> https://github.com/miurahr/gdal/tree/ogr_frmt_gml_fgdgmlv4
>
> There is an issue to prepare many tests and it should be done
> before merge.
>
> https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX#
>
> best regards,
>
> Hiroshi
> RD of OpenStreetMap Foundation Japan
>
> On 2015年10月08日 21:41, Ko Nagase wrote:
>> Hi Even,
>>
>>> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
>>> before April or May next year (but as usual, the earlier it is integrated, the
>>> better.)
>> Okay.
>> Thanks for confirmation and useful information!
>>
>> Regards,
>>
>> Ko
>>
>> 2015-10-08 21:30 GMT+09:00 Even Rouault <[hidden email]>:
>>>> So, could you wait for a while ?
>>> Ko-san,
>>>
>>> Sure, I was just seeking for news. GDAL 2.1 will probably not be released
>>> before April or May next year (but as usual, the earlier it is integrated, the
>>> better.)
>>>
>>> Even
>>>
>



--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: [hidden email]
web: http://georepublic.co.jp
_______________________________________________
OSGeoJapan-discuss mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
12
Loading...