[gdal-dev] Python3 continuous testing

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Python3 continuous testing

Even Rouault-2
Hi,

Just to notify you that I've added a python3 branch for Travis-CI testing :
https://travis-ci.org/rouault/gdal_coverage/branches

Before today's commits, Python 3 bindings had been broken in trunk for a few
months without anyone noticying/complaining. Now this should no longer happen.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Tamas Szekeres
Hi Even,

I've noticed build issue on Windows using the latest (stable) Python3.x versions, hope that this fix will solve that problem too.

Thanks,

Tamas
 

2015-01-27 13:44 GMT+01:00 Even Rouault <[hidden email]>:
Hi,

Just to notify you that I've added a python3 branch for Travis-CI testing :
https://travis-ci.org/rouault/gdal_coverage/branches

Before today's commits, Python 3 bindings had been broken in trunk for a few
months without anyone noticying/complaining. Now this should no longer happen.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Even Rouault-2
Le mardi 27 janvier 2015 14:49:01, Tamas Szekeres a écrit :
> Hi Even,
>
> I've noticed build issue on Windows using the latest (stable) Python3.x
> versions, hope that this fix will solve that problem too.

Hi Tamas,

Thas would be surprising since my commits solved runtime failure, not build
failure. Which error do you get ?

I don't see any error when looking at :
http://build.gisinternals.com/sdk/build-output/vc10-20150127-6-56-46-62-vc10-
dev.txt

>
> Thanks,
>
> Tamas
>
> 2015-01-27 13:44 GMT+01:00 Even Rouault <[hidden email]>:
> > Hi,
> >
> > Just to notify you that I've added a python3 branch for Travis-CI testing
> > : https://travis-ci.org/rouault/gdal_coverage/branches
> >
> > Before today's commits, Python 3 bindings had been broken in trunk for a
> > few
> > months without anyone noticying/complaining. Now this should no longer
> > happen.
> >
> > Even
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > _______________________________________________
> > gdal-dev mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/gdal-dev

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Tamas Szekeres
Hi Even,

I get this output from a Python 3.4 build (VS2012):


E:\builds\Python34\python.exe setup.py build
running build
running build_py
creating build
creating build\lib.win32-3.4
copying gdal.py -> build\lib.win32-3.4
copying ogr.py -> build\lib.win32-3.4
copying osr.py -> build\lib.win32-3.4
copying gdalconst.py -> build\lib.win32-3.4
copying gdalnumeric.py -> build\lib.win32-3.4
creating build\lib.win32-3.4\osgeo
copying osgeo\gdal.py -> build\lib.win32-3.4\osgeo
copying osgeo\gdalconst.py -> build\lib.win32-3.4\osgeo
copying osgeo\gdalnumeric.py -> build\lib.win32-3.4\osgeo
copying osgeo\gdal_array.py -> build\lib.win32-3.4\osgeo
copying osgeo\ogr.py -> build\lib.win32-3.4\osgeo
copying osgeo\osr.py -> build\lib.win32-3.4\osgeo
copying osgeo\__init__.py -> build\lib.win32-3.4\osgeo
Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py build\lib.win32-3.4\osgeo\gdalconst.py build\lib.win32-3.4\osgeo\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
Skipping implicit fixer: ws_comma
Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py build\lib.win32-3.4\osgeo\gdalconst.py build\lib.win32-3.4\osgeo\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
Skipping implicit fixer: ws_comma
running build_ext
building 'osgeo._gdal' extension
creating build\temp.win32-3.4
creating build\temp.win32-3.4\Release
creating build\temp.win32-3.4\Release\extensions
C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../../port -I../../gcore -I../../alg -I../../ogr/ -IE:\builds\Python34\include -IE:\builds\Python34\include -IE:\builds\Python34\lib\site-packages\numpy\core\include /Tpextensions/gdal_wrap.cpp /Fobuild\temp.win32-3.4\Release\extensions/gdal_wrap.obj
gdal_wrap.cpp
extensions/gdal_wrap.cpp(2451) : error C3861: 'PyCObject_Import': identifier not found
extensions/gdal_wrap.cpp(2521) : error C3861: 'PyCObject_FromVoidPtr': identifier not found
extensions/gdal_wrap.cpp(2544) : error C3861: 'PyCObject_AsVoidPtr': identifier not found
extensions/gdal_wrap.cpp(2549) : error C3861: 'PyCObject_FromVoidPtr': identifier not found
C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE\xlocale(336) : warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
e:\sdk\vc11\gdal-trunk\gdal\port\cpl_config.h(16) : warning C4005: 'HAVE_SNPRINTF' : macro redefinition
        e:\builds\python34\include\pyerrors.h(450) : see previous definition of 'HAVE_SNPRINTF'
e:\sdk\vc11\gdal-trunk\gdal\port\cpl_config.h(62) : warning C4005: 'HAVE_DIRECT_H' : macro redefinition
        e:\builds\python34\include\pyconfig.h(467) : see previous definition of 'HAVE_DIRECT_H'
../../gcore\gdal_priv.h(547) : warning C4251: 'GDALColorTable::aoEntries' : class 'std::vector<_Ty>' needs to have dll-interface to be used by clients of class 'GDALColorTable'
        with
        [
            _Ty=GDALColorEntry
        ]
../../gcore\gdal_priv.h(932) : warning C4251: 'GDALDriverManager::oMapNameToDrivers' : class 'std::map<_Kty,_Ty>' needs to have dll-interface to be used by clients of class 'GDALDriverManager'
        with
        [
            _Kty=CPLString,
            _Ty=GDALDriver *
        ]
extensions/gdal_wrap.cpp(3003) : warning C4244: 'argument' : conversion from 'GIntBig' to 'Py_ssize_t', possible loss of data
extensions/gdal_wrap.cpp(3016) : warning C4244: 'argument' : conversion from 'GIntBig' to 'Py_ssize_t', possible loss of data
extensions/gdal_wrap.cpp(3019) : warning C4244: 'return' : conversion from 'GIntBig' to 'int', possible loss of data
extensions/gdal_wrap.cpp(4136) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
extensions/gdal_wrap.cpp(4137) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
extensions/gdal_wrap.cpp(4218) : warning C4244: '=' : conversion from 'GIntBig' to 'int', possible loss of data
extensions/gdal_wrap.cpp(4392) : warning C4244: 'argument' : conversion from 'GIntBig' to 'Py_ssize_t', possible loss of data
extensions/gdal_wrap.cpp(4414) : warning C4244: 'argument' : conversion from 'GIntBig' to 'size_t', possible loss of data
extensions/gdal_wrap.cpp(4419) : warning C4244: 'argument' : conversion from 'GIntBig' to 'size_t', possible loss of data
extensions/gdal_wrap.cpp(4422) : warning C4244: 'argument' : conversion from 'GIntBig' to 'size_t', possible loss of data
extensions/gdal_wrap.cpp(4790) : warning C4244: 'argument' : conversion from 'GIntBig' to 'Py_ssize_t', possible loss of data
extensions/gdal_wrap.cpp(4813) : warning C4244: 'argument' : conversion from 'GIntBig' to 'size_t', possible loss of data
extensions/gdal_wrap.cpp(4840) : warning C4244: 'argument' : conversion from 'GIntBig' to 'Py_ssize_t', possible loss of data



builds at gisinternals use older python releases which don't produce this problem.


Thanks,

Tamas
 

2015-01-27 14:56 GMT+01:00 Even Rouault <[hidden email]>:
Le mardi 27 janvier 2015 14:49:01, Tamas Szekeres a écrit :
> Hi Even,
>
> I've noticed build issue on Windows using the latest (stable) Python3.x
> versions, hope that this fix will solve that problem too.

Hi Tamas,

Thas would be surprising since my commits solved runtime failure, not build
failure. Which error do you get ?

I don't see any error when looking at :
<a href="http://build.gisinternals.com/sdk/build-output/vc10-20150127-6-56-46-62-vc10- dev.txt" target="_blank">http://build.gisinternals.com/sdk/build-output/vc10-20150127-6-56-46-62-vc10-
dev.txt

>
> Thanks,
>
> Tamas
>
> 2015-01-27 13:44 GMT+01:00 Even Rouault <[hidden email]>:
> > Hi,
> >
> > Just to notify you that I've added a python3 branch for Travis-CI testing
> > : https://travis-ci.org/rouault/gdal_coverage/branches
> >
> > Before today's commits, Python 3 bindings had been broken in trunk for a
> > few
> > months without anyone noticying/complaining. Now this should no longer
> > happen.
> >
> > Even
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > _______________________________________________
> > gdal-dev mailing list
> > [hidden email]
> > http://lists.osgeo.org/mailman/listinfo/gdal-dev

--
Spatialys - Geospatial professional services
http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Even Rouault-2
Le mardi 27 janvier 2015 23:18:45, Tamas Szekeres a écrit :

> Hi Even,
>
> I get this output from a Python 3.4 build (VS2012):
>
>
> E:\builds\Python34\python.exe setup.py build
> running build
> running build_py
> creating build
> creating build\lib.win32-3.4
> copying gdal.py -> build\lib.win32-3.4
> copying ogr.py -> build\lib.win32-3.4
> copying osr.py -> build\lib.win32-3.4
> copying gdalconst.py -> build\lib.win32-3.4
> copying gdalnumeric.py -> build\lib.win32-3.4
> creating build\lib.win32-3.4\osgeo
> copying osgeo\gdal.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdalconst.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdalnumeric.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdal_array.py -> build\lib.win32-3.4\osgeo
> copying osgeo\ogr.py -> build\lib.win32-3.4\osgeo
> copying osgeo\osr.py -> build\lib.win32-3.4\osgeo
> copying osgeo\__init__.py -> build\lib.win32-3.4\osgeo
> Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py
> build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py
> build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py
> build\lib.win32-3.4\osgeo\gdalconst.py
> build\lib.win32-3.4\osgeo\gdalnumeric.py
> build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py
> build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
> Skipping implicit fixer: ws_comma
> Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py
> build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py
> build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py
> build\lib.win32-3.4\osgeo\gdalconst.py
> build\lib.win32-3.4\osgeo\gdalnumeric.py
> build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py
> build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
> Skipping implicit fixer: ws_comma
> running build_ext
> building 'osgeo._gdal' extension
> creating build\temp.win32-3.4
> creating build\temp.win32-3.4\Release
> creating build\temp.win32-3.4\Release\extensions
> C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe /c
> /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../../port -I../../gcore -I../../alg
> -I../../ogr/ -IE:\builds\Python34\include -IE:\builds\Python34\include
> -IE:\builds\Python34\lib\site-packages\numpy\core\include
> /Tpextensions/gdal_wrap.cpp
> /Fobuild\temp.win32-3.4\Release\extensions/gdal_wrap.obj
> gdal_wrap.cpp
> extensions/gdal_wrap.cpp(2451) : error C3861: 'PyCObject_Import':
> identifier not found
> extensions/gdal_wrap.cpp(2521) : error C3861: 'PyCObject_FromVoidPtr':
> identifier not found
> extensions/gdal_wrap.cpp(2544) : error C3861: 'PyCObject_AsVoidPtr':
> identifier not found
> extensions/gdal_wrap.cpp(2549) : error C3861: 'PyCObject_FromVoidPtr':
> identifier not found

--> This is SWIG generated code, so perhaps you need to update to a newer SWIG
that supports Python 3.4 ?

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Tamas Szekeres
In reply to this post by Tamas Szekeres
Hi Even,

I'm always confused about the GDAL-python bindings what is the best approach to use. The SWIG generated output is also included in the bindings, but as far as I remember it wasn't always up to date. Also the windows makefile will generate the bindings by default and there's no option to bypass this step as far as I remember.

On the other hand, I'm not quite sure which one is the suggested SWIG version to generate the bindings, I also remember the cases when the recent version wasn't sufficient and we had to use an earlier SWIG version to compile. Not sure how it depends on the Python version either. 

I think we should either remove the SWIG generated output from the repository or modify the makefiles not to generate the output every time.

Best regards,

Tamas

 

2015-01-28 9:59 GMT+01:00 Even Rouault <[hidden email]>:
Le mardi 27 janvier 2015 23:18:45, Tamas Szekeres a écrit :
> Hi Even,
>
> I get this output from a Python 3.4 build (VS2012):
>
>
> E:\builds\Python34\python.exe setup.py build
> running build
> running build_py
> creating build
> creating build\lib.win32-3.4
> copying gdal.py -> build\lib.win32-3.4
> copying ogr.py -> build\lib.win32-3.4
> copying osr.py -> build\lib.win32-3.4
> copying gdalconst.py -> build\lib.win32-3.4
> copying gdalnumeric.py -> build\lib.win32-3.4
> creating build\lib.win32-3.4\osgeo
> copying osgeo\gdal.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdalconst.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdalnumeric.py -> build\lib.win32-3.4\osgeo
> copying osgeo\gdal_array.py -> build\lib.win32-3.4\osgeo
> copying osgeo\ogr.py -> build\lib.win32-3.4\osgeo
> copying osgeo\osr.py -> build\lib.win32-3.4\osgeo
> copying osgeo\__init__.py -> build\lib.win32-3.4\osgeo
> Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py
> build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py
> build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py
> build\lib.win32-3.4\osgeo\gdalconst.py
> build\lib.win32-3.4\osgeo\gdalnumeric.py
> build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py
> build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
> Skipping implicit fixer: ws_comma
> Fixing build\lib.win32-3.4\gdal.py build\lib.win32-3.4\ogr.py
> build\lib.win32-3.4\osr.py build\lib.win32-3.4\gdalconst.py
> build\lib.win32-3.4\gdalnumeric.py build\lib.win32-3.4\osgeo\gdal.py
> build\lib.win32-3.4\osgeo\gdalconst.py
> build\lib.win32-3.4\osgeo\gdalnumeric.py
> build\lib.win32-3.4\osgeo\gdal_array.py build\lib.win32-3.4\osgeo\ogr.py
> build\lib.win32-3.4\osgeo\osr.py build\lib.win32-3.4\osgeo\__init__.py
> Skipping implicit fixer: ws_comma
> running build_ext
> building 'osgeo._gdal' extension
> creating build\temp.win32-3.4
> creating build\temp.win32-3.4\Release
> creating build\temp.win32-3.4\Release\extensions
> C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe /c
> /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../../port -I../../gcore -I../../alg
> -I../../ogr/ -IE:\builds\Python34\include -IE:\builds\Python34\include
> -IE:\builds\Python34\lib\site-packages\numpy\core\include
> /Tpextensions/gdal_wrap.cpp
> /Fobuild\temp.win32-3.4\Release\extensions/gdal_wrap.obj
> gdal_wrap.cpp
> extensions/gdal_wrap.cpp(2451) : error C3861: 'PyCObject_Import':
> identifier not found
> extensions/gdal_wrap.cpp(2521) : error C3861: 'PyCObject_FromVoidPtr':
> identifier not found
> extensions/gdal_wrap.cpp(2544) : error C3861: 'PyCObject_AsVoidPtr':
> identifier not found
> extensions/gdal_wrap.cpp(2549) : error C3861: 'PyCObject_FromVoidPtr':
> identifier not found

--> This is SWIG generated code, so perhaps you need to update to a newer SWIG
that supports Python 3.4 ?

--
Spatialys - Geospatial professional services
http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Python3 continuous testing

Even Rouault-2
Hi Tamas,

>
> I'm always confused about the GDAL-python bindings what is the best
> approach to use. The SWIG generated output is also included in the
> bindings, but as far as I remember it wasn't always up to date.

As far as I'm concerned, I try to make sure to commit updated generated files
(for Python). And making sure they are sync'ed with the .i is part of the
release process when we cut a version.

> Also the
> windows makefile will generate the bindings by default and there's no
> option to bypass this step as far as I remember.
>
> On the other hand, I'm not quite sure which one is the suggested SWIG
> version to generate the bindings,

Me neither. Your below error seems to suggest that another SWIG version as the
one you use (2.0.4 apparently) should be used (provided recent SWIG versions
have caught up with Python 3.4). Perhaps open a ticket if you don't have time
to investigate on that.

> I also remember the cases when the recent
> version wasn't sufficient and we had to use an earlier SWIG version to
> compile. Not sure how it depends on the Python version either.

I somehow remember an issue (something like one language needed a particular
SWIG version, but that didn't work with other languages), but I'm not sure if
it was with Python or another language.

>
> I think we should either remove the SWIG generated output from the
> repository or modify the makefiles not to generate the output every time.

Sounds reasonable. We've had such discussion in the past and I think the
outcome was that requiring people to have swig available to build the Python
bindings could be a pain for them. Beyond the pain, if we indeed need
particular/very recent swig for Python, then having the generated files can
make sense.

The Unix makefile has a 'generate' target that generates the bindings with SWIG
and a 'build' target that compiles generated bindings . Perhaps similar
approach could be taken for the Windows makefile (I don't anticipate myself
trying that, so I let you experimenting if you wish)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev