Quantcast

[gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?

Even Rouault-2

Hi,

 

Currently the default mode of linking GDAL with proj.4 is to use dynamic loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the reason for that was that it could have make it easier to use an alternate projection engine, but apparently nobody cared enough to plug a new one, and it could be done with standard linking. One downside of the current mechanism (besides code complication) is that it requires to list the exact library name of proj4 in GDAL source code, which can change depending on the soname of proj4. And this can cause subtle issues like

https://trac.osgeo.org/gdal/ticket/6881

 

So I'd suggest just keeping standard linking mechanism, and renaming the current --with-static-proj4 configure flag as --with-proj4, as the current name is confusing, and making it the default behaviour.

 

Any thoughts ?

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Kurt Schwehr-2
+1

On May 6, 2017 4:58 AM, "Even Rouault" <[hidden email]> wrote:

Hi,

 

Currently the default mode of linking GDAL with proj.4 is to use dynamic loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the reason for that was that it could have make it easier to use an alternate projection engine, but apparently nobody cared enough to plug a new one, and it could be done with standard linking. One downside of the current mechanism (besides code complication) is that it requires to list the exact library name of proj4 in GDAL source code, which can change depending on the soname of proj4. And this can cause subtle issues like

https://trac.osgeo.org/gdal/ticket/6881

 

So I'd suggest just keeping standard linking mechanism, and renaming the current --with-static-proj4 configure flag as --with-proj4, as the current name is confusing, and making it the default behaviour.

 

Any thoughts ?

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Even Rouault-2
In reply to this post by Even Rouault-2

On samedi 6 mai 2017 13:58:06 CEST Even Rouault wrote:

> Hi,

>

> Currently the default mode of linking GDAL with proj.4 is to use dynamic

> loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the

> reason for that was that it could have make it easier to use an alternate

> projection engine, but apparently nobody cared enough to plug a new one,

> and it could be done with standard linking. One downside of the current

> mechanism (besides code complication) is that it requires to list the exact

> library name of proj4 in GDAL source code, which can change depending on

> the soname of proj4. And this can cause subtle issues like

> https://trac.osgeo.org/gdal/ticket/6881

>

> So I'd suggest just keeping standard linking mechanism, and renaming the

> current --with- static-proj4 configure flag as --with-proj4, as the current

> name is confusing, and making it the default behaviour.

>

> Any thoughts ?

 

I'm intending to drop support for proj.4 < 4.8 too to do some cleanups

 

>

> Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Zachary Flamig
In reply to this post by Kurt Schwehr-2
+1

I’ve been tricked by this several times where I think I have gdal successfully compiled and then go to reproject something only to find out it can’t :-(

Zac


On May 6, 2017, at 10:20 AM, Kurt Schwehr <[hidden email]> wrote:

+1

On May 6, 2017 4:58 AM, "Even Rouault" <[hidden email]> wrote:
Hi,

 

Currently the default mode of linking GDAL with proj.4 is to use dynamic loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the reason for that was that it could have make it easier to use an alternate projection engine, but apparently nobody cared enough to plug a new one, and it could be done with standard linking. One downside of the current mechanism (besides code complication) is that it requires to list the exact library name of proj4 in GDAL source code, which can change depending on the soname of proj4. And this can cause subtle issues like

 

So I'd suggest just keeping standard linking mechanism, and renaming the current --with-static-proj4 configure flag as --with-proj4, as the current name is confusing, and making it the default behaviour.

 

Any thoughts ?

 

Even

 

--
Spatialys - Geospatial professional services

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Ivan Lucena-4
In reply to this post by Kurt Schwehr-2

-1


We have the options to build drivers against static and dynamic libraries of its SDKs, like HDF4,5 and openjpeg for example, using the same "--with-driver-name" 


Why not keep that same option for proj4?


--with-proj4=<static or dynamic>-library-path


I have projects where I let user to decide if they want to download and install proj4 while using GDAL and I would prefer to keep that option. 


Maybe with a compiler option we can basically "remove" the dynamic load when static linked proj4 was used. Config can translate the --with-proj4 into a C++ define, right?


Regards,

Ivan



From: gdal-dev <[hidden email]> on behalf of Kurt Schwehr <[hidden email]>
Sent: Saturday, May 6, 2017 11:20:34 AM
To: Even Rouault
Cc: gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?
 
+1

On May 6, 2017 4:58 AM, "Even Rouault" <[hidden email]> wrote:

Hi,

 

Currently the default mode of linking GDAL with proj.4 is to use dynamic loading mechanism (dlopen on Unix, LoadLibary on Windows). I believe the reason for that was that it could have make it easier to use an alternate projection engine, but apparently nobody cared enough to plug a new one, and it could be done with standard linking. One downside of the current mechanism (besides code complication) is that it requires to list the exact library name of proj4 in GDAL source code, which can change depending on the soname of proj4. And this can cause subtle issues like

https://trac.osgeo.org/gdal/ticket/6881

 

So I'd suggest just keeping standard linking mechanism, and renaming the current --with-static-proj4 configure flag as --with-proj4, as the current name is confusing, and making it the default behaviour.

 

Any thoughts ?

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Even Rouault-2

On samedi 6 mai 2017 17:04:33 CEST Ivan Lucena wrote:

> -1

>

>

> We have the options to build drivers against static and dynamic libraries of

> its SDKs, like HDF4,5 and openjpeg for example, using the same

> "--with-driver-name"

>

>

> Why not keep that same option for proj4?

 

I think you are confusing things. For a good reason since the --with-static-proj4 name is confusing. --with-static-proj4 can link against libproj.a or libproj.so depending on which is available, which is classic build time linking. My suggestion is to keep this as the only option to use proj.4, and rename it --with-proj4 for more clarity and consistency

 

When this option is not specified, proj4 is optionaly loaded through dlopen()/LoadLibrary(), which is runtime linking and what I'm proposing to drop for simplicity.

 

HDF4, 5, openjpeg are only supported through build time linking.

 

>

>

> --with-proj4=<static or dynamic>-library-path

>

>

> I have projects where I let user to decide if they want to download and

> install proj4 while using GDAL and I would prefer to keep that option.

>

 

Well, that seem to be a rather particular use case. Given the small size of libproj regarding libgdal, you could always build against proj.

 

After my cleanup and removal of pre-4.8 support, ogrct.cpp is now down to 1110 lines against 1311 before, and much easier to follow. And that save testing behaviour in --with-static-proj4 and --without-static-proj4 cases

 

>

> Maybe with a compiler option we can basically "remove" the dynamic load when

> static linked proj4 was used.

 

That's already the case. If currently you use --with-static-proj4, dynamic loading at runtime is disabled.

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Ivan Lucena-4

HI Even,


I understand the good intention of cleaning up code but that should not remove functionality. You are breaking backward compatibility. What if someone updates GDAL in some installation, proj4 is there and it will not going to be used?


I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to have the options to let the user decided if they want to add proj4 shared libraries.


Regards,

Ivan






From: Even Rouault <[hidden email]>
Sent: Saturday, May 6, 2017 1:35:18 PM
To: Ivan Lucena
Cc: Kurt Schwehr; gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?
 

On samedi 6 mai 2017 17:04:33 CEST Ivan Lucena wrote:

> -1

>

>

> We have the options to build drivers against static and dynamic libraries of

> its SDKs, like HDF4,5 and openjpeg for example, using the same

> "--with-driver-name"

>

>

> Why not keep that same option for proj4?

 

I think you are confusing things. For a good reason since the --with-static-proj4 name is confusing. --with-static-proj4 can link against libproj.a or libproj.so depending on which is available, which is classic build time linking. My suggestion is to keep this as the only option to use proj.4, and rename it --with-proj4 for more clarity and consistency

 

When this option is not specified, proj4 is optionaly loaded through dlopen()/LoadLibrary(), which is runtime linking and what I'm proposing to drop for simplicity.

 

HDF4, 5, openjpeg are only supported through build time linking.

 

>

>

> --with-proj4=<static or dynamic>-library-path

>

>

> I have projects where I let user to decide if they want to download and

> install proj4 while using GDAL and I would prefer to keep that option.

>

 

Well, that seem to be a rather particular use case. Given the small size of libproj regarding libgdal, you could always build against proj.

 

After my cleanup and removal of pre-4.8 support, ogrct.cpp is now down to 1110 lines against 1311 before, and much easier to follow. And that save testing behaviour in --with-static-proj4 and --without-static-proj4 cases

 

>

> Maybe with a compiler option we can basically "remove" the dynamic load when

> static linked proj4 was used.

 

That's already the case. If currently you use --with-static-proj4, dynamic loading at runtime is disabled.

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Even Rouault-2

Ivan,

 

> I understand the good intention of cleaning up code but that should not

> remove functionality. You are breaking backward compatibility. What if

> someone updates GDAL in some installation, proj4 is there and it will not

> going to be used?

 

Well that would be documented in the release notes... There are a number of breaks in each new release. That's not really different. And the way of solving in the case you mention is rather simple.

 

> I my case, I *cannot* distribute proj4 into my GDAL build

 

I've the feeling you're abusing -1 for a reason which you mentionned privately to me but in my opinion isn't really valid for the rest of the community. But this is not an important enough topic for me to fight about and create useless tensions among contributors.

 

So I'm dropping the ball on this. For posterity, my patch is at:

https://trac.osgeo.org/gdal/ticket/6882

 

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Andrew C Aitchison-2
In reply to this post by Even Rouault-2
On Sat, 6 May 2017, Even Rouault wrote:

> I'm intending to drop support for proj.4 < 4.8 too to do some cleanups

Red Hat / Centos / Scientific Linux 6 use proj4 v4.8.x
so I have no problem with this.

--
Andrew C. Aitchison Cambridge, UK
  [hidden email]


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Frank Warmerdam
In reply to this post by Even Rouault-2
Even,

I'm +0 on this change.

On the one hand, I *like* the fact that the dlopen() approach means
that adding libproj.so after the fact means that GDAL starts
supporting coordinate system conversion without a rebuild.  On the
other hand, it isn't clear why we do this only for libproj.

I have some appreciation for Ivan's stated concerns with regard to
regression, but I don't think that should hold back cleanup in major
new versions.

Best regards,
Frank


On Sat, May 6, 2017 at 12:09 PM, Even Rouault
<[hidden email]> wrote:

> Ivan,
>
>
>
>> I understand the good intention of cleaning up code but that should not
>
>> remove functionality. You are breaking backward compatibility. What if
>
>> someone updates GDAL in some installation, proj4 is there and it will not
>
>> going to be used?
>
>
>
> Well that would be documented in the release notes... There are a number of
> breaks in each new release. That's not really different. And the way of
> solving in the case you mention is rather simple.
>
>
>
>> I my case, I *cannot* distribute proj4 into my GDAL build
>
>
>
> I've the feeling you're abusing -1 for a reason which you mentionned
> privately to me but in my opinion isn't really valid for the rest of the
> community. But this is not an important enough topic for me to fight about
> and create useless tensions among contributors.
>
>
>
> So I'm dropping the ball on this. For posterity, my patch is at:
>
> https://trac.osgeo.org/gdal/ticket/6882
>
>
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
>
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/gdal-dev



--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows |
and watch the world go round - Rush    | Geospatial Software Developer
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Howard Butler-3
I'm +1 on this. I too found it confusing that Proj.4 worked in this way and no other libraries did in GDAL.


> I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to have the options to let the user decided if they want to add proj4 shared libraries.

Ivan,

I don't understand the scenario here. It isn't a licensing thing to prevent you from distributing Proj.4, and what do you do in the case of all of the other shared libraries that are linked at compile time that GDAL uses?

Howard

>
> On May 7, 2017, at 3:01 AM, Frank Warmerdam <[hidden email]> wrote:
>
> Even,
>
> I'm +0 on this change.
>
> On the one hand, I *like* the fact that the dlopen() approach means
> that adding libproj.so after the fact means that GDAL starts
> supporting coordinate system conversion without a rebuild.  On the
> other hand, it isn't clear why we do this only for libproj.
>
> I have some appreciation for Ivan's stated concerns with regard to
> regression, but I don't think that should hold back cleanup in major
> new versions.
>
> Best regards,
> Frank
>
>
> On Sat, May 6, 2017 at 12:09 PM, Even Rouault
> <[hidden email]> wrote:
>> Ivan,
>>
>>
>>
>>> I understand the good intention of cleaning up code but that should not
>>
>>> remove functionality. You are breaking backward compatibility. What if
>>
>>> someone updates GDAL in some installation, proj4 is there and it will not
>>
>>> going to be used?
>>
>>
>>
>> Well that would be documented in the release notes... There are a number of
>> breaks in each new release. That's not really different. And the way of
>> solving in the case you mention is rather simple.
>>
>>
>>
>>> I my case, I *cannot* distribute proj4 into my GDAL build
>>
>>
>>
>> I've the feeling you're abusing -1 for a reason which you mentionned
>> privately to me but in my opinion isn't really valid for the rest of the
>> community. But this is not an important enough topic for me to fight about
>> and create useless tensions among contributors.
>>
>>
>>
>> So I'm dropping the ball on this. For posterity, my patch is at:
>>
>> https://trac.osgeo.org/gdal/ticket/6882
>>
>>
>>
>>
>>
>> Even
>>
>>
>>
>> --
>>
>> Spatialys - Geospatial professional services
>>
>> http://www.spatialys.com
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> [hidden email]
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
> light and sound - activate the windows |
> and watch the world go round - Rush    | Geospatial Software Developer
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping dlopen/LoadLibrary loading of proj.4 ?

Ivan Lucena-4

Hi Howard,


Yep. It is a license issue but I am not going to get into the details.


But apart from that, lets think about other scenarios:


An application is using GDAL and Proj4 and someone decide to update GDAL to get some bug fixes.


But GDAL is build *without* static Proj4 and therefore is blind to the presence of Proj4 shared library.


That would be a problem. Right?


Anyway. If the change makes sense and it is for the better. And it is for a new release (no backport).


+0 ( I can't vote anyway)


Best regards,


Ivan




From: gdal-dev <[hidden email]> on behalf of Howard Butler <[hidden email]>
Sent: Sunday, May 7, 2017 9:47:26 AM
To: Frank Warmerdam
Cc: gdal dev
Subject: Re: [gdal-dev] Dropping dlopen/LoadLibrary loading of proj.4 ?
 
I'm +1 on this. I too found it confusing that Proj.4 worked in this way and no other libraries did in GDAL.


> I my case, I *cannot* distribute proj4 into my GDAL build and I *need* to have the options to let the user decided if they want to add proj4 shared libraries.

Ivan,

I don't understand the scenario here. It isn't a licensing thing to prevent you from distributing Proj.4, and what do you do in the case of all of the other shared libraries that are linked at compile time that GDAL uses?

Howard

>
> On May 7, 2017, at 3:01 AM, Frank Warmerdam <[hidden email]> wrote:
>
> Even,
>
> I'm +0 on this change.
>
> On the one hand, I *like* the fact that the dlopen() approach means
> that adding libproj.so after the fact means that GDAL starts
> supporting coordinate system conversion without a rebuild.  On the
> other hand, it isn't clear why we do this only for libproj.
>
> I have some appreciation for Ivan's stated concerns with regard to
> regression, but I don't think that should hold back cleanup in major
> new versions.
>
> Best regards,
> Frank
>
>
> On Sat, May 6, 2017 at 12:09 PM, Even Rouault
> <[hidden email]> wrote:
>> Ivan,
>>
>>
>>
>>> I understand the good intention of cleaning up code but that should not
>>
>>> remove functionality. You are breaking backward compatibility. What if
>>
>>> someone updates GDAL in some installation, proj4 is there and it will not
>>
>>> going to be used?
>>
>>
>>
>> Well that would be documented in the release notes... There are a number of
>> breaks in each new release. That's not really different. And the way of
>> solving in the case you mention is rather simple.
>>
>>
>>
>>> I my case, I *cannot* distribute proj4 into my GDAL build
>>
>>
>>
>> I've the feeling you're abusing -1 for a reason which you mentionned
>> privately to me but in my opinion isn't really valid for the rest of the
>> community. But this is not an important enough topic for me to fight about
>> and create useless tensions among contributors.
>>
>>
>>
>> So I'm dropping the ball on this. For posterity, my patch is at:
>>
>> https://trac.osgeo.org/gdal/ticket/6882
>>
>>
>>
>>
>>
>> Even
>>
>>
>>
>> --
>>
>> Spatialys - Geospatial professional services
>>
>> http://www.spatialys.com
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> [hidden email]
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
> light and sound - activate the windows |
> and watch the world go round - Rush    | Geospatial Software Developer
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Loading...