Releasing 4.8.0 ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Releasing 4.8.0 ?

Emmanuel Séguin
Hi everyone,

Experiencing thread-safety issues while using Proj4 4.7.0 in our own WMS
software, we (IGNF) have done a test with the trunk version of Proj4 =>
New Proj4 API thread-safe functions solved our problems. As we're
generally happy about the tests done using this trunk version, we were
thinking about releasing our software with a new version of Proj4. This
leads us to the subject of this message :

How far is the trunk version of Proj4 from the 4.8.0 release ? And how
can this milestone be reached ?

The 4.8.0 milestone indicator
(http://trac.osgeo.org/proj/milestone/4.8.0) shows only two active
tickets yet to be solved.
However, there's a more substantial list of 46 active bug tickets
waiting to be solved (http://trac.osgeo.org/proj/report/1).

Do you think all the active bug tickets need to be fixed before
releasing 4.8.0 ?

Which one of them do you think must be absolutely fixed in the 4.8.0
release ?

Thanx for you opinions,

Best regards,

Manu

--
Emmanuel Séguin
SIEL - Pôle technique du Géoportail
Institut Géographique National
2/4, avenue Pasteur - 94165 Saint Mandé Cedex
Phone : +33 1.43.98.80.00 (ext : 72.21)
http://fr.linkedin.com/in/emmmanuelseguin


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Releasing 4.8.0 ?

Frank Warmerdam
2011/8/5 Emmanuel Séguin <[hidden email]>:
> How far is the trunk version of Proj4 from the 4.8.0 release ? And how
> can this milestone be reached ?

Emmanuel,

Good question.  I made a push for a 4.8.0 release about six
weeks ago, but got bogged down in the number of open bugs
and ultimately failed to bring forward even a beta.

> The 4.8.0 milestone indicator
> (http://trac.osgeo.org/proj/milestone/4.8.0) shows only two active
> tickets yet to be solved.
> However, there's a more substantial list of 46 active bug tickets
> waiting to be solved (http://trac.osgeo.org/proj/report/1).
>
> Do you think all the active bug tickets need to be fixed before
> releasing 4.8.0 ?

I certainly do not feel the bug backlog needs to be cleared before
a release.

> Which one of them do you think must be absolutely fixed in the 4.8.0
> release ?

That is a very good question.  I'd appreciate folks identifying bugs
(on list, or possibly by setting the milestone to 4.8.0 in Trac) that
they think are important to fix before we proceed with a release.
Some I have trouble judging the importance of.

I'd like to have a PROJ 4.8.0 release completed in August.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Releasing 4.8.0 ?

support.mn
In reply to this post by Emmanuel Séguin
Hello,

Before thread safe measures in library could somebody check
and test the performance of all versions since we do not want
to have any back steps in speed. (And maybe put a switch to
remove such measures if they are not required if they really do
add some overhead?)

My suggestion would be that if somebody needs thread safety
he adds some locking mechanism above Proj.4 so that the threads
do not mess up (write) the data and reading only by many threads
is not a problem? What exactly are the changes made in Proj.4
from 4.6 to 4.7?

Regards: Janne.

----------------------------------------------------

Emmanuel Séguin [[hidden email]] kirjoitti:

> Hi everyone,
>
> Experiencing thread-safety issues while using Proj4 4.7.0 in our own WMS
> software, we (IGNF) have done a test with the trunk version of Proj4 =>
> New Proj4 API thread-safe functions solved our problems. As we're
> generally happy about the tests done using this trunk version, we were
> thinking about releasing our software with a new version of Proj4. This
> leads us to the subject of this message :
>
> How far is the trunk version of Proj4 from the 4.8.0 release ? And how
> can this milestone be reached ?
>
> The 4.8.0 milestone indicator
> (http://trac.osgeo.org/proj/milestone/4.8.0) shows only two active
> tickets yet to be solved.
> However, there's a more substantial list of 46 active bug tickets
> waiting to be solved (http://trac.osgeo.org/proj/report/1).
>
> Do you think all the active bug tickets need to be fixed before
> releasing 4.8.0 ?
>
> Which one of them do you think must be absolutely fixed in the 4.8.0
> release ?
>
> Thanx for you opinions,
>
> Best regards,
>
> Manu
>
> --
> Emmanuel Séguin
> SIEL - Pôle technique du Géoportail
> Institut Géographique National
> 2/4, avenue Pasteur - 94165 Saint Mandé Cedex
> Phone : +33 1.43.98.80.00 (ext : 72.21)
> http://fr.linkedin.com/in/emmmanuelseguin
>
>
> _______________________________________________
> Proj mailing list
> [hidden email]
> http://lists.maptools.org/mailman/listinfo/proj
>

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Releasing 4.8.0 ?

Frank Warmerdam
On Sun, Aug 7, 2011 at 6:28 AM,  <[hidden email]> wrote:
> Hello,
>
> Before thread safe measures in library could somebody check
> and test the performance of all versions since we do not want
> to have any back steps in speed. (And maybe put a switch to
> remove such measures if they are not required if they really do
> add some overhead?)

Janne,

Doing some timetests between versions does seem worthwhile.  If
this is a major concern for you, I would encourage you to do so.  Perhaps
compare 4.6.0, 4.7.0 and trunk.

I will note that while I am concerned about performance, my
goal with PROJ.4 isn't necessarily to be the fastest projections
library.  I don't accept that any peformance regression is unacceptable.

You can configure --without-mutex on unix-like systems to
disable use of locking where it is currently found (access to
datum shift grids, and the "init file" cache).

> My suggestion would be that if somebody needs thread safety
> he adds some locking mechanism above Proj.4 so that the threads
> do not mess up (write) the data and reading only by many threads
> is not a problem? What exactly are the changes made in Proj.4
> from 4.6 to 4.7?

Your suggested approach was used in the past, but it was an
unnecessary limitation in most cases so it was decided (by me
at least) that PROJ.4 should become multi-thread safe.

This has been accomplished by two main means.

1) Implement an exclusion lock around access to some
shared resources, in particular the datum shift grids and
the cache of initiations loaded from init files (like 'epsg').

2) Implement an execution context object, visible at the
application layer, that can be used to segregate different
threads.  The most important thing put in this thread was
the "pj_errno", but some other things, like error handlers
as well.  This allows us to implement reentrancy without
mutual exclusion locks.

There was also a bit of cleanup of other unnecessary static
data I think.   Some of this work, the exclusion locks in
particular, was already done in 4.7.0.  The context work is
in 4.8.0 and requires using slightly different function entry
points to get full advantage.

The introduction of a context should likely have been
vetted by an RFC process.  I am still willing to do so
if there are several members of the community who would
like us to have a chance to consider whether or not to go
this direction.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Releasing 4.8.0 ?

support.mn
In reply to this post by Emmanuel Séguin
Frank,

if you are only using local variables in stack.. there are no thread
problems since each thread has it's own variables.. the point is
just to remove all globals and instead of referencing globals
calling access functions to protected locals. Basically each
thread should have it's own variables in stack or where ever.

I have not tested the speed of 4.7 but I had the feeling after switching
back to 4.6 that all proj.4 operations had an increase of speed and
that is why I am suspicious about the 4.7 coding? Especially the
startup initialization of projections was much faster in 4.6.

Since I do not need thread safety even if my projects mostly are
multi threaded (only one thread handles the projections) I would
not like to have any drawbacks in speed. So that I really prefer a
switch that would remove all unnecessary thread related overhead.

Frank, can you run "Diff" for 4.6 and 4.7 (or 4.8)? I don't have the tool
installed right now here.

http://en.wikipedia.org/wiki/Diff
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/diff.html

Regards: Janne.

--------------------------------

Frank Warmerdam [[hidden email]] kirjoitti:

> On Sun, Aug 7, 2011 at 6:28 AM,  <[hidden email]> wrote:
> > Hello,
> >
> > Before thread safe measures in library could somebody check
> > and test the performance of all versions since we do not want
> > to have any back steps in speed. (And maybe put a switch to
> > remove such measures if they are not required if they really do
> > add some overhead?)
>
> Janne,
>
> Doing some timetests between versions does seem worthwhile.  If
> this is a major concern for you, I would encourage you to do so.  Perhaps
> compare 4.6.0, 4.7.0 and trunk.
>
> I will note that while I am concerned about performance, my
> goal with PROJ.4 isn't necessarily to be the fastest projections
> library.  I don't accept that any peformance regression is unacceptable.
>
> You can configure --without-mutex on unix-like systems to
> disable use of locking where it is currently found (access to
> datum shift grids, and the "init file" cache).
>
> > My suggestion would be that if somebody needs thread safety
> > he adds some locking mechanism above Proj.4 so that the threads
> > do not mess up (write) the data and reading only by many threads
> > is not a problem? What exactly are the changes made in Proj.4
> > from 4.6 to 4.7?
>
> Your suggested approach was used in the past, but it was an
> unnecessary limitation in most cases so it was decided (by me
> at least) that PROJ.4 should become multi-thread safe.
>
> This has been accomplished by two main means.
>
> 1) Implement an exclusion lock around access to some
> shared resources, in particular the datum shift grids and
> the cache of initiations loaded from init files (like 'epsg').
>
> 2) Implement an execution context object, visible at the
> application layer, that can be used to segregate different
> threads.  The most important thing put in this thread was
> the "pj_errno", but some other things, like error handlers
> as well.  This allows us to implement reentrancy without
> mutual exclusion locks.
>
> There was also a bit of cleanup of other unnecessary static
> data I think.   Some of this work, the exclusion locks in
> particular, was already done in 4.7.0.  The context work is
> in 4.8.0 and requires using slightly different function entry
> points to get full advantage.
>
> The introduction of a context should likely have been
> vetted by an RFC process.  I am still willing to do so
> if there are several members of the community who would
> like us to have a chance to consider whether or not to go
> this direction.
>
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Software Developer
> _______________________________________________
> Proj mailing list
> [hidden email]
> http://lists.maptools.org/mailman/listinfo/proj
>

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj