Transformation pipelines - your opinion?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Transformation pipelines - your opinion?

Thomas Knudsen
All,

I have recently opened a new pull request over at
https://github.com/OSGeo/proj.4/pull/388 ...

It introduces projection pipelines (i.e. multi-step transformations),
and, at the end-user facing side of things, a new 3D transformation
program called “tran”, which directly supports transformation
pipelines.

While the pull request expands the capabilities of proj.4
dramatically, the code is not very intrusive: Technically, the
pipeline driver is just a new projection, and hence a transformation
pipeline is introduced with the incantation “+proj=pipeline”.

I agree that at first, this may seem a bit odd, so I hope to initiate
a discussion here about e.g. the syntax, and aboult potentially useful
pipeline primitives: The code is still in its infancy, so a lot will
happen before it is ready for merging into the proj.4 master branch
(although I hope it can be done in time for version 4.10)

You can contribute to the discussion here and/or over at the github
page already mentioned. Also, you can read more about syntactic
details at my proj.4 page http://thomasknudsen.net/proj4.html, where
you can also find instructions for downloading a recent Windows build
of the “tran” tool.

/thomas
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

Howard Butler-3

> On Jun 4, 2016, at 10:46 PM, Thomas Knudsen <[hidden email]> wrote:
>
> All,
>
> I have recently opened a new pull request over at
> https://github.com/OSGeo/proj.4/pull/388 ...
>
> It introduces projection pipelines (i.e. multi-step transformations),
> and, at the end-user facing side of things, a new 3D transformation
> program called “tran”, which directly supports transformation
> pipelines.
>
> While the pull request expands the capabilities of proj.4
> dramatically, the code is not very intrusive: Technically, the
> pipeline driver is just a new projection, and hence a transformation
> pipeline is introduced with the incantation “+proj=pipeline”.
>
> I agree that at first, this may seem a bit odd, so I hope to initiate
> a discussion here about e.g. the syntax, and aboult potentially useful
> pipeline primitives: The code is still in its infancy, so a lot will
> happen before it is ready for merging into the proj.4 master branch
> (although I hope it can be done in time for version 4.10)
>
> You can contribute to the discussion here and/or over at the github
> page already mentioned. Also, you can read more about syntactic
> details at my proj.4 page http://thomasknudsen.net/proj4.html, where
> you can also find instructions for downloading a recent Windows build
> of the “tran” tool.

This looks like a really great addition! I've been watching the pull request with excitement that this approach will provide more avenues for these kinds of challenging problems. As a point cloud and LiDAR person, these capabilities will be very useful, and the momentum you and Kristian have been building to improve Proj.4 maintenance and feature set is greatly appreciated.

Carry on!

Howard
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

NDavid
Sorry for the (very) late reply,

I've read the discussion about this pipelines feature at github and your proj4 page and
I've definitely see such coordinate transformation pipeline as very useful.
BUT I'm more incline to keep only cartographic projection inside proj4 and to implement
pipeline and other coordinate transformation outside proj4 in another library/librairies.
And perhaps with a more C++ style than ANSI C.  

Some potential coordinate transform that, I think, could fit into a pipeline are
- cartographic projection (of course)
- datum conversion with planar gridshift or use of geoid.
- unit transform
- cartesian <-> polar <-> cylindric coordinate
- temporal transform (gps time to utc time ? )
- trajectory georeferencing (for point cloud, mobile mapping camera)
- conversion between sensor coordinate system and platform/IMU
- ..
Some of this coordinate transformation need additional data (trajectory files, grid files,
datum/ellispoid dictionnary) and if implement inside proj4 that could lead to insert
into proj4 some dependencies or making it a bigger library.
And so people who are only interrested in cartographic transformation functions of
proj4 will have to pay for something they don't want.
I know that pj_transform/cs2cs/pj_apply_gridshift are not only about cartographic
projection but also about datum transform. I understand to keep such function
inside proj4 for historical and compatibility reasons but I don't think this is
their "right" place.

 just my 2 cents about this pipeline proposal.
Nicolas
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

Thomas Knudsen
Nicolas, I understand, and respect, your opinion, but i disagree for a number of reasons. Let me start with the simplest, and the one I think will probably also be the most convincing counterargument for your major concern (“...people who are only interrested in cartographic transformation functions of proj4 will have to pay for something they don't want...”):

If you take an architectural look at the proj.4 library, it consists of (currently) 146 different projections, each adding in the range of 2-10 kilobytes to the library footprint.

The pipeline functionality is typical in that respect: it is organized as a (small) number of additional projections (yes really: the pipeline driver itself is, architectually speaking, just another projection). And, as you can see in the (edited) compilation result below, the current weight of the pipeline package is only approximately twice the weight of the probably most used projection (the Engsager extended transverse mercator, etmerc):

$ gcc -I. -W -Wall -Wextra -pedantic -O2 -c PJ_pipeline.c PJ_horner.c PJ_cart.c PJ_helmert.c proj_etmerc.c
$ dir *.o

 2.655 horner.o
 3.576 PJ_cart.o
 6.465 PJ_helmert.o
 8.382 PJ_pipeline.o
10.733 proj_etmerc.o

Let us assume that the final pipeline functionality may end up weighing 3 times that, it is still only 6 times the weight of etmerc, and all in all less than 10% of the total weight of the library on my test platform (gcc under Windows 7).

For these, say, 60 kB, you buy an infrastructure for implementing a large number of transformations through use of external parameter files, rather than by letting them add to the direct weight of the library, by implementing them as hard coded projections.

My colleague Kristian Evers and I are in the process of adding proj.4 support for a number of Danish, Faroese and Greenlandic systems, each of which (due to a very large number of parameters) will weigh a substantial fraction of 60 kB if not implemented using pipelines and external parameter files.

Hence, I believe that even if you have no use for the pipeline functionality, it will cost you only a modest amount of additional weight - much less weight than the addition of, say, 10 extra hard coded projections

Now, add to this, that the conceptual simplicity of a projection library in comparison to the complexity of a full geodetic framework, really is an illusion: While in theory projections are simple (in the sense “mathematically well defined”) transformations from angular to linear coordinates, in reality they are not at all simple, if you want them to relate to any kind of real world.

Evidently, if you only need to relate a latitude/longitude pair in a given horizontal datum, to a set of projected coordinates in the same horizontal datum, a projection library is all you need.

But that is very seldom the case, as also indicated by the fact that NAD27-to-NAD83 transformation was part of the proj bundle right from the start, and the fact that sponsors during the years have found value in sponsoring Frank Warmerdam’s work on implementing first horizontal datum shifts, later on also vertical.

The latter also hints at the fact that today, the vast majority of coordinate-capture is done by GPS/GNSS, and hence natively 3D, although not necessarily referred to a system that makes much sense, except for low accuracy work.

To preserve the high geometrical accuracy from the global system, when transforming to a regional reference system (e.g. ETRS89) and vertical datum (e.g. NAP/EVRS), you need access to more fundamental geodetic functionality (although not much more than already included in the pipeline package), before you can finally transform your latitude, longitude, and elevation data into traditional map coordinates, using the projection functionality already existing in proj.4.

When moving away from the stable parts of tectonic plates, you also need support for intra plate deformation models. This is the case for large parts of Scandinavia, which are heavily influenced by post-glacial uplift. Using the pipeline metaphor, this can be expressed in fairly simple terms, making it accessible to end users through direct support from their domain specific software, using proj.4 for georeferencing.

From airborne LiDAR mapping, we have learned the lesson, that provision of high accuracy geodata results in unexpected and innovative uses, leading to request for even higher accuracy. In other words: accuracy is addictive. And without proper geodetic handling of transformations, you will never consistently approach an accuracy much better than metre level (consistent with the difference between global and plate fixed coordinates).

Geodetic transformations are typically not algorithmically complex, compared to what is already included in proj.4 - we just need a platform for handling the metadata and stringing together elementary transformations. As already hinted at above, the overall weight of the functionality will be only a fraction of the existing library.

Form the geodetic side, I see proj.4 with the pipeline extension as the only viable road to a successful dissemination of the functionality required. And from the proj.4 side, I see extended geodetic functionality as the only viable road for continued relevance in an increasingly always-connected 3D society.

/Thomas

2016-10-13 10:41 GMT+02:00 NDavid <[hidden email]>:
Sorry for the (very) late reply,

I've read the discussion about this pipelines feature at github and your
proj4 page and
I've definitely see such coordinate transformation pipeline as very useful.
BUT I'm more incline to keep only cartographic projection inside proj4 and
to implement
pipeline and other coordinate transformation outside proj4 in another
library/librairies.
And perhaps with a more C++ style than ANSI C.

Some potential coordinate transform that, I think, could fit into a pipeline
are
- cartographic projection (of course)
- datum conversion with planar gridshift or use of geoid.
- unit transform
- cartesian <-> polar <-> cylindric coordinate
- temporal transform (gps time to utc time ? )
- trajectory georeferencing (for point cloud, mobile mapping camera)
- conversion between sensor coordinate system and platform/IMU
- ..
Some of this coordinate transformation need additional data (trajectory
files, grid files,
datum/ellispoid dictionnary) and if implement inside proj4 that could lead
to insert
into proj4 some dependencies or making it a bigger library.
And so people who are only interrested in cartographic transformation
functions of
proj4 will have to pay for something they don't want.
I know that pj_transform/cs2cs/pj_apply_gridshift are not only about
cartographic
projection but also about datum transform. I understand to keep such
function
inside proj4 for historical and compatibility reasons but I don't think this
is
their "right" place.

 just my 2 cents about this pipeline proposal.
Nicolas



--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Transformation-pipelines-your-opinion-tp5269960p5290527.html
Sent from the PROJ.4 mailing list archive at Nabble.com.
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

support-2

Hello,

I 100 % agree with Nicolas to keep all parts that are not required to be inside Proj.4 outside it. Later it is much easier to read and handle such parts as required. You should just add a layer above Proj4 which could be used if your code required and omitted if more simple functionality is only required. As Nicolas wrote:

"BUT I'm more incline to keep only cartographic projection inside proj4 and to implement pipeline and other coordinate transformation outside proj4 in another library/librairies."

The same separate library idea should be ok with the syntax scanner idea for Proj.4 which is not yet implemented since it does not require to be inside Proj.4 at all. Since syntax scanning for errors is totally an independent step.. and can as well be omitted if not required.

You can also do what ever you wish if you rename you end product something else than Proj.4 .. to keep it consistent (compare "libproj" for an example of such projects).

Regards: Janne.

--------------------------------------------

Thomas Knudsen kirjoitti 2016-10-21 15:55:

Nicolas, I understand, and respect, your opinion, but i disagree for a number of reasons. Let me start with the simplest, and the one I think will probably also be the most convincing counterargument for your major concern ("...people who are only interrested in cartographic transformation functions of proj4 will have to pay for something they don't want..."):
 
If you take an architectural look at the proj.4 library, it consists of (currently) 146 different projections, each adding in the range of 2-10 kilobytes to the library footprint.
 
The pipeline functionality is typical in that respect: it is organized as a (small) number of additional projections (yes really: the pipeline driver itself is, architectually speaking, just another projection). And, as you can see in the (edited) compilation result below, the current weight of the pipeline package is only approximately twice the weight of the probably most used projection (the Engsager extended transverse mercator, etmerc):
 
$ gcc -I. -W -Wall -Wextra -pedantic -O2 -c PJ_pipeline.c PJ_horner.c PJ_cart.c PJ_helmert.c proj_etmerc.c
$ dir *.o
 
 2.655 horner.o
 3.576 PJ_cart.o
 6.465 PJ_helmert.o
 8.382 PJ_pipeline.o
10.733 proj_etmerc.o
 
Let us assume that the final pipeline functionality may end up weighing 3 times that, it is still only 6 times the weight of etmerc, and all in all less than 10% of the total weight of the library on my test platform (gcc under Windows 7).
 
For these, say, 60 kB, you buy an infrastructure for implementing a large number of transformations through use of external parameter files, rather than by letting them add to the direct weight of the library, by implementing them as hard coded projections.
 
My colleague Kristian Evers and I are in the process of adding proj.4 support for a number of Danish, Faroese and Greenlandic systems, each of which (due to a very large number of parameters) will weigh a substantial fraction of 60 kB if not implemented using pipelines and external parameter files.
 
Hence, I believe that even if you have no use for the pipeline functionality, it will cost you only a modest amount of additional weight - much less weight than the addition of, say, 10 extra hard coded projections
 
Now, add to this, that the conceptual simplicity of a projection library in comparison to the complexity of a full geodetic framework, really is an illusion: While in theory projections are simple (in the sense "mathematically well defined") transformations from angular to linear coordinates, in reality they are not at all simple, if you want them to relate to any kind of real world.
 
Evidently, if you only need to relate a latitude/longitude pair in a given horizontal datum, to a set of projected coordinates in the same horizontal datum, a projection library is all you need.
 
But that is very seldom the case, as also indicated by the fact that NAD27-to-NAD83 transformation was part of the proj bundle right from the start, and the fact that sponsors during the years have found value in sponsoring Frank Warmerdam's work on implementing first horizontal datum shifts, later on also vertical.
 
The latter also hints at the fact that today, the vast majority of coordinate-capture is done by GPS/GNSS, and hence natively 3D, although not necessarily referred to a system that makes much sense, except for low accuracy work.
 
To preserve the high geometrical accuracy from the global system, when transforming to a regional reference system (e.g. ETRS89) and vertical datum (e.g. NAP/EVRS), you need access to more fundamental geodetic functionality (although not much more than already included in the pipeline package), before you can finally transform your latitude, longitude, and elevation data into traditional map coordinates, using the projection functionality already existing in proj.4.
 
When moving away from the stable parts of tectonic plates, you also need support for intra plate deformation models. This is the case for large parts of Scandinavia, which are heavily influenced by post-glacial uplift. Using the pipeline metaphor, this can be expressed in fairly simple terms, making it accessible to end users through direct support from their domain specific software, using proj.4 for georeferencing.
 
From airborne LiDAR mapping, we have learned the lesson, that provision of high accuracy geodata results in unexpected and innovative uses, leading to request for even higher accuracy. In other words: accuracy is addictive. And without proper geodetic handling of transformations, you will never consistently approach an accuracy much better than metre level (consistent with the difference between global and plate fixed coordinates).
 
Geodetic transformations are typically not algorithmically complex, compared to what is already included in proj.4 - we just need a platform for handling the metadata and stringing together elementary transformations. As already hinted at above, the overall weight of the functionality will be only a fraction of the existing library.
 
Form the geodetic side, I see proj.4 with the pipeline extension as the only viable road to a successful dissemination of the functionality required. And from the proj.4 side, I see extended geodetic functionality as the only viable road for continued relevance in an increasingly always-connected 3D society.
 
/Thomas

2016-10-13 10:41 GMT+02:00 NDavid <[hidden email]>:
Sorry for the (very) late reply,

I've read the discussion about this pipelines feature at github and your
proj4 page and
I've definitely see such coordinate transformation pipeline as very useful.
BUT I'm more incline to keep only cartographic projection inside proj4 and
to implement
pipeline and other coordinate transformation outside proj4 in another
library/librairies.
And perhaps with a more C++ style than ANSI C.

Some potential coordinate transform that, I think, could fit into a pipeline
are
- cartographic projection (of course)
- datum conversion with planar gridshift or use of geoid.
- unit transform
- cartesian <-> polar <-> cylindric coordinate
- temporal transform (gps time to utc time ? )
- trajectory georeferencing (for point cloud, mobile mapping camera)
- conversion between sensor coordinate system and platform/IMU
- ..
Some of this coordinate transformation need additional data (trajectory
files, grid files,
datum/ellispoid dictionnary) and if implement inside proj4 that could lead
to insert
into proj4 some dependencies or making it a bigger library.
And so people who are only interrested in cartographic transformation
functions of
proj4 will have to pay for something they don't want.
I know that pj_transform/cs2cs/pj_apply_gridshift are not only about
cartographic
projection but also about datum transform. I understand to keep such
function
inside proj4 for historical and compatibility reasons but I don't think this
is
their "right" place.

 just my 2 cents about this pipeline proposal.
Nicolas



--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Transformation-pipelines-your-opinion-tp5269960p5290527.html
Sent from the PROJ.4 mailing list archive at Nabble.com.
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj



_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

Thomas Knudsen
Janne,

I have already elaborated on the illusory nature of the perception of projections as "clean, beautiful mathematical entities" as opposed to the "ugly geodetic real world references".

As already mentioned, proj's history indicates that without geodesy, a projection library is built on a solid foundation of thin air: While mathematically beautiful, without a geodetic foundation, projected coordinates will never refer to a real world.

The abstraction of a coordinate without an explicit system is leaky - the Earth being dynamic, there is no "perfect, eternally stable" backdrop that can be used as pivot gateway between all other systems.

Reference frames are real things in the real world, with all the imperfections of reality. Mathematical coordinate systems are abstract things in an abstract world, that do not suffer from these imperfections. Unfortunately, however, they do not in any way relate to the world we're living in.

A large amount of geodetic functionality already exists in proj, and has since its inception. It could, however, probably have been implemented in a more straightforward way if native 3D support had been part of proj at an earlier point in time.

That being said, I invite you to reread my former input, and my pull requests: What I am currently adding to proj.4 is a minor tool kit of building blocks for implementing a large family of transformations using mostly external parameter files, rather than adding them directly to the library code.

I think this is exactly what you're advocating, if I interpret correctly your vague ranting about not having anything in the library that does not need to be there. Please note, however, that "need" is a highly subjective term - your needs are apparently different than mine.

But in general, the world and its needs evolve, and so does proj. If you do not like the direction proj evolves, the source code is open, you're free to adapt it in almost any way you find necessary to suit your needs.

/Thomas






2016-10-24 8:50 GMT+02:00 <[hidden email]>:

Hello,

I 100 % agree with Nicolas to keep all parts that are not required to be inside Proj.4 outside it. Later it is much easier to read and handle such parts as required. You should just add a layer above Proj4 which could be used if your code required and omitted if more simple functionality is only required. As Nicolas wrote:

"BUT I'm more incline to keep only cartographic projection inside proj4 and to implement pipeline and other coordinate transformation outside proj4 in another library/librairies."

The same separate library idea should be ok with the syntax scanner idea for Proj.4 which is not yet implemented since it does not require to be inside Proj.4 at all. Since syntax scanning for errors is totally an independent step.. and can as well be omitted if not required.

You can also do what ever you wish if you rename you end product something else than Proj.4 .. to keep it consistent (compare "libproj" for an example of such projects).

Regards: Janne.

--------------------------------------------

Thomas Knudsen kirjoitti 2016-10-21 15:55:

Nicolas, I understand, and respect, your opinion, but i disagree for a number of reasons. Let me start with the simplest, and the one I think will probably also be the most convincing counterargument for your major concern ("...people who are only interrested in cartographic transformation functions of proj4 will have to pay for something they don't want..."):
 
If you take an architectural look at the proj.4 library, it consists of (currently) 146 different projections, each adding in the range of 2-10 kilobytes to the library footprint.
 
The pipeline functionality is typical in that respect: it is organized as a (small) number of additional projections (yes really: the pipeline driver itself is, architectually speaking, just another projection). And, as you can see in the (edited) compilation result below, the current weight of the pipeline package is only approximately twice the weight of the probably most used projection (the Engsager extended transverse mercator, etmerc):
 
$ gcc -I. -W -Wall -Wextra -pedantic -O2 -c PJ_pipeline.c PJ_horner.c PJ_cart.c PJ_helmert.c proj_etmerc.c
$ dir *.o
 
 2.655 horner.o
 3.576 PJ_cart.o
 6.465 PJ_helmert.o
 8.382 PJ_pipeline.o
10.733 proj_etmerc.o
 
Let us assume that the final pipeline functionality may end up weighing 3 times that, it is still only 6 times the weight of etmerc, and all in all less than 10% of the total weight of the library on my test platform (gcc under Windows 7).
 
For these, say, 60 kB, you buy an infrastructure for implementing a large number of transformations through use of external parameter files, rather than by letting them add to the direct weight of the library, by implementing them as hard coded projections.
 
My colleague Kristian Evers and I are in the process of adding proj.4 support for a number of Danish, Faroese and Greenlandic systems, each of which (due to a very large number of parameters) will weigh a substantial fraction of 60 kB if not implemented using pipelines and external parameter files.
 
Hence, I believe that even if you have no use for the pipeline functionality, it will cost you only a modest amount of additional weight - much less weight than the addition of, say, 10 extra hard coded projections
 
Now, add to this, that the conceptual simplicity of a projection library in comparison to the complexity of a full geodetic framework, really is an illusion: While in theory projections are simple (in the sense "mathematically well defined") transformations from angular to linear coordinates, in reality they are not at all simple, if you want them to relate to any kind of real world.
 
Evidently, if you only need to relate a latitude/longitude pair in a given horizontal datum, to a set of projected coordinates in the same horizontal datum, a projection library is all you need.
 
But that is very seldom the case, as also indicated by the fact that NAD27-to-NAD83 transformation was part of the proj bundle right from the start, and the fact that sponsors during the years have found value in sponsoring Frank Warmerdam's work on implementing first horizontal datum shifts, later on also vertical.
 
The latter also hints at the fact that today, the vast majority of coordinate-capture is done by GPS/GNSS, and hence natively 3D, although not necessarily referred to a system that makes much sense, except for low accuracy work.
 
To preserve the high geometrical accuracy from the global system, when transforming to a regional reference system (e.g. ETRS89) and vertical datum (e.g. NAP/EVRS), you need access to more fundamental geodetic functionality (although not much more than already included in the pipeline package), before you can finally transform your latitude, longitude, and elevation data into traditional map coordinates, using the projection functionality already existing in proj.4.
 
When moving away from the stable parts of tectonic plates, you also need support for intra plate deformation models. This is the case for large parts of Scandinavia, which are heavily influenced by post-glacial uplift. Using the pipeline metaphor, this can be expressed in fairly simple terms, making it accessible to end users through direct support from their domain specific software, using proj.4 for georeferencing.
 
From airborne LiDAR mapping, we have learned the lesson, that provision of high accuracy geodata results in unexpected and innovative uses, leading to request for even higher accuracy. In other words: accuracy is addictive. And without proper geodetic handling of transformations, you will never consistently approach an accuracy much better than metre level (consistent with the difference between global and plate fixed coordinates).
 
Geodetic transformations are typically not algorithmically complex, compared to what is already included in proj.4 - we just need a platform for handling the metadata and stringing together elementary transformations. As already hinted at above, the overall weight of the functionality will be only a fraction of the existing library.
 
Form the geodetic side, I see proj.4 with the pipeline extension as the only viable road to a successful dissemination of the functionality required. And from the proj.4 side, I see extended geodetic functionality as the only viable road for continued relevance in an increasingly always-connected 3D society.
 
/Thomas

2016-10-13 10:41 GMT+02:00 NDavid <[hidden email]>:
Sorry for the (very) late reply,

I've read the discussion about this pipelines feature at github and your
proj4 page and
I've definitely see such coordinate transformation pipeline as very useful.
BUT I'm more incline to keep only cartographic projection inside proj4 and
to implement
pipeline and other coordinate transformation outside proj4 in another
library/librairies.
And perhaps with a more C++ style than ANSI C.

Some potential coordinate transform that, I think, could fit into a pipeline
are
- cartographic projection (of course)
- datum conversion with planar gridshift or use of geoid.
- unit transform
- cartesian <-> polar <-> cylindric coordinate
- temporal transform (gps time to utc time ? )
- trajectory georeferencing (for point cloud, mobile mapping camera)
- conversion between sensor coordinate system and platform/IMU
- ..
Some of this coordinate transformation need additional data (trajectory
files, grid files,
datum/ellispoid dictionnary) and if implement inside proj4 that could lead
to insert
into proj4 some dependencies or making it a bigger library.
And so people who are only interrested in cartographic transformation
functions of
proj4 will have to pay for something they don't want.
I know that pj_transform/cs2cs/pj_apply_gridshift are not only about
cartographic
projection but also about datum transform. I understand to keep such
function
inside proj4 for historical and compatibility reasons but I don't think this
is
their "right" place.

 just my 2 cents about this pipeline proposal.
Nicolas



--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Transformation-pipelines-your-opinion-tp5269960p5290527.html
Sent from the PROJ.4 mailing list archive at Nabble.com.
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj



_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Transformation pipelines - your opinion?

Even Rouault-2
In reply to this post by Thomas Knudsen
Hi,

I can understand Nicolas' point about making proj do one thing well as being
in the line of the Unix philosophy. But I think some other things must be
considered.
Libraries with too small scope have a less chance of attracting a sufficiently
big and healthy development community, making them a more risky choice for
adoption. A new library also implies more packaging effort, infrastructure
needs, etc.
Proj.4 is definitely one of the place that has the most momentum to address geo
related transform. As Thomas pointed out, cartographic projections are only a
part of the overall problem.
Regarding the size of the library, proj is really small w.r.t today's
standards. The Linux libc is 1.8 MB large whereas proj fits in 340 KB... It's
far from being cluttered.

Even


> Nicolas, I understand, and respect, your opinion, but i disagree for a
> number of reasons. Let me start with the simplest, and the one I think will
> probably also be the most convincing counterargument for your major concern
> (“...people who are only interrested in cartographic transformation
> functions of proj4 will have to pay for something they don't want...”):
>
> If you take an architectural look at the proj.4 library, it consists of
> (currently) 146 different projections, each adding in the range of 2-10
> kilobytes to the library footprint.
>
> The pipeline functionality is typical in that respect: it is organized as a
> (small) number of additional projections (yes really: the pipeline driver
> itself is, architectually speaking, just another projection). And, as you
> can see in the (edited) compilation result below, the current weight of the
> pipeline package is only approximately twice the weight of the probably
> most used projection (the Engsager extended transverse mercator, etmerc):
>
> $ gcc -I. -W -Wall -Wextra -pedantic -O2 -c PJ_pipeline.c PJ_horner.c
> PJ_cart.c PJ_helmert.c proj_etmerc.c
> $ dir *.o
>
>  2.655 horner.o
>  3.576 PJ_cart.o
>  6.465 PJ_helmert.o
>  8.382 PJ_pipeline.o
> 10.733 proj_etmerc.o
>
> Let us assume that the final pipeline functionality may end up weighing 3
> times that, it is still only 6 times the weight of etmerc, and all in all
> less than 10% of the total weight of the library on my test platform (gcc
> under Windows 7).
>
> For these, say, 60 kB, you buy an infrastructure for implementing a large
> number of transformations through use of external parameter files, rather
> than by letting them add to the direct weight of the library, by
> implementing them as hard coded projections.
>
> My colleague Kristian Evers and I are in the process of adding proj.4
> support for a number of Danish, Faroese and Greenlandic systems, each of
> which (due to a very large number of parameters) will weigh a substantial
> fraction of 60 kB if not implemented using pipelines and external parameter
> files.
>
> Hence, I believe that even if you have no use for the pipeline
> functionality, it will cost you only a modest amount of additional weight -
> much less weight than the addition of, say, 10 extra hard coded projections
>
> Now, add to this, that the conceptual simplicity of a projection library in
> comparison to the complexity of a full geodetic framework, really is an
> illusion: While in theory projections are simple (in the sense
> “mathematically well defined”) transformations from angular to linear
> coordinates, in reality they are not at all simple, if you want them to
> relate to any kind of real world.
>
> Evidently, if you only need to relate a latitude/longitude pair in a given
> horizontal datum, to a set of projected coordinates in the same horizontal
> datum, a projection library is all you need.
>
> But that is very seldom the case, as also indicated by the fact that
> NAD27-to-NAD83 transformation was part of the proj bundle right from the
> start, and the fact that sponsors during the years have found value in
> sponsoring Frank Warmerdam’s work on implementing first horizontal datum
> shifts, later on also vertical.
>
> The latter also hints at the fact that today, the vast majority of
> coordinate-capture is done by GPS/GNSS, and hence natively 3D, although not
> necessarily referred to a system that makes much sense, except for low
> accuracy work.
>
> To preserve the high geometrical accuracy from the global system, when
> transforming to a regional reference system (e.g. ETRS89) and vertical
> datum (e.g. NAP/EVRS), you need access to more fundamental geodetic
> functionality (although not much more than already included in the pipeline
> package), before you can finally transform your latitude, longitude, and
> elevation data into traditional map coordinates, using the projection
> functionality already existing in proj.4.
>
> When moving away from the stable parts of tectonic plates, you also need
> support for intra plate deformation models. This is the case for large
> parts of Scandinavia, which are heavily influenced by post-glacial uplift.
> Using the pipeline metaphor, this can be expressed in fairly simple terms,
> making it accessible to end users through direct support from their domain
> specific software, using proj.4 for georeferencing.
>
> From airborne LiDAR mapping, we have learned the lesson, that provision of
> high accuracy geodata results in unexpected and innovative uses, leading to
> request for even higher accuracy. In other words: accuracy is addictive.
> And without proper geodetic handling of transformations, you will never
> consistently approach an accuracy much better than metre level (consistent
> with the difference between global and plate fixed coordinates).
>
> Geodetic transformations are typically not algorithmically complex,
> compared to what is already included in proj.4 - we just need a platform
> for handling the metadata and stringing together elementary
> transformations. As already hinted at above, the overall weight of the
> functionality will be only a fraction of the existing library.
>
> Form the geodetic side, I see proj.4 with the pipeline extension as the
> only viable road to a successful dissemination of the functionality
> required. And from the proj.4 side, I see extended geodetic functionality
> as the only viable road for continued relevance in an increasingly
> always-connected 3D society.
>
> /Thomas
>
> 2016-10-13 10:41 GMT+02:00 NDavid <[hidden email]>:
> > Sorry for the (very) late reply,
> >
> > I've read the discussion about this pipelines feature at github and your
> > proj4 page and
> > I've definitely see such coordinate transformation pipeline as very
> > useful. BUT I'm more incline to keep only cartographic projection inside
> > proj4 and to implement
> > pipeline and other coordinate transformation outside proj4 in another
> > library/librairies.
> > And perhaps with a more C++ style than ANSI C.
> >
> > Some potential coordinate transform that, I think, could fit into a
> > pipeline
> > are
> > - cartographic projection (of course)
> > - datum conversion with planar gridshift or use of geoid.
> > - unit transform
> > - cartesian <-> polar <-> cylindric coordinate
> > - temporal transform (gps time to utc time ? )
> > - trajectory georeferencing (for point cloud, mobile mapping camera)
> > - conversion between sensor coordinate system and platform/IMU
> > - ..
> > Some of this coordinate transformation need additional data (trajectory
> > files, grid files,
> > datum/ellispoid dictionnary) and if implement inside proj4 that could
> > lead to insert
> > into proj4 some dependencies or making it a bigger library.
> > And so people who are only interrested in cartographic transformation
> > functions of
> > proj4 will have to pay for something they don't want.
> > I know that pj_transform/cs2cs/pj_apply_gridshift are not only about
> > cartographic
> > projection but also about datum transform. I understand to keep such
> > function
> > inside proj4 for historical and compatibility reasons but I don't think
> > this
> > is
> > their "right" place.
> >
> >  just my 2 cents about this pipeline proposal.
> >
> > Nicolas
> >
> >
> >
> > --
> > View this message in context: http://osgeo-org.1560.x6.
> > nabble.com/Transformation-pipelines-your-opinion-tp5269960p5290527.html
> > Sent from the PROJ.4 mailing list archive at Nabble.com.
> > _______________________________________________
> > Proj mailing list
> > [hidden email]
> > http://lists.maptools.org/mailman/listinfo/proj

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj