embedding GeoTools is always a pain

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

embedding GeoTools is always a pain

teeschke
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the complexity and pitfalls of this process. Two Scenarios to describe the problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI library is required but not hosted in the maven central or boundless repo. So I must use another repository (osgeo/webda) which seems to be the last one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is not a pleasure and from my point of view not state-of-the-art because all other libraries are available at maven central and are able to manage their own dependencies. But in case of GeoTools I always need to look into reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all modules into the pom to not run into problems I always got or is there a how-to I missed? :)

Thanks and kind regards,
Tee
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Mark Prins
On 24-11-15 23:35, teeschke wrote:

> Dear GeoTools users and developers,
>
> after embedding GeoTools in my n-th project I am very annoyed about the
> complexity and pitfalls of this process. Two Scenarios to describe the
> problem:
>
> First: I wanted to parse a KML file and try to follow the Geometry Guide on
> http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
> 1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
> because I learned my lessons from projects before
> 2. trying to resolve the KMLConfiguration
> a) adding gt-xml and gt-geometry modules as dependencies which is not
> documented but seems to be meaningful because the how-to is about this
> b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
> the xsd folder and it sounds meaningful as well
> --> lucky me: it was the right one
> Sidenote: I needed to scroll through the boundless repository manually to
> scan for this modules
>
> Second Scenario: whenever I want to do something with grid files - the JAI
> library is required but not hosted in the maven central or boundless repo.
> So I must use another repository (osgeo/webda) which seems to be the last
> one which is hosting the JAI library :)

see item 7 in the quickstart:
http://docs.geotools.org/latest/userguide/tutorial/quickstart/maven.html 
basically you should use the osgeo repo unless you want to live on the edge

In my experience any IDE with Maven support (eclipse, netbeans) also has
support for Maven indexes making it possible to do local lookups.

It would be helpful if of osgeo was running a proper maven repository so
it could provide an index, but I guess the boundless repo can serve that
purpose. I would agree that hosting in maven central is the optimal
solution, but that also requires building javadoc for each artifact.

-M


------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

jody.garnett
In reply to this post by teeschke
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.

If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Andy Turner

Hi Jody, Hi everyone,


Many moons ago I started developing a raster data calculator based on the Raster class of GeoTools. I developed this so that it tiles/chunks 2D square celled rasters into chunks of the same size, but configurable rows and columns. I called the library Grids. Every Grid in Grids currently either holds ints or doubles, but I had envisaged developing ones for boolean and BigDecimal too. Every Grid shares a Grids_Environment object for memory handling, so if when processing the system runs low on memory, various data/tiles/chunks can be swapped to file space. There can be quite a lot involved in being clever about swapping data to and from different memory stores. Anyway, the library can handle multiple large grids on modest resources for many operations. Perhaps this can be evolved to be the JAI replacement.


In Grids there are various ways of storing chunks. They can be stored as HashMaps or Arrays or in other ways depending on their size. At present these can be stored as JAI images too, so currently there is a dependency on JAI for these, but it is not necessary at all.


Here is a link to the latest Grids on GitHub:

https://github.com/agdturner/grids

There is a bit more history of grids here:

http://www.geog.leeds.ac.uk/people/a.turner/src/andyt/java/grids/

Any questions I'll be happy to try to answer them and provide examples etc.


Cheers,


Andy




From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Andy Turner

One thing I think it is worth pointing out about Grids (as it is fairly fundamental) is that the underlying coordinate system is based on BigDecimal. The calculations for where a grid value is located are based on BigDecimal arithmetic. This is because otherwise there can be problems when dealing with rasters with very large numbers of rows and columns. Although Grids will handle multiple rasters with very large numbers of rows and columns, there are non memory/disk resource restrictions on how many rows and columns any grid can have and this will vary depending on how the definition of the chunks (data model used to store these).


I have not parallelised anything much in grids, but many grid operations can be parallelised and I have thought about this in respect of memory handling.


Some of the memory handling catches OutOfMemoryErrors to deal with problems if they do come up. It is much better though to test available memory to judge if the grids processing component is running low. In the Generic library upon which Grids depends there are useful functions for this.




From: Andy Turner <[hidden email]>
Sent: 29 November 2015 19:27
To: Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 

Hi Jody, Hi everyone,


Many moons ago I started developing a raster data calculator based on the Raster class of GeoTools. I developed this so that it tiles/chunks 2D square celled rasters into chunks of the same size, but configurable rows and columns. I called the library Grids. Every Grid in Grids currently either holds ints or doubles, but I had envisaged developing ones for boolean and BigDecimal too. Every Grid shares a Grids_Environment object for memory handling, so if when processing the system runs low on memory, various data/tiles/chunks can be swapped to file space. There can be quite a lot involved in being clever about swapping data to and from different memory stores. Anyway, the library can handle multiple large grids on modest resources for many operations. Perhaps this can be evolved to be the JAI replacement.


In Grids there are various ways of storing chunks. They can be stored as HashMaps or Arrays or in other ways depending on their size. At present these can be stored as JAI images too, so currently there is a dependency on JAI for these, but it is not necessary at all.


Here is a link to the latest Grids on GitHub:

https://github.com/agdturner/grids

There is a bit more history of grids here:

http://www.geog.leeds.ac.uk/people/a.turner/src/andyt/java/grids/

Any questions I'll be happy to try to answer them and provide examples etc.


Cheers,


Andy




From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Neal Dillman
In reply to this post by jody.garnett

Hello Jody,

 

I would suggest it might be worth talking to the folks at IDR Solutions.  They have a replacement that could work if an agreement could be worked out.  Information here: https://www.idrsolutions.com/jdeli

 

Regards,

Neal

 

PS:  If this works out perhaps that will get the GeoTiff issue moved up ;)

 

From: Jody Garnett [mailto:[hidden email]]
Sent: Sunday, November 29, 2015 10:41 AM
To: teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

 

We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.

 

If you have any suggestions on parties interested in the above tasks we would love to know.

 

 

--

Jody Garnett

 

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:

Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users

 


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Peter Borissow
In reply to this post by Andy Turner
I can definitely sympathize with Tee's situation. In the latest release of geotools (14.1) there are 223 different jar files. Chances are you don't need all of them for your project. In fact, if you're a noob and load all the jar files into a project without maven you'll get all sorts of errors.

Personally, I like my projects to have as small a footprint as possible with a minimal set of jars/dependencies. In the case of geotools, that means removing as many jars as necessary. I don't use maven so I have to prune the jars manually.

First, I separate out all the jars into categories (raster/image jars, database jars, vector jars, ogc/web related jars, xml/schema, commons, etc). Then I start adding the jars I think I need and add more jars as needed. This is an time consuming process but it keeps my final deployment size to a minimum.

Again, I don't use Maven and I don't know if it would help with this process. However, it would definitely help folks like me if the jars could be binned in some way. I can draft a list of bins if the packaging folks are willing to entertain this idea.


Thanks,
Peter



PS. Love geotools! Thanks for everyone's hard work.



From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users



------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Jes Wulfsberg Nielsen

You really want an automated dependency manager for that, so you can just point at the functionality you want, and get the needed jars assembled automatically.

 

If you don’t want Maven, you could take a look at the leaner Ivy, which uses the same definitions and repositories as Maven, but only handles the dependency management.

(In fairness, if you only use Maven for dependency management, it’s not bad. Most of its bad reputation comes from people obsessively trying to use it as its own programming language).

 

JWN

 

From: Peter Borissow [mailto:[hidden email]]
Sent: 3. december 2015 11:59
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

 

I can definitely sympathize with Tee's situation. In the latest release of geotools (14.1) there are 223 different jar files. Chances are you don't need all of them for your project. In fact, if you're a noob and load all the jar files into a project without maven you'll get all sorts of errors.

 

Personally, I like my projects to have as small a footprint as possible with a minimal set of jars/dependencies. In the case of geotools, that means removing as many jars as necessary. I don't use maven so I have to prune the jars manually.

 

First, I separate out all the jars into categories (raster/image jars, database jars, vector jars, ogc/web related jars, xml/schema, commons, etc). Then I start adding the jars I think I need and add more jars as needed. This is an time consuming process but it keeps my final deployment size to a minimum.

 

Again, I don't use Maven and I don't know if it would help with this process. However, it would definitely help folks like me if the jars could be binned in some way. I can draft a list of bins if the packaging folks are willing to entertain this idea.

 

 

Thanks,

Peter

 

 

PS. Love geotools! Thanks for everyone's hard work.

 


From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

 

We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.

Replace JAI · geotools/geotools Wiki · GitHub

geotools - Official GeoTools repository ... You can clone with HTTPS

 

 

If you have any suggestions on parties interested in the above tasks we would love to know.

 


--

Jody Garnett

 

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:

Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users

 

 

------------------------------------------------------------------------------

 

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users

 


------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: Grids

Peter Borissow
In reply to this post by Andy Turner
Hi Andy-
    Sounds interesting. Does the Grids library support no_data and random access to a region of interest? Does the input raster need to be tiled? Is JAI required? Any other dependencies? Any known limitations?

Also, it would be interesting to have a couple simple examples on your github page for how to load in a raster and access pixel values at a given point on the image.


Thanks,
Peter


From: Andy Turner <[hidden email]>
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Sent: Sunday, November 29, 2015 2:55 PM
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

One thing I think it is worth pointing out about Grids (as it is fairly fundamental) is that the underlying coordinate system is based on BigDecimal. The calculations for where a grid value is located are based on BigDecimal arithmetic. This is because otherwise there can be problems when dealing with rasters with very large numbers of rows and columns. Although Grids will handle multiple rasters with very large numbers of rows and columns, there are non memory/disk resource restrictions on how many rows and columns any grid can have and this will vary depending on how the definition of the chunks (data model used to store these).

I have not parallelised anything much in grids, but many grid operations can be parallelised and I have thought about this in respect of memory handling.

Some of the memory handling catches OutOfMemoryErrors to deal with problems if they do come up. It is much better though to test available memory to judge if the grids processing component is running low. In the Generic library upon which Grids depends there are useful functions for this.





From: Andy Turner <[hidden email]>
Sent: 29 November 2015 19:27
To: Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
Hi Jody, Hi everyone,

Many moons ago I started developing a raster data calculator based on the Raster class of GeoTools. I developed this so that it tiles/chunks 2D square celled rasters into chunks of the same size, but configurable rows and columns. I called the library Grids. Every Grid in Grids currently either holds ints or doubles, but I had envisaged developing ones for boolean and BigDecimal too. Every Grid shares a Grids_Environment object for memory handling, so if when processing the system runs low on memory, various data/tiles/chunks can be swapped to file space. There can be quite a lot involved in being clever about swapping data to and from different memory stores. Anyway, the library can handle multiple large grids on modest resources for many operations. Perhaps this can be evolved to be the JAI replacement.

In Grids there are various ways of storing chunks. They can be stored as HashMaps or Arrays or in other ways depending on their size. At present these can be stored as JAI images too, so currently there is a dependency on JAI for these, but it is not necessary at all.

Here is a link to the latest Grids on GitHub:
There is a bit more history of grids here:
Any questions I'll be happy to try to answer them and provide examples etc.

Cheers,

Andy



From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users



------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: Grids

Andy Turner

Yes, no_data values can be set. In calculations, no_data values are treated as I have thought was appropriate, for example in generalisations that calculate averages etc.

 

You can request the value of the Grid at a point and there are utility type methods that get all values within a set distance in coordinate space.


The input data does not need to be tiled. I load input data into Grids using an ESRI ASCII grid format. Typically this tiles the data the number of rows and the number of columns in each tile (chunk) can be set and don't have to be the same.


You can do everything in Grids without JAI, so it is not really required, but more of an optional for storage of data in chunks. However, not all the Grids source will compile without it. I did want to separate this out completely and have the JAI data store as a sort of plugin, but have not got around to this - and from the sounds of it, it might be best to factor JAI out completely anyway.


Grids depends on my Generic library (https://github.com/agdturner/generic) which helps handle the swapping of data (amongst other things, in particular calculations using BigDecimal arithmetic). The Generic library itself depends on Apache Commons Math. Again, that is not much of a dependency and could be factored out.


I don't have a lot of time right now, but I'll work on some examples and reply in due course.


Cheers,


Andy




From: Peter Borissow <[hidden email]>
Sent: 03 December 2015 14:50
To: Andy Turner; Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: Grids
 
Hi Andy-
    Sounds interesting. Does the Grids library support no_data and random access to a region of interest? Does the input raster need to be tiled? Is JAI required? Any other dependencies? Any known limitations?

Also, it would be interesting to have a couple simple examples on your github page for how to load in a raster and access pixel values at a given point on the image.


Thanks,
Peter


From: Andy Turner <[hidden email]>
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Sent: Sunday, November 29, 2015 2:55 PM
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

One thing I think it is worth pointing out about Grids (as it is fairly fundamental) is that the underlying coordinate system is based on BigDecimal. The calculations for where a grid value is located are based on BigDecimal arithmetic. This is because otherwise there can be problems when dealing with rasters with very large numbers of rows and columns. Although Grids will handle multiple rasters with very large numbers of rows and columns, there are non memory/disk resource restrictions on how many rows and columns any grid can have and this will vary depending on how the definition of the chunks (data model used to store these).

I have not parallelised anything much in grids, but many grid operations can be parallelised and I have thought about this in respect of memory handling.

Some of the memory handling catches OutOfMemoryErrors to deal with problems if they do come up. It is much better though to test available memory to judge if the grids processing component is running low. In the Generic library upon which Grids depends there are useful functions for this.





From: Andy Turner <[hidden email]>
Sent: 29 November 2015 19:27
To: Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
Hi Jody, Hi everyone,

Many moons ago I started developing a raster data calculator based on the Raster class of GeoTools. I developed this so that it tiles/chunks 2D square celled rasters into chunks of the same size, but configurable rows and columns. I called the library Grids. Every Grid in Grids currently either holds ints or doubles, but I had envisaged developing ones for boolean and BigDecimal too. Every Grid shares a Grids_Environment object for memory handling, so if when processing the system runs low on memory, various data/tiles/chunks can be swapped to file space. There can be quite a lot involved in being clever about swapping data to and from different memory stores. Anyway, the library can handle multiple large grids on modest resources for many operations. Perhaps this can be evolved to be the JAI replacement.

In Grids there are various ways of storing chunks. They can be stored as HashMaps or Arrays or in other ways depending on their size. At present these can be stored as JAI images too, so currently there is a dependency on JAI for these, but it is not necessary at all.

Here is a link to the latest Grids on GitHub:
https://github.com/agdturner/grids
agdturner/grids
grids - A Java spatial raster data handling library.


There is a bit more history of grids here:
Any questions I'll be happy to try to answer them and provide examples etc.

Cheers,

Andy



From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users



------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

Peter Borissow
In reply to this post by Jes Wulfsberg Nielsen
Yes, an automated dependency manager would be great! My IDE (Netbeans) will flag missing dependencies in my source code without Maven or Ivy. Problem is that it doesn't catch various inter-dependencies within the jar files. Unfortunately, those inter-dependencies only become obvious at runtime.

In the absence of an automated dependency manager, it would be *really nice* to break up the the binary distribution of GeoTools into sub-directories. That would help users pick and choose which jars/libs they need for their specific needs. So instead of having 200+ different jars in your project, you might only need a dozen or so.

For example, I tend to organize the jars into the following directory structure:

/core
 - gt-api-14.1.jar
 - gt-opengis-14.1.jar
 - gt-metadata-14.1.jar
 - gt-referencing-14.1.jar
 - gt-geometry-14.1.jar
 - jsr-275-1.0-beta-2.jar
 - jts-1.13.jar
 ...

/epsg
 - gt-epsg-extension-14.1.jar
 - gt-epsg-wkt-14.1.jar
 ...

/epsg/driver
 - gt-epsg-hsql-14.1.jar
 - gt-epsg-postgresql-14.1.jar
 ...


/raster
 - gt-coverage-14.1.jar
 - gt-grid-14.1.jar
 - jai_core-1.1.3.jar
 ...

/raster/processing
 - gt-image-14.1.jar
 - gt-imagemosaic-14.1.jar
 - gt-imagepyramid-14.1.jar

/raster/drivers
 - gt-geotiff-14.1.jar
 - gdal-1.8.1.jar
 - gt-gtopo30-14.1.jar
 - gt-imageio-ext-.jar
 ...

/vector/drivers
 - gt-geojson-14.1.jar
 - gt-shapefile-14.1.jar
 - gt-vpf-14.1.jar
 ...

/database/drivers
 - gt-jdbc-14.1.jar
 ...

/database/drivers/h2
 - gt-jdbc-h2-14.1.jar
 - h2-1.1.119.jar

/database/drivers/sqlite
 - gt-jdbc-spatialite-14.1.jar
 - sqlite-jdbc-3.8.6.jar

/database/drivers/postgis
 - gt-jdbc-postgis-14.1.jar
 - postgresql-9.4-1201-jdbc41.jar

/web
 - gt-wfs-14.1.jar
 - net.opengis.wfs-14.1.jar
 - gt-wms-14.1.jar


Something like that...

Then, when I need to create a new app, I pick which jars I think I need based on their functionality. I only include the database drivers that I need, I might not need to do anything with rasters so I can exclude that directory wholesale, etc.

I realize that the original thread was maven-specific so I apologize if my comments are steering the conversation off course. Just throwing out an idea to help make GeoTools more accessible to new users and casual users like me :-)

Thanks,
Peter







From: Jes Wulfsberg Nielsen <[hidden email]>
To: GeoTools Users <[hidden email]>
Sent: Thursday, December 3, 2015 7:30 AM
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

You really want an automated dependency manager for that, so you can just point at the functionality you want, and get the needed jars assembled automatically.
 
If you don’t want Maven, you could take a look at the leaner Ivy, which uses the same definitions and repositories as Maven, but only handles the dependency management.
(In fairness, if you only use Maven for dependency management, it’s not bad. Most of its bad reputation comes from people obsessively trying to use it as its own programming language).
 
JWN


From: Peter Borissow [mailto:[hidden email]]
Sent: 3. december 2015 11:59
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
I can definitely sympathize with Tee's situation. In the latest release of geotools (14.1) there are 223 different jar files. Chances are you don't need all of them for your project. In fact, if you're a noob and load all the jar files into a project without maven you'll get all sorts of errors.
 
Personally, I like my projects to have as small a footprint as possible with a minimal set of jars/dependencies. In the case of geotools, that means removing as many jars as necessary. I don't use maven so I have to prune the jars manually.
 
First, I separate out all the jars into categories (raster/image jars, database jars, vector jars, ogc/web related jars, xml/schema, commons, etc). Then I start adding the jars I think I need and add more jars as needed. This is an time consuming process but it keeps my final deployment size to a minimum.
 
Again, I don't use Maven and I don't know if it would help with this process. However, it would definitely help folks like me if the jars could be binned in some way. I can draft a list of bins if the packaging folks are willing to entertain this idea.
 
 
Thanks,
Peter
 
 
PS. Love geotools! Thanks for everyone's hard work.
 

From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS
 
 
If you have any suggestions on parties interested in the above tasks we would love to know.
 

--
Jody Garnett
 
On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:

Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
 
 
------------------------------------------------------------------------------
 
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
 

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users



------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: Grids

Peter Borissow
In reply to this post by Andy Turner
Thanks Andy-
    I appreciate the info. Something to consider in the future. Maybe the dev team should take a look at it as they investigate replacing JAI?

Looking forward to some examples/docs whenever you get to it.

Take Care,
Peter


From: Andy Turner <[hidden email]>
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>; Peter Borissow <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Sent: Thursday, December 3, 2015 10:18 AM
Subject: Re: Grids

Yes, no_data values can be set. In calculations, no_data values are treated as I have thought was appropriate, for example in generalisations that calculate averages etc.
 
You can request the value of the Grid at a point and there are utility type methods that get all values within a set distance in coordinate space.

The input data does not need to be tiled. I load input data into Grids using an ESRI ASCII grid format. Typically this tiles the data the number of rows and the number of columns in each tile (chunk) can be set and don't have to be the same.

You can do everything in Grids without JAI, so it is not really required, but more of an optional for storage of data in chunks. However, not all the Grids source will compile without it. I did want to separate this out completely and have the JAI data store as a sort of plugin, but have not got around to this - and from the sounds of it, it might be best to factor JAI out completely anyway.

Grids depends on my Generic library (https://github.com/agdturner/generic) which helps handle the swapping of data (amongst other things, in particular calculations using BigDecimal arithmetic). The Generic library itself depends on Apache Commons Math. Again, that is not much of a dependency and could be factored out.

I don't have a lot of time right now, but I'll work on some examples and reply in due course.

Cheers,

Andy



From: Peter Borissow <[hidden email]>
Sent: 03 December 2015 14:50
To: Andy Turner; Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: Grids
 
Hi Andy-
    Sounds interesting. Does the Grids library support no_data and random access to a region of interest? Does the input raster need to be tiled? Is JAI required? Any other dependencies? Any known limitations?

Also, it would be interesting to have a couple simple examples on your github page for how to load in a raster and access pixel values at a given point on the image.


Thanks,
Peter


From: Andy Turner <[hidden email]>
To: Jody Garnett <[hidden email]>; teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Sent: Sunday, November 29, 2015 2:55 PM
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

One thing I think it is worth pointing out about Grids (as it is fairly fundamental) is that the underlying coordinate system is based on BigDecimal. The calculations for where a grid value is located are based on BigDecimal arithmetic. This is because otherwise there can be problems when dealing with rasters with very large numbers of rows and columns. Although Grids will handle multiple rasters with very large numbers of rows and columns, there are non memory/disk resource restrictions on how many rows and columns any grid can have and this will vary depending on how the definition of the chunks (data model used to store these).

I have not parallelised anything much in grids, but many grid operations can be parallelised and I have thought about this in respect of memory handling.

Some of the memory handling catches OutOfMemoryErrors to deal with problems if they do come up. It is much better though to test available memory to judge if the grids processing component is running low. In the Generic library upon which Grids depends there are useful functions for this.





From: Andy Turner <[hidden email]>
Sent: 29 November 2015 19:27
To: Jody Garnett; teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
Hi Jody, Hi everyone,

Many moons ago I started developing a raster data calculator based on the Raster class of GeoTools. I developed this so that it tiles/chunks 2D square celled rasters into chunks of the same size, but configurable rows and columns. I called the library Grids. Every Grid in Grids currently either holds ints or doubles, but I had envisaged developing ones for boolean and BigDecimal too. Every Grid shares a Grids_Environment object for memory handling, so if when processing the system runs low on memory, various data/tiles/chunks can be swapped to file space. There can be quite a lot involved in being clever about swapping data to and from different memory stores. Anyway, the library can handle multiple large grids on modest resources for many operations. Perhaps this can be evolved to be the JAI replacement.

In Grids there are various ways of storing chunks. They can be stored as HashMaps or Arrays or in other ways depending on their size. At present these can be stored as JAI images too, so currently there is a dependency on JAI for these, but it is not necessary at all.

Here is a link to the latest Grids on GitHub:
https://github.com/agdturner/grids
agdturner/grids
grids - A Java spatial raster data handling library.


There is a bit more history of grids here:
Any questions I'll be happy to try to answer them and provide examples etc.

Cheers,

Andy



From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users





------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

jody.garnett
In reply to this post by Neal Dillman
That is interesting, JDeli does not appear to be open source - which is the same trouble we have with JAI.

Of note the ImageIO code does not seem to have such a difficult license, it is mostly the JAI interfaces that are a problem.

--
Jody Garnett

On 29 November 2015 at 12:26, Neal Dillman <[hidden email]> wrote:

Hello Jody,

 

I would suggest it might be worth talking to the folks at IDR Solutions.  They have a replacement that could work if an agreement could be worked out.  Information here: https://www.idrsolutions.com/jdeli

 

Regards,

Neal

 

PS:  If this works out perhaps that will get the GeoTiff issue moved up ;)

 

From: Jody Garnett [mailto:[hidden email]]
Sent: Sunday, November 29, 2015 10:41 AM
To: teeschke <[hidden email]>
Cc: GeoTools Users <[hidden email]>
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain

 

We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.

 

If you have any suggestions on parties interested in the above tasks we would love to know.

 

 

--

Jody Garnett

 

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:

Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users

 



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users
Reply | Threaded
Open this post in threaded view
|

Re: embedding GeoTools is always a pain

jody.garnett
In reply to this post by Peter Borissow
Drafting bins would be fine, I think we can control the assembly step (where the final zip is produced) but I am not quite sure how.

It is too bad you do not use maven, or ant+ivy, as it makes managing java dependencies easier and is integrated in most IDEs now.

--
Jody Garnett

On 3 December 2015 at 02:58, Peter Borissow <[hidden email]> wrote:
I can definitely sympathize with Tee's situation. In the latest release of geotools (14.1) there are 223 different jar files. Chances are you don't need all of them for your project. In fact, if you're a noob and load all the jar files into a project without maven you'll get all sorts of errors.

Personally, I like my projects to have as small a footprint as possible with a minimal set of jars/dependencies. In the case of geotools, that means removing as many jars as necessary. I don't use maven so I have to prune the jars manually.

First, I separate out all the jars into categories (raster/image jars, database jars, vector jars, ogc/web related jars, xml/schema, commons, etc). Then I start adding the jars I think I need and add more jars as needed. This is an time consuming process but it keeps my final deployment size to a minimum.

Again, I don't use Maven and I don't know if it would help with this process. However, it would definitely help folks like me if the jars could be binned in some way. I can draft a list of bins if the packaging folks are willing to entertain this idea.


Thanks,
Peter



PS. Love geotools! Thanks for everyone's hard work.



From: Jody Garnett <[hidden email]>
Sent: 29 November 2015 18:40
To: teeschke
Cc: GeoTools Users
Subject: Re: [Geotools-gt2-users] embedding GeoTools is always a pain
 
We are starting to look at replacing JAI, and will be doing design and fundraising over the course of 2016. JAI is not open source so we are unable to place it on maven central. The second part is service provider interface is a pain in some environments, and the "service registry" we use looks to be unsupported in Java 9.
Replace JAI · geotools/geotools Wiki · GitHub
geotools - Official GeoTools repository ... You can clone with HTTPS


If you have any suggestions on parties interested in the above tasks we would love to know.


--
Jody Garnett

On 24 November 2015 at 14:35, teeschke <[hidden email]> wrote:
Dear GeoTools users and developers,

after embedding GeoTools in my n-th project I am very annoyed about the
complexity and pitfalls of this process. Two Scenarios to describe the
problem:

First: I wanted to parse a KML file and try to follow the Geometry Guide on
http://docs.geotools.org/stable/userguide/library/xml/geometry.html#kml-parser
1. add boundless repository and include gt-api, gt-main and gt-epsg-hsql
because I learned my lessons from projects before
2. trying to resolve the KMLConfiguration
a) adding gt-xml and gt-geometry modules as dependencies which is not
documented but seems to be meaningful because the how-to is about this
b) adding the xsd/gt-xsd-kml module as dependencies because I stumbled into
the xsd folder and it sounds meaningful as well
--> lucky me: it was the right one
Sidenote: I needed to scroll through the boundless repository manually to
scan for this modules

Second Scenario: whenever I want to do something with grid files - the JAI
library is required but not hosted in the maven central or boundless repo.
So I must use another repository (osgeo/webda) which seems to be the last
one which is hosting the JAI library :)

At the end of the day all this searching and trying needs a lot of time, is
not a pleasure and from my point of view not state-of-the-art because all
other libraries are available at maven central and are able to manage their
own dependencies. But in case of GeoTools I always need to look into
reference projects to find out how I was getting it the last time.

What are your experience with scenarios like this? Are you adding all
modules into the pom to not run into problems I always got or is there a
how-to I missed? :)

Thanks and kind regards,
Tee




--
View this message in context: http://osgeo-org.1560.x6.nabble.com/embedding-GeoTools-is-always-a-pain-tp5238366.html
Sent from the geotools-gt2-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741551&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users


------------------------------------------------------------------------------

_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
GeoTools-GT2-Users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-gt2-users