Does rasdaman CE solve an open source geospatial problem?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Does rasdaman CE solve an open source geospatial problem?

Edzer Pebesma-2
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
--
Edzer Pebesma
Institute for Geoinformatics  (ifgi),  University of Münster
Heisenbergstraße 2, 48149 Münster, Germany; +49 251 83 33081
Journal of Statistical Software:   http://www.jstatsoft.org/
Computers & Geosciences:   http://elsevier.com/locate/cageo/
Spatial Statistics Society http://www.spatialstatistics.info


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Peter Baumann
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Peter Baumann
oh, just looking at the subject again:

several service providers believe indeed rasdaman community does offer a significant advantage:
- see the download figures on www.rasdaman.org
- concretely, see www.planetserver.eu which is running rasdaman community on - I believe - about 20 TB of Planetary Science data.

-Peter


On 05/21/2016 09:56 AM, Peter Baumann wrote:
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Cameron Shorter
Hi Edzer,
Thank you for raising this topic questioning the value of radsaman community edition. It is pertinent considering recent discussions about Rasdaman incubation.

Peter, your comments about programmers wanting to get paid for their work is valid, but does not provide justification for OSGeo promoting a proprietary business model. OSGeo is in the business of promoting open source software, and helping people who create open source software.

Rasdaman's business model is in a grey zone. It provides a community edition and a proprietary edition. This is often referred to as an "open core" business model, or sometimes less favorably called "crippleware".
I think Rasdaman is the only OSGeo (proposed) project which provides an open core model. All prior projects have been pure open source.

Although a an open core model deviates from OSGeo's original principles, one could argue that Rasdaman community edition stands on its own as a valuable, quality open source geospatial application by itself, worthy of OSGeo promotion.

Edzer's comments appear to counter this argument. Edzer, I understand you suggest Rasdaman community edition is of little value for real world problems?
Extending from this, OSGeo endorsement of Rasdaman should be questioned and potentially withdrawn.

I'd be interested to hear opinions of others in the field as to whether Rasdaman community version is of value for real-world production systems by itself.

A deeper question for the greater OSGeo community is should OSGeo endorse Open Core business models?

Warm regards, Cameron

On 21/05/2016 6:07 pm, Peter Baumann wrote:
oh, just looking at the subject again:

several service providers believe indeed rasdaman community does offer a significant advantage:
- see the download figures on www.rasdaman.org
- concretely, see www.planetserver.eu which is running rasdaman community on - I believe - about 20 TB of Planetary Science data.

-Peter


On 05/21/2016 09:56 AM, Peter Baumann wrote:
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)




_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Cameron Shorter,
Software and Data Solutions Manager
LISAsoft
Suite 112, Jones Bay Wharf,
26 - 32 Pirrama Rd, Pyrmont NSW 2009

P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099

_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

geowolf
On Sat, May 21, 2016 at 2:06 PM, Cameron Shorter <[hidden email]> wrote:
I'd be interested to hear opinions of others in the field as to whether Rasdaman community version is of value for real-world production systems by itself.

I guess that would depend on the needs of the production system itself, and I'm not sure the OSGeo inclusion should be
based on how capable the software is... in short, I'd stay away from this criteria.

However I believe this discussion raise an important point, it does not make sense to talk about Rasdaman without qualification,
it makes sense to talk about Rasdaman CE and Rasdaman Enterprise, any discussion about the abilities of the software,
benchmarks and the like, should be properly qualified by the version of it.

I agree OSGeo should be mostly concerned about the CE version, and how open it actually is. 
Like, if someone develops an extension to it that mimics some of the enterprise functionality, will it be welcomed in the project?
That is the most important aspect, the "open core" model has a clear tendency to keep the core version limited to
show better value in the paid for version, but the community around the core version should not be pressured to stay away
from improvements that would hurt the money making brother of it.

While we are on the topic, there are also gray areas in which this model tends to fall into, which can ruffle some feathers as well.
Say someone develops an improvement for the core version that is competitive, to some level, with some functionality of the enterprise version,
and the company decides to "donate" the equivalent code because it's better developed, better tested, faster, more scalable
and so on.
This tends to hurt feelings on both sides, the community developer may feel walked over, his/her code dumped in the thrash,
the company may feel like it's been held hostage of some ransom, by having to donate its superior code over the 
lower quality one offered by the community (of course, it could be the case that the community offers something better,
but if that point is reached, the death bell is probably starting to sing for the open core approach).

To some extend that's unavoidable I'm afraid, but I believe that OSGeo should at least be concerned that the
community version is treated like a full open source project and everybody feels empowered to pitch into it according
to their needs and abilities.

Then there is another topic of discussion, which is whether OSGeo agrees with a dual licensing model to start with,
given the obvious tension that it generates between a open community and the company benefiting from it.
Peter cites the need to make a honest living doing open source, and the open core module as a way to reach that objective.
As a long time open source developer involved in single licence software I never felt like I was underpaid or 
mistreated, even when I have clear vision of companies making money off work that sometimes I did over holidays...
I simply knew what I was getting myself into, and what the consequences of an open license are.
That said, I do appreciate that people want to get an honest living and work in open source at the same time, it's
simply too good of an opportunity to miss.

Cheers
Andrea



--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime 
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

 

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


-------------------------------------------------------

_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

geowolf
On Sat, May 21, 2016 at 2:36 PM, Andrea Aime <[hidden email]> wrote:
On Sat, May 21, 2016 at 2:06 PM, Cameron Shorter <[hidden email]> wrote:
I'd be interested to hear opinions of others in the field as to whether Rasdaman community version is of value for real-world production systems by itself.

I guess that would depend on the needs of the production system itself, and I'm not sure the OSGeo inclusion should be
based on how capable the software is... in short, I'd stay away from this criteria.

Thinking about it, I believe this one needs some clarification. What I'm trying to say is that Rasdaman CE should be evaluated
for it own merits and selected based on them, instead of being compared with its commercial brother and rejected because
we find it lacking in the comparison.
Most projects graduated years ago with a lot less functionality that they have today, we should be more concerned by
how useful the project is today and how well it can grow in the future.

I also stress that whether or not "open core" projects are natural members of OSGeo, or not, is a separate topic, but at the
same time very important one. There is a lot of articles about "open core" to read on the internet... I've tried to search "Open
core is good" on Google to give it a "positive" slant, I cannot really say the result of the search agree on... positive :-p
It could just well be that the proponents have not spent enough time writing about its benefits.

Cheers
Andrea


--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime 
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.

 

The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility  for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.


-------------------------------------------------------

_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Dave McIlhagga-2
In reply to this post by Cameron Shorter
I think OSGeo should stay out of the business of weighing in on how companies decide to commercialize technologies, so long as they respect the licensing terms.

We and many companies use MapServer for instance at the heart of our proprietary stacks, which is simply another flavor of the open core model.

The more important issue here is whether the structure of a project's licensing and PSC provides a commercial advantage to one company over any other.

The concern I have with the benevolent dictator model in discussion here is that reasonable advancements members of the PSC may want to pursue for the project that may erode from the commercial interests of the benevolent dictator could be denied, and that doesn't feel like a fair playing field for the PSC to operate in, or a scenario OSGeo should endorse.

Dave






Sent from mobile

On May 21, 2016, at 8:06 AM, Cameron Shorter <[hidden email]> wrote:

Hi Edzer,
Thank you for raising this topic questioning the value of radsaman community edition. It is pertinent considering recent discussions about Rasdaman incubation.

Peter, your comments about programmers wanting to get paid for their work is valid, but does not provide justification for OSGeo promoting a proprietary business model. OSGeo is in the business of promoting open source software, and helping people who create open source software.

Rasdaman's business model is in a grey zone. It provides a community edition and a proprietary edition. This is often referred to as an "open core" business model, or sometimes less favorably called "crippleware".
I think Rasdaman is the only OSGeo (proposed) project which provides an open core model. All prior projects have been pure open source.

Although a an open core model deviates from OSGeo's original principles, one could argue that Rasdaman community edition stands on its own as a valuable, quality open source geospatial application by itself, worthy of OSGeo promotion.

Edzer's comments appear to counter this argument. Edzer, I understand you suggest Rasdaman community edition is of little value for real world problems?
Extending from this, OSGeo endorsement of Rasdaman should be questioned and potentially withdrawn.

I'd be interested to hear opinions of others in the field as to whether Rasdaman community version is of value for real-world production systems by itself.

A deeper question for the greater OSGeo community is should OSGeo endorse Open Core business models?

Warm regards, Cameron

On 21/05/2016 6:07 pm, Peter Baumann wrote:
oh, just looking at the subject again:

several service providers believe indeed rasdaman community does offer a significant advantage:
- see the download figures on www.rasdaman.org
- concretely, see www.planetserver.eu which is running rasdaman community on - I believe - about 20 TB of Planetary Science data.

-Peter


On 05/21/2016 09:56 AM, Peter Baumann wrote:
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)




_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Cameron Shorter,
Software and Data Solutions Manager
LISAsoft
Suite 112, Jones Bay Wharf,
26 - 32 Pirrama Rd, Pyrmont NSW 2009

P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Rob Emanuele
In reply to this post by Peter Baumann
Peter,

It's impressive that 20 TB is being served out by the community edition. However, the point that Edzer brings up (and that I was unaware of) causes me even more concern regarding the language used in service of promoting the potential OSGeo version of rasdaman around it's comparison to other projects.

Are the performance benchmarks that you were siting as sources based on queries that would benefit from distributed processing (which is a very large class of queries)? Are the benchmarks using the open source rasdaman or the proprietary version? If you are going to compare yourself to a distributed processing engine such as Apache Spark, you should be comparing it fairly - Apache Spark does not claim to be a fast solution in a single machine case. If you are comparing it to the proprietary version of rasdaman, then I'm wondering why you are referencing those numbers when talking about the open source version, which has been the subject of these threads of conversation. Google Earth Engine may beat us all out in performance, but that doesn't directly contribute much to the open source geospatial community because it lacks that key openness.

If rasdaman CE does not scale horizontally, there are use cases and example queries with which I am very confident rasdaman would be outperformed by systems constructed at their core with Apache Spark. If I've misunderstood something, or if this is somehow not the case, please let me know.

Best,
Rob

On Sat, May 21, 2016 at 4:07 AM, Peter Baumann <[hidden email]> wrote:
oh, just looking at the subject again:

several service providers believe indeed rasdaman community does offer a significant advantage:
- see the download figures on www.rasdaman.org
- concretely, see www.planetserver.eu which is running rasdaman community on - I believe - about 20 TB of Planetary Science data.

-Peter



On 05/21/2016 09:56 AM, Peter Baumann wrote:
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: <a href="tel:%2B49-421-200-3178" value="+494212003178" target="_blank">+49-421-200-3178, fax: <a href="tel:%2B49-421-200-493178" value="+49421200493178" target="_blank">+49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: <a href="tel:%2B49-173-5837882" value="+491735837882" target="_blank">+49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: <a href="tel:%2B49-421-200-3178" value="+494212003178" target="_blank">+49-421-200-3178, fax: <a href="tel:%2B49-421-200-493178" value="+49421200493178" target="_blank">+49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: <a href="tel:%2B49-173-5837882" value="+491735837882" target="_blank">+49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Dave McIlhagga-2
In reply to this post by Dave McIlhagga-2
Actually for some historical context here, the reason the rules around establishing PSC were established the way they are, was to ensure that Autodesk’s MapGuide project would not be dominated in the “Benevolent Dictator Model” that may otherwise have occurred.

The PSC could not comprise more than 50% Autodesk employees, and there was no chair who had the deciding vote. Autodesk willing went along with this to ensure transparency and fairness going forward.

Regardless of the outcome, this then provided the project the fair opportunity to grow with the involvement of any individual or company participating without anyone having an unfair advantage in this playing field.

To me this is one of the more fundamental reasons the foundation exists — and compromising on it would be a dangerous precedent for the future.

Dave





On May 21, 2016, at 12:36 PM, Dave McIlhagga <[hidden email]> wrote:

I think OSGeo should stay out of the business of weighing in on how companies decide to commercialize technologies, so long as they respect the licensing terms.

We and many companies use MapServer for instance at the heart of our proprietary stacks, which is simply another flavor of the open core model.

The more important issue here is whether the structure of a project's licensing and PSC provides a commercial advantage to one company over any other.

The concern I have with the benevolent dictator model in discussion here is that reasonable advancements members of the PSC may want to pursue for the project that may erode from the commercial interests of the benevolent dictator could be denied, and that doesn't feel like a fair playing field for the PSC to operate in, or a scenario OSGeo should endorse.

Dave






Sent from mobile

On May 21, 2016, at 8:06 AM, Cameron Shorter <[hidden email]> wrote:

Hi Edzer,
Thank you for raising this topic questioning the value of radsaman community edition. It is pertinent considering recent discussions about Rasdaman incubation.

Peter, your comments about programmers wanting to get paid for their work is valid, but does not provide justification for OSGeo promoting a proprietary business model. OSGeo is in the business of promoting open source software, and helping people who create open source software.

Rasdaman's business model is in a grey zone. It provides a community edition and a proprietary edition. This is often referred to as an "open core" business model, or sometimes less favorably called "crippleware".
I think Rasdaman is the only OSGeo (proposed) project which provides an open core model. All prior projects have been pure open source.

Although a an open core model deviates from OSGeo's original principles, one could argue that Rasdaman community edition stands on its own as a valuable, quality open source geospatial application by itself, worthy of OSGeo promotion.

Edzer's comments appear to counter this argument. Edzer, I understand you suggest Rasdaman community edition is of little value for real world problems?
Extending from this, OSGeo endorsement of Rasdaman should be questioned and potentially withdrawn.

I'd be interested to hear opinions of others in the field as to whether Rasdaman community version is of value for real-world production systems by itself.

A deeper question for the greater OSGeo community is should OSGeo endorse Open Core business models?

Warm regards, Cameron

On 21/05/2016 6:07 pm, Peter Baumann wrote:
oh, just looking at the subject again:

several service providers believe indeed rasdaman community does offer a significant advantage:
- see the download figures on www.rasdaman.org
- concretely, see www.planetserver.eu which is running rasdaman community on - I believe - about 20 TB of Planetary Science data.

-Peter


On 05/21/2016 09:56 AM, Peter Baumann wrote:
Hm, first of all: this is opening a different thread, talking about functionality of rasdaman community. Next, it is based on assumptions - without details (because off topic): conclusions are wrong.

But to respond to the core message emphasized in the first paragraph: I respectfully disagree. In particular, such a position does not benefit the open source community very much as I am trying to explain below.

TL;DR:

You have a strong expertise in Geoinformatics, I know something about Computer Science. This is where we can talk as professors and scientists. Your statement is about economics, industry etc. Having an opinion there (and articulate it) is fair, but in these fields our opinion weighs not more than anyone else's in the street. We should not attempt to attain importance through inapplicable roles.

Let us look at a professor. They have a conveniently high salary which is paid by society, that is: tax payers. Nobody can influence what a professor does and how much return s/he generates for society.

A single open source developer (or a small group, whatever) do not experience this convenience. They have a dream where they invest, they try to not make money for getting richer than a professor ;-) but merely for their economic survival. Some (in particular scientists) enjoy the money rain coming from publicly funded projects (again: the tax payer subsidizes), but most in the community have to struggle hard. They face reluctant customers, competition by the giants in the market, and many more obstacles.

From the cosy place of a lifelong position with a secured salary and decent retirement funds it is easy to say that all software should be free like free beer (quote from below: "can be reproduced by other scientists without prohibitive license costs").

If the open source movement cannibalizes itself it will make it all so easy for the big players to maintain their dominance, they will silently applaud. Quoting Jeroen:
> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A THREAT, THEY ARE A NECESSITY.

That said: It is entirely ok to have the opinion you have. Others, though, may disagree. I am one of those.

respectfully,
Peter



On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
As a scientist, I teach my students that for doing science it is a
requirement to work with open source software, because only then
workflows are fully transparent and can be reproduced by other
scientists without prohibitive license costs. Currently, working
with large amounts of earth observation (EO) or climate model data
typically requires to download these data tile by tile, stitch them
together, and go through all of them. Array databases may simplify
this substantially: after ingesting the tiles, they can directly
work on the whole data as a multi-dimensinal array ("data cube").
Computations on these array are typically embarassingly parallel,
and scale up with the number of cores in a cluster.

Rasdaman is an array data base that comes in two flavours, the open
source community edition (CE) and the commercial enterprise edition
(EE). The differences between the two are clear [1]. When I want to
use rasdaman CE (open source) for scalable image analysis, I get
stuck waiting for one core to finish everything [1]. This is not
going to solve any problems related to computing on large data,
and is not scalable. The bold claim that rasdaman.org opens with
("This worldwide leading array analytics engine distinguishes itself
by its flexibility, performance, and scalability") is not true for
the CE advertised. This has been mentioned in the past on mailing
lists [2,3], but the typical answer from Peter Baumann diverts into
other arguments. Also the benchmark graph (photo from an AGU poster)
[4] that Peter sent this week [5] must refer to the enterprise
edition, since Spark and Hive both scale, but rasdaman CE does not [3].

I assume that on the discussions on this list, ONLY the open
source community edition is considered, compared, and discussed,
as a potential future OSGeo project.

OSGeo supports the needs of the open source geospatial community [6].

Given

 * the bold claims and continuing confusion about whether,
   and which, rasdaman is scalable,
 * the need for OSGeo to give good advice to prospective users
   about technologies that do scale EO data analysis,
 * the current (unfilled!) needs of scientists for good, open source
   software for such analysis, and
 * the potential conflict of interest of its creator [7],

I wonder wether OSGeo should recommend rasdaman CE to the open
source geospatial community.


[1] http://rasdaman.org/wiki/Features
[2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
[3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
[4]
https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
[5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
[6] http://www.osgeo.org/content/faq/foundation_faq.html
[7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)



-- 
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)




_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

-- 
Cameron Shorter,
Software and Data Solutions Manager
LISAsoft
Suite 112, Jones Bay Wharf,
26 - 32 Pirrama Rd, Pyrmont NSW 2009

P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Edzer Pebesma-2
In reply to this post by Peter Baumann
Thanks, Peter, for the interesting opinions, but I see them as further
divergence into other matters, not a discussion of my statement.

I'd love to discuss all issues you mention some time. Did I ever tell
you I owe my professor position to my open source contributions? Are
there others on this list with similar experiences?

On 21/05/16 09:56, Peter Baumann wrote:

> Hm, first of all: this is opening a different thread, talking about
> functionality of rasdaman community. Next, it is based on assumptions -
> without details (because off topic): conclusions are wrong.
>
> But to respond to the core message emphasized in the first paragraph: I
> respectfully disagree. In particular, such a position does not benefit
> the open source community very much as I am trying to explain below.
>
> TL;DR:
>
> You have a strong expertise in Geoinformatics, I know something about
> Computer Science. This is where we can talk as professors and
> scientists. Your statement is about economics, industry etc. Having an
> opinion there (and articulate it) is fair, but in these fields our
> opinion weighs not more than anyone else's in the street. We should not
> attempt to attain importance through inapplicable roles.
>
> Let us look at a professor. They have a conveniently high salary which
> is paid by society, that is: tax payers. Nobody can influence what a
> professor does and how much return s/he generates for society.
>
> A single open source developer (or a small group, whatever) do not
> experience this convenience. They have a dream where they invest, they
> try to not make money for getting richer than a professor ;-) but merely
> for their economic survival. Some (in particular scientists) enjoy the
> money rain coming from publicly funded projects (again: the tax payer
> subsidizes), but most in the community have to struggle hard. They face
> reluctant customers, competition by the giants in the market, and many
> more obstacles.
>
> From the cosy place of a lifelong position with a secured salary and
> decent retirement funds it is easy to say that all software should be
> free like free beer (quote from below: "can be reproduced by other
> scientists without prohibitive license costs").
>
> If the open source movement cannibalizes itself it will make it all so
> easy for the big players to maintain their dominance, they will silently
> applaud. Quoting Jeroen:
>> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
> THREAT, THEY ARE A NECESSITY.
>
> That said: It is entirely ok to have the opinion you have. Others,
> though, may disagree. I am one of those.
>
> respectfully,
> Peter
>
>
>
> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
>> As a scientist, I teach my students that for doing science it is a
>> requirement to work with open source software, because only then
>> workflows are fully transparent and can be reproduced by other
>> scientists without prohibitive license costs. Currently, working
>> with large amounts of earth observation (EO) or climate model data
>> typically requires to download these data tile by tile, stitch them
>> together, and go through all of them. Array databases may simplify
>> this substantially: after ingesting the tiles, they can directly
>> work on the whole data as a multi-dimensinal array ("data cube").
>> Computations on these array are typically embarassingly parallel,
>> and scale up with the number of cores in a cluster.
>>
>> Rasdaman is an array data base that comes in two flavours, the open
>> source community edition (CE) and the commercial enterprise edition
>> (EE). The differences between the two are clear [1]. When I want to
>> use rasdaman CE (open source) for scalable image analysis, I get
>> stuck waiting for one core to finish everything [1]. This is not
>> going to solve any problems related to computing on large data,
>> and is not scalable. The bold claim that rasdaman.org opens with
>> ("This worldwide leading array analytics engine distinguishes itself
>> by its flexibility, performance, and scalability") is not true for
>> the CE advertised. This has been mentioned in the past on mailing
>> lists [2,3], but the typical answer from Peter Baumann diverts into
>> other arguments. Also the benchmark graph (photo from an AGU poster)
>> [4] that Peter sent this week [5] must refer to the enterprise
>> edition, since Spark and Hive both scale, but rasdaman CE does not [3].
>>
>> I assume that on the discussions on this list, ONLY the open
>> source community edition is considered, compared, and discussed,
>> as a potential future OSGeo project.
>>
>> OSGeo supports the needs of the open source geospatial community [6].
>>
>> Given
>>
>>  * the bold claims and continuing confusion about whether,
>>    and which, rasdaman is scalable,
>>  * the need for OSGeo to give good advice to prospective users
>>    about technologies that do scale EO data analysis,
>>  * the current (unfilled!) needs of scientists for good, open source
>>    software for such analysis, and
>>  * the potential conflict of interest of its creator [7],
>>
>> I wonder wether OSGeo should recommend rasdaman CE to the open
>> source geospatial community.
>>
>>
>> [1] http://rasdaman.org/wiki/Features
>> [2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
>> [3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
>> [4]
>> https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
>>
>>
>> _______________________________________________
>> Discuss mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/discuss
>
> --
> Dr. Peter Baumann
>  - Professor of Computer Science, Jacobs University Bremen
>    www.faculty.jacobs-university.de/pbaumann
>    mail: [hidden email]
>    tel: +49-421-200-3178, fax: +49-421-200-493178
>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>    www.rasdaman.com, mail: [hidden email]
>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>
>
--
Edzer Pebesma
Institute for Geoinformatics  (ifgi),  University of Münster
Heisenbergstraße 2, 48149 Münster, Germany; +49 251 83 33081
Journal of Statistical Software:   http://www.jstatsoft.org/
Computers & Geosciences:   http://elsevier.com/locate/cageo/
Spatial Statistics Society http://www.spatialstatistics.info


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Edzer Pebesma-2
In reply to this post by Cameron Shorter
Cameron, my comment is in-line:

On 21/05/16 14:06, Cameron Shorter wrote:

> Hi Edzer,
> Thank you for raising this topic questioning the value of radsaman
> community edition. It is pertinent considering recent discussions about
> Rasdaman incubation.
>
> Peter, your comments about programmers wanting to get paid for their
> work is valid, but does not provide justification for OSGeo promoting a
> proprietary business model. OSGeo is in the business of promoting open
> source software, and helping people who create open source software.
>
> Rasdaman's business model is in a grey zone. It provides a community
> edition and a proprietary edition. This is often referred to as an "open
> core" business model, or sometimes less favorably called "crippleware".
> I think Rasdaman is the only OSGeo (proposed) project which provides an
> open core model. All prior projects have been pure open source.
>
> Although a an open core model deviates from OSGeo's original principles,
> one could argue that Rasdaman community edition stands on its own as a
> valuable, quality open source geospatial application by itself, worthy
> of OSGeo promotion.
>
> Edzer's comments appear to counter this argument. Edzer, I understand
> you suggest Rasdaman community edition is of little value for real world
> problems?
Yes; the example Peter mentions, http://planetserver.eu/ does solve a
problem: providing a download service for large scale imagery, and it is
good if rasdaman CE can do this for 20 Tb, with sufficient performance.
There are however several other open source technologies that can also
do this, potentially much simpler (e.g. thredds data server, maybe even
gdal VRT).

The problems where array data management systems really come into play
is scalable computing: doing something useful with large data sets
without having to download all the data first ("bringing the
computations to the data, instead of data to the computations") - this
is what much of the big data scalability fanfare is about, not about
serving or downloading data. And this is where rasdaman CE does not
scale - only EE seems to do so.

The longer we (the open source community) wait with a good solution to
this problem, the more ground we loose to Google Earth Engine, a very
nice service that is bad for science. See also the last slide ("Google
Earth Engine is Evil") in the excellent presentation by Jordi Inglada,
held last week at the Living Planet Symposium:

http://jordiinglada.net/stok/LivingPlanet/LandCoverSlides.pdf

You might see this as a call to arms, and it is, but a new thread will
be needed to follow up on this.

> Extending from this, OSGeo endorsement of Rasdaman should be questioned
> and potentially withdrawn.
>
> I'd be interested to hear opinions of others in the field as to whether
> Rasdaman community version is of value for real-world production systems
> by itself.
>
> A deeper question for the greater OSGeo community is should OSGeo
> endorse Open Core business models?
>
> Warm regards, Cameron
>
> On 21/05/2016 6:07 pm, Peter Baumann wrote:
>> oh, just looking at the subject again:
>>
>> several service providers believe indeed rasdaman community does offer
>> a significant advantage:
>> - see the download figures on www.rasdaman.org
>> - concretely, see www.planetserver.eu which is running rasdaman
>> community on - I believe - about 20 TB of Planetary Science data.
>>
>> -Peter
>>
>>
>> On 05/21/2016 09:56 AM, Peter Baumann wrote:
>>> Hm, first of all: this is opening a different thread, talking about
>>> functionality of rasdaman community. Next, it is based on assumptions
>>> - without details (because off topic): conclusions are wrong.
>>>
>>> But to respond to the core message emphasized in the first paragraph:
>>> I respectfully disagree. In particular, such a position does not
>>> benefit the open source community very much as I am trying to explain
>>> below.
>>>
>>> TL;DR:
>>>
>>> You have a strong expertise in Geoinformatics, I know something about
>>> Computer Science. This is where we can talk as professors and
>>> scientists. Your statement is about economics, industry etc. Having
>>> an opinion there (and articulate it) is fair, but in these fields our
>>> opinion weighs not more than anyone else's in the street. We should
>>> not attempt to attain importance through inapplicable roles.
>>>
>>> Let us look at a professor. They have a conveniently high salary
>>> which is paid by society, that is: tax payers. Nobody can influence
>>> what a professor does and how much return s/he generates for society.
>>>
>>> A single open source developer (or a small group, whatever) do not
>>> experience this convenience. They have a dream where they invest,
>>> they try to not make money for getting richer than a professor ;-)
>>> but merely for their economic survival. Some (in particular
>>> scientists) enjoy the money rain coming from publicly funded projects
>>> (again: the tax payer subsidizes), but most in the community have to
>>> struggle hard. They face reluctant customers, competition by the
>>> giants in the market, and many more obstacles.
>>>
>>> From the cosy place of a lifelong position with a secured salary and
>>> decent retirement funds it is easy to say that all software should be
>>> free like free beer (quote from below: "can be reproduced by other
>>> scientists without prohibitive license costs").
>>>
>>> If the open source movement cannibalizes itself it will make it all
>>> so easy for the big players to maintain their dominance, they will
>>> silently applaud. Quoting Jeroen:
>>> > NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
>>> THREAT, THEY ARE A NECESSITY.
>>>
>>> That said: It is entirely ok to have the opinion you have. Others,
>>> though, may disagree. I am one of those.
>>>
>>> respectfully,
>>> Peter
>>>
>>>
>>>
>>> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
>>>> As a scientist, I teach my students that for doing science it is a
>>>> requirement to work with open source software, because only then
>>>> workflows are fully transparent and can be reproduced by other
>>>> scientists without prohibitive license costs. Currently, working
>>>> with large amounts of earth observation (EO) or climate model data
>>>> typically requires to download these data tile by tile, stitch them
>>>> together, and go through all of them. Array databases may simplify
>>>> this substantially: after ingesting the tiles, they can directly
>>>> work on the whole data as a multi-dimensinal array ("data cube").
>>>> Computations on these array are typically embarassingly parallel,
>>>> and scale up with the number of cores in a cluster.
>>>>
>>>> Rasdaman is an array data base that comes in two flavours, the open
>>>> source community edition (CE) and the commercial enterprise edition
>>>> (EE). The differences between the two are clear [1]. When I want to
>>>> use rasdaman CE (open source) for scalable image analysis, I get
>>>> stuck waiting for one core to finish everything [1]. This is not
>>>> going to solve any problems related to computing on large data,
>>>> and is not scalable. The bold claim that rasdaman.org opens with
>>>> ("This worldwide leading array analytics engine distinguishes itself
>>>> by its flexibility, performance, and scalability") is not true for
>>>> the CE advertised. This has been mentioned in the past on mailing
>>>> lists [2,3], but the typical answer from Peter Baumann diverts into
>>>> other arguments. Also the benchmark graph (photo from an AGU poster)
>>>> [4] that Peter sent this week [5] must refer to the enterprise
>>>> edition, since Spark and Hive both scale, but rasdaman CE does not [3].
>>>>
>>>> I assume that on the discussions on this list, ONLY the open
>>>> source community edition is considered, compared, and discussed,
>>>> as a potential future OSGeo project.
>>>>
>>>> OSGeo supports the needs of the open source geospatial community [6].
>>>>
>>>> Given
>>>>
>>>>  * the bold claims and continuing confusion about whether,
>>>>    and which, rasdaman is scalable,
>>>>  * the need for OSGeo to give good advice to prospective users
>>>>    about technologies that do scale EO data analysis,
>>>>  * the current (unfilled!) needs of scientists for good, open source
>>>>    software for such analysis, and
>>>>  * the potential conflict of interest of its creator [7],
>>>>
>>>> I wonder wether OSGeo should recommend rasdaman CE to the open
>>>> source geospatial community.
>>>>
>>>>
>>>> [1] http://rasdaman.org/wiki/Features
>>>> [2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
>>>> [3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
>>>> [4]
>>>> https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
>>>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
>>>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
>>>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
>>>>
>>>>
>>>> _______________________________________________
>>>> Discuss mailing list
>>>> [hidden email]
>>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>>
>>> --
>>> Dr. Peter Baumann
>>>  - Professor of Computer Science, Jacobs University Bremen
>>>    www.faculty.jacobs-university.de/pbaumann
>>>    mail: [hidden email]
>>>    tel: +49-421-200-3178, fax: +49-421-200-493178
>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>    www.rasdaman.com, mail: [hidden email]
>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>>
>>>
>>
>> --
>> Dr. Peter Baumann
>>  - Professor of Computer Science, Jacobs University Bremen
>>    www.faculty.jacobs-university.de/pbaumann
>>    mail: [hidden email]
>>    tel: +49-421-200-3178, fax: +49-421-200-493178
>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>    www.rasdaman.com, mail: [hidden email]
>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>
>>
>>
>>
>> _______________________________________________
>> Discuss mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/discuss
>
> --
> Cameron Shorter,
> Software and Data Solutions Manager
> LISAsoft
> Suite 112, Jones Bay Wharf,
> 26 - 32 Pirrama Rd, Pyrmont NSW 2009
>
> P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
>
--
Edzer Pebesma
Institute for Geoinformatics  (ifgi),  University of Münster
Heisenbergstraße 2, 48149 Münster, Germany; +49 251 83 33081
Journal of Statistical Software:   http://www.jstatsoft.org/
Computers & Geosciences:   http://elsevier.com/locate/cageo/
Spatial Statistics Society http://www.spatialstatistics.info


_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss

signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Does rasdaman CE solve an open source geospatial problem?

Bruce Bannerman-3
In reply to this post by Edzer Pebesma-2
Hi Edzer,


Thank you for your timely email.

I agree that the functionality of the community version has been constrained and have noted your comments below on aa earlier thread on the Rasdaman lists that proceeded down a similar line with direct responses to issues avoided through diverting the discussion around the nuances of the query used.

I have also noted that Rasdaman Gmbh has been gradually reducing the difference between the two versions by porting some functionality across to the Community version, just not the performance aspects that matter.


I tend to be quite pragmatic on these issues. The approach that I took was that we need to focus on getting open project governance with many parties involved as per [b1].


I considered that once we got a wider representation and a truely open project we could begin to address these types of issues within an appropriate framework.

In part, this is why I deliberately chose the wording at [2], in order to expose and to force the issue.


This approach has clearly not worked and I cannot see the governance issue resolved to the benefit of a truely open community as thinks currently stand.


On reflecting through the past archives of the Incubation list relating to Rasdaman Incubation, I see that these have been issues right from the beginning.


We should not dictate to a commercial organisation how they run their business. They will need to make decisions that are appropriate for their future growth.


IMO, I see an organisation that is trying to find a viable way to make a living from open source. They have chosen this approach. Many organisations have this issue. Other organisations take alternate approaches.


It is time to call this for what it is and pull the plug on incubation. Perhaps market forces will dictate an alternate approach for Rasdaman, but I doubt it.



After (sadly) watching how this has played out and the intransigence adopted, I’m personally treating any further investment of my time into Rasdaman in its current structure to be a very poor investment. Though this could quickly change if the governance issue is addressed constructively.


I have therefore shelved any plans that I have for use of Rasdaman. I’d say that Thredds and PyWPS will meet my immediate needs.


I agree with you that this is a lost opportunity. The project has so much potential.

As has been discussed recently on this list, there is always the option to fork Rasdaman, but I do not think that this is something to be undertaken lightly.


Bruce

[b1] http://lists.osgeo.org/pipermail/incubator/2016-May/003050.html

[b2] http://lists.osgeo.org/pipermail/incubator/2016-April/002996.html

 



On 22/05/2016, 23:23, "Discuss on behalf of Edzer Pebesma"
<[hidden email] on behalf of
[hidden email]> wrote:

>Cameron, my comment is in-line:
>
>On 21/05/16 14:06, Cameron Shorter wrote:
>> Hi Edzer,
>> Thank you for raising this topic questioning the value of radsaman
>> community edition. It is pertinent considering recent discussions about
>> Rasdaman incubation.
>>
>> Peter, your comments about programmers wanting to get paid for their
>> work is valid, but does not provide justification for OSGeo promoting a
>> proprietary business model. OSGeo is in the business of promoting open
>> source software, and helping people who create open source software.
>>
>> Rasdaman's business model is in a grey zone. It provides a community
>> edition and a proprietary edition. This is often referred to as an "open
>> core" business model, or sometimes less favorably called "crippleware".
>> I think Rasdaman is the only OSGeo (proposed) project which provides an
>> open core model. All prior projects have been pure open source.
>>
>> Although a an open core model deviates from OSGeo's original principles,
>> one could argue that Rasdaman community edition stands on its own as a
>> valuable, quality open source geospatial application by itself, worthy
>> of OSGeo promotion.
>>
>> Edzer's comments appear to counter this argument. Edzer, I understand
>> you suggest Rasdaman community edition is of little value for real world
>> problems?
>
>Yes; the example Peter mentions, http://planetserver.eu/ does solve a
>problem: providing a download service for large scale imagery, and it is
>good if rasdaman CE can do this for 20 Tb, with sufficient performance.
>There are however several other open source technologies that can also
>do this, potentially much simpler (e.g. thredds data server, maybe even
>gdal VRT).
>
>The problems where array data management systems really come into play
>is scalable computing: doing something useful with large data sets
>without having to download all the data first ("bringing the
>computations to the data, instead of data to the computations") - this
>is what much of the big data scalability fanfare is about, not about
>serving or downloading data. And this is where rasdaman CE does not
>scale - only EE seems to do so.
>
>The longer we (the open source community) wait with a good solution to
>this problem, the more ground we loose to Google Earth Engine, a very
>nice service that is bad for science. See also the last slide ("Google
>Earth Engine is Evil") in the excellent presentation by Jordi Inglada,
>held last week at the Living Planet Symposium:
>
>http://jordiinglada.net/stok/LivingPlanet/LandCoverSlides.pdf
>
>You might see this as a call to arms, and it is, but a new thread will
>be needed to follow up on this.
>
>> Extending from this, OSGeo endorsement of Rasdaman should be questioned
>> and potentially withdrawn.
>>
>> I'd be interested to hear opinions of others in the field as to whether
>> Rasdaman community version is of value for real-world production systems
>> by itself.
>>
>> A deeper question for the greater OSGeo community is should OSGeo
>> endorse Open Core business models?
>>
>> Warm regards, Cameron
>>
>> On 21/05/2016 6:07 pm, Peter Baumann wrote:
>>> oh, just looking at the subject again:
>>>
>>> several service providers believe indeed rasdaman community does offer
>>> a significant advantage:
>>> - see the download figures on www.rasdaman.org
>>> - concretely, see www.planetserver.eu which is running rasdaman
>>> community on - I believe - about 20 TB of Planetary Science data.
>>>
>>> -Peter
>>>
>>>
>>> On 05/21/2016 09:56 AM, Peter Baumann wrote:
>>>> Hm, first of all: this is opening a different thread, talking about
>>>> functionality of rasdaman community. Next, it is based on assumptions
>>>> - without details (because off topic): conclusions are wrong.
>>>>
>>>> But to respond to the core message emphasized in the first paragraph:
>>>> I respectfully disagree. In particular, such a position does not
>>>> benefit the open source community very much as I am trying to explain
>>>> below.
>>>>
>>>> TL;DR:
>>>>
>>>> You have a strong expertise in Geoinformatics, I know something about
>>>> Computer Science. This is where we can talk as professors and
>>>> scientists. Your statement is about economics, industry etc. Having
>>>> an opinion there (and articulate it) is fair, but in these fields our
>>>> opinion weighs not more than anyone else's in the street. We should
>>>> not attempt to attain importance through inapplicable roles.
>>>>
>>>> Let us look at a professor. They have a conveniently high salary
>>>> which is paid by society, that is: tax payers. Nobody can influence
>>>> what a professor does and how much return s/he generates for society.
>>>>
>>>> A single open source developer (or a small group, whatever) do not
>>>> experience this convenience. They have a dream where they invest,
>>>> they try to not make money for getting richer than a professor ;-)
>>>> but merely for their economic survival. Some (in particular
>>>> scientists) enjoy the money rain coming from publicly funded projects
>>>> (again: the tax payer subsidizes), but most in the community have to
>>>> struggle hard. They face reluctant customers, competition by the
>>>> giants in the market, and many more obstacles.
>>>>
>>>> From the cosy place of a lifelong position with a secured salary and
>>>> decent retirement funds it is easy to say that all software should be
>>>> free like free beer (quote from below: "can be reproduced by other
>>>> scientists without prohibitive license costs").
>>>>
>>>> If the open source movement cannibalizes itself it will make it all
>>>> so easy for the big players to maintain their dominance, they will
>>>> silently applaud. Quoting Jeroen:
>>>> > NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
>>>> THREAT, THEY ARE A NECESSITY.
>>>>
>>>> That said: It is entirely ok to have the opinion you have. Others,
>>>> though, may disagree. I am one of those.
>>>>
>>>> respectfully,
>>>> Peter
>>>>
>>>>
>>>>
>>>> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
>>>>> As a scientist, I teach my students that for doing science it is a
>>>>> requirement to work with open source software, because only then
>>>>> workflows are fully transparent and can be reproduced by other
>>>>> scientists without prohibitive license costs. Currently, working
>>>>> with large amounts of earth observation (EO) or climate model data
>>>>> typically requires to download these data tile by tile, stitch them
>>>>> together, and go through all of them. Array databases may simplify
>>>>> this substantially: after ingesting the tiles, they can directly
>>>>> work on the whole data as a multi-dimensinal array ("data cube").
>>>>> Computations on these array are typically embarassingly parallel,
>>>>> and scale up with the number of cores in a cluster.
>>>>>
>>>>> Rasdaman is an array data base that comes in two flavours, the open
>>>>> source community edition (CE) and the commercial enterprise edition
>>>>> (EE). The differences between the two are clear [1]. When I want to
>>>>> use rasdaman CE (open source) for scalable image analysis, I get
>>>>> stuck waiting for one core to finish everything [1]. This is not
>>>>> going to solve any problems related to computing on large data,
>>>>> and is not scalable. The bold claim that rasdaman.org opens with
>>>>> ("This worldwide leading array analytics engine distinguishes itself
>>>>> by its flexibility, performance, and scalability") is not true for
>>>>> the CE advertised. This has been mentioned in the past on mailing
>>>>> lists [2,3], but the typical answer from Peter Baumann diverts into
>>>>> other arguments. Also the benchmark graph (photo from an AGU poster)
>>>>> [4] that Peter sent this week [5] must refer to the enterprise
>>>>> edition, since Spark and Hive both scale, but rasdaman CE does not
>>>>>[3].
>>>>>
>>>>> I assume that on the discussions on this list, ONLY the open
>>>>> source community edition is considered, compared, and discussed,
>>>>> as a potential future OSGeo project.
>>>>>
>>>>> OSGeo supports the needs of the open source geospatial community [6].
>>>>>
>>>>> Given
>>>>>
>>>>>  * the bold claims and continuing confusion about whether,
>>>>>    and which, rasdaman is scalable,
>>>>>  * the need for OSGeo to give good advice to prospective users
>>>>>    about technologies that do scale EO data analysis,
>>>>>  * the current (unfilled!) needs of scientists for good, open source
>>>>>    software for such analysis, and
>>>>>  * the potential conflict of interest of its creator [7],
>>>>>
>>>>> I wonder wether OSGeo should recommend rasdaman CE to the open
>>>>> source geospatial community.
>>>>>
>>>>>
>>>>> [1] http://rasdaman.org/wiki/Features
>>>>> [2]
>>>>>https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
>>>>> [3]
>>>>>https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
>>>>> [4]
>>>>>
>>>>>https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd
>>>>>4/attachment.jpg
>>>>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
>>>>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
>>>>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Discuss mailing list
>>>>> [hidden email]
>>>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>>>
>>>> --
>>>> Dr. Peter Baumann
>>>>  - Professor of Computer Science, Jacobs University Bremen
>>>>    www.faculty.jacobs-university.de/pbaumann
>>>>    mail: [hidden email]
>>>>    tel: <a href="tel:%2B49-421-200-3178" value="+494212003178">+49-421-200-3178, fax: <a href="tel:%2B49-421-200-493178" value="+49421200493178">+49-421-200-493178
>>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>>    www.rasdaman.com, mail: [hidden email]
>>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: <a href="tel:%2B49-173-5837882" value="+491735837882">+49-173-5837882
>>>> "Si forte in alienas manus oberraverit hec peregrina epistola
>>>>incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur
>>>>cui soli destinata, nec preripiat quisquam non sibi parata." (mail
>>>>disclaimer, AD 1083)
>>>>
>>>>
>>>
>>> --
>>> Dr. Peter Baumann
>>>  - Professor of Computer Science, Jacobs University Bremen
>>>    www.faculty.jacobs-university.de/pbaumann
>>>    mail: [hidden email]
>>>    tel: <a href="tel:%2B49-421-200-3178" value="+494212003178">+49-421-200-3178, fax: <a href="tel:%2B49-421-200-493178" value="+49421200493178">+49-421-200-493178
>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>    www.rasdaman.com, mail: [hidden email]
>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: <a href="tel:%2B49-173-5837882" value="+491735837882">+49-173-5837882
>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
>>>ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
>>>destinata, nec preripiat quisquam non sibi parata." (mail disclaimer,
>>>AD 1083)
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Discuss mailing list
>>> [hidden email]
>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>
>> --
>> Cameron Shorter,
>> Software and Data Solutions Manager
>> LISAsoft
>> Suite 112, Jones Bay Wharf,
>> 26 - 32 Pirrama Rd, Pyrmont NSW 2009
>>
>> P <a href="tel:%2B61%202%209009%205000" value="+61290095000">+61 2 9009 5000,  W www.lisasoft.com,  F <a href="tel:%2B61%202%209009%205099" value="+61290095099">+61 2 9009 5099
>>
>
>--
>Edzer Pebesma
>Institute for Geoinformatics  (ifgi),  University of Münster
>Heisenbergstraße 2, 48149 Münster, Germany; <a href="tel:%2B49%20251%2083%2033081" value="+492518333081">+49 251 83 33081
>Journal of Statistical Software:   http://www.jstatsoft.org/
>Computers & Geosciences:   http://elsevier.com/locate/cageo/
>Spatial Statistics Society http://www.spatialstatistics.info
>



_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

project bashing -- was: Re: Does rasdaman CE solve an open source geospatial problem?

Peter Baumann
In reply to this post by Edzer Pebesma-2
Wow. Now that OSGeo is going to evict our project, suddenly a bashing tornado
starts from the fundamentalist angle: "they don't obey, therefore we proclaim
their technology is bad. Burn their project!"

A technically oriented discussion, unbiased and out of sheer interest, I love to
conduct. But this thread is neither unbiased nor driven by genuine (technical)
interest, rather intending to damage an open-source project with hundreds of
expert person months invested and spinoff contributions to several other
open-source projects (such as GDAL, MapServer, OpenSSL). Appreciated by
large-scale data centers.

Based on a surprisingly superficial understanding of array databases in some
contributions, based on unproven assumptions, not taking time to read (nor
understand) the manifold publications in the community explaining details. Based
on a fundamentalist bias against business.

My 15-hours days are calling on me with other duties. So we continue awaiting
faithfully the final OSGeo verdict on incubation, whatever it is after 6 years.

-Peter


On 05/22/2016 03:23 PM, Edzer Pebesma wrote:

> Cameron, my comment is in-line:
>
> On 21/05/16 14:06, Cameron Shorter wrote:
>> Hi Edzer,
>> Thank you for raising this topic questioning the value of radsaman
>> community edition. It is pertinent considering recent discussions about
>> Rasdaman incubation.
>>
>> Peter, your comments about programmers wanting to get paid for their
>> work is valid, but does not provide justification for OSGeo promoting a
>> proprietary business model. OSGeo is in the business of promoting open
>> source software, and helping people who create open source software.
>>
>> Rasdaman's business model is in a grey zone. It provides a community
>> edition and a proprietary edition. This is often referred to as an "open
>> core" business model, or sometimes less favorably called "crippleware".
>> I think Rasdaman is the only OSGeo (proposed) project which provides an
>> open core model. All prior projects have been pure open source.
>>
>> Although a an open core model deviates from OSGeo's original principles,
>> one could argue that Rasdaman community edition stands on its own as a
>> valuable, quality open source geospatial application by itself, worthy
>> of OSGeo promotion.
>>
>> Edzer's comments appear to counter this argument. Edzer, I understand
>> you suggest Rasdaman community edition is of little value for real world
>> problems?
> Yes; the example Peter mentions, http://planetserver.eu/ does solve a
> problem: providing a download service for large scale imagery, and it is
> good if rasdaman CE can do this for 20 Tb, with sufficient performance.
> There are however several other open source technologies that can also
> do this, potentially much simpler (e.g. thredds data server, maybe even
> gdal VRT).
>
> The problems where array data management systems really come into play
> is scalable computing: doing something useful with large data sets
> without having to download all the data first ("bringing the
> computations to the data, instead of data to the computations") - this
> is what much of the big data scalability fanfare is about, not about
> serving or downloading data. And this is where rasdaman CE does not
> scale - only EE seems to do so.
>
> The longer we (the open source community) wait with a good solution to
> this problem, the more ground we loose to Google Earth Engine, a very
> nice service that is bad for science. See also the last slide ("Google
> Earth Engine is Evil") in the excellent presentation by Jordi Inglada,
> held last week at the Living Planet Symposium:
>
> http://jordiinglada.net/stok/LivingPlanet/LandCoverSlides.pdf
>
> You might see this as a call to arms, and it is, but a new thread will
> be needed to follow up on this.
>
>> Extending from this, OSGeo endorsement of Rasdaman should be questioned
>> and potentially withdrawn.
>>
>> I'd be interested to hear opinions of others in the field as to whether
>> Rasdaman community version is of value for real-world production systems
>> by itself.
>>
>> A deeper question for the greater OSGeo community is should OSGeo
>> endorse Open Core business models?
>>
>> Warm regards, Cameron
>>
>> On 21/05/2016 6:07 pm, Peter Baumann wrote:
>>> oh, just looking at the subject again:
>>>
>>> several service providers believe indeed rasdaman community does offer
>>> a significant advantage:
>>> - see the download figures on www.rasdaman.org
>>> - concretely, see www.planetserver.eu which is running rasdaman
>>> community on - I believe - about 20 TB of Planetary Science data.
>>>
>>> -Peter
>>>
>>>
>>> On 05/21/2016 09:56 AM, Peter Baumann wrote:
>>>> Hm, first of all: this is opening a different thread, talking about
>>>> functionality of rasdaman community. Next, it is based on assumptions
>>>> - without details (because off topic): conclusions are wrong.
>>>>
>>>> But to respond to the core message emphasized in the first paragraph:
>>>> I respectfully disagree. In particular, such a position does not
>>>> benefit the open source community very much as I am trying to explain
>>>> below.
>>>>
>>>> TL;DR:
>>>>
>>>> You have a strong expertise in Geoinformatics, I know something about
>>>> Computer Science. This is where we can talk as professors and
>>>> scientists. Your statement is about economics, industry etc. Having
>>>> an opinion there (and articulate it) is fair, but in these fields our
>>>> opinion weighs not more than anyone else's in the street. We should
>>>> not attempt to attain importance through inapplicable roles.
>>>>
>>>> Let us look at a professor. They have a conveniently high salary
>>>> which is paid by society, that is: tax payers. Nobody can influence
>>>> what a professor does and how much return s/he generates for society.
>>>>
>>>> A single open source developer (or a small group, whatever) do not
>>>> experience this convenience. They have a dream where they invest,
>>>> they try to not make money for getting richer than a professor ;-)
>>>> but merely for their economic survival. Some (in particular
>>>> scientists) enjoy the money rain coming from publicly funded projects
>>>> (again: the tax payer subsidizes), but most in the community have to
>>>> struggle hard. They face reluctant customers, competition by the
>>>> giants in the market, and many more obstacles.
>>>>
>>>> From the cosy place of a lifelong position with a secured salary and
>>>> decent retirement funds it is easy to say that all software should be
>>>> free like free beer (quote from below: "can be reproduced by other
>>>> scientists without prohibitive license costs").
>>>>
>>>> If the open source movement cannibalizes itself it will make it all
>>>> so easy for the big players to maintain their dominance, they will
>>>> silently applaud. Quoting Jeroen:
>>>>> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
>>>> THREAT, THEY ARE A NECESSITY.
>>>>
>>>> That said: It is entirely ok to have the opinion you have. Others,
>>>> though, may disagree. I am one of those.
>>>>
>>>> respectfully,
>>>> Peter
>>>>
>>>>
>>>>
>>>> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
>>>>> As a scientist, I teach my students that for doing science it is a
>>>>> requirement to work with open source software, because only then
>>>>> workflows are fully transparent and can be reproduced by other
>>>>> scientists without prohibitive license costs. Currently, working
>>>>> with large amounts of earth observation (EO) or climate model data
>>>>> typically requires to download these data tile by tile, stitch them
>>>>> together, and go through all of them. Array databases may simplify
>>>>> this substantially: after ingesting the tiles, they can directly
>>>>> work on the whole data as a multi-dimensinal array ("data cube").
>>>>> Computations on these array are typically embarassingly parallel,
>>>>> and scale up with the number of cores in a cluster.
>>>>>
>>>>> Rasdaman is an array data base that comes in two flavours, the open
>>>>> source community edition (CE) and the commercial enterprise edition
>>>>> (EE). The differences between the two are clear [1]. When I want to
>>>>> use rasdaman CE (open source) for scalable image analysis, I get
>>>>> stuck waiting for one core to finish everything [1]. This is not
>>>>> going to solve any problems related to computing on large data,
>>>>> and is not scalable. The bold claim that rasdaman.org opens with
>>>>> ("This worldwide leading array analytics engine distinguishes itself
>>>>> by its flexibility, performance, and scalability") is not true for
>>>>> the CE advertised. This has been mentioned in the past on mailing
>>>>> lists [2,3], but the typical answer from Peter Baumann diverts into
>>>>> other arguments. Also the benchmark graph (photo from an AGU poster)
>>>>> [4] that Peter sent this week [5] must refer to the enterprise
>>>>> edition, since Spark and Hive both scale, but rasdaman CE does not [3].
>>>>>
>>>>> I assume that on the discussions on this list, ONLY the open
>>>>> source community edition is considered, compared, and discussed,
>>>>> as a potential future OSGeo project.
>>>>>
>>>>> OSGeo supports the needs of the open source geospatial community [6].
>>>>>
>>>>> Given
>>>>>
>>>>>  * the bold claims and continuing confusion about whether,
>>>>>    and which, rasdaman is scalable,
>>>>>  * the need for OSGeo to give good advice to prospective users
>>>>>    about technologies that do scale EO data analysis,
>>>>>  * the current (unfilled!) needs of scientists for good, open source
>>>>>    software for such analysis, and
>>>>>  * the potential conflict of interest of its creator [7],
>>>>>
>>>>> I wonder wether OSGeo should recommend rasdaman CE to the open
>>>>> source geospatial community.
>>>>>
>>>>>
>>>>> [1] http://rasdaman.org/wiki/Features
>>>>> [2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
>>>>> [3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
>>>>> [4]
>>>>> https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
>>>>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
>>>>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
>>>>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Discuss mailing list
>>>>> [hidden email]
>>>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>>> --
>>>> Dr. Peter Baumann
>>>>  - Professor of Computer Science, Jacobs University Bremen
>>>>    www.faculty.jacobs-university.de/pbaumann
>>>>    mail: [hidden email]
>>>>    tel: +49-421-200-3178, fax: +49-421-200-493178
>>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>>    www.rasdaman.com, mail: [hidden email]
>>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>>>
>>>>
>>> --
>>> Dr. Peter Baumann
>>>  - Professor of Computer Science, Jacobs University Bremen
>>>    www.faculty.jacobs-university.de/pbaumann
>>>    mail: [hidden email]
>>>    tel: +49-421-200-3178, fax: +49-421-200-493178
>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>    www.rasdaman.com, mail: [hidden email]
>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Discuss mailing list
>>> [hidden email]
>>> http://lists.osgeo.org/mailman/listinfo/discuss
>> --
>> Cameron Shorter,
>> Software and Data Solutions Manager
>> LISAsoft
>> Suite 112, Jones Bay Wharf,
>> 26 - 32 Pirrama Rd, Pyrmont NSW 2009
>>
>> P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
>>

--
Dr. Peter Baumann
 - Professor of Computer Science, Jacobs University Bremen
   www.faculty.jacobs-university.de/pbaumann
   mail: [hidden email]
   tel: +49-421-200-3178, fax: +49-421-200-493178
 - Executive Director, rasdaman GmbH Bremen (HRB 26793)
   www.rasdaman.com, mail: [hidden email]
   tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
"Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis
dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec
preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)




_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss
Reply | Threaded
Open this post in threaded view
|

Re: project bashing -- was: Re: Does rasdaman CE solve an open source geospatial problem?

Cameron Shorter
Hi Peter,
I hear that you sound offended. I'm sorry about that.

The business model being proposed by Rasdaman is different to all OSGeo
projects before, and is resulting in the OSGeo community questioning the
definition of what OSGeo supports.

I haven't heard anyone question the technical merit of Rasdaman
enterprise. I haven't heard anyone question your right to license
Rasdaman any way you wish.

People have been debating whether supporting Rasdaman's business model
fits with OSGeo's principles. The general advise I'm hearing is that
Rasdaman is not. These principles have been developed and refined by
many people who have led successful open source projects, and are in
place to ensure OSGeo projects have the best chance of being successful.

Advise you have received is provided as constructive criticism. I
appreciate that it is not advise you want to year, however I'm hopeful
that you consider and apply it.

Warm regards,
Cameron

On 24/05/2016 1:09 am, Peter Baumann wrote:

> Wow. Now that OSGeo is going to evict our project, suddenly a bashing tornado
> starts from the fundamentalist angle: "they don't obey, therefore we proclaim
> their technology is bad. Burn their project!"
>
> A technically oriented discussion, unbiased and out of sheer interest, I love to
> conduct. But this thread is neither unbiased nor driven by genuine (technical)
> interest, rather intending to damage an open-source project with hundreds of
> expert person months invested and spinoff contributions to several other
> open-source projects (such as GDAL, MapServer, OpenSSL). Appreciated by
> large-scale data centers.
>
> Based on a surprisingly superficial understanding of array databases in some
> contributions, based on unproven assumptions, not taking time to read (nor
> understand) the manifold publications in the community explaining details. Based
> on a fundamentalist bias against business.
>
> My 15-hours days are calling on me with other duties. So we continue awaiting
> faithfully the final OSGeo verdict on incubation, whatever it is after 6 years.
>
> -Peter
>
>
> On 05/22/2016 03:23 PM, Edzer Pebesma wrote:
>> Cameron, my comment is in-line:
>>
>> On 21/05/16 14:06, Cameron Shorter wrote:
>>> Hi Edzer,
>>> Thank you for raising this topic questioning the value of radsaman
>>> community edition. It is pertinent considering recent discussions about
>>> Rasdaman incubation.
>>>
>>> Peter, your comments about programmers wanting to get paid for their
>>> work is valid, but does not provide justification for OSGeo promoting a
>>> proprietary business model. OSGeo is in the business of promoting open
>>> source software, and helping people who create open source software.
>>>
>>> Rasdaman's business model is in a grey zone. It provides a community
>>> edition and a proprietary edition. This is often referred to as an "open
>>> core" business model, or sometimes less favorably called "crippleware".
>>> I think Rasdaman is the only OSGeo (proposed) project which provides an
>>> open core model. All prior projects have been pure open source.
>>>
>>> Although a an open core model deviates from OSGeo's original principles,
>>> one could argue that Rasdaman community edition stands on its own as a
>>> valuable, quality open source geospatial application by itself, worthy
>>> of OSGeo promotion.
>>>
>>> Edzer's comments appear to counter this argument. Edzer, I understand
>>> you suggest Rasdaman community edition is of little value for real world
>>> problems?
>> Yes; the example Peter mentions, http://planetserver.eu/ does solve a
>> problem: providing a download service for large scale imagery, and it is
>> good if rasdaman CE can do this for 20 Tb, with sufficient performance.
>> There are however several other open source technologies that can also
>> do this, potentially much simpler (e.g. thredds data server, maybe even
>> gdal VRT).
>>
>> The problems where array data management systems really come into play
>> is scalable computing: doing something useful with large data sets
>> without having to download all the data first ("bringing the
>> computations to the data, instead of data to the computations") - this
>> is what much of the big data scalability fanfare is about, not about
>> serving or downloading data. And this is where rasdaman CE does not
>> scale - only EE seems to do so.
>>
>> The longer we (the open source community) wait with a good solution to
>> this problem, the more ground we loose to Google Earth Engine, a very
>> nice service that is bad for science. See also the last slide ("Google
>> Earth Engine is Evil") in the excellent presentation by Jordi Inglada,
>> held last week at the Living Planet Symposium:
>>
>> http://jordiinglada.net/stok/LivingPlanet/LandCoverSlides.pdf
>>
>> You might see this as a call to arms, and it is, but a new thread will
>> be needed to follow up on this.
>>
>>> Extending from this, OSGeo endorsement of Rasdaman should be questioned
>>> and potentially withdrawn.
>>>
>>> I'd be interested to hear opinions of others in the field as to whether
>>> Rasdaman community version is of value for real-world production systems
>>> by itself.
>>>
>>> A deeper question for the greater OSGeo community is should OSGeo
>>> endorse Open Core business models?
>>>
>>> Warm regards, Cameron
>>>
>>> On 21/05/2016 6:07 pm, Peter Baumann wrote:
>>>> oh, just looking at the subject again:
>>>>
>>>> several service providers believe indeed rasdaman community does offer
>>>> a significant advantage:
>>>> - see the download figures on www.rasdaman.org
>>>> - concretely, see www.planetserver.eu which is running rasdaman
>>>> community on - I believe - about 20 TB of Planetary Science data.
>>>>
>>>> -Peter
>>>>
>>>>
>>>> On 05/21/2016 09:56 AM, Peter Baumann wrote:
>>>>> Hm, first of all: this is opening a different thread, talking about
>>>>> functionality of rasdaman community. Next, it is based on assumptions
>>>>> - without details (because off topic): conclusions are wrong.
>>>>>
>>>>> But to respond to the core message emphasized in the first paragraph:
>>>>> I respectfully disagree. In particular, such a position does not
>>>>> benefit the open source community very much as I am trying to explain
>>>>> below.
>>>>>
>>>>> TL;DR:
>>>>>
>>>>> You have a strong expertise in Geoinformatics, I know something about
>>>>> Computer Science. This is where we can talk as professors and
>>>>> scientists. Your statement is about economics, industry etc. Having
>>>>> an opinion there (and articulate it) is fair, but in these fields our
>>>>> opinion weighs not more than anyone else's in the street. We should
>>>>> not attempt to attain importance through inapplicable roles.
>>>>>
>>>>> Let us look at a professor. They have a conveniently high salary
>>>>> which is paid by society, that is: tax payers. Nobody can influence
>>>>> what a professor does and how much return s/he generates for society.
>>>>>
>>>>> A single open source developer (or a small group, whatever) do not
>>>>> experience this convenience. They have a dream where they invest,
>>>>> they try to not make money for getting richer than a professor ;-)
>>>>> but merely for their economic survival. Some (in particular
>>>>> scientists) enjoy the money rain coming from publicly funded projects
>>>>> (again: the tax payer subsidizes), but most in the community have to
>>>>> struggle hard. They face reluctant customers, competition by the
>>>>> giants in the market, and many more obstacles.
>>>>>
>>>>>  From the cosy place of a lifelong position with a secured salary and
>>>>> decent retirement funds it is easy to say that all software should be
>>>>> free like free beer (quote from below: "can be reproduced by other
>>>>> scientists without prohibitive license costs").
>>>>>
>>>>> If the open source movement cannibalizes itself it will make it all
>>>>> so easy for the big players to maintain their dominance, they will
>>>>> silently applaud. Quoting Jeroen:
>>>>>> NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
>>>>> THREAT, THEY ARE A NECESSITY.
>>>>>
>>>>> That said: It is entirely ok to have the opinion you have. Others,
>>>>> though, may disagree. I am one of those.
>>>>>
>>>>> respectfully,
>>>>> Peter
>>>>>
>>>>>
>>>>>
>>>>> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
>>>>>> As a scientist, I teach my students that for doing science it is a
>>>>>> requirement to work with open source software, because only then
>>>>>> workflows are fully transparent and can be reproduced by other
>>>>>> scientists without prohibitive license costs. Currently, working
>>>>>> with large amounts of earth observation (EO) or climate model data
>>>>>> typically requires to download these data tile by tile, stitch them
>>>>>> together, and go through all of them. Array databases may simplify
>>>>>> this substantially: after ingesting the tiles, they can directly
>>>>>> work on the whole data as a multi-dimensinal array ("data cube").
>>>>>> Computations on these array are typically embarassingly parallel,
>>>>>> and scale up with the number of cores in a cluster.
>>>>>>
>>>>>> Rasdaman is an array data base that comes in two flavours, the open
>>>>>> source community edition (CE) and the commercial enterprise edition
>>>>>> (EE). The differences between the two are clear [1]. When I want to
>>>>>> use rasdaman CE (open source) for scalable image analysis, I get
>>>>>> stuck waiting for one core to finish everything [1]. This is not
>>>>>> going to solve any problems related to computing on large data,
>>>>>> and is not scalable. The bold claim that rasdaman.org opens with
>>>>>> ("This worldwide leading array analytics engine distinguishes itself
>>>>>> by its flexibility, performance, and scalability") is not true for
>>>>>> the CE advertised. This has been mentioned in the past on mailing
>>>>>> lists [2,3], but the typical answer from Peter Baumann diverts into
>>>>>> other arguments. Also the benchmark graph (photo from an AGU poster)
>>>>>> [4] that Peter sent this week [5] must refer to the enterprise
>>>>>> edition, since Spark and Hive both scale, but rasdaman CE does not [3].
>>>>>>
>>>>>> I assume that on the discussions on this list, ONLY the open
>>>>>> source community edition is considered, compared, and discussed,
>>>>>> as a potential future OSGeo project.
>>>>>>
>>>>>> OSGeo supports the needs of the open source geospatial community [6].
>>>>>>
>>>>>> Given
>>>>>>
>>>>>>   * the bold claims and continuing confusion about whether,
>>>>>>     and which, rasdaman is scalable,
>>>>>>   * the need for OSGeo to give good advice to prospective users
>>>>>>     about technologies that do scale EO data analysis,
>>>>>>   * the current (unfilled!) needs of scientists for good, open source
>>>>>>     software for such analysis, and
>>>>>>   * the potential conflict of interest of its creator [7],
>>>>>>
>>>>>> I wonder wether OSGeo should recommend rasdaman CE to the open
>>>>>> source geospatial community.
>>>>>>
>>>>>>
>>>>>> [1] http://rasdaman.org/wiki/Features
>>>>>> [2] https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
>>>>>> [3] https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
>>>>>> [4]
>>>>>> https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd4/attachment.jpg
>>>>>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
>>>>>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
>>>>>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Discuss mailing list
>>>>>> [hidden email]
>>>>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>>>> --
>>>>> Dr. Peter Baumann
>>>>>   - Professor of Computer Science, Jacobs University Bremen
>>>>>     www.faculty.jacobs-university.de/pbaumann
>>>>>     mail: [hidden email]
>>>>>     tel: +49-421-200-3178, fax: +49-421-200-493178
>>>>>   - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>>>     www.rasdaman.com, mail: [hidden email]
>>>>>     tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>>>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>>>>
>>>>>
>>>> --
>>>> Dr. Peter Baumann
>>>>   - Professor of Computer Science, Jacobs University Bremen
>>>>     www.faculty.jacobs-university.de/pbaumann
>>>>     mail: [hidden email]
>>>>     tel: +49-421-200-3178, fax: +49-421-200-493178
>>>>   - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>>>>     www.rasdaman.com, mail: [hidden email]
>>>>     tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>>>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Discuss mailing list
>>>> [hidden email]
>>>> http://lists.osgeo.org/mailman/listinfo/discuss
>>> --
>>> Cameron Shorter,
>>> Software and Data Solutions Manager
>>> LISAsoft
>>> Suite 112, Jones Bay Wharf,
>>> 26 - 32 Pirrama Rd, Pyrmont NSW 2009
>>>
>>> P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
>>>

--
Cameron Shorter,
Software and Data Solutions Manager
LISAsoft
Suite 112, Jones Bay Wharf,
26 - 32 Pirrama Rd, Pyrmont NSW 2009

P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099

_______________________________________________
Discuss mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/discuss