Use REST/curl to publish single GeoTIFF programmatically

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Use REST/curl to publish single GeoTIFF programmatically

Panagiotis Kountouris

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis





_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

Daniele Romagnoli-3
Hi Panagiotis,
REST is used to avoid the need to go through the GUI.
please, see my feedbacks inlined below...  

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?
(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

Afterwards, you may add new granules or update the mosaic, via POST.

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.
Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff

curl -v -u admin:geoserver -XPUT -d file://D:/data/geotiff/sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF2/external.geotiff

Quoting form the GeoServer doc:
These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

file—Uploads a file from a local source. The body of the request is the file itself.
url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.
external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

Whilst the extension parameter
specifies the type of coverage store. The following extensions are supported:

Extension Coverage store
geotiff         GeoTIFF
worldimage Georeferenced image (JPEG, PNG, TIFF)
imagemosaic Image mosaic

Please, let us know.
Regards,
Daniele
 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis



_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

Daniele Romagnoli-3


On Mon, Apr 1, 2019 at 2:22 PM Daniele Romagnoli <[hidden email]> wrote:
Hi Panagiotis,
REST is used to avoid the need to go through the GUI.
please, see my feedbacks inlined below...  

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?
(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

Afterwards, you may add new granules or update the mosaic, via POST.

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.
Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

In reference to these 2 commands, I forgot to say that they do the same thing, the first phisically uploading the file whilst the second one simply refers to an already existing file, as reported in the quote. So you just need to use one of the twos.
 
curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff

curl -v -u admin:geoserver -XPUT -d file://D:/data/geotiff/sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF2/external.geotiff

Quoting form the GeoServer doc:
These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

file—Uploads a file from a local source. The body of the request is the file itself.
url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.
external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

Whilst the extension parameter
specifies the type of coverage store. The following extensions are supported:

Extension Coverage store
geotiff         GeoTIFF
worldimage Georeferenced image (JPEG, PNG, TIFF)
imagemosaic Image mosaic

Please, let us know.
Regards,
Daniele
 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis



_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

Panagiotis Kountouris
In reply to this post by Daniele Romagnoli-3

Dear Daniele,

first I would like to thank you for your quick response and the very informative and helpful email. That cleared up lot of things! Secondly accept my apologies for taking me some time to apply your suggestions into my case.

For the ImageMosaic: I think (unfortunately I didn't use code revision yet since I am experimenting before go to production; hence not sure if that was my bug) I mixed up how I define the http request and the end points. Thanks to your email is much clearer now.

To update the mosaic (either add new granules or update the existing ones) my understanding is that I actually do not need to delete any previous shape index (that is the *.properties and *.shp files). Using the

curl -v -u admin:pswd -XPOST -H "Content-type: text/plain" -d "file:///data/geoserver/data/mosaic" "http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic"

seems it does the job with no extra action.

Regarding the Geotiff maybe my finding could be relevant for other users so I am posting my solution:

I discovered (by comparing layer properties using the gui as well) that after the curl command

curl -v -u admin:pswd -XPUT --data-binary "file:///data/geoserver/data/sic/test.tiff" "http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/external.geotiff"


the declared srs was undefined. Hence I had to force to use a projection by:

curl -v -u admin:pswd -XPUT -H "Content-type: text/xml" -d "<coverage><srs>EPSG:XXXX</srs><projectionPolicy>FORCE_DECLARED</projectionPolicy><enabled>true</enabled></coverage>" $SIC_URL/coverages/${SIC_FILE%.*}

Then the Geotiff layer was displayed with not problem :)

Many thanks again for your help!

Best regards

Panagiotis


On 01.04.19 14:22, Daniele Romagnoli wrote:
Hi Panagiotis,
REST is used to avoid the need to go through the GUI.
please, see my feedbacks inlined below...  

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?
(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

Afterwards, you may add new granules or update the mosaic, via POST.

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.
Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff


Quoting form the GeoServer doc:
These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

file—Uploads a file from a local source. The body of the request is the file itself.
url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.
external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

Whilst the extension parameter
specifies the type of coverage store. The following extensions are supported:

Extension Coverage store
geotiff         GeoTIFF
worldimage Georeferenced image (JPEG, PNG, TIFF)
imagemosaic Image mosaic

Please, let us know.
Regards,
Daniele
 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis



_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--
Regards,
Daniele Romagnoli
==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
==

Ing. Daniele Romagnoli
Senior Software Engineer

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
Italy
phone: +39 0584 962313
fax:      +39 0584 1660272


-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
-- 
Dr. Panagiotis Kountouris

Chief Research Officer
Drift & Noise Polar Services GmbH
www.driftnoise.com
Stavendamm 17
28195 Bremen, Germany

Office: +49 421 223 70 660
Mobile: +49 176 79 07 11 36


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

David Osborn
In reply to this post by Daniele Romagnoli-3

Good Afternoon,

 

I am attempting exactly the same thing:

 

I have a workspace and datastore configured and, to get things started, I placed a .tiff file in the store so that I could test everything. That worked fine and the layer preview displayed correctly. As the tiff is a satellite rainfall radar image, I want to periodically replace it with a new one, downloaded from the Met Office and converted to a GeoTiff using GDAL_TRANSLATE.

 

To upload the new file, I use the following command:

 

"\Program Files\Curl\bin\curl.exe" -v -u admin:geoserver -XPUT -H "Content-type:image/tiff" --data-binary @Rainfall.tif http://localhost:8080/geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff

 

I note that I appear to need to use the “file.geotiff” expression at the end of the destination url, which necessitated me changing the filename in the data store from .tiff.to .geotiff.

 

The result is that a temporary file is created in the data folder but the pre-existing file is not replaced. The log shows this:

 

Caused by: java.io.IOException: Failed to move C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff.a3ba2d0f-9fc2-43dc-a269-8ebaa40500b6.tmp - unable to remove existing: C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff

                at org.geoserver.platform.resource.Files.move(Files.java:403)

                at org.geoserver.platform.resource.FileSystemResourceStore$FileSystemResource$2.close(FileSystemResourceStore.java:280)

                at org.geoserver.rest.util.RESTUtils.handleBinUpload(RESTUtils.java:128)

                at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:73)

                ... 106 more

2019-04-04 18:40:00,216 WARN [annotation.ExceptionHandlerExceptionResolver] - Resolved [org.geoserver.rest.RestException 500 INTERNAL_SERVER_ERROR: Error while storing uploaded file:]

 

The command line output looks like this:

 

*   Trying ::1...

* TCP_NODELAY set

* Connected to localhost (::1) port 8080 (#0)

* Server auth using Basic with user 'admin'

> PUT /geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff HTTP/1.1

> Host: localhost:8080

> Authorization: Basic YWRtaW46Z2Vvc2VydmVy

> User-Agent: curl/7.56.1

> Accept: */*

> Content-type:image/tiff

> Content-Length: 1001270

> Expect: 100-continue

> 

< HTTP/1.1 100 Continue

* We are completely uploaded and fine

< HTTP/1.1 500 Server Error

< X-Frame-Options: SAMEORIGIN

< Transfer-Encoding: chunked

< Server: Jetty(9.4.12.v20180830)

< 

Error while storing uploaded file:* Connection #0 to host localhost left intact

 

If I try to delete the existing GeoTiff, I get an error that the file is in use by Java.exe.

 

Clearly I’m making a basic mistake here, but as a novice I’m struggling to see what is wrong. The basic requirement is, like Panagiotis, to replace an existing file with an updated one.

 

Thanks for any help

 

David

 

 

From: Daniele Romagnoli [mailto:[hidden email]]
Sent: 01 April 2019 14:01
To: Panagiotis Kountouris
Cc: [hidden email]
Subject: Re: [Geoserver-users] Use REST/curl to publish single GeoTIFF programmatically

 

 

 

On Mon, Apr 1, 2019 at 2:22 PM Daniele Romagnoli <[hidden email]> wrote:

Hi Panagiotis,

REST is used to avoid the need to go through the GUI.

please, see my feedbacks inlined below...  

 

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?

(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

 

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

 

Afterwards, you may add new granules or update the mosaic, via POST.

 

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.

Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

 

In reference to these 2 commands, I forgot to say that they do the same thing, the first phisically uploading the file whilst the second one simply refers to an already existing file, as reported in the quote. So you just need to use one of the twos.

 

curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff

 

curl -v -u admin:geoserver -XPUT -d file://D:/data/geotiff/sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF2/external.geotiff

 

Quoting form the GeoServer doc:

These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

 

file—Uploads a file from a local source. The body of the request is the file itself.

url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.

external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

 

Whilst the extension parameter

specifies the type of coverage store. The following extensions are supported:

 

Extension Coverage store

geotiff         GeoTIFF

worldimage Georeferenced image (JPEG, PNG, TIFF)

imagemosaic Image mosaic

 

Please, let us know.

Regards,

Daniele

 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis

 

 

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.



_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

Panagiotis Kountouris

Hi David,

my work around for this is the following:

1. create workspace, store and publish data via the GUI. Thats because I was lazy and wanted to get things quickly running.

2. Use rest interface to update the image. In bash the following works for me:

YOUR_FILE=test.tiff

DELETE_STORE_CMD=http://localhost:8080/geoserver/rest/workspaces/yourworkspace/coveragestores/yourstore?recurse=true

DATA_DIR=file:///your_data_path

DATA_URL=http://localhost:8080/geoserver/rest/workspaces/yourworkspace/coveragestores/yourstore/


I remove the old physical file:

rm /path_to_your_file/test.tiff

Then cp your new file to your data path.

I delete the store:

curl -v -u admin:geoserver -XDELETE $DELETE_STORE_CMD

I set again the store with the new GeoTIFF file

curl -v -u admin:geoserver -XPUT -H "Content-type: image/tiff" --data-binary $DATA_DIR/$YOUR_FILE $DATA_URL/external.geotiff


I then need to set the declared srs and works fine. To set the srs please see also  http://osgeo-org.1560.x6.nabble.com/Use-REST-curl-to-publish-single-GeoTIFF-programmatically-td5400133.html

Hope that helps

Best,

Panagiotis


On 05.04.19 13:42, David Osborn wrote:

Good Afternoon,

 

I am attempting exactly the same thing:

 

I have a workspace and datastore configured and, to get things started, I placed a .tiff file in the store so that I could test everything. That worked fine and the layer preview displayed correctly. As the tiff is a satellite rainfall radar image, I want to periodically replace it with a new one, downloaded from the Met Office and converted to a GeoTiff using GDAL_TRANSLATE.

 

To upload the new file, I use the following command:

 

"\Program Files\Curl\bin\curl.exe" -v -u admin:geoserver -XPUT -H "Content-type:image/tiff" --data-binary @Rainfall.tif http://localhost:8080/geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff

 

I note that I appear to need to use the “file.geotiff” expression at the end of the destination url, which necessitated me changing the filename in the data store from .tiff.to .geotiff.

 

The result is that a temporary file is created in the data folder but the pre-existing file is not replaced. The log shows this:

 

Caused by: java.io.IOException: Failed to move C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff.a3ba2d0f-9fc2-43dc-a269-8ebaa40500b6.tmp - unable to remove existing: C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff

                at org.geoserver.platform.resource.Files.move(Files.java:403)

                at org.geoserver.platform.resource.FileSystemResourceStore$FileSystemResource$2.close(FileSystemResourceStore.java:280)

                at org.geoserver.rest.util.RESTUtils.handleBinUpload(RESTUtils.java:128)

                at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:73)

                ... 106 more

2019-04-04 18:40:00,216 WARN [annotation.ExceptionHandlerExceptionResolver] - Resolved [org.geoserver.rest.RestException 500 INTERNAL_SERVER_ERROR: Error while storing uploaded file:]

 

The command line output looks like this:

 

*   Trying ::1...

* TCP_NODELAY set

* Connected to localhost (::1) port 8080 (#0)

* Server auth using Basic with user 'admin'

> PUT /geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff HTTP/1.1

> Host: localhost:8080

> Authorization: Basic YWRtaW46Z2Vvc2VydmVy

> User-Agent: curl/7.56.1

> Accept: */*

> Content-type:image/tiff

> Content-Length: 1001270

> Expect: 100-continue

> 

< HTTP/1.1 100 Continue

* We are completely uploaded and fine

< HTTP/1.1 500 Server Error

< X-Frame-Options: SAMEORIGIN

< Transfer-Encoding: chunked

< Server: Jetty(9.4.12.v20180830)

< 

Error while storing uploaded file:* Connection #0 to host localhost left intact

 

If I try to delete the existing GeoTiff, I get an error that the file is in use by Java.exe.

 

Clearly I’m making a basic mistake here, but as a novice I’m struggling to see what is wrong. The basic requirement is, like Panagiotis, to replace an existing file with an updated one.

 

Thanks for any help

 

David

 

 

From: Daniele Romagnoli [[hidden email]]
Sent: 01 April 2019 14:01
To: Panagiotis Kountouris
Cc: [hidden email]
Subject: Re: [Geoserver-users] Use REST/curl to publish single GeoTIFF programmatically

 

 

 

On Mon, Apr 1, 2019 at 2:22 PM Daniele Romagnoli <[hidden email]> wrote:

Hi Panagiotis,

REST is used to avoid the need to go through the GUI.

please, see my feedbacks inlined below...  

 

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?

(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

 

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

 

Afterwards, you may add new granules or update the mosaic, via POST.

 

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.

Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

 

In reference to these 2 commands, I forgot to say that they do the same thing, the first phisically uploading the file whilst the second one simply refers to an already existing file, as reported in the quote. So you just need to use one of the twos.

 

curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff

 

 

Quoting form the GeoServer doc:

These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

 

file—Uploads a file from a local source. The body of the request is the file itself.

url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.

external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

 

Whilst the extension parameter

specifies the type of coverage store. The following extensions are supported:

 

Extension Coverage store

geotiff         GeoTIFF

worldimage Georeferenced image (JPEG, PNG, TIFF)

imagemosaic Image mosaic

 

Please, let us know.

Regards,

Daniele

 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url>file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis

 

 

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


  



_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Reply | Threaded
Open this post in threaded view
|

Re: Use REST/curl to publish single GeoTIFF programmatically

David Osborn

Hello Panagiotis,

 

Thank you for that information. I will give it a try and see how I go. The server is running on Windows, but I think that I can make the necessary adjustments.

 

Best Wishes


David

 

 

From: Panagiotis Kountouris [mailto:[hidden email]]
Sent: 05 April 2019 14:02
To: David Osborn
Cc: 'Daniele Romagnoli'; [hidden email]
Subject: Re: [Geoserver-users] Use REST/curl to publish single GeoTIFF programmatically

 

Hi David,

my work around for this is the following:

1. create workspace, store and publish data via the GUI. Thats because I was lazy and wanted to get things quickly running.

2. Use rest interface to update the image. In bash the following works for me:

YOUR_FILE=test.tiff

DELETE_STORE_CMD=http://localhost:8080/geoserver/rest/workspaces/yourworkspace/coveragestores/yourstore?recurse=true

DATA_DIR=<a href="file:///\\your_data_path">file:///your_data_path

DATA_URL=http://localhost:8080/geoserver/rest/workspaces/yourworkspace/coveragestores/yourstore/

 

I remove the old physical file:

rm /path_to_your_file/test.tiff

Then cp your new file to your data path.

I delete the store:

curl -v -u admin:geoserver -XDELETE $DELETE_STORE_CMD

I set again the store with the new GeoTIFF file

curl -v -u admin:geoserver -XPUT -H "Content-type: image/tiff" --data-binary $DATA_DIR/$YOUR_FILE $DATA_URL/external.geotiff

 

I then need to set the declared srs and works fine. To set the srs please see also  http://osgeo-org.1560.x6.nabble.com/Use-REST-curl-to-publish-single-GeoTIFF-programmatically-td5400133.html

Hope that helps

Best,

Panagiotis

 

On 05.04.19 13:42, David Osborn wrote:

Good Afternoon,

 

I am attempting exactly the same thing:

 

I have a workspace and datastore configured and, to get things started, I placed a .tiff file in the store so that I could test everything. That worked fine and the layer preview displayed correctly. As the tiff is a satellite rainfall radar image, I want to periodically replace it with a new one, downloaded from the Met Office and converted to a GeoTiff using GDAL_TRANSLATE.

 

To upload the new file, I use the following command:

 

"\Program Files\Curl\bin\curl.exe" -v -u admin:geoserver -XPUT -H "Content-type:image/tiff" --data-binary @Rainfall.tif http://localhost:8080/geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff

 

I note that I appear to need to use the “file.geotiff” expression at the end of the destination url, which necessitated me changing the filename in the data store from .tiff.to .geotiff.

 

The result is that a temporary file is created in the data folder but the pre-existing file is not replaced. The log shows this:

 

Caused by: java.io.IOException: Failed to move C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff.a3ba2d0f-9fc2-43dc-a269-8ebaa40500b6.tmp - unable to remove existing: C:\Program Files (x86)\GeoServer 2.15.0\data_dir\data\ManxUtilities\RainfallRadar\RainfallRadar.geotiff

                at org.geoserver.platform.resource.Files.move(Files.java:403)

                at org.geoserver.platform.resource.FileSystemResourceStore$FileSystemResource$2.close(FileSystemResourceStore.java:280)

                at org.geoserver.rest.util.RESTUtils.handleBinUpload(RESTUtils.java:128)

                at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:73)

                ... 106 more

2019-04-04 18:40:00,216 WARN [annotation.ExceptionHandlerExceptionResolver] - Resolved [org.geoserver.rest.RestException 500 INTERNAL_SERVER_ERROR: Error while storing uploaded file:]

 

The command line output looks like this:

 

*   Trying ::1...

* TCP_NODELAY set

* Connected to localhost (::1) port 8080 (#0)

* Server auth using Basic with user 'admin'

> PUT /geoserver/rest/workspaces/ManxUtilities/coveragestores/RainfallRadar/file.geotiff HTTP/1.1

> Host: localhost:8080

> Authorization: Basic YWRtaW46Z2Vvc2VydmVy

> User-Agent: curl/7.56.1

> Accept: */*

> Content-type:image/tiff

> Content-Length: 1001270

> Expect: 100-continue

< HTTP/1.1 100 Continue

* We are completely uploaded and fine

< HTTP/1.1 500 Server Error

< X-Frame-Options: SAMEORIGIN

< Transfer-Encoding: chunked

< Server: Jetty(9.4.12.v20180830)

Error while storing uploaded <a href="file:///\\*">file:* Connection #0 to host localhost left intact

 

If I try to delete the existing GeoTiff, I get an error that the file is in use by Java.exe.

 

Clearly I’m making a basic mistake here, but as a novice I’m struggling to see what is wrong. The basic requirement is, like Panagiotis, to replace an existing file with an updated one.

 

Thanks for any help

 

David

 

 

From: Daniele Romagnoli [[hidden email]]
Sent: 01 April 2019 14:01
To: Panagiotis Kountouris
Cc: [hidden email]
Subject: Re: [Geoserver-users] Use REST/curl to publish single GeoTIFF programmatically

 

 

 

On Mon, Apr 1, 2019 at 2:22 PM Daniele Romagnoli <[hidden email]> wrote:

Hi Panagiotis,

REST is used to avoid the need to go through the GUI.

please, see my feedbacks inlined below...  

 

On Fri, Mar 29, 2019 at 11:13 AM Panagiotis Kountouris <[hidden email]> wrote:

Good day list members,

I am trying (amateur level) to use the REST interface, to publish an image-mosaic (works) and a single geotiff (still not working) layers via a WMS service.

I would kindly ask your feedback if this is the correct way to do that, and what am i doing wrong for the single geotiff case:

For the Image mosaic: I created with the GUI a workspace ("MYWSP") and a datastore (MYMSC) using the ImageMosaic plugin. I set also a data location folder where it is updated regularly. For some reason I was not able to just update the mosaic. So my workaround was to delete the coveragestore, and the files which contain the shp information and the *.properties of the mosaic. Then, I recreate it, every time there is new data available:

curl -v -u admin:pswd -XDELETE http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC?recurse=true

curl -v -u admin:pswd -XPUT -H "Content-type: text/plain" -d <a href="file:///\\data\geoserver\data\mosaic">file:///data/geoserver/data/mosaic http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYMSC/external.imagemosaic

This seems to work fine although I am not sure this is the proper way..

Could you please provide some more details about "For some reason I was not able to just update the mosaic"?

(As a note, when doing multiple attempts when creating a new imageMosaic via REST make sure you don't have partially configured or already existing layers/stores).  

 

In theory, once you have a workspace defined, you should be able to create a new ImageMosaic via PUT.

 

Afterwards, you may add new granules or update the mosaic, via POST.

 

 

For the single geotiff: I created a datastore (MYGT) using the GeoTIFF plugin. With the GUI i can create also a layer "mylayer" which displays the single geotiff file. I tried to update via REST the layer but no success:

curl -v -u admin:pswd -XPUT -H "Content-type: image/tiff" --data-binary <a href="file:///\\data\geoserver\data\sic\test.tiff">file:///data/geoserver/data/sic/test.tiff http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff

I tried also the file.geotiff instead of the external.geotiff with no success. Its unclear to me how one should use those extensions, and I could not find any related information.

In order to configure a GeoTIFF you just need to send the CURL command with no need to do other things through the GUI.

Provided that you already have a defined workspace (MYWSP), the sample commands I tested right now are:

 

In reference to these 2 commands, I forgot to say that they do the same thing, the first phisically uploading the file whilst the second one simply refers to an already existing file, as reported in the quote. So you just need to use one of the twos.

 

curl -v -u admin:geoserver -XPUT --data-binary @sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF/file.geotiff

 

curl -v -u admin:geoserver -XPUT -d <a href="file:///D:\data\geotiff\sampledata.tif">file://D:/data/geotiff/sampledata.tif http://localhost:8080/geoserver/rest/workspaces/MYWSP/coveragestores/MYGEOTIFF2/external.geotiff

 

Quoting form the GeoServer doc:

These endpoints (file, url, and external) allow a file containing either spatial data [...] to be added (via a PUT request) into an existing data store, or will create a new data store if it doesn’t already exist. The three endpoints are used to specify the method that is used to upload the file:

 

file—Uploads a file from a local source. The body of the request is the file itself.

url—Uploads a file from an remote source. The body of the request is a URL pointing to the file to upload. This URL must be visible from the server.

external—Uses an existing file on the server. The body of the request is the absolute path to the existing file.

 

Whilst the extension parameter

specifies the type of coverage store. The following extensions are supported:

 

Extension Coverage store

geotiff         GeoTIFF

worldimage Georeferenced image (JPEG, PNG, TIFF)

imagemosaic Image mosaic

 

Please, let us know.

Regards,

Daniele

 

Here is the output from the last curl command:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
* Server auth using Basic with user 'admin'
> PUT /geoserver/rest/workspaces/MYWSP/coveragestores/MYGT/external.geotiff HTTP/1.1
> Host: localhost:8080
> Authorization: Basic YWRtaW46czNjcjN0
> User-Agent: curl/7.52.1
> Accept: */*
> Content-type: image/tiff
> Content-Length: 72
>
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 201
< X-Frame-Options: SAMEORIGIN
< Content-Disposition: inline;filename=f.txt
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 29 Mar 2019 08:56:34 GMT
<
<coverageStore>
  <name>MYGT</name>
  <description>test data</description>
  <type>GeoTIFF</type>
  <enabled>true</enabled>
  <workspace>
    <name>MYWSP</name>
  </workspace>
  <__default>false</__default>
  <url><a href="file:///\\data\geoserver\data\sic\test.tif">file:/data/geoserver/data/sic/test.tif</url>
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

It looks that it has a 201 response, but in fact the layer is either broken or not publish, and I can not visualize it (I use qgis and loading as WMS layer). I am pretty sure I miss something here.. I would be grateful for any help on how to proceed with those issues...

Best regards

Panagiotis

 

 

_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.


 

--

Regards,

Daniele Romagnoli

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.

==

 

Ing. Daniele Romagnoli

Senior Software Engineer

 

GeoSolutions S.A.S.

Via di Montramito 3/A

55054  Massarosa (LU)

Italy

phone: +39 0584 962313

fax:      +39 0584 1660272

 

 

-------------------------------------------------------

 

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

 

This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.

 


_______________________________________________
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:
- Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


[hidden email]
https://lists.sourceforge.net/lists/listinfo/geoserver-users