[gdal-dev] Slope obtained with GDAL has weird lines

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Slope obtained with GDAL has weird lines

Danilo da Rosa
Hi!

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Joaquim Luis

Ah, let me guess. The dem grid was interpolated from contour lines and you seeing the contours again.

 

From: gdal-dev <[hidden email]> On Behalf Of Danilo da Rosa
Sent: Monday, March 16, 2020 11:09 PM
To: [hidden email]
Subject: [gdal-dev] Slope obtained with GDAL has weird lines

 

Hi!

 

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

alexhagenzanker
Another guess:

The vertical resolution of your DEM is low, say height is measured by 1m intervals, and your horizontal resolution is relatively high, say the pixel size is 25m. Then the minimum slope is 1/25. For many pixels in flatter area, this means adjacent pixels are of exactly


From: gdal-dev <[hidden email]> on behalf of Joaquim Manuel Freire Luís <[hidden email]>
Sent: Tuesday, March 17, 2020 12:32:13 AM
To: Danilo da Rosa <[hidden email]>; [hidden email] <[hidden email]>
Subject: Re: [gdal-dev] Slope obtained with GDAL has weird lines
 

Ah, let me guess. The dem grid was interpolated from contour lines and you seeing the contours again.

 

From: gdal-dev <[hidden email]> On Behalf Of Danilo da Rosa
Sent: Monday, March 16, 2020 11:09 PM
To: [hidden email]
Subject: [gdal-dev] Slope obtained with GDAL has weird lines

 

Hi!

 

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

alexhagenzanker
Oops, message completed below...


From: gdal-dev <[hidden email]> on behalf of Alex HighViz <[hidden email]>
Sent: Tuesday, 17 March 2020, 06:29
To: Danilo da Rosa; [hidden email]; Joaquim Manuel Freire Luís
Subject: Re: [gdal-dev] Slope obtained with GDAL has weird lines

Another guess:

The vertical resolution of your DEM is low, say height is measured by 1m intervals, and your horizontal resolution is relatively high, say the pixel size is 25m. Then the minimum slope is 1/25. For many pixels in flatter area, this means adjacent pixels are of exactly same height, and for some it is the 1m step that is recorded as a contour line. Basically your data has the profile of a terraced rice paddy.

Could that be it? Alex


From: gdal-dev <[hidden email]> on behalf of Joaquim Manuel Freire Luís <[hidden email]>
Sent: Tuesday, March 17, 2020 12:32:13 AM
To: Danilo da Rosa <[hidden email]>; [hidden email] <[hidden email]>
Subject: Re: [gdal-dev] Slope obtained with GDAL has weird lines
 

Ah, let me guess. The dem grid was interpolated from contour lines and you seeing the contours again.

 

From: gdal-dev <[hidden email]> On Behalf Of Danilo da Rosa
Sent: Monday, March 16, 2020 11:09 PM
To: [hidden email]
Subject: [gdal-dev] Slope obtained with GDAL has weird lines

 

Hi!

 

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines



_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Michael Patrick
In reply to this post by Joaquim Luis
> I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84. ...   Does anyone have an idea of how to solve it?
 
Get better elevation data ... you didn't mention where in the world you are, but if you tell me, I can probably make some suggestions.

 
> Ah, let me guess. The dem grid was interpolated from contour lines and you seeing the contours again.

Can confirm, and they used a fairly crappy interpolation method also, it's not just terraced, it's wrinkled along the contour lines.

Michael Patrick
Data Ferret

Virus-free. www.avast.com

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Martin Chapman-3
In reply to this post by Danilo da Rosa
A slope map indicates the maximum rate of change between a pixel elevation value and its neighboring 8 pixels.  In other words, the output value for each pixel represents the steepest slope, or decent to the neighboring elevation values.  The output values are given as either percent slope or degrees from 0 (flat) to 90 (vertical) depending on the command line switches you choose.  

Therefore, the output raster values no longer represent elevation values from the input but rather slope (steepness) values and should be interpreted differently.  A common way to visually interpret them would be to color code them with a color ramp where each color in the ramp represents a range of degrees somewhere between 0 and 90.  That way a user can look at the image and quickly identify steep slopes versus not so steep slopes and flat areas.  To create a color ramp that matches changes in steepness you will need to use a statistical method on the slope values such as standard deviations or jenks natural breaks to group similar slope values together and where they change.  In other words, you will want your color ramp to change colors where flat areas start to get steeper.  I hope that makes sense.

Another way to use the results would be to write an algorithm to identify areas that are within a certain range of steepness so you could identify a good place to build a road or perhaps lay train tracks that must not exceed a certain steepness.  

I hope that helps.

Best regards,
Martin Chapman

On Mar 16, 2020, at 5:09 PM, Danilo da Rosa <[hidden email]> wrote:


Hi!

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Danilo da Rosa
Thanks for your answers Joaquim, Alex and Martin!

I think the problem is height is measured by 1m intervals, thatś why I see 1 meter contour lines in the slope result. The dem grid was not interpolated from contours, at least not by me, the data comes from Mapzen terrain tiles, I don't know exactly how they are producing it.

Do you think it would be a good idea to do some kind of interpolation to smooth the DEM file or the slope file? Do you have any recommendations on how to do that using gdal? The idea is to use the gdaldem color-relief command to generate a coloured and easy to read map. The problem is that this lines makes the map more difficult to understand, which is a priority in this case.

Thanks again!
Danilo


El mar., 17 mar. 2020 a las 4:50, Martin Chapman (<[hidden email]>) escribió:
A slope map indicates the maximum rate of change between a pixel elevation value and its neighboring 8 pixels.  In other words, the output value for each pixel represents the steepest slope, or decent to the neighboring elevation values.  The output values are given as either percent slope or degrees from 0 (flat) to 90 (vertical) depending on the command line switches you choose.  

Therefore, the output raster values no longer represent elevation values from the input but rather slope (steepness) values and should be interpreted differently.  A common way to visually interpret them would be to color code them with a color ramp where each color in the ramp represents a range of degrees somewhere between 0 and 90.  That way a user can look at the image and quickly identify steep slopes versus not so steep slopes and flat areas.  To create a color ramp that matches changes in steepness you will need to use a statistical method on the slope values such as standard deviations or jenks natural breaks to group similar slope values together and where they change.  In other words, you will want your color ramp to change colors where flat areas start to get steeper.  I hope that makes sense.

Another way to use the results would be to write an algorithm to identify areas that are within a certain range of steepness so you could identify a good place to build a road or perhaps lay train tracks that must not exceed a certain steepness.  

I hope that helps.

Best regards,
Martin Chapman

On Mar 16, 2020, at 5:09 PM, Danilo da Rosa <[hidden email]> wrote:


Hi!

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

David Strip-2
On 3/17/2020 2:24 PM, Danilo da Rosa wrote:
Do you think it would be a good idea to do some kind of interpolation to smooth the DEM file or the slope file? Do you have any recommendations on how to do that using gdal? The idea is to use the gdaldem color-relief command to generate a coloured and easy to read map. The problem is that this lines makes the map more difficult to understand, which is a priority in this case.

as has already been pointed out, the root of your problem is that your DEM appears to have been created from contours with no interpolation, resulting in a series of terraces with steps between each level, giving the appearance that the terrain consists of flat areas bounded by 90° slopes. Interpolating the existing DEM would relieve this problem, but there is no reason to believe the interpolation represents the actual surface. You would be better off starting with a new DEM that accurately represents the region of interest.  I used the SRTM downloader plug-in QGIS to download a new DEM. I saved this DEM in a projected coordinate system to get away from the problems of dealing with degrees (as issue pointed out by others). I then used the slope tool (which actually just calls gdal) to create this slope image:

The slope ranges from 0 (white) to about 30° (black).
SRTM data is 1 arc-second, or about 30 meters, which is coarser than you previous data, but at least it's the actual surface. There are higher resolution DEMs for much of the US on the US National Map. Maryland (where your patch appears to lie) is covered by 1/9 arc-second data and parts are covered by 1 meter DEMs derived (I think) from LIDAR.

You state your goal is a colored, easy to read map. I've seen articles about combining hillshade, elevation coloring, and slope, though I've more often seen just hillshade + elevation (the latter part being hypsometric tinting). I think you will find it easier to achieve this goal in Qgis, which will allow you to experiment interactively with the various parameters in tinting,  hill-shading, and combining the layers. For most of these operations Qgis uses gdal behind the curtain, so you can even see the gdal calls if you want to replicate the results from the command line.

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Martin Chapman-3
In reply to this post by Danilo da Rosa

Danilo,

 

Resampling only occurs when an image is being resized or warped.  If the dimensions are not changing then resampling is not really appropriate.  If you do need to resample, the utilities called gdal_translate.exe and gdalwarp.exe may be used depending on your needs.  Other utilities also provide resample options but they may not fit your purposes.

 

As a general rule you should NOT use nearest neighbor to resample elevation (floating point) data because it can drastically alter the elevation values from the original values and creates a staircase like effect in the output imagery.  Use something like cubic or bilinear.  That said, if the input data is bad then nothing you do will fix it.  Maybe look on the web for the USGS 3Dep elevation data if you are in the USA or SRTM 1 arc second data for continuous worldwide coverage.  Maybe the source data you are using is just bad or has been altered.

 

You should definitely NOT resample the slope output.  I think your source data may just be bad but it’s hard to know.  Download other sources and compare the output.  If the output looks similar then review your code or command line options.

 

Best regards,

Martin Chapman

 

From: gdal-dev [mailto:[hidden email]] On Behalf Of Danilo da Rosa
Sent: Tuesday, March 17, 2020 2:25 PM
To: [hidden email]
Subject: Re: [gdal-dev] Slope obtained with GDAL has weird lines

 

Thanks for your answers Joaquim, Alex and Martin!

 

I think the problem is height is measured by 1m intervals, thatś why I see 1 meter contour lines in the slope result. The dem grid was not interpolated from contours, at least not by me, the data comes from Mapzen terrain tiles, I don't know exactly how they are producing it.

 

Do you think it would be a good idea to do some kind of interpolation to smooth the DEM file or the slope file? Do you have any recommendations on how to do that using gdal? The idea is to use the gdaldem color-relief command to generate a coloured and easy to read map. The problem is that this lines makes the map more difficult to understand, which is a priority in this case.

 

Thanks again!

Danilo

 

 

El mar., 17 mar. 2020 a las 4:50, Martin Chapman (<[hidden email]>) escribió:

A slope map indicates the maximum rate of change between a pixel elevation value and its neighboring 8 pixels.  In other words, the output value for each pixel represents the steepest slope, or decent to the neighboring elevation values.  The output values are given as either percent slope or degrees from 0 (flat) to 90 (vertical) depending on the command line switches you choose.  

 

Therefore, the output raster values no longer represent elevation values from the input but rather slope (steepness) values and should be interpreted differently.  A common way to visually interpret them would be to color code them with a color ramp where each color in the ramp represents a range of degrees somewhere between 0 and 90.  That way a user can look at the image and quickly identify steep slopes versus not so steep slopes and flat areas.  To create a color ramp that matches changes in steepness you will need to use a statistical method on the slope values such as standard deviations or jenks natural breaks to group similar slope values together and where they change.  In other words, you will want your color ramp to change colors where flat areas start to get steeper.  I hope that makes sense.

 

Another way to use the results would be to write an algorithm to identify areas that are within a certain range of steepness so you could identify a good place to build a road or perhaps lay train tracks that must not exceed a certain steepness.  

 

I hope that helps.

 

Best regards,

Martin Chapman



On Mar 16, 2020, at 5:09 PM, Danilo da Rosa <[hidden email]> wrote:



Hi!

 

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Slope obtained with GDAL has weird lines

Danilo da Rosa
Thanks for your answers Martin and David.

Yes, you are right, I will use an other data source without that problem. An other user sent me an article about Release of NASADEM Data Products at https://lpdaac.usgs.gov/news/release-nasadem-data-products/.

I think that using this data would be the best option.

Thanks again!
Danilo

El mar., 17 mar. 2020 a las 19:50, Martin Chapman (<[hidden email]>) escribió:

Danilo,

 

Resampling only occurs when an image is being resized or warped.  If the dimensions are not changing then resampling is not really appropriate.  If you do need to resample, the utilities called gdal_translate.exe and gdalwarp.exe may be used depending on your needs.  Other utilities also provide resample options but they may not fit your purposes.

 

As a general rule you should NOT use nearest neighbor to resample elevation (floating point) data because it can drastically alter the elevation values from the original values and creates a staircase like effect in the output imagery.  Use something like cubic or bilinear.  That said, if the input data is bad then nothing you do will fix it.  Maybe look on the web for the USGS 3Dep elevation data if you are in the USA or SRTM 1 arc second data for continuous worldwide coverage.  Maybe the source data you are using is just bad or has been altered.

 

You should definitely NOT resample the slope output.  I think your source data may just be bad but it’s hard to know.  Download other sources and compare the output.  If the output looks similar then review your code or command line options.

 

Best regards,

Martin Chapman

 

From: gdal-dev [mailto:[hidden email]] On Behalf Of Danilo da Rosa
Sent: Tuesday, March 17, 2020 2:25 PM
To: [hidden email]
Subject: Re: [gdal-dev] Slope obtained with GDAL has weird lines

 

Thanks for your answers Joaquim, Alex and Martin!

 

I think the problem is height is measured by 1m intervals, thatś why I see 1 meter contour lines in the slope result. The dem grid was not interpolated from contours, at least not by me, the data comes from Mapzen terrain tiles, I don't know exactly how they are producing it.

 

Do you think it would be a good idea to do some kind of interpolation to smooth the DEM file or the slope file? Do you have any recommendations on how to do that using gdal? The idea is to use the gdaldem color-relief command to generate a coloured and easy to read map. The problem is that this lines makes the map more difficult to understand, which is a priority in this case.

 

Thanks again!

Danilo

 

 

El mar., 17 mar. 2020 a las 4:50, Martin Chapman (<[hidden email]>) escribió:

A slope map indicates the maximum rate of change between a pixel elevation value and its neighboring 8 pixels.  In other words, the output value for each pixel represents the steepest slope, or decent to the neighboring elevation values.  The output values are given as either percent slope or degrees from 0 (flat) to 90 (vertical) depending on the command line switches you choose.  

 

Therefore, the output raster values no longer represent elevation values from the input but rather slope (steepness) values and should be interpreted differently.  A common way to visually interpret them would be to color code them with a color ramp where each color in the ramp represents a range of degrees somewhere between 0 and 90.  That way a user can look at the image and quickly identify steep slopes versus not so steep slopes and flat areas.  To create a color ramp that matches changes in steepness you will need to use a statistical method on the slope values such as standard deviations or jenks natural breaks to group similar slope values together and where they change.  In other words, you will want your color ramp to change colors where flat areas start to get steeper.  I hope that makes sense.

 

Another way to use the results would be to write an algorithm to identify areas that are within a certain range of steepness so you could identify a good place to build a road or perhaps lay train tracks that must not exceed a certain steepness.  

 

I hope that helps.

 

Best regards,

Martin Chapman



On Mar 16, 2020, at 5:09 PM, Danilo da Rosa <[hidden email]> wrote:



Hi!

 

This may be a very basic question, but I can't find the answer anyware.

I am working with this DEM obtained using terraincache, it uses mapzen data. The SRC is WGS 84.

DEM File in QGIS

I ran this GDAL command to get the slope:

gdaldem slope -s 111120 dem.tif slope.tif (I had to do scaling because of the SRC)

The resulting image looks fine but it's not smooth, it has weird lines over it, like contour lines.

Slope file on QGIS

You can download the DEM file here: https://gofile.io/?c=Du7WnM

Does anyone have an idea of how to solve it?

Thanks in advance for your help,

Danilo

P.S. I made this question here too: https://gis.stackexchange.com/questions/353983/slope-obtained-with-gdal-has-weird-lines

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev