Building image pyramids -> qgis crashing

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Building image pyramids -> qgis crashing

Mikhail Umorin-2
I want to speed up layer rendering so I want to build image pyramids. I use DOQ files, convert them to GeoTiff using mrsidecode and import those raster tiffs as layers. When I try to build the pyramids from layer properties 50 % of the time I qgis crashes:

<numerous error messages of different kind>
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: /home/mike/GIS/Dallas/DOQ-NAIP/d329602_3.tif:DumpModeDecode: Not enough data for scanline 1152
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: /home/mike/GIS/Dallas/DOQ-NAIP/d329602_3.tif:DumpModeDecode: Not enough data for scanline 1280
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: /home/mike/GIS/Dallas/DOQ-NAIP/d329602_3.tif:DumpModeDecode: Not enough data for scanline 1408
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: /home/mike/GIS/Dallas/DOQ-NAIP/d329602_3.tif:DumpModeDecode: Not enough data for scanline 1536
ERROR 1: TIFFReadEncodedTile() failed.
Warning 1: TIFFReadDirectory:/home/mike/GIS/Dallas/DOQ-NAIP/d329602_3.tif: Wrong "StripByteCounts" field, ignoring and calculating from imagelength
Segmentation fault

The <numerous error messages of different kinds> also come up when qgis does *not* crush when building pyramids, but sooner or later I get segfault with seemingly no difference in preceeding <numerous error messages of different kinds>

What is the problem and/or how can I track it down?

Mikhail.

P.S. QGis compiled from 0.9.1 source w/ gcc 4.2.2 on AMD Opteron  Linux kernel 2.6.22.18-server-1mdv #1 SMP, same problem also exists in Intel Xeon dual core same Linux


_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Oleg Gusev
Am Samstag 16 Februar 2008 00:57:28 schrieb Mikhail Umorin:
>
> ERROR 1: TIFFReadEncodedTile() failed.
...
>
> What is the problem and/or how can I track it down?
>
It's a longstanding gdal bug for
updating compressed .tiff tiles. The only
known workaround is to disable compression.

 Oleg.
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Mikhail Umorin-2

Spasibo za otvet (Thank you for the answer)

It's a longstanding gdal bug for
updating compressed .tiff tiles. The only
known workaround is to disable compression.

How do I do that?

M.

_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Maciej Sieczka - old
In reply to this post by Oleg Gusev
Oleg Gusev pisze:
> Am Samstag 16 Februar 2008 00:57:28 schrieb Mikhail Umorin:
>> ERROR 1: TIFFReadEncodedTile() failed.
> ...
>> What is the problem and/or how can I track it down?

> It's a longstanding gdal bug for
> updating compressed .tiff tiles. The only
> known workaround is to disable compression.

A way to avoid such problems in future would be to modify QGIS not to
build internal overviews (at least as an option). It could build
external overviews instead, in .aux files.

The benefits of external overviews:

1. The input raster doesn't need to be uncompressed.
2. The input data are not modified.
3. Input data remain compatible with software that don't tolerate
internal overviews.
4. Ergo - no need to duplicate data for safety/usability reasons before
building overviews.
5. Other software support external overviews that GDAL creates - eg.
ArcGIS, ERDAS Imagine.

What do developers think about it? Should I make an enhacement request?

Maciek

P.S.

For now, to create a external overwiev from CLI use eg.:
gdaladdo --config USE_RRD YES some.tif 2 4 8 16
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Oleg Gusev
In reply to this post by Mikhail Umorin-2
Am Sonntag 17 Februar 2008 01:56:55 schrieb Mikhail Umorin:
>
>> It's a longstanding gdal bug for
> >updating compressed .tiff tiles. The only
> >known workaround is to disable compression.
>
> How do I do that?
>
I'm usually using gdaladdo to create overviews, but
gdaladdo also has its own share of problems. The tile size is
hardcoded at 128x128 pixels, and sometimes it segfaults
on non-8bit paletted geotiffs. So this is more like a trial-and-error
solution.

 Oleg.

_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Maciej Sieczka - old
In reply to this post by Mikhail Umorin-2
Mikhail Umorin pisze:

> It's a longstanding gdal bug for
> updating compressed .tiff tiles. The only
> known workaround is to disable compression.

In Trac [1] Frank says the bug is fixed in "GDAL 1.4.4 with internal
libtiff, or libtiff 3.9 (from CVS)".

I confirm. Checked in QGIS trunk r8190, built against GDAL 1.5.0+SVN
2008-03-04 (internal libtiff). No crashes building overviews in QGIS
whatsoever. Checked with several Byte and Uint16 GeoTIFFs, all having
colortables, compressed with lzw, deflate or packbits.

Maciek

[1]https://trac.osgeo.org/qgis/ticket/605
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Oleg Gusev-2
Am Samstag 08 März 2008 18:20:34 schrieb Maciej Sieczka:
>
> In Trac [1] Frank says the bug is fixed in "GDAL 1.4.4 with internal
> libtiff, or libtiff 3.9 (from CVS)".
>
> I confirm. Checked in QGIS trunk r8190, built against GDAL 1.5.0+SVN
> 2008-03-04 (internal libtiff). No crashes building overviews in QGIS
> whatsoever.

 Hi Maciek,

  please try to build the overviews for a tiled compressed 3 band geotiff
  (gdal_translate -co compress=deflate -co tiled=yes)
  It segfaults for most images i've tried using gdal-1.5.0

Oleg.
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Maciej Sieczka - old
Oleg Gusev pisze:
> Am Samstag 08 März 2008 18:20:34 schrieb Maciej Sieczka:

>> In Trac [1] Frank says the bug is fixed in "GDAL 1.4.4 with internal
>> libtiff, or libtiff 3.9 (from CVS)".
>>
>> I confirm. Checked in QGIS trunk r8190, built against GDAL 1.5.0+SVN
>> 2008-03-04 (internal libtiff). No crashes building overviews in QGIS
>> whatsoever.

>   please try to build the overviews for a tiled compressed 3 band geotiff
>   (gdal_translate -co compress=deflate -co tiled=yes)
>   It segfaults for most images i've tried using gdal-1.5.0

It does not crash for me even with 3 band, deflated, tiled tiff.

I'm attaching a smallish one for you to try if it crashes your QGIS.

Maciek


_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user

slope_rgb_dfl_tls.tif (41K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Oleg Gusev-2
Am Samstag 08 März 2008 19:39:38 schrieb Maciej Sieczka:
>
> I'm attaching a smallish one for you to try if it crashes your QGIS.
>
This file is probably too small to cause any problems, but i routinely
get segfaults with bigger .tiff files.

 Oleg.

---------------------------------------
$ gdalinfo --version
GDAL 1.5.0, released 2007/12/18

$ gdalinfo FOO.tif
Driver: GTiff/GeoTIFF
Files: FOO.tif
Size is 8183, 5613
...
Band 1 Block=8183x1 Type=Byte, ColorInterp=Red
  NoData Value=255
Band 2 Block=8183x1 Type=Byte, ColorInterp=Green
  NoData Value=255
Band 3 Block=8183x1 Type=Byte, ColorInterp=Blue
  NoData Value=255

$ gdaladdo  FOO.tif 2 4 8 16 32
0...10...20.Warning 1: TIFFReadDirectory:FOO.tif: Wrong "StripByteCounts"
field, ignoring and calculating from imagelength
ERROR 1: FOO.tif:DumpModeDecode: Not enough data for scanline 256
ERROR 1: TIFFReadEncodedTile() failed.
...
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: IReadBlock failed at X offset 8, Y offset 19
ERROR 1: GetBlockRef failed at X block offset 8, Y block offset 19
ERROR 1: FOO.tif:DumpModeDecode: Not enough data for scanline 1152
...
Segmentation fault
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Maciej Sieczka - old
Oleg Gusev pisze:
> Am Samstag 08 März 2008 19:39:38 schrieb Maciej Sieczka:

>> I'm attaching a smallish one for you to try if it crashes your QGIS.

> This file is probably too small to cause any problems, but i routinely
> get segfaults with bigger .tiff files.

OK. Can you put one online and link it in the bug report [1]?

[1]https://trac.osgeo.org/qgis/ticket/605

Maciek
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

pcav
In reply to this post by Oleg Gusev-2
Oleg Gusev ha scritto:
> Am Samstag 08 März 2008 19:39:38 schrieb Maciej Sieczka:
>> I'm attaching a smallish one for you to try if it crashes your QGIS.
>>
> This file is probably too small to cause any problems, but i routinely
> get segfaults with bigger .tiff files.

I confirm the crash. The resulting raster is quite reddish (?).
Better move this discussion to the trac, IMHO.
pc
--
Paolo Cavallini, see: http://www.faunalia.it/pc
Io voto per il software libero:
http://elezioni.softwarelibero.it/info/iniziativa
Noi ci troviamo con parecchie difficoltà con NGI http://www.ngi.it/
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user
Reply | Threaded
Open this post in threaded view
|

Re: Building image pyramids -> qgis crashing

Maciej Sieczka - old
Paolo Cavallini pisze:

> I confirm the crash.

Paolo

What GDAL version are you using?

Maciek
_______________________________________________
Qgis-user mailing list
[hidden email]
http://lists.qgis.org/cgi-bin/mailman/listinfo/qgis-user